DALL-E 2 + Representation 🔍
“To DALL-E, or not to DALL-E? That is the question.”
Hi there EInsighters! In anticipation of our next EQUATION Issue, we're diving into the topic of generative AI. Our focus this month is on DALL-E 2 + Representation.
As always, I’m your curator, Idil, and I'll be taking you on a journey through one of the most talked about tech scandals in the world, while also drawing on the EInsight of our selected EI Experts.
So let’s dive right in - shall we?
Generative AI has taken the world by storm and just about everyone and anyone is talking about it. DALL-E 2 has been a crowd favorite with people sharing AI-generated images of hilarious animals and cartoons on social media, but turns out not all is right when it comes to photorealistic images. OpenAI’s “Red Team”, a group of external researchers brought together to call out potential risks and limitations associated with DALL-E 2, warned the company not to go public with that feature due to its harmful consequences. The result? AI-generated photorealistic images of “lawyers” showed white men in suits, and the prompt “personal assistant” resulted in images of women. The Red Team found that in addition to replicating societal stereotypes regarding gender, the system over-represented white people and Western traditions. Although OpenAI has taken steps to reduce some forms of bias, there is still a long way to go to achieve representation for people of colour as well as the LGBTQ+ community.
So could it have been avoided?
Well, I’ve asked just that to our EI Experts, and this is what they had to say...
It comes as no surprise that OpenAI’s DALL-E 2 is composed of state-of-the-art Machine Learning techniques with a history of impressive empirical accuracy.
As Flavio puts it, “Good accuracy results from the capability to generalise well given appropriate training sets. For example, given (1) some concept [e.g. whatever it is that characterises a mechanical engineer working on an offshore oil rig] and (2) a lot of labelled examples [e.g. “whatever it is that characterises a mechanical engineer working on an offshore oil rig” is PRESENT in images such-and-such and ABSENT in images such-and-such], DALL-E 2 generates a function which can take as input any image and return PRESENT or ABSENT. This function then is accurate if, in most cases, it returns PRESENT when there is a representative of the target concept in the image and, in most cases, it returns ABSENT when the target concept is not in the image.”
“Based on this function, DALL-E 2 then summarises the function in a visual way. In other words, it generates a stereotype of the implicit characterisation of the concept that was built using examples and a function-generating algorithm.”
The real question then is - can an algorithm be created to be 100% bias-proof? Is complete representation possible, or will it forever be every developer’s white whale?
“Irrespective of how accurate an algorithm is, the universality of the obtained stereotypes will also depend on (1) the universality of the set of examples, and (2) the fine-grained characterisation of each element in the set as a representative of PRESENCE or ABSENCE of the target concept. Evidently, it is IMPOSSIBLE to build an absolutely universal set of examples in which every element has been characterised with an infinite set of attributes comprising everything that can possibly point to the target concept, therefore every set of examples is doomed to be (1) context-dependent and (2) biased.
This sad conclusion does not indicate, however, that Machine Learning and all that should be kicked straight to the garbage bin. The tools that have been developed in recent years have great potential to transform the world and, indeed, do so in a positive way. What it does indicate is that these tools are, by design, limited to the context and planned use that was considered at the time they were designed. Such limitations should be always explicit and clear to users, to ensure proper interpretation of results and their limits.”
Are there any steps companies could take today to help reduce bias and improve representation in the tools they develop?
Anna reiterates the need for companies to build diverse teams -
“Diverse teams will always help. When you have a diverse group of people and you’re about to deploy a feature/product/service, the chances are somebody will raise their hand and say, “Listen, this is not representing me. Let’s not launch yet, wait a bit, and make sure it includes me”. But of course, there are a range of things that need to happen first for that person to speak up.”
Most companies fall into the trap of hiring for the sake of “diversity” as a means of filling a certain quota in order to look good in mainstream media. But the reality is, being in the room doesn’t mean much if you aren’t provided with the necessary support and tools to enable you to make real change. More often than not, companies don’t know how to utilise diverse teams to their advantage because they haven’t implemented the required processes.
“Companies needs to cultivate an inclusive culture so that people who come from minority and under-represented groups feel safe to speak up without the fear that they might be punished for doing so. Companies need to work on two levels to accomplish this - there should be channels that enable employees to speak up directly as well as channels for whistleblowers to provide anonymous feedback.”
Just to make sure we’re all on the same page, what exactly do we mean by a team that's "diverse"?
“Diverse essentially means that the entire society is represented, which is a really difficult thing to do because society itself is very complex. To make things even more complicated, when focusing on the team aspect, we often overlook the individual aspect. Being a diverse team is not enough; we need to individually acknowledge our own self. As obvious as it may sound, you need to understand that you’re not “normal” - what you are is an individual that comes from a certain culture with different preferences, ways of going about things, standards, and so on. You need to be able to understand that what may be normal to you may not be normal for the entire population. Being aware of oneself is the first step towards diversity. It is understanding that what I am is not “normal”; it’s just one way of being. And once you accept that then teams can start the transformation towards understanding and embracing diversity to achieve success.”
Going back to DALL-E 2, there was an interesting statement made by OpenAI’s Chief Technology Officer, Mira Murati, in The Washington Post where she said the reason OpenAI did not follow the Red Team’s warning was because “removing the feature would prevent the company from figuring out how to do it safely”.
“If you already have a team that is analysing the feature, and they’re telling you very clearly that there are some safety issues, why are you deploying it even in small groups? Once the genie’s out of the bottle, it’s out there. Other groups can take it and use it however they want; they don’t have to follow terms and conditions, and the code can be copied, reused, changed, and so on… To me, it sets off several alarms. There must have been other pressures from different stakeholders within the company, something else that pushed people into sending that feature out. It simply does not make sense that in order to figure out how to do something safely, you need to first send it out to the public. It’s much safer to invite as many teams like the Red Team to be part of the innovation process from the definition.”
For those wanting to know more about when to involve an ethics team in the development of a product, check out this EQUATION article co-authored by Anna and another one of our EI Experts, Divyansh Agarwal.
So our technology seems to be advancing at an exponential speed - to what extent are our laws keeping up?
As always, we turn to Abigail to give us a breakdown of the legal side of things -
“There are countless legal ramifications. The DALL-E 2 system is loaded with massive amounts of images taken from the internet which could possibly be copyrighted or trademarked. With this, there is a danger of trademark and copyright infringement. Where the image generated is very similar to a work protected by copyright or a trademark, there is a potential risk of copyright and trademark infringement as well. Under the doctrine of fair use, there might be a potential exception to infringement in such circumstances. However, it can be argued that publicly available data is not explicitly upheld by U.S. legal precedent as fair use.
Then of course, the risk of data privacy also exists as the system is being trained on enormous amounts of personal data. In situations where consent is not obtained, there may be a breach of data privacy. Unfair representation and discrimination are also issues which may give rise to legal challenges. Furthermore, under tort law, there may be potential liability in the event that a system-generated image causes damage or injury to a person or property. It would then be an issue of trying to demonstrate causality which is difficult considering the different parties involved - from the software developers to the owners of the AI model, to the manufacturer of the product, to the designer… Who should be held responsible?
In addition, there are already questions concerning who owns the AI-generated image. The U.S. copyright law does not grant copyright protection to AI-generated images as they are not created by a human. For now, you cannot copyright your DALL-E 2 generations in the majority of countries. Indeed, we already have copyright laws in existence, but since AI image ownership disputes have not yet been contested, new legal arguments and legislation will be introduced. So, as AI continually evolves, there are many things to watch out for. Maybe employment replacement by AI isn't our primary concern. Perhaps we ought to consider the legal implications and how they will give rise to a brand-new body of law. When you don't have the solution to something, litigation frequently results, and from that, case law is created. Consequently, many of the answers to the questions we have with generative AI would be found in court cases.”
So until new case law emerges, what can we do?
Tomas suggests we turn to ethics -
“Ethics is primarily about responsibility. This responsibility is a shared responsibility between the legislator, the developer and the user.”
“One of the elements of responsibility that emerges in just about every AI system is the responsibility not to harm others (no harm principle). This principle invites to consider possible negative consequences when developing an AI system. In the case of DALL-E 2, it is not inconceivable to create content that causes harm to others. From the outset, OpenAI was aware of potential negative consequences of DALL-E 2. During the testing phase, their Red Team pointed out that there were potential dangers associated with the AI system. The same team then formulated a number of recommendations, not all of which were followed by the company. The AI was fitted with a number of filters that excluded sexually explicit content, violence and other potentially harmful inputs from the system, but OpenAI did not follow the warning of the Red Team about generating photorealistic faces. The only thing still stopping users from posting photorealistic faces on social media are now the terms of services that bring in the principle of human oversight in the use of the generated photos. The question, however, is whether that is enough to ensure that no harm is done. Shifting freedom and responsibility to the users not only shows great faith in humanity, but perhaps also a form of naivety and shifting own responsibility towards the users.”
“Stating that it is an AI-generated image is not enough here from an ethical point of view. After all, it does not take into account two premises. First, the impact of an image is not just a rational fact. Images also affect people subconsciously. Just think of what advertising tries to do. In this sense, images can also be used manipulatively. The manipulative use of images is not unique to images generated by AI, but the manipulative potential increases. A second premise that is too little taken into account is that mentioning that something is AI-generated only makes sense at the point when people also realise its scope. What percentage of the population can assess that this potentially means the image is fake? And does the rational consideration that the image should not be believed also completely negate the impact of the image?
Another element of responsibility that emerges clearly is the element of bias. In itself, countering discrimination can be seen as an element of the no harm principle cited above. Nevertheless, it deserves special attention in the context of AI. After all, the output of an AI system depends heavily on the input. If there is bias in the database used, there will also be bias in the output of the AI system. Often these are historical or social biases that are magnified even more sharply. Training algorithms on ever larger databases has certainly not reduced the risk of bias. Well-known examples are the exclusive portrayal of engineers as white men or the fact that nurses are shown as female by default. Sources of OpenAI point to the fact that bias is often of a deeply contextual nature, making it difficult to measure and mitigate the actual downstream harms resulting from use of DALL-E 2. A lot of work still has to be done there.”
But here’s the good news - you don’t have to do it all on your own. Take advantage of companies like Ethical Intelligence to place ethics at the forefront of your innovation process to ensure alignment between your values, technology and business objectives. Build a better tomorrow with the help of our EI Expert Network.
So what should your business do to avoid a similar tech scandal?
Clarify your product’s limitations to your users to ensure proper interpretation of results
Consult ethics experts early on in the innovation process
Check out our previous EInsight Issue on Deepfakes
Make sure to subscribe to our quarterly tech ethics magazine, EQUATION.
Coming Soon - “The Generative Revolution: Ethics in the New Wave of AI”
A massive thank you to our incredible EI Experts for their contribution in the making of this month's issue: Dr. Flavio S. Correa da Silva, Anna Danés, Abigail Ichoku, and Dr. Tomas Folens.
That's all for now, EInsighters! See you next month...
Liked this month's issue? Share it with a friend!
Have a tech scandal you want us to cover? Let me know!