Speaker 1: Okay, make a photo of a CEO and enter all
Speaker 1: right results. Oh, it made four different pictures and all
Speaker 1: of them appear to be light skinned men in suits. Okay,
Speaker 1: do that again, photo of a ce Oh and for
Speaker 1: more people who look like men to me. This is
Speaker 1: my very first time using an AI image generator, is
Speaker 1: through a free website on my laptop. Essentially, it's an
Speaker 1: AI model like chat GPT, but instead of creating AI
Speaker 1: generated text, it produces AI generated images. I type into
Speaker 1: a search box what I want to see, and a
Speaker 1: few seconds later it produces pictures that the model believes
Speaker 1: are what I'm asking asking for. All right, let's do
Speaker 1: this again. Photo of a physician and looks to me
Speaker 1: like men, but in lab coats and scrubs. Oh, and
Speaker 1: you know they're doctors because they all have stethoscopes around
Speaker 1: their necks. I requested images for several different jobs and
Speaker 1: repeated the requests at least ten times for each one.
Speaker 1: The results were eye opening. Almost all the images of
Speaker 1: CEOs and doctors the model produced, at least when I
Speaker 1: was using it, appeared to me to be men. All
Speaker 1: nurses and almost all teachers appeared to be women. By
Speaker 1: the way, I'm saying appeared to be women or men,
Speaker 1: because these images are people who don't actually exist, and
Speaker 1: so identifiers like gender, race, and ethnicity are subjective and
Speaker 1: we'll talk more about that a bit later. Most images
Speaker 1: of attorneys the model look to me to be light
Speaker 1: skinned men. Images of scientists appeared to be more diverse,
Speaker 1: but they also mostly looked like men, and this is
Speaker 1: really weird. When the model did generate pictures of attorneys
Speaker 1: or scientists who looked to me like women, they were
Speaker 1: very often shown dressed in traditional men's clothing, like business
Speaker 1: suits and wearing neckties, as though the model couldn't fully
Speaker 1: accept the concept of a woman in those professions. AI
Speaker 1: sometimes generates a distorted version of reality that doesn't look
Speaker 1: like the world we live in, and it can perpetuate
Speaker 1: gender and racial stereotypes. This matters because, as we know,
Speaker 1: AI is fast working its way into our lives, and
Speaker 1: AI generated images that can make us believe something artificial
Speaker 1: is actually something real, maybe especially influential and potentially harmful.
Speaker 2: So we're looking at a situation where we're generating more
Speaker 2: and more content via AI, more and more of these
Speaker 2: synthetic images. Those images become a part of the body
Speaker 2: of images, the body of work that's on the internet,
Speaker 2: they are more biased than reality. And then in the
Speaker 2: future those images get fed back into future AI systems,
Speaker 2: so that you end up in this nasty cycle where
Speaker 2: the bias is getting worse and worse and being fed
Speaker 2: back into future systems which are then less diverse.
Speaker 3: So there was a recent eurocal report that suggested that
Speaker 3: by twenty twenty six, ninety percent of all online content
Speaker 3: could be artificially generated. What happens when ninety percent of
Speaker 3: all online images are images reinforcing those stereotypes.
Speaker 1: Bloomberg's Dina Bass and Leonardo Nicoletti dug deep into the
Speaker 1: data to find out why the results look like this
Speaker 1: and what can be done to fix the shortcomings of
Speaker 1: this rapidly emerging technology. I'm wes Kasova today on the
Speaker 1: Big take. You can't trust your eyes when it comes
Speaker 1: to AI, like, maybe you can start by just giving
Speaker 1: us an overview of what you found in this investigation.
Speaker 3: This investigation essentially looks at generative AI, which is a
Speaker 3: new type of AI. It's like chat GPT where you
Speaker 3: ask it a question and it just answers you or
Speaker 3: gives you the information you need. In our case, we
Speaker 3: use stable diffusion, which is similar to chat GPT, but
Speaker 3: it's instead of text to text, which means like using
Speaker 3: texts to generate more text, it's text to image where
Speaker 3: you ask it a question or give it a description,
Speaker 3: and then it will generate an image for you of
Speaker 3: what you're looking for. And well, you know that gives
Speaker 3: us lots of possibilities and opens lots of doors for work,
Speaker 3: for design, for artistic advertising, lots of purposes. What we
Speaker 3: found is that it also has very strong bias is
Speaker 3: against people of color, women in general. And so what
Speaker 3: we wanted to show through this piece is really how
Speaker 3: biased is generative AI, and specifically generative AI that creates
Speaker 3: images and visual representations. And you know to what extent
Speaker 3: are these biased ingrained in this technology? And you know
Speaker 3: what are the potential implications of that.
Speaker 1: This is significant because it's different from the kind of
Speaker 1: facial recognition that we already know about, Is that right?
Speaker 4: It is?
Speaker 2: So somewhere around twenty eighteen we started finding out that
Speaker 2: facial recognition software had significant racial and gender biases. And
Speaker 2: what that software is You have a picture, an image,
Speaker 2: and the AI scans it and tries to predict what's
Speaker 2: in it?
Speaker 4: You know, what am I looking at? Is it a
Speaker 4: black cat, Is it a cheeseburger? Is it a white woman?
Speaker 4: What am I looking at?
Speaker 2: In twenty eighteen, a couple of researchers join Bulmini, Timnique Cabrew,
Speaker 2: and then Deboraji combined to do some work called gender sheets,
Speaker 2: where they ran a bunch of the popular facial recognition programs,
Speaker 2: ran tests on them and found that their performance was
Speaker 2: significantly worse on people of color and significantly worse on
Speaker 2: women of color. So we've known that that's an issue,
Speaker 2: and it's wrapped up in real world scenarios. There have
Speaker 2: been situations in the US where black men have been
Speaker 2: mistakenly arrested because they were flagged by facial recognition software.
Speaker 2: It turns out it was some completely different person, So
Speaker 2: we know that's a problem. Generative AI is a new
Speaker 2: type of AI, and it's a new wrinkle. So instead
Speaker 2: of AI that scans existing pictures, it's creating new ones
Speaker 2: and that we found also has significant racial and gender biases.
Speaker 2: So the additional, you know, significant issue that this raises
Speaker 2: is we're now using artificial intelligence to create massive volumes
Speaker 2: of new content. Then put out into the world for use.
Speaker 2: That new content is demonstrating racial and gender bias, and
Speaker 2: we're adding to a body of content out there, using
Speaker 2: it for reports, using it for clip art, for presentations,
Speaker 2: and it is significantly biased.
Speaker 1: And Leo, how did you go about finding this bias
Speaker 1: in this new form of generative AI?
Speaker 3: As a half reporter but also half former scientists, academic
Speaker 3: and just coder. The fact that many of these models
Speaker 3: generative AI models are open source was actually very useful
Speaker 3: for just researchers in general, but also reporters because it
Speaker 3: gives the possibility for anybody to download the generative AI model,
Speaker 3: in our case, Stable Diffusion and ask it to generate images.
Speaker 3: And so what I did is I simply went on
Speaker 3: the hugging Face platform, which is this really interesting and
Speaker 3: very useful platform that has come out recently that hosts
Speaker 3: all of these models, including source versions of GPT for example,
Speaker 3: and stable Diffusion, and I downloaded the model, and then
Speaker 3: I wrote some code to basically iterate through a series
Speaker 3: of very well known high paying and low paying jobs
Speaker 3: and also different criminalized activities, and just ask the model
Speaker 3: a very simple question, can you generate color photograph of blank?
Speaker 3: And Blank is a judge, an engineer, and a janitor,
Speaker 3: a housekeeper, a fast food worker. For professions, for example,
Speaker 3: and for criminalized activities, we looked at three of them,
Speaker 3: so Blank would be a terrorist, a drug dealer, or
Speaker 3: an inmate. I let my computer run for actually an
Speaker 3: entire month, because it's very computationally heavy to generate thousands
Speaker 3: of images for each of those keywords. So that was
Speaker 3: the first step.
Speaker 1: So you would just say to it, make me a
Speaker 1: picture of a CEO and then see what it came
Speaker 1: up with.
Speaker 3: Exactly. But the idea was to do that exact same
Speaker 3: thing thousands of times, so that instead of, you know,
Speaker 3: having anecdotal evidence that the AI might be biased, we
Speaker 3: would actually gather a database of images of the same
Speaker 3: thing over and over and over. Basically that would allow
Speaker 3: us as reporters and as data scientists to then analyze
Speaker 3: those thousands of images and actually find a pattern across
Speaker 3: those images. So that's exactly what we did.
Speaker 1: And what is the pattern that you found when you
Speaker 1: typed in ceo, when you typed in fast food worker
Speaker 1: all of the other things you mentioned, and then asked
Speaker 1: it to show you pictures thousands of times? What did
Speaker 1: it turn up.
Speaker 3: So the pattern is a very stark pattern. It's that
Speaker 3: for high paying professions, the generative AI model is overwhelmingly
Speaker 3: generating pictures of white men, and for low paying professions
Speaker 3: it's overwhelmingly generated more pictures of women and darker skinned people.
Speaker 3: So in our analysis we couldn't really talk about race
Speaker 3: because race is very hard to quantify in images, especially
Speaker 3: when you have images of fake people essentially that can't
Speaker 3: really self identify, So you can't say this is a
Speaker 3: black person or this is an Asian person. But what
Speaker 3: you can do is rigorous scientific analysis where you do
Speaker 3: things like average all the pixels of a person's skin
Speaker 3: across all of the images of one profession, and for example,
Speaker 3: doing that, what we found is that the pattern of
Speaker 3: darker skinned subjects being overrepresented in low paying professions and
Speaker 3: lighter skin subjects being overrepresented in high paying professions. And
Speaker 3: the same goes for criminalized activities, where you have darker
Speaker 3: skin tones constantly and systematically being represented in criminalized activities.
Speaker 1: Tina Leo was talking about stable diffusion. Exactly what is
Speaker 1: that and how does it work?
Speaker 2: So stable diffusion is a text to image program that
Speaker 2: is open source. It's distributed by a company called Stability Ai,
Speaker 2: and the version that we used is Leao mentioned hosted
Speaker 2: on hugging Face, which is basically a repository of open
Speaker 2: source AI model. So some of your listeners may have
Speaker 2: heard of GitHub, which is a repository of programming code.
Speaker 2: Hugging Face tries to be sort of like a version
Speaker 2: of that for AI models, and a lot of your
Speaker 2: listeners may have actually heard of a different image generation program,
Speaker 2: which is open AI's Dolli. Dolli to the second version
Speaker 2: of it came out in wider distribution last year and
Speaker 2: around July. It was announced a bit earlier than that,
Speaker 2: and that was also very popular and attracted a lot
Speaker 2: of attention. Stable Diffusion followed that and came out as
Speaker 2: an open source version, and because it was open source,
Speaker 2: it's been very widely used. In order to use the
Speaker 2: open ai version, you for you know, commercial applications, you
Speaker 2: have to work with open Ai.
Speaker 4: You have to pay for that, and so it's a
Speaker 4: little different.
Speaker 2: I just want to talk for a minute about why
Speaker 2: we did not look at open AI's Dolli, and that's
Speaker 2: because it's not open source, so we can't tell what
Speaker 2: is in the training data.
Speaker 4: For DOLLI in a way that we.
Speaker 2: Can for stable diffusion, and there are greater limits on
Speaker 2: what you can do with it.
Speaker 4: It was sort of difficult to look at the bias
Speaker 4: there and.
Speaker 1: Dan, by open source, what exactly do you mean.
Speaker 2: So it's basically the opposite of proprietary software.
Speaker 4: It's freely distributed. It's openly distributed.
Speaker 2: Anyone can download it, use it, and in the case
Speaker 2: of AI models, you have greater freedom to play with
Speaker 2: it to tweak different parts of the AI model to
Speaker 2: what you need.
Speaker 3: You can see exactly the code or the data that
Speaker 3: is going behind an AI model, and you can see
Speaker 3: the different versions of the model over time, and that's
Speaker 3: very important for people who are trying to improve these
Speaker 3: things because you can basically have some sort of version control,
Speaker 3: so control fork. The previous version used to be like this,
Speaker 3: and now we've improved it, and now we can see
Speaker 3: clearly the difference between the new version and the previous version. Actually,
Speaker 3: for this story, we did interview prominent academics within this field,
Speaker 3: and they've all really stressed this point that one of
Speaker 3: the only ways to address the problem of bias is
Speaker 3: to start by having open source models, because then those
Speaker 3: models can be taken by other academics or other organizations
Speaker 3: that are also transparent, and whatever they do to them
Speaker 3: to quote unquote improve them is now again made transparent,
Speaker 3: made very publicly known, and available to yet more academics
Speaker 3: to improve upon it.
Speaker 4: Again, there's also greater auditability.
Speaker 2: The reason that we were able to run this experiment
Speaker 2: on stable diffusion is that it's open source. So you know,
Speaker 2: we obviously found some significant problems, but there is that auditability.
Speaker 2: You don't have that with open aies Dolly and so again.
Speaker 2: Open AI has said that they're taking steps to address
Speaker 2: representation and make sure that the outputs are representative, but
Speaker 2: you kind of have to trust them because you don't
Speaker 2: know what they're doing.
Speaker 1: After the break, what's the data set behind these AI
Speaker 1: generated images? We know from everything we've been hearing all
Speaker 1: about chat GPT now GPT for that it takes as
Speaker 1: its source enormous amounts of data that exists on the Internet.
Speaker 1: What is the source material for generative AI when it
Speaker 1: comes to images.
Speaker 3: So, the source material for most generative AIS models, these
Speaker 3: so called large language models, it's basically the entire Internet.
Speaker 3: In simple terms, it's everything that's been posted on the
Speaker 3: Internet in the past ten fifteen years. The way that
Speaker 3: works is that there is a data set called Lyon,
Speaker 3: which basically collected URLs to images or texts for the
Speaker 3: past fifteen years all over the Internet.
Speaker 2: When you're training on data from across the entire Internet,
Speaker 2: as most of us know, there's a fair amount of
Speaker 2: unsavory stuff out there on the Internet. And you know,
Speaker 2: there's been some academic work done on the earlier version
Speaker 2: of this data set, this line On data set that
Speaker 2: found pornography, violence, again, racial and gender bias. When certain
Speaker 2: terms that were associated with certain races were used, it
Speaker 2: was much more likely to bring up an image that
Speaker 2: was sexualized.
Speaker 4: So there are a lot of problems.
Speaker 2: Within that data set, and it is an openly available,
Speaker 2: open source data set, and so the viewpoint of the
Speaker 2: people behind it is, look, you know, you should use
Speaker 2: this for academic work. If you're using this in a
Speaker 2: commercial product, you've got to actually take some responsibility for
Speaker 2: the content, and we've made some not Safe.
Speaker 4: For work filters. There are steps you can take, but you.
Speaker 2: Know, when you're training a model on a large volume
Speaker 2: of data from across the entire Internet, there is a
Speaker 2: lot of unsavory stuff in there, and there are way
Speaker 2: way too many images in this data set for anybody
Speaker 2: to go through it and make sure that they're cleaning
Speaker 2: it up.
Speaker 1: Diana, what does Stable Diffusion say about your findings about
Speaker 1: this bias in their data?
Speaker 2: We reached out to Stable Diffusion and explained what we
Speaker 2: were finding, and they sent us in an email state
Speaker 2: informous spokesperson saying that quote, all AI models have inherent
Speaker 2: biases that are representative of the data sets they're trained on,
Speaker 2: and by open sourcing our models, we aim to support
Speaker 2: the AI community and collaborate to improve bias evaluation techniques
Speaker 2: and develop solutions beyond the basic prompt modification. The company
Speaker 2: also told us that, you know, they have sort of
Speaker 2: an initiative to developed some open source models that will
Speaker 2: be trained on data sets that are specific to different
Speaker 2: countries and cultures, and so part of the argument the
Speaker 2: company was making is that the open source nature of
Speaker 2: what they're doing will enable them to address some of
Speaker 2: these issues by getting more and more data that is
Speaker 2: more diverse than what they currently have.
Speaker 1: Dina, we can see why bias would be so harmful,
Speaker 1: especially when it comes to images, which are very powerful.
Speaker 1: What are some of the real world downsides that we
Speaker 1: see with the possibility of fake images, bias and images
Speaker 1: being proliferated all over the world.
Speaker 2: There is an issue of deep fakes, things that are
Speaker 2: meant to mislead people, misinformation that you can't tell as
Speaker 2: AI generated. With the specific issue of bias. There's a
Speaker 2: number of issues that crop up here. One is a
Speaker 2: representation one. So if we're going to start using all
Speaker 2: of these synthetic generated images for brochures, for advertisements, for
Speaker 2: marketing materials, and we're already seeing what happens when the
Speaker 2: marketing materials have all the CEOs be white men, doesn't
Speaker 2: that worse in the situation that we already have. You know,
Speaker 2: one of the things that we found in this experiment
Speaker 2: was that the bias in the unstable diffusion was actually
Speaker 2: worse than the real world. So we know that there
Speaker 2: are fewer female CEOs, but the number of female CEOs
Speaker 2: that were being generated in these experiments was even lower
Speaker 2: than the real world. So we're looking at a situation
Speaker 2: where we're generating more and more content via AI, more
Speaker 2: and more of these synthetic images. Those images become a
Speaker 2: part of the body of images, the body of work
Speaker 2: that's on the internet, they are more biased than reality,
Speaker 2: and then in the future those images get fed back
Speaker 2: into future AI systems, so that you end up in
Speaker 2: this nasty cycle where the bias is getting worse and
Speaker 2: worse and being fed back into future systems which are
Speaker 2: then less diverse.
Speaker 3: So there was a recent eurocal report that suggested that
Speaker 3: by twenty twenty six, ninety percent of all online content
Speaker 3: could be artificially generated. What happens when ninety percent of
Speaker 3: all online images are images reinforcing those stereotypes? One of
Speaker 3: the main impacts can really affect people's mental health and
Speaker 3: how they project themselves into the world and you know,
Speaker 3: what kind of jobs that they see themselves doing in life.
Speaker 3: So that's a really big issue that can definitely be
Speaker 3: reinforced by this problem.
Speaker 1: When we come back. How can artificial intelligence become more intelligent?
Speaker 1: Diina earlier layout said that these open source models have
Speaker 1: one advantage, which is that everybody is able to kind
Speaker 1: of work on them and improve them. And if you are, say,
Speaker 1: had an advertising agency making a brochure and you ask
Speaker 1: it to create a CEO and it's a white male,
Speaker 1: can't you say no, that's not the image I'm looking
Speaker 1: for that there's a certain amount of responsibility of people
Speaker 1: who are generating these images not to just simply accept
Speaker 1: what the generative AI bot spits out.
Speaker 2: When we talk about AI bias, a lot of the
Speaker 2: quote unquote blame for it gets put on the data sets.
Speaker 2: There needs to also be accountability from users at all levels,
Speaker 2: and that includes the people that are creating the models,
Speaker 2: the researchers that are working on the models, who have
Speaker 2: their own biases that get kind of imprinted on these models,
Speaker 2: and it includes the people that are using them.
Speaker 4: At the end of the day, it's not totally clear.
Speaker 2: To me that you can currently use these models that
Speaker 2: effectively to even specify in that way and get the
Speaker 2: output you want.
Speaker 1: So do you know what can actually be done to
Speaker 1: fix this? We talked earlier about how there's a lot
Speaker 1: of work being done to improve these models.
Speaker 2: One of the things that needs to be done is
Speaker 2: increased diversification of the data set to be a way
Speaker 2: to get data from other countries, other cultures, and there
Speaker 2: needs to be to be clear a way.
Speaker 4: To do that that's ethical.
Speaker 2: There have been projects or companies that have tried to
Speaker 2: source a more diverse set of data, but they've done
Speaker 2: it in unethical ways. They've tried to get images of people,
Speaker 2: and they've done it without consent. This is sort of
Speaker 2: cropped up in the facial recognition era when people are
Speaker 2: trying to fix those systems, just as a question about
Speaker 2: the largeness of all of these models. So the current
Speaker 2: trend in AI is that bigger is better, that the
Speaker 2: only way to do these kind of foundational models is
Speaker 2: to have the sum total of the Internet dumped into
Speaker 2: the training data. There are people working on ways to
Speaker 2: do better smaller models, in which case you have greater
Speaker 2: control over what is in the data set and you
Speaker 2: can do things that are more targeted. If we move
Speaker 2: to optimizing the technology where you don't just have to
Speaker 2: add more volume in order to have a better performing
Speaker 2: algorithm or model, that could help as well.
Speaker 1: LAO is somebody who is deep in this data and
Speaker 1: watching how it's developing very rapidly. What are you watching
Speaker 1: for as this keeps unfolding.
Speaker 3: One of the most interesting developments is really the open
Speaker 3: source versus closed source models, And you know which are
Speaker 3: going to become the status quo because you know it's
Speaker 3: not clear right now. It's very easy to use closed
Speaker 3: source models in some way because they have better user interfaces,
Speaker 3: and they market it better, and you know, it's it's
Speaker 3: for profits, so they have all these ways to kind
Speaker 3: of like get really mainstream. But at the same time,
Speaker 3: the open source models are being used by millions of people,
Speaker 3: not just people you know that are using them like
Speaker 3: as a developers or researchers. And then we also see
Speaker 3: private companies adapting open source models as opposed to closed
Speaker 3: source models because they actually recognize the fact that they
Speaker 3: can and build on top of those models within their systems.
Speaker 3: It's very unclear and it will be interesting to see
Speaker 3: if five ten years from now, generative bi has become
Speaker 3: completely a closed source thing because it's easier to regulate.
Speaker 3: You can just regulate private companies and tell them what
Speaker 3: to do, or it's become an open source thing because
Speaker 3: there's more transparency and it's easier to see if things
Speaker 3: are getting better or not.
Speaker 1: Leo Dina, thanks so much for coming on the show.
Speaker 4: Thank you, Wes, thank you for having us.
Speaker 1: Thanks for listening to us here at the Big Take.
Speaker 1: It's a daily podcast from Bloomberg and iHeartRadio. For more
Speaker 1: shows from iHeartRadio, visit the iHeartRadio, app, Apple Podcasts or
Speaker 1: wherever you listen, and we'd love to hear from you.
Speaker 1: Email us questions or comments to Big Take at Bloomberg
Speaker 1: dot net. The supervising producer of The Big Take is
Speaker 1: Vicky burg Alina. Our senior producer is Catherine Fink. Frederica
Speaker 1: Romanello is our producer. Our associate producer is Zenobsiiti. Raphael
Speaker 1: M Seely is our engineer. Our original music was composed
Speaker 1: by Leo Sidrin. I'm West Kasova. We'll be back tomorrow
Speaker 1: with another Big Take