Speaker 1: Welcome to tech Stuff, a production from iHeartRadio. Hey there,
Speaker 1: and welcome to tech Stuff. I'm your host, Jonathan Strickland.
Speaker 1: I'm an executive producer with iHeart Podcasts. And how the
Speaker 1: tech are you? So? Imagine for a moment that you
Speaker 1: are in school. Some of y'all might actually be in school,
Speaker 1: but others, like me, we have to satisfy ourselves by
Speaker 1: having that occasional stress dream where we imagine that we're
Speaker 1: in school and it's time to take a final and
Speaker 1: we haven't gone to class all year, and also we
Speaker 1: can't remember our locker combination. I don't know about you,
Speaker 1: but I still occasionally get those dreams. And I'm almost
Speaker 1: fifty years old at this point. Anyway, you're in school,
Speaker 1: you're in English class, and you've been given the dreaded
Speaker 1: term paper assignment. You're told you need to go to
Speaker 1: the library and you have to gather resources and read
Speaker 1: up and form your thesis and write your paper while
Speaker 1: making verifiable citations all the way through. So off you
Speaker 1: go to the library. However, you discover, horror of horrors,
Speaker 1: that all the resource books have disappeared. They're none in
Speaker 1: their place. Are other student term papers? Now? Some of
Speaker 1: those term papers are pretty good, some of them are terrible.
Speaker 1: Nearly all of them do have a list of references
Speaker 1: at the end, But the problem is that you don't
Speaker 1: have access to those references. You only have access to
Speaker 1: the term papers, which, in a way, you could say
Speaker 1: is a filtered view of those references. But you have
Speaker 1: no way of knowing if the student who wrote the
Speaker 1: term papers you've pulled out did a proper citation. You
Speaker 1: don't know if the student understood the source material. You
Speaker 1: don't know if they have made a valid reference using
Speaker 1: that source. You don't know if the student didn't understand
Speaker 1: the source and thus misconstrued the information, either accidentally or
Speaker 1: on purpose, or if the student is just outright plagiarizing
Speaker 1: the source material or making stuff up. So how do
Speaker 1: you think your own term paper would turn out? Probably
Speaker 1: it'd be a challenge to write a good term paper.
Speaker 1: It definitely would be difficult or almost impossible to support
Speaker 1: your thesis using citations, because all you would have access
Speaker 1: to would be other term papers. Chances are you'd have
Speaker 1: a pretty lousy grade by the end of that assignment. Now,
Speaker 1: I started off this episode with that analogy because today
Speaker 1: we're going to talk about what happens when AI models
Speaker 1: train off stuff that was generated by other or sometimes
Speaker 1: even the same but earlier versions of AI models. So
Speaker 1: when bots make stuff that other bots consume, and then
Speaker 1: those other bots make new stuff and the cycle goes on.
Speaker 1: Where are the humans in this picture. Maybe they're in
Speaker 1: an actual library, because the online resources will all have
Speaker 1: become practically useless. So if we want to actually learn anything,
Speaker 1: we're gonna need to go back to the basics. So
Speaker 1: we're going to talk about an idea called model collapse,
Speaker 1: as in large language models LMS and other types of
Speaker 1: AI models. We're going to build to that. However, first up,
Speaker 1: let's explore the tendency of AI models to produce wrong
Speaker 1: or misleading results, regardless of whether the material used to
Speaker 1: train that AI model came from AI or humans. This
Speaker 1: is something I've talked about in past episodes, but it's
Speaker 1: an important part to kind of build toward our understanding
Speaker 1: of what model collapse is. Now. In past episodes, I've
Speaker 1: talked about the issue of AI hallucinations, also sometimes called confabulations.
Speaker 1: Some people prefer confabulations to hallucinations. This is the tendency
Speaker 1: for generative AI to mistakenly include untrue or misleading information,
Speaker 1: or to insert stuff that does not belong into whatever
Speaker 1: it is that's creating, whether that's an image or text
Speaker 1: or what an so. One fairly recent example of this
Speaker 1: was when Google's AI augmented search tool suggested that you
Speaker 1: add a non toxic glue to your pizza ingredients if
Speaker 1: you want to solve the irritating issue of cheese slip
Speaker 1: sladden away off your ding dang dern pizza. Clearly this
Speaker 1: answer is not acceptable. Adding glue, non toxic or otherwise
Speaker 1: is not a way of making good eats. I'm pretty
Speaker 1: sure Alton Brown would agree with me, and actually I
Speaker 1: would argue this is one of the less egregious cases
Speaker 1: of AI providing a bad answer. It's famous because it
Speaker 1: got a lot of traction. It went viral for how
Speaker 1: bad the answer was. But in the grand scheme of things,
Speaker 1: there are other examples that were far more potentially harmful.
Speaker 1: So why does AI do this sometimes? Well, there are
Speaker 1: a few different contributing factors that lead AI to making
Speaker 1: these mistakes. By the way, the reason why some people
Speaker 1: prefer confabulations as opposed to hallucinations. Hallucination sounds like the
Speaker 1: AI I has somehow been tricked into thinking something is
Speaker 1: what it isn't right, like the idea that you hallucinate
Speaker 1: your seeing or hearing or experiencing something that's not really there.
Speaker 1: Confabulation suggests that the AI is inventing something. It is confabulating,
Speaker 1: it is creating an answer where there was none, and
Speaker 1: so some people prefer the second one because they but
Speaker 1: it puts more of the onus on the AI model itself.
Speaker 1: So one of the factors that contributes to AI making mistakes.
Speaker 1: And you know, large language models and like are in
Speaker 1: part focused on pattern recognition, and this can lead to issues. Now,
Speaker 1: recognizing patterns is what gives these models the ability to
Speaker 1: form relevant and coherent responses to queries, and obviously pattern
Speaker 1: recognition is important otherwise you're just gonna perceive everything is
Speaker 1: being random and meaningless and then really, this whole conversation
Speaker 1: doesn't mean anything either, or if the whole universe is meaningless,
Speaker 1: then what are we even doing here? But I don't
Speaker 1: want you to go down that path of existential dread.
Speaker 1: So sometimes AI will detect a pattern where there really
Speaker 1: isn't a pattern. And we humans do this too, you know,
Speaker 1: we sometimes experience like paradolia. For example. That's when we
Speaker 1: perceive something meaningful within an otherwise meaningless thing, like we
Speaker 1: see a pattern where there is none. So if you
Speaker 1: were to look at the clouds and you say that
Speaker 1: one of them looks very like a whale, that's paradolia.
Speaker 1: It's also a reference to Hamlet the infamous face on Mars,
Speaker 1: which was really just a hill with some shadows cast
Speaker 1: on it. Because the angle of the image, that was
Speaker 1: another example of paradolia, people began to think that there
Speaker 1: was actually a big sculpted face on Mars. There's not.
Speaker 1: It's a hill. The shadows hit the hill in a
Speaker 1: specific way that made it look kind of like the
Speaker 1: face of an enormous statue, something like the Sphinx, something
Speaker 1: along those lines. But in fact it was just a hill.
Speaker 1: And if you took another image from a different angle,
Speaker 1: which people have done, the illusion of a face disappears.
Speaker 1: So again, that was us inventing a pattern where there
Speaker 1: was none. Now, much of the time we humans can
Speaker 1: recognize when the things we see, you know, the shapes
Speaker 1: of faces or whatever it may be, aren't actually there. Right,
Speaker 1: we can recognize, oh, that looks like a blah, blah blah,
Speaker 1: but we know it's not actually a real image of that.
Speaker 1: It just happens. Now. Sometimes we don't recognize this. Sometimes
Speaker 1: there are ties where people will assume that what they're
Speaker 1: seeing is an actual image made with intent and intelligence,
Speaker 1: perhaps not by humans but by something. So there are
Speaker 1: all those stories of people going bonkers because they believe
Speaker 1: they saw an image of like the Virgin Mary in
Speaker 1: a potato chip or whatever. And machines don't necessarily have
Speaker 1: any checks against fall hits when it comes to pattern recognition,
Speaker 1: and then they might act on a perceived pattern, which
Speaker 1: means the machines produce bad results. What's more, machines conceive
Speaker 1: patterns where we can't. Like sometimes there are patterns present
Speaker 1: that we cannot perceive because maybe the dataset is far
Speaker 1: too large or far too complicated, and so we can't
Speaker 1: perceive where the pattern is. It's just beyond our abilities
Speaker 1: to do so. But sometimes machines can detect those patterns,
Speaker 1: and sometimes they are meaningful. So it can be really tricky.
Speaker 1: If a machine thinks it's found a pattern, it can
Speaker 1: be hard for people to verify or discredit that because
Speaker 1: it's on a scale that we humans are not really
Speaker 1: well equipped to handle with generative AI. This can mean
Speaker 1: that the AI model correctly identifies that it needs to
Speaker 1: use a specific syntax to craft a response to whatever
Speaker 1: query or direction it was given, and it can thus
Speaker 1: put together a sentence that grammatically makes sense. What's happening
Speaker 1: is it's essentially statistically analyzing the structure of hundreds of
Speaker 1: millions of sentences, as well as the role that certain
Speaker 1: words play within those sentences, so that it quote unquote
Speaker 1: knows how to write a grammatically correct response, and ultimately
Speaker 1: it's using statistics to pick what should be the most
Speaker 1: correct word in each position of that sentence. So ideally,
Speaker 1: it's pulling information from various sources that are related to
Speaker 1: whatever it is you're asking about and pulling the words
Speaker 1: together in a way that makes logical sense and is accurate,
Speaker 1: and it's a correct answer to whatever your question is.
Speaker 1: But that doesn't always happen right. Sometimes it can't find
Speaker 1: the right word. Sometimes it finds a different word that
Speaker 1: it thinks is right, but it's not. And the real
Speaker 1: problem is it will present this to you authoritatively as
Speaker 1: if the AI is absolutely certain this is the right answer,
Speaker 1: when in fact it's wrong and the AI has no
Speaker 1: way of knowing it's wrong. It's not purposefully trying to
Speaker 1: mislead you, and at least not necessarily. Maybe it was
Speaker 1: given direction to try and do that, but that's another matter.
Speaker 1: It's just trying to complete its task and failing to
Speaker 1: do so accurately. Sometimes the word or a series of
Speaker 1: words can be wrong. Therefore, now grammatically it could be correct,
Speaker 1: but factually it could be completely made up. And why
Speaker 1: this all happens. It does get really complicated. It's not
Speaker 1: necessarily due to just one specific flaw. It's not always
Speaker 1: the case that, oh, that data point didn't appear in
Speaker 1: the data set for some reason, and so the computer
Speaker 1: made something up. There are other issues that could also
Speaker 1: be at play. So, for example, one possible reason for
Speaker 1: hallucinations is something that's called overfitting. IBM defines this as
Speaker 1: what happens quote when an algorith rhythm fits too closely
Speaker 1: or even exactly to its training data, resulting in a
Speaker 1: model that can't make accurate predictions or conclusions from any
Speaker 1: data other than the training data. End quote. That's from
Speaker 1: a piece on IBM dot com. It's titled what is overfitting?
Speaker 1: Sometimes models get so complex or they're trained so closely
Speaker 1: on a specific data set that they start to pick
Speaker 1: up more noise than signal. They give significance to insignificant things.
Speaker 1: I think of this kind of like the character Dracks
Speaker 1: in the Guardians of the Galaxy movies. Drags takes things literally,
Speaker 1: so if you use a saying or an idiom on him,
Speaker 1: he's likely to interpret what you're saying as being what
Speaker 1: you mean. So if you say, oh, that's like throwing
Speaker 1: the baby out with the bathwater, he would assume you're
Speaker 1: talking about something you have literally done before in your life,
Speaker 1: that you have literally thrown out a baby with bathwater,
Speaker 1: and he would not understand you were using an analog
Speaker 1: to describe getting rid of important stuff along with the
Speaker 1: unimportant stuff you want to get rid of. If a
Speaker 1: model has been overfitted, if it's been trained too much
Speaker 1: on a relatively narrow set of data, it might have
Speaker 1: trouble taking what it has learned and generalizing those learnings
Speaker 1: towards something else that's outside the data set. And rather
Speaker 1: than saying I'm sorry, I don't know the answer to that,
Speaker 1: it could produce an answer that follows the statistical rules
Speaker 1: that the model is set to In other words, it'll
Speaker 1: create something that grammatically makes sense, but it won't necessarily
Speaker 1: be relevant or you know, thematically or irrelevance makes sense.
Speaker 1: So in this way, an AI model can become like
Speaker 1: that stereotypical person in the car who absolutely refuses to
Speaker 1: pull over and ask for directions when they get lost,
Speaker 1: because that would be showing weakness. No, gush, darn. It
Speaker 1: will somehow reason our way out of taking that wrong
Speaker 1: turn forty five minutes ago. That'll fix everything. Except it
Speaker 1: doesn't fix everything, and it can make things worse. But
Speaker 1: it's not just pattern recognition that can trip up AI models.
Speaker 1: Another issue is bias. I've talked about bias in other episodes,
Speaker 1: but it's really important that we understand what we mean
Speaker 1: when we're talking bias and how it can happen, because
Speaker 1: I think a lot of people get tripped up. They
Speaker 1: think it's a machine, right, it doesn't possess opinions. How
Speaker 1: can it have bias? Well, we'll explore that in just
Speaker 1: a couple of moments, but first let's take a quick
Speaker 1: break to think our sponsors. How can an AI model
Speaker 1: have bias? Well, the answer is that the machines that
Speaker 1: AI runs on the algorithms that AI is built upon.
Speaker 1: All this stuff, it didn't just pop out of nowhere. Ultimately,
Speaker 1: this stuff was designed, built, and programmed by human beings.
Speaker 1: Even if you have had a piece of software that
Speaker 1: was designed by AI, while the AI that designed it
Speaker 1: in turn had been designed by humans at least somewhere
Speaker 1: down the line once you trace it back far enough so.
Speaker 1: Human beings absolutely do have biases, and those biases can
Speaker 1: make their way into the routines and processes of machines.
Speaker 1: MIT has a great introduction to AI hallucinations and bias
Speaker 1: on a web page that has the fitting title when
Speaker 1: AI Gets It Wrong, Addressing AI hallucinations and bias now.
Speaker 1: In that article, the author points out that AI has
Speaker 1: had issues with bias for years and uses the example
Speaker 1: of image analysis. The author cites a project called Gender Shades.
Speaker 1: This was led by Joi Adowa Buomini, and I apologize
Speaker 1: for my pronunciation of the name. But the project examined
Speaker 1: how an AI powered gender classification tool performed when presented
Speaker 1: with subjects of varying genders, ethnicities, and skin tones from
Speaker 1: the IARPA Janus benchmark A data set or IJBA. This
Speaker 1: is a database of facial images taken from various angles
Speaker 1: and lighting conditions of lots of different people. It's used
Speaker 1: as a government benchmark for testing stuff like facial recognition technologies. Now.
Speaker 1: The project also used a gender classification benchmark from Adance,
Speaker 1: and this was in part to try and address shortcomings
Speaker 1: with the IJB dash A benchmark set. Plus due to
Speaker 1: the limitations of both of these data sets, which I'll
Speaker 1: talk about in just a moment, the project also outlines
Speaker 1: a process to create a better data set for the
Speaker 1: purposes of training technologies like facial recognition and gender classification.
Speaker 1: The project aimed to test several gender classifier programs from
Speaker 1: companies Microsoft and IBM, among others, all with regard to
Speaker 1: quote gender, skin type, and the intersection of skin type
Speaker 1: and gender end quote. So Joy found that the data
Speaker 1: sets from IJB dah A skewed male and lighter skin
Speaker 1: tones skewed heavily male and lighter skin tones. In fact,
Speaker 1: she said between seventy nine point six percent and eighty
Speaker 1: six point twenty four percent of all the images in
Speaker 1: the database were of people with lighter skin tones, and
Speaker 1: fewer than twenty five percent of all the images were
Speaker 1: of women or female presenting people worse, Yet, only four
Speaker 1: point four percent of all the images were of female
Speaker 1: presenting people who had dark skin Adiance's data set had
Speaker 1: a better distribution of photos, at least between genders. Female
Speaker 1: presenting people made up fifty two percent of the images
Speaker 1: in Aightiance's data set, but again, lighter skin tones made
Speaker 1: up the majority of these images. Less than fifteen percent
Speaker 1: of all the images in that data set contained people
Speaker 1: of darker skin tones. So I'm sure you can already
Speaker 1: see where this is going. If you train an AI
Speaker 1: model on data that has a disproportionate emphasis on certain factors,
Speaker 1: such as certain genders or certain skin tones, then you
Speaker 1: would expect the AI to be better at handling cases
Speaker 1: that fall into those categories, Right Like, if most of
Speaker 1: the data you've fed to your AI model is of
Speaker 1: men who have a lighter skin tone, then when you
Speaker 1: are serving the AI model a picture of someone who's
Speaker 1: male presenting and has a lighter skin tone, chances are
Speaker 1: the tools going to work better. If you are instead
Speaker 1: feeding it images of people who fall outside those majority cases,
Speaker 1: the AI tool is probably not going to work as
Speaker 1: well with them, and that's exactly what Joy found in
Speaker 1: her research. She discovered that gender classification tools from all
Speaker 1: of the providers performed better with lighter skinned men than
Speaker 1: with any other group. They perform the worst with darker
Speaker 1: skinned women. Thus we have a bias in the system.
Speaker 1: The data that folks use to train these systems had
Speaker 1: that bias, and it unsurprisingly affects how the AI does
Speaker 1: its job. Now, this isn't just a curiosity for research labs.
Speaker 1: Of course, around the world, various organizations and companies are
Speaker 1: making use of facial recognition tools and gender classification tools.
Speaker 1: There are numerous stories of law enforcement agencies getting into
Speaker 1: hot water for relying on this kind of technology. So
Speaker 1: we know that this technology isn't reliable, particularly if someone
Speaker 1: belongs to a group that's outside of lighter skinned men,
Speaker 1: and the data being used to train these tools is limited.
Speaker 1: That's why we're having these issues, or one of the
Speaker 1: main reasons why we're having these issues. So it stands
Speaker 1: to reason we should not employ those tools for anything
Speaker 1: really at all, other than maybe working to make them better.
Speaker 1: But we definitely shouldn't be using them for things like
Speaker 1: law enforcement, for example. At least we should not use
Speaker 1: them until we can address the problem of bias generative
Speaker 1: AI can actually have similar issues with bias that MIT.
Speaker 1: Article that I mentioned earlier in this episode cites another
Speaker 1: article by Leonardo Nicoletti and Dina Bass titled humans are biased.
Speaker 1: Generative AI is even worse. This piece appeared in Bloomberg.
Speaker 1: So this article explores how a generative AI platform called
Speaker 1: stable Diffusion had a tendency to make assumptions based on
Speaker 1: racial and gender stereotypes, thus repeating and even amplifying those stereotypes.
Speaker 1: Nicoletti and Bass performed and in formal test with stable Diffusion,
Speaker 1: a pretty thorough one, but still informal. They asked stable
Speaker 1: Diffusion to generate images of people who were working one
Speaker 1: of fourteen different jobs. Now, half of those jobs belonged
Speaker 1: to what they called high paying positions, like things that
Speaker 1: you would typically associate as a high paying job. The
Speaker 1: other half typically were too low paying jobs, and actually
Speaker 1: a little less than half of them were low paying jobs.
Speaker 1: Three of them actually fell into the category of crime,
Speaker 1: so like you know, thief or something like that. The
Speaker 1: two had Stable Diffusion generate more than five thousand images
Speaker 1: total so that they could really compare. They didn't want
Speaker 1: to just create, you know, a single image each that's
Speaker 1: a terrible test. They wanted to see, all right, is
Speaker 1: this something that's actually appearing over and over again when
Speaker 1: we make use of this tool, or is it possible
Speaker 1: that you know, you run fourteen tests and it just
Speaker 1: happens to go along with racial stereotypes. Nope. They classified
Speaker 1: the generated images based off of the Fitzpatrick's skin scale.
Speaker 1: This is actually a skin pigmentation metric that's used by
Speaker 1: dermatologists as well as like other researchers, and the scale
Speaker 1: goes from one to six, so one would be very
Speaker 1: light skinned and six would be very dark skinned. The
Speaker 1: researchers found that stable diffusion was far more likely to
Speaker 1: create a person with a lighter skin tone for positions
Speaker 1: that traditionally fall into the higher paid categories, and that
Speaker 1: it was more likely to generate someone with a darker
Speaker 1: skin tone for lower paid or criminal categories. What's more,
Speaker 1: stable diffusion generated images of people appearing to be men
Speaker 1: or male presenting for most of those higher paid positions.
Speaker 1: It was very rare for it to generate the image
Speaker 1: of a female presenting person in the role of one
Speaker 1: of these traditionally higher paid jobs. So the AI was
Speaker 1: perpetuating and amplifying these racial and gender stereotypes. This actually
Speaker 1: reminds me of a classic riddle that was intended to
Speaker 1: reveal bias. I'm sure most of you have heard this
Speaker 1: before or some variation. So the riddle typically goes something
Speaker 1: like this. A father and a son are in a
Speaker 1: terrible car accident, and the father tragically dies at the scene.
Speaker 1: The son is badly injured. EMTs arrived. They rushed the
Speaker 1: boy to a surgical ward. The surgeon on duty looks
Speaker 1: at the boy and says, I can't operate on him,
Speaker 1: he's my son. Well, how could that be true? Now?
Speaker 1: The obvious answer is the surgeon is the boy's mother.
Speaker 1: And I think a lot of people arrive at that
Speaker 1: conclusion much more easily today than they did when I
Speaker 1: was a kid. Like when I was a kid, the
Speaker 1: sexist stereotype was that all real quote unquote real doctors
Speaker 1: and surgeons were men and women they were nurses or administrators. Right,
Speaker 1: That was the stereotype that people kind of believed in.
Speaker 1: But I'm sure most of y'all understood this answer, or
Speaker 1: you've been exposed to this riddle numerous times. I mean,
Speaker 1: it is a meme at this point, but again, back
Speaker 1: in my day, a lot of folks would likely get
Speaker 1: stumped by this, or they would say something dumb like, oh,
Speaker 1: it turns out the surgeon was the real dad and
Speaker 1: the father who died at the scene had been the
Speaker 1: adopted father he adopted the boy, or something along those lines,
Speaker 1: which reveals the bias of the listener. It reminds the
Speaker 1: listener to think critically and be aware of sexist stereotypes.
Speaker 1: So AI can produce the wrong results due to bias
Speaker 1: built into the underlying model and end up making these
Speaker 1: same mistakes right, Like if you say surgeon, it may
Speaker 1: mistakenly just believe ah, you meant man. It has to
Speaker 1: be a man that I generate in this image because
Speaker 1: the user said surgeon, so that means man. That's a
Speaker 1: real problem. With enough work and attention, we can actually
Speaker 1: create training materials that minimize bias and can help reverse
Speaker 1: this trend. But even doing that is not enough to
Speaker 1: eliminate errors in generative AI. There are other problems we
Speaker 1: have to look out for. So what happens when you
Speaker 1: have an AI model, like a large language model, for example,
Speaker 1: and part of the massive amount of material that it's
Speaker 1: training itself on includes data sets that were generated by
Speaker 1: other AI. When an AI image generator is pulling images
Speaker 1: that were made by other image generators and then training
Speaker 1: itself on that, or you know, even if it's pulling
Speaker 1: images that an earlier version of that very same generator
Speaker 1: had created, the mistakes that exist in those AI generated images,
Speaker 1: or you know, it's if we're not talking images like
Speaker 1: in text or whatever, those things can become like you
Speaker 1: would argue, oh, those things are noise, right, that's those
Speaker 1: are mistakes. But AI doesn't know that they're mistakes. They don't.
Speaker 1: It doesn't know that it's noise. If you're training it
Speaker 1: on the data, it thinks it's significant. And if it
Speaker 1: thinks it's significant, it's going to incorporate it and perhaps
Speaker 1: even dial it up quite a bit. So a great
Speaker 1: way of illustrating this, in my opinion, is to talk
Speaker 1: about fingers. I mean, I'm sure all of you out
Speaker 1: there have experienced seeing AI generated images that hilariously get
Speaker 1: the fingers totally wrong. A lot of AI image generators
Speaker 1: have real problems with fingers, So you might have folks
Speaker 1: and images who wind up with way too many fingers,
Speaker 1: like seven or eight perrand, or maybe they have not
Speaker 1: enough fingers, or maybe all their fingers are thumbs, or
Speaker 1: maybe they bend in unnatural ways, or they all look
Speaker 1: like long strands of spaghetti. These are clearly miss you know,
Speaker 1: image generators have identified fingers are appendages, and these appendages
Speaker 1: attached to hands. But the machines don't really follow the
Speaker 1: rules when it comes to portraying those fingers, and they
Speaker 1: do the best they can, and sometimes the best they
Speaker 1: can is hilariously bad. But if image generator models train
Speaker 1: on material that was created by AI, those weird fingers
Speaker 1: are seen as a feature, not a bug. Like the
Speaker 1: AI model doesn't know, Oh, fingers don't actually look like that,
Speaker 1: that's wrong. It just says, ah, this is how fingers
Speaker 1: sometimes look based upon these images I've been trained on,
Speaker 1: which means the next generation of image generators will stress
Speaker 1: these features more instead of correcting for them, which means
Speaker 1: you're going to get some really weird images as a result.
Speaker 1: And this process can repeat itself, and it gets worse
Speaker 1: and worse each time. It's like making a copy of
Speaker 1: a copy of a copy. You eventually reach a point
Speaker 1: where the copy you have produced is illegible or doesn't
Speaker 1: look enough like the original at all for you to
Speaker 1: even easily say, oh, this is a copy of that.
Speaker 1: That can be a real problem. And of course this
Speaker 1: is just one example the fingers in AI. That's an
Speaker 1: easy mark to hit, right, but there are countless other examples.
Speaker 1: In a paper titled The Curse of Recursion Training on
Speaker 1: Generated Data Makes Models Forget, a group of researchers from
Speaker 1: the University of Cambridge, Oxford University, Imperial College London, the
Speaker 1: University of Edinburgh, and the University of Toronto present an
Speaker 1: argument of a pretty bleak future if AI researchers don't
Speaker 1: take the proper measures to head it off. We're going
Speaker 1: to talk more about that in just a moment, but
Speaker 1: first let's take another quick break to thank our sponsors. Okay,
Speaker 1: before the break, I mentioned this paper, the cursor recursion
Speaker 1: Training on Generated Data makes Models Forget. It's a great article.
Speaker 1: It does get very technical at one point, but the
Speaker 1: researchers did a great job explaining the top level problem
Speaker 1: and the potential outcome of that problem in a way
Speaker 1: that I think anyone could find accessible. When you get
Speaker 1: to the actual analysis part, that's when it gets really technical.
Speaker 1: But the summary, the conclusions, all of that, I think
Speaker 1: is easy to understand. So in that paper, the researchers say, quote,
Speaker 1: we discover that learning from data produced by other models
Speaker 1: causes model collapse, a degenerative process whereby over time models
Speaker 1: forget the true underlying data distribution end quote. So essentially,
Speaker 1: these AI models will quote unquote for get information while
Speaker 1: simultaneously a set of learned behaviors they have created through synthesizing.
Speaker 1: All this information will begin to converge and lead to
Speaker 1: a broken model that's no longer really useful. It won't
Speaker 1: present anything that's of real value. So the researchers argue
Speaker 1: that quote the use of llm's at scale to publish
Speaker 1: content on the Internet will pollute the collection of data
Speaker 1: to train them endo quote. That's bad news, and it's
Speaker 1: definitely going to be an issue, particularly with sites that
Speaker 1: fall into the content farm category, because it's already happening right.
Speaker 1: There are already websites out there that have turned to
Speaker 1: AI generation to flesh out the articles that they have
Speaker 1: in their database, and these articles are of a varying quality,
Speaker 1: and all of those getting scooped up in a future
Speaker 1: AI model session and used side by side with articles
Speaker 1: that were researched written and edited by human beings and
Speaker 1: therefore potentially at least of higher quality. I'm not saying
Speaker 1: that all human written articles and edited articles are great.
Speaker 1: They're not. There's some bad stuff out there that human
Speaker 1: beings have written. But with those steps in place, you
Speaker 1: have the potential for really great work. With AI. You
Speaker 1: don't necessarily get that. You hope you get it, but
Speaker 1: there's no guarantee and there aren't enough I would say
Speaker 1: safety valves to make sure that things don't go off
Speaker 1: the rails. So getting back to content farms, if you
Speaker 1: are unfamiliar with that term, well don't worry. You've almost
Speaker 1: certainly come across a content farm at some point in
Speaker 1: the past. So these are sites that just churn out
Speaker 1: an enormous amount of content, typically in an effort to
Speaker 1: tap into the sweet sweet waters of SEO, which stands
Speaker 1: for search engine optimization. So for a lot of websites
Speaker 1: out there, the majority of traffic coming to the website
Speaker 1: comes courtesy of a search engine. And when I say
Speaker 1: a search engine, you might as well fill in the
Speaker 1: name Google there, because that's the big one. I mean.
Speaker 1: There are other search engines out there, and some of
Speaker 1: them do contribute to this too, but Google commands somewhere
Speaker 1: between eighty and like ninety five percent of the search market.
Speaker 1: Exactly where that it falls is a matter of debate.
Speaker 1: Like I looked at a few different Internet analytics sites, right,
Speaker 1: and they had different percentages, but there was always above
Speaker 1: eighty percent, and some as high as like ninety two
Speaker 1: or ninety three. So it's safe to say that Google
Speaker 1: dominates the search space. You know, technically it may not
Speaker 1: be a monopoly, but effectively it kind of is. So
Speaker 1: sites that depend on traffic from search naturally want to
Speaker 1: find ways for their pay is to rank high in
Speaker 1: search results and to appear in more search results. Now
Speaker 1: that's actually easier said than done. Google has changed its
Speaker 1: page ranking algorithm a few different times, and some search
Speaker 1: results are dependent upon who is doing the searching. That
Speaker 1: means that you and I might each search for the
Speaker 1: exact same thing, maybe we word it the exact same way,
Speaker 1: but we'll end up getting different results. Google says, quote
Speaker 1: personalization is only used in your results if it can
Speaker 1: provide more relevant and helpful information end quote. So presumably
Speaker 1: it doesn't happen all the time. That means that in
Speaker 1: some cases you and I will get identical results depending
Speaker 1: upon what it is we're searching for, and in other
Speaker 1: cases we will get very different search results. I do
Speaker 1: know this makes SEO a much larger challenge because it's
Speaker 1: impossible to be all things to all people. You know,
Speaker 1: you can only do the best you can to try
Speaker 1: and show up for any given search query. It is
Speaker 1: super duper hard if you're dependent upon human writers and
Speaker 1: editors to generate all the stuff that you're shoving out
Speaker 1: in an effort to get clicks. So most of your
Speaker 1: traffic is coming from search. We talked about this already.
Speaker 1: You need to have lots of stuff on your site
Speaker 1: that people could be searching for so that traffic comes
Speaker 1: your way, and that way you can make money through
Speaker 1: web advertising. Essentially, you could try to be reactionary, right,
Speaker 1: You could try to generate new content as things capture
Speaker 1: of the public interest, but you run the danger of
Speaker 1: getting to the party too late and that by the
Speaker 1: time you have something up, no one's talking about it
Speaker 1: anymore and you're not really seeing any real traffic from that.
Speaker 1: What if instead you could just kind of open up
Speaker 1: a fire hose of content using generative AI. Well, if
Speaker 1: you just had AI write a whole bunch of articles
Speaker 1: in the style that you've established for your company, and maybe,
Speaker 1: if you're feeling a little cautious, you'll even employ a
Speaker 1: couple of human being editors to take on the job
Speaker 1: of reading over these generated articles and to correct any
Speaker 1: mistakes that were made, and perhaps even tweak a couple
Speaker 1: of things here and there to make it sound more
Speaker 1: human if necessary. But now you can push out way
Speaker 1: more content without having to wait on human writers to
Speaker 1: research and write everything. Plus, AI does not complain if
Speaker 1: you assign it to write a suite of articles about
Speaker 1: gluten free skincare products. By the way, I'm using my
Speaker 1: real world life experience with that last example. I once
Speaker 1: got that writing assignment. It was dumb then and it's
Speaker 1: dumb now, But I guess people were searching for it,
Speaker 1: so I got an assignment to write it. Now. I
Speaker 1: would like to think that the site I was writing for,
Speaker 1: which was how Stuffworks dot Com, wasn't really a content farm.
Speaker 1: I would love to think that, and I would argue
Speaker 1: that for many years when I wrote there, it did
Speaker 1: not qualify as a content farm. We did try to
Speaker 1: write in depth, authoritative articles about all sorts of stuff,
Speaker 1: like whether we were talking about technology or society, or
Speaker 1: money or entertainment, whatever it might be. We applied rigor,
Speaker 1: you know, journalistic rigor, toward the research and writing and
Speaker 1: editing of those pieces. Over time, things changed where we
Speaker 1: started to cater more toward ad deals, where we would
Speaker 1: get this big ad deal with a company like a
Speaker 1: you know, cosmetics company, for example, and we would suddenly
Speaker 1: have hundreds of articles assigned in the field of cosmetics,
Speaker 1: articles that were incredibly niche like, there was no way
Speaker 1: that we're going to drive a ton of traffic. But
Speaker 1: collectively then these articles could get a lot of traffic.
Speaker 1: Not a single one, but across the board. If someone
Speaker 1: happened to be searching for this thing, they could find
Speaker 1: their way to our article and that would be another
Speaker 1: click coming our way. It was a very much a
Speaker 1: shotgun approach to writing content. I hated it. There were
Speaker 1: articles I wrote that I am not at all. It's
Speaker 1: not that I'm not proud of the work I did.
Speaker 1: I'm not proud of getting the assignment, like it was
Speaker 1: a joke in my opinion, But that's what we were
Speaker 1: trying to do in order to survive. Because again, how
Speaker 1: stuff works was like one of these websites in that
Speaker 1: most of the traffic coming through How Stuff Works came
Speaker 1: through a search engine. Someone was looking to learn how
Speaker 1: something worked and they got sent our way. People weren't,
Speaker 1: as a rule, just coming to How Stuff Works to
Speaker 1: peruse the website. We always wanted that that was what
Speaker 1: our goal was, to create a destination website that people
Speaker 1: would want to go to just to see, oh, what's
Speaker 1: new on the site, But we never really achieved that.
Speaker 1: It's a really hard thing to do. There are people
Speaker 1: who do it and it's amazing, but it's not easy
Speaker 1: to replicate. So instead we wrote tons of articles about
Speaker 1: stuff that people were searching for, and that just kind
Speaker 1: of was our mo at that point. Anyway, if you're
Speaker 1: using AI to create these kinds of articles, it's going
Speaker 1: to generate a lot of stuff that's just not very good.
Speaker 1: But then who cares? Like you don't necessarily care if
Speaker 1: the material is good. If the only traffic you're really
Speaker 1: getting on your website is coming from search engines, you
Speaker 1: just need it to show up in the search engines. Now,
Speaker 1: if the search engine is able to determine, hey, this
Speaker 1: is low quality content, and it disincentivizes people visiting by
Speaker 1: making it go further down the search results, then you're
Speaker 1: going to have a problem, and a lot of content
Speaker 1: farms ran into that problem. Google downgraded content farms in
Speaker 1: their search algorithm. Other sites like duck duck go removed
Speaker 1: websites that were considered content farms because the people running
Speaker 1: duck dot go realized, Hey, these sites aren't inviting anything
Speaker 1: of real value to visitors. Why are we even serving
Speaker 1: it up. That's not really a good use of anyone's time.
Speaker 1: But if you're in a space where the jig isn't
Speaker 1: up yet, you might as well just go ahead and
Speaker 1: create as much garbage as you can because you just
Speaker 1: want the clicks. You don't care if people actually think
Speaker 1: the articles are of good quality or that they're going
Speaker 1: to learn anything useful. You don't even necessarily care if
Speaker 1: the articles are accurate. You care that people are clicking
Speaker 1: on the articles. So if that's your perspective, ultimately, then
Speaker 1: the goal for you is to push as much of
Speaker 1: this stuff out the door as you possibly can, generate
Speaker 1: it as fast as possible, get it online as quickly
Speaker 1: as you can, and hope that starts to rank in
Speaker 1: search so that people flood in to read about whatever
Speaker 1: it is you're writing about. But it's not just people
Speaker 1: who are going to your links, is it. There are
Speaker 1: bots crawling the web now. Some of them are calling
Speaker 1: the web in order to index those web pages for
Speaker 1: the purposes of things like search engines, but other bots
Speaker 1: are there to scrape data for the purposes of training
Speaker 1: the next generation of large language models. Essentially, at this point,
Speaker 1: bots are reading articles that were written by other bots,
Speaker 1: and so when the next large language model launches, it
Speaker 1: does so on a data set that has been polluted
Speaker 1: by bot generated information. That means the next generation will
Speaker 1: be even worse, and so on, and eventually we arrive
Speaker 1: at a point where the Internet, this amazing invention that
Speaker 1: provides access to practically all of human knowledge, becomes absolutely
Speaker 1: infested with junk that is inaccurate and increasingly nonsensical, and
Speaker 1: we render this incredible invention useless. This isn't just speculation either.
Speaker 1: We have examples of companies turning to AI to generate articles.
Speaker 1: C Net famously did this early in the days of
Speaker 1: generative AI, and cnet properly got roasted for doing it,
Speaker 1: first roasted for not presenting it in a way that
Speaker 1: was transparent, and then also for including articles that just
Speaker 1: had outright wrong information in them and publishing them as
Speaker 1: if they were vetted pieces that editors had gone through.
Speaker 1: How stuff works. Again, my old employer where I got
Speaker 1: that Skincare writing assignment once upon a time, they've done
Speaker 1: this too. They laid off their human writers, they stopped
Speaker 1: giving assignments to freelancers. Later on they laid off the
Speaker 1: entire editorial staff after the editors protested this move toward
Speaker 1: AI generated content. This trend is happening. Not only are
Speaker 1: talented people being put out of work, which is bad
Speaker 1: enough already. These editors and writers, they believed in what
Speaker 1: they were doing. Yeah, sometimes the assignments stank, sometimes they
Speaker 1: were not good, but the writers and editors still believed
Speaker 1: in doing as good a job as they possibly could.
Speaker 1: But their replacements, the AI, they're just making the Internet
Speaker 1: worse by generating unreliable and terrible content. Then no one
Speaker 1: actually wants to read unless they just happen to put
Speaker 1: that particular set of terms into a search engine and
Speaker 1: the search engine couldn't find anything better to serve them. Again,
Speaker 1: it's as if you needed to learn something important, but
Speaker 1: all you have access to are just sloppily written articles
Speaker 1: by people who had no understanding or passion about the
Speaker 1: subject matter they were writing on, and there were no
Speaker 1: editors to steer the writer toward creating a more accurate
Speaker 1: or informative piece. It gets pretty darn bleak. Is it inevitable? Though? No,
Speaker 1: it's not inevitable. This future happens if the people who
Speaker 1: are training the AI models allow it to happen, but
Speaker 1: with careful stewardship. By guiding the AI models so that
Speaker 1: they don't pull training data from garbage sites and they
Speaker 1: really focus on reputable sources, it's possible to avoid these
Speaker 1: issues at least in some part. I mean some things
Speaker 1: like hallucinations, confabulations, that kind of stuff that can happen anyway,
Speaker 1: but you can at least limit it. That's not really
Speaker 1: what we're seeing right now, though, because at the moment,
Speaker 1: companies are rushing into the AI space right they are
Speaker 1: pushing so hard to create large language models that dwarf
Speaker 1: the previous generation's capabilities. So to do that, they have
Speaker 1: to seek out training data from all across the internet.
Speaker 1: You have to train these AI models on tons and
Speaker 1: tons of information to make them useful. The more data
Speaker 1: you have access to the better. Social platforms have provided
Speaker 1: a popular source of information. We know that Reddit has
Speaker 1: struck deals with open ai, for example, in order to
Speaker 1: crawl Reddit to pull information. But you know what, social
Speaker 1: platforms are also really popular with bots, not just with people.
Speaker 1: So even this approach brings with it the risk of
Speaker 1: AI training on other AI generated data, which again leads
Speaker 1: to model collapse. Further down the road, I might one
Speaker 1: day do a much more in depth episode about this paper,
Speaker 1: the curse of recursion. Training on generated data makes models forget.
Speaker 1: I've given a very high level summary of what the
Speaker 1: researchers say in that paper, but it might benefit us
Speaker 1: to take a much closer look at what they found
Speaker 1: and their conclusions. So I may revisit this topic in
Speaker 1: the future, but for now, I think it's just good
Speaker 1: to remember that AI does have the potential to do
Speaker 1: great things. I mean, it can potentially augment our work
Speaker 1: efforts and let us accomplish goals more quickly and efficiently
Speaker 1: and accurately. But AI also has the potential to make
Speaker 1: things miserable and churn out content that no one wants
Speaker 1: to see other than other bots, and creating a cynical
Speaker 1: cycle that ultimately could make the Internet into a cluttered,
Speaker 1: practically useless mess. So which way are we going to go?
Speaker 1: I think my answer day to day depends on how
Speaker 1: optimistic I feel, but at the very least, I think
Speaker 1: knowing about the risks is important. That's it for today's episode.
Speaker 1: I hope you are all well. I will try to
Speaker 1: get away from AI topics. I know I've been covering
Speaker 1: a lot of that recently, and that'd be nice to
Speaker 1: kind of branch into other areas of tech, So I'm
Speaker 1: going to try and do that. It's just AI stuff
Speaker 1: just keeps on happening, y'all. But I will talk to
Speaker 1: you again really soon. Tech Stuff is an iHeartRadio production.
Speaker 1: For more podcasts from iHeartRadio, visit the iHeartRadio app, Apple Podcasts,
Speaker 1: or wherever you listen to your favorite shows.