Speaker 1: Welcome to tech Stuff, a production from iHeartRadio. Hey there,
Speaker 1: and welcome to tech Stuff. I'm your host, Jonathan Strickland.
Speaker 1: I'm an executive producer with iHeart Podcasts. And how the
Speaker 1: tech are you well. I just got back from celebrating
Speaker 1: my birthday. Thank y'all for all of you who are
Speaker 1: wishing me a happy birthday. And here in the United States,
Speaker 1: we're about to have our national holiday celebrating the fourth
Speaker 1: of July. I realize Fourth of July happens everywhere, not
Speaker 1: just in the US, but we celebrate it here in
Speaker 1: the US, and as such, there's very limited time to
Speaker 1: get everything done, and I really wasn't able to pull
Speaker 1: an episode together in time, and I apologize for that,
Speaker 1: but I thought I would bring an older episode to
Speaker 1: y'all so that we can still have an episode to
Speaker 1: listen to today. And typically I would have one of my
Speaker 1: Fireworks episodes play on this day, because Fireworks has a
Speaker 1: very close association with the Fourth of July here in
Speaker 1: the United States. But I've done that for several years
Speaker 1: in a row, and I've thought it might be nice
Speaker 1: to have a break from Fireworks instead. I thought I
Speaker 1: would focus on something that continues to be a very
Speaker 1: important topic in tech, and that is artificial intelligence. And
Speaker 1: AI is incredibly impressive, but there are also lots of
Speaker 1: challenges with AI, and those are ranging from the technological
Speaker 1: side to the social side right and how we implement AI.
Speaker 1: One thing I thought that we don't really get to
Speaker 1: talk about very much is the concept of forgetting with AI.
Speaker 1: We have a lot of generative AI out there that
Speaker 1: is drawing upon huge resources of information, but AI can
Speaker 1: also quote unquote forget. So this episode originally published on
Speaker 1: July thirty first of twenty twenty three. It is called
Speaker 1: Machine Learning and Catastrophic Forgetting. And I think it's a
Speaker 1: useful thing to reflect upon as we see more and
Speaker 1: more headlines about tech companies and their investment increasingly astronomical
Speaker 1: investment in artificial intelligence. I hope you enjoy so. Over
Speaker 1: this past weekend, I was listening to the podcast The
Speaker 1: Skeptics Guide to the Universe, which I have no connection to.
Speaker 1: I just listened to it, and it included a section
Speaker 1: on AI that referenced something I don't think I had
Speaker 1: heard of before, which is really talking more about my
Speaker 1: oversight than anything else. Maybe I did hear about it
Speaker 1: but then I forgot about it, you know, catastrophically. So
Speaker 1: the thing they talked about was catastrophic forgetting in artificial intelligence,
Speaker 1: specifically in machine learning systems built on artificial neural networks. Now,
Speaker 1: before we talk about catastrophic forgetting, which as I mentioned,
Speaker 1: is related to neural networks and machine learning, we really
Speaker 1: need to do a quick reminder, not a quick reminder.
Speaker 1: We need to do a full reminder on how all
Speaker 1: this works. And that's going to require us to do
Speaker 1: a whole lot of remembering. Not a catastrophic amount, but
Speaker 1: a lot. So the history of artificial intelligence as a
Speaker 1: discipline is one of intense and important debates in fields
Speaker 1: like computer science. Now, I have often talked about how
Speaker 1: AI can be seen as the convergence of several other
Speaker 1: disciplines into its own field. And there's more than one
Speaker 1: way to approach the challenge of artificial intelligence. And in
Speaker 1: the history of AI, we actually saw that play out,
Speaker 1: and some would argue the way it played out means
Speaker 1: that we're actually just now playing catch up. So different
Speaker 1: schools of thought pushed these different approaches forward as this
Speaker 1: should be the prevailing methodology we use to develop artificial intelligence.
Speaker 1: This is important because the development of AI does not
Speaker 1: exist in a vacuum, right. It exists in our real world.
Speaker 1: Research requires funding, and when you've got different sides arguing
Speaker 1: that their approach to artificial intelligence is superior and that
Speaker 1: the alternatives are not just inferior, but potentially limited to
Speaker 1: the point of being useless, well you've got a metaphorical
Speaker 1: wrestling match going on. The winner takes home the big
Speaker 1: prize of getting funding for their research, and the loser
Speaker 1: has to scrabble for whatever they can find, and often
Speaker 1: they will see their work languish as a result. By
Speaker 1: the way, this is why I often bring stuff up
Speaker 1: in this podcast that is outside the realm of tech.
Speaker 1: I've received a lot of messages over the years from
Speaker 1: folks saying that I should leave out stuff like money
Speaker 1: or politics. Politics is the big one. But to me,
Speaker 1: that doesn't make sense because tech exists within our world,
Speaker 1: a world that is largely shaped by money and politics.
Speaker 1: I don't think we can separate the tech from all
Speaker 1: of that because I believe that if you were to
Speaker 1: somehow magically remove those influences, If somehow money and politics
Speaker 1: never played a part in the development of technology, our
Speaker 1: tech would look very different from what it does today.
Speaker 1: Not necessarily better or worse, but different. I mean, think
Speaker 1: about Thomas Edison. He was very much driven by financial success,
Speaker 1: like his work in tech was really mostly about making
Speaker 1: lots of money. And without the making lots of money part,
Speaker 1: you don't really have his drive to really bring together
Speaker 1: the brightest minds of his generation and set them to
Speaker 1: work on creating incredible technology. So I think we have
Speaker 1: to take all these things into consideration. Anyway, that's a
Speaker 1: total rabbit trail, and I apology. Let's get back to
Speaker 1: our story. It really begins around nineteen forty three when
Speaker 1: a pair of researchers at the University of Chicago first
Speaker 1: proposed the concept of the basic unit of a neural network.
Speaker 1: Those researchers were Warren McCullough and Walter Pets, And in fact,
Speaker 1: they demonstrate their idea by showing a simple electrical circuit
Speaker 1: the very basis for what would become a neural network.
Speaker 1: So their proposal was a system that would use those
Speaker 1: simple circuits to mimic the neurons that we have in
Speaker 1: our noggins. So our brain consists of a bunch of
Speaker 1: these neurons, and you might wonder how much is a bunch. Well,
Speaker 1: we're talking about on average, around one hundred billion neurons
Speaker 1: in the human brain. These neurons interconnect with each other.
Speaker 1: It's not just a one to one, right, You've got
Speaker 1: these interconnections between all these different neurons, not with every
Speaker 1: neuron connected to every other neuron, but lots of interconnections.
Speaker 1: And if we're looking at just the connections, you would
Speaker 1: count more than one hundred trillion of them in the
Speaker 1: typical human brain. And these connections in our brains make
Speaker 1: up neural circuits. Those circuits light up, and that represents
Speaker 1: us doing lots of different stuff, from experiencing the world
Speaker 1: around us so perception to thinking about a past memory.
Speaker 1: You know that typically is like recreating the same pathway
Speaker 1: over and over, and sometimes we don't recreate it exactly correctly,
Speaker 1: and our memory ends up not being a perfect representation
Speaker 1: of the thing that we actually experienced. This is why
Speaker 1: things like eyewitness testimony is not always very reliable, because
Speaker 1: our memories aren't infallible. They can trick us and we
Speaker 1: can have all those pathways light up. When we learn
Speaker 1: a new skill, we start forming new pathways, and then
Speaker 1: as we practice this skill, we start to reinforce those pathways.
Speaker 1: So McCulla and Pitts propose that we create machines capable
Speaker 1: of doing essentially a similar thing that our brains do,
Speaker 1: so kind of a neuromimicry, not exactly one to one
Speaker 1: the way our brains work, but inspired by the way
Speaker 1: our brains work. Now, we would be limited by what
Speaker 1: the technology of the day would be able to do,
Speaker 1: because there's no feasible way we could create a massive
Speaker 1: electrical system with one hundred billion individual simple circuits with
Speaker 1: more than one hundred trillion connections between them. That would
Speaker 1: be beyond our capability. It would be beyond our resources.
Speaker 1: We could, however, create systems that used interconnected circuits to
Speaker 1: process information and to teach such a system to do
Speaker 1: specific tasks. Now, in nineteen forty nine, Donald Hebb wrote
Speaker 1: a book about biological neurons, and he titled this book
Speaker 1: the Organization of Behavior and suggested neural pathways get stronger
Speaker 1: with additional use, kind of like you know, if you
Speaker 1: exercise your muscles, you build strength over time, while so
Speaker 1: is the same with neural pathways, and if you don't
Speaker 1: use those muscles, well, then your muscles get weaker. Well,
Speaker 1: same with neural pathways. If you end up learning a skill,
Speaker 1: but then over a great amount of time you no
Speaker 1: longer practice that skill, you're going to lose some of
Speaker 1: your ability, maybe not all of it, but at least
Speaker 1: some of it. And you have to you know, like
Speaker 1: I think about wrestlers who come back from from retirement,
Speaker 1: professional wrestlers, they call it ring rust. You got to
Speaker 1: knock off the ring rust and get back into step
Speaker 1: and kind of get back into your groove. And it
Speaker 1: takes a little time. Typically sometimes you know, you can
Speaker 1: get back into the game faster than others, but you
Speaker 1: get the idea. And also heb ended up proposing the
Speaker 1: concept of cells that fire together wire together, meaning that
Speaker 1: neurons that fire at the same time end up strengthening
Speaker 1: faster than other neurons do. So when you get into
Speaker 1: that system, you can actually reinforce those pathways. And for
Speaker 1: AI this would be really important. And it wasn't very
Speaker 1: long after Donald Habb had published this work that researchers
Speaker 1: in the field of AI tried to apply that concept
Speaker 1: that philosophy to computer science. By the mid nineteen fifties,
Speaker 1: the burgeoning computer science lab and AI lab at MIT
Speaker 1: was building out neural networks based on Hebb's ideas. Meanwhile,
Speaker 1: another computer scientist named Frank Rosenblatt was looking at primitive
Speaker 1: neural systems and he started with flies like house flies.
Speaker 1: He wanted to explore systems that were involved when a
Speaker 1: fly would quickly move away after detecting a possible threat,
Speaker 1: like instantly, or at least appear to us to instantly
Speaker 1: react to something. So, for example, a fly swatter coming
Speaker 1: at it, like you might be moving the fly swater
Speaker 1: very quickly, and yet the fly is able to move
Speaker 1: super fast with no perceivable delay. Right, we know that
Speaker 1: we have a delay from when we perceive something to
Speaker 1: when we can act on something. Like if you've ever
Speaker 1: been in a fender bender in a car accident, you
Speaker 1: know that that there's a delay between when you see
Speaker 1: the issue when you can hit the brake, and that
Speaker 1: can lead to accidents. Well, with flies, that delay seems
Speaker 1: to be super super small. So Rosenblatt was really interested
Speaker 1: in exploring the neurological reasons for that. How can that happen?
Speaker 1: It has to be really simple, right, There has to
Speaker 1: be a simple and more or less direct pathway that
Speaker 1: exists to allow a fly to react to detecting a
Speaker 1: potential threat like that, and if you could replicate that
Speaker 1: with electronics, you could have a very simple but potentially
Speaker 1: powerful artificial intelligence system. So he came up with this
Speaker 1: system that would be based off that very simple direct
Speaker 1: pathway that you would see in something like a fly,
Speaker 1: and he called it the perceptron. So he went back
Speaker 1: to the simple circuit design that was proposed by Pitts
Speaker 1: and McCullough and he built out the Mark one perceptron
Speaker 1: or perceptron. I guess I should say, so let's talk
Speaker 1: about a perceptron, like not big P, but a little
Speaker 1: P perceptron. This is probably what we would call a
Speaker 1: neural node in a modern neural network. So the purpose
Speaker 1: of the perceptron was to accept inputs and produce an
Speaker 1: output based on some threshold, Like if the inputs meet
Speaker 1: a certain threshold, one output would be produced. If they
Speaker 1: failed to do so, a different output would be produced.
Speaker 1: The inputs, in turn would be assigned weights, which would
Speaker 1: factor into the output the perceptron would generate. So when
Speaker 1: we're talking weights, I mean weights as in like how
Speaker 1: heavy something is or in this case, how much impact
Speaker 1: that thing has, So we're talking about how much impact
Speaker 1: one input has relative to other inputs. Let me use
Speaker 1: a really mundane human example to kind of explain what
Speaker 1: this means. Let's say that your friend asks you to
Speaker 1: go see a movie with them, and it's going to
Speaker 1: be playing tonight at nine pm. But you've had a
Speaker 1: really busy day and you might not be able to
Speaker 1: even eat dinner until around nine pm. And if you
Speaker 1: go see this movie, it might mean having to skip
Speaker 1: dinner or to try and eat something really fast and
Speaker 1: unhealthy before you go to the movie. What's more, you
Speaker 1: got a really big day tomorrow and you feel like
Speaker 1: you really need to be well rested for it. However,
Speaker 1: at the same time, you haven't seen this friend in ages,
Speaker 1: and you really like this person and you've wanted to
Speaker 1: hang with them for a really long time. Plus the
Speaker 1: movie they're suggesting is one you've really wanted to see
Speaker 1: and you haven't gone yet. Well, you would likely assign
Speaker 1: at least unconsciously weights to each of these factors before
Speaker 1: you make your decision. You know, if getting some dinner
Speaker 1: without having to rush, and also to be really well
Speaker 1: rested for tomorrow are really important to you, you'll probably
Speaker 1: reluctantly decline the offer. But if you really crave some
Speaker 1: time with your friend and you really want to see
Speaker 1: that movie before all the spoilers come out on Facebook
Speaker 1: or whatever, maybe you'll say yes. Your decision depends upon
Speaker 1: the weights you assign those factors, those inputs, even if
Speaker 1: you don't consciously think about it that way. Well, the
Speaker 1: Perceptron system worked in a similar way, produced outputs by
Speaker 1: taking the inputs into consideration, including each input's weight. Moreover,
Speaker 1: the more you submitted inputs, the more the system would
Speaker 1: quote unquote learn how to weight each of those inputs,
Speaker 1: all with the goal of bringing the actual output that
Speaker 1: the process or you know, generates closer to the one
Speaker 1: you want it to generate. Okay, I just said a
Speaker 1: lot there. We've got some more to get through. But
Speaker 1: before we get to that, let's take a quick break,
Speaker 1: all right. Before the break, we were talking about inputs
Speaker 1: and weights and the idea of getting an output that
Speaker 1: is close to what you want the system to do.
Speaker 1: That's not a guarantee, right, The system could generate an
Speaker 1: output that's quote unquote wrong, you know, depending on whatever
Speaker 1: task you've set this machine learning system to learn, and
Speaker 1: that gets a bit conceptual. So let's talk about a
Speaker 1: simple example that I love to use. If you've been
Speaker 1: listening to texta for a while, you've heard this before,
Speaker 1: and that's talking about pictures of cats. Because cats ruled
Speaker 1: the Internet. I don't know if they still do. They
Speaker 1: won't talk to me, so just knock things off shelves. Anyway.
Speaker 1: If your goal is to tea each a computer system
Speaker 1: to differentiate photos that include a cat from photos that
Speaker 1: do not include a cat, well, you would need to
Speaker 1: train the system, and part of that includes feeding the
Speaker 1: system a whole bunch of photographs. Some of those would
Speaker 1: have cats in them, some would not, and chances are
Speaker 1: the system would misidentify photos. Maybe a significant number of
Speaker 1: those photos. You would probably have false positives where the
Speaker 1: system thinks there's a cat there and there's not, and
Speaker 1: false negatives where it doesn't think there's a cat there
Speaker 1: but there is. At that point, your goal is to
Speaker 1: try and teach the system to close the gap between
Speaker 1: the actual results it produces and what you want it
Speaker 1: to produce. In some systems, that means you might have
Speaker 1: to go in manually to adjust the input weights to
Speaker 1: increase the weight of one input versus another in an
Speaker 1: effort to cut down on mistakes. So the perceptron was interesting,
Speaker 1: but it was very limited in complexity. It was essentially
Speaker 1: a single layer where you'd feed a bunch of inputs
Speaker 1: in and you would get an output. So it was
Speaker 1: suitable for a subset of computational challenges, but anything beyond
Speaker 1: that was well beyond its own reach as a single
Speaker 1: layer network. By the late nineteen fifties, other researchers had
Speaker 1: created new neural networks that were multi layered. So a
Speaker 1: node or neuron didn't just accept inputs, it would generate
Speaker 1: outputs that then would become inputs for another layer down.
Speaker 1: So instead of just having one layer of nodes, you
Speaker 1: would have multiple layers of nodes. Typically you would have
Speaker 1: one at the quote unquote top of the network, and
Speaker 1: you would have outputs at the bottom, and the ones
Speaker 1: in between would be often referred to as hidden layers,
Speaker 1: and who knows how many there would be. So anyway
Speaker 1: you would feed data to the system, the initial nodes
Speaker 1: would generate information as outputs that would become inputs for
Speaker 1: the next layer down, which would then continue the process
Speaker 1: and so on and so forth until you get to
Speaker 1: the output. So now you had artificial neural networks that
Speaker 1: could tackle more complex challenges, and you would have multiple
Speaker 1: steps in the process. Didn't necessarily mean they were automatically
Speaker 1: better than the perceptron, was just that they were able
Speaker 1: to tackle more complicated tasks. What followed is something that
Speaker 1: will probably sound really familiar to you if you ever
Speaker 1: follow technology or fads, the hype around machine learning and
Speaker 1: artificial intelligence, and keep in mind this is like the
Speaker 1: nineteen sixties. It grew beyond the technology's actual capabilities. At
Speaker 1: that time. People started to project what this technology would
Speaker 1: be able to do, and they did so thinking it
Speaker 1: was going to be in a very short turnaround, like
Speaker 1: we're right on the very precipice of a monstrous breakthrough
Speaker 1: that will bring the science fiction future into the present.
Speaker 1: So when it was realized that we weren't at that, like,
Speaker 1: that's not how progress typically works. It's usually much more
Speaker 1: gradual and humble than that, well, then enthusiasm around AI
Speaker 1: began to take a hit. And as I mentioned already,
Speaker 1: a big part of AI research really comes down to funding,
Speaker 1: and it gets really challenging to secure funding when public
Speaker 1: opinion dims on a technology. We've seen this happen lots
Speaker 1: of times, right, like three D television was a fad
Speaker 1: that was pushed. Now, granted, that one, you could argue
Speaker 1: was more of an example of manufacturing companies that make
Speaker 1: televisions trying to push a technology on consumers and the
Speaker 1: consumers just weren't interested. You could argue that was the
Speaker 1: case there. But virtual reality in the nineteen nineties definitely
Speaker 1: followed this pathway. There was this excitement around virtual reality.
Speaker 1: Then that excitement faded to almost nothing when people realized
Speaker 1: that the actual state of the art of the technology
Speaker 1: was far below where they expected it to be. And
Speaker 1: suddenly people who are working in VR couldn't get funding
Speaker 1: for their work and they kind of had to scrounge
Speaker 1: around in order to keep the development going at all.
Speaker 1: And then eventually we would see that come back around again.
Speaker 1: You could argue that NFTs recently went through this too,
Speaker 1: where the hype went well beyond what NFTs could actually do.
Speaker 1: I've been really down on NFTs in general. I do
Speaker 1: think that there are potential legitimate uses for NFTs, but
Speaker 1: I think the early examples were frivolous and almost solely
Speaker 1: centered around speculation, as in like financial speculation and as
Speaker 1: a result, there was nothing for it to do other
Speaker 1: than to create a bubble that would ultimately burst, which
Speaker 1: is what happened. And maybe NFTs will recover from that
Speaker 1: and become something that's more fundamentally useful in the Internet
Speaker 1: in the future or in digital commerce in the future.
Speaker 1: But it's going to have to get over the catastrophe
Speaker 1: that happened when the rug was pulled out from underneath
Speaker 1: n FTS. And that was all predictable and preventable. But
Speaker 1: like I've said before, like I've lifted the joke from
Speaker 1: Peter Cook, we've learned from our mistakes. We can repeat
Speaker 1: them almost exactly. Anyway, This same sort of hype cycle
Speaker 1: activity happened with neural networks and machine learning in the
Speaker 1: nineteen sixties. Then enter Marvin Minsky and Seymour Pappart of
Speaker 1: MIT's AI lab. They were leading that lab at the time.
Speaker 1: In nineteen sixty nine, they co authored a book titled Perceptrons.
Speaker 1: They were actually critical of that artificial neural network approach
Speaker 1: to AI and machine learning. They were concerned that the
Speaker 1: limitations of the technology meant that you would need an
Speaker 1: unrealistically huge system of artificial neurons. Perhaps then using that
Speaker 1: system to compute an infinite number of variations of the
Speaker 1: same process or task if you wanted to train the
Speaker 1: weights so that they were of the optimal value. So,
Speaker 1: in other words, they thought, it's too impractical and it's
Speaker 1: going to take too much compute time, and you're never
Speaker 1: going to achieve the result you want. You're never going
Speaker 1: to get to that most perfect system. And they believed
Speaker 1: it just had fundamental inescapable flaws. They had different systems
Speaker 1: in mind. Now Minski and Separate tried to push their
Speaker 1: systems forward, and I could do a full episode about
Speaker 1: them too, and their ideas were not bad. They were different.
Speaker 1: It was a different approach. But this also meant that
Speaker 1: researchers who had been pushing the development of our artificial
Speaker 1: neural networks felt forced to move on to different projects
Speaker 1: because financial support for anything connected to the concept of
Speaker 1: neural networks effectively disappeared, right like funding just dropped for that.
Speaker 1: Because here you had these experts in computer science saying, yeah,
Speaker 1: this approach, while interesting, has already hit an insurmountable obstacle
Speaker 1: and it's not going to go any further. It's gone
Speaker 1: as far as it can go. And so a lot
Speaker 1: of computer scientists blamed Minsky and Separate for essentially demolishing
Speaker 1: funding for neural networks for more than a decade, and
Speaker 1: in fact, this would become an era that retrospectively, computer
Speaker 1: scientists would reference as the AI Winter got all Game
Speaker 1: of Thrones up in here. Now. In nineteen eighty two,
Speaker 1: there was a hint of spring thawing out that AI
Speaker 1: Winter researchers in Japan were starting to resurrect work on
Speaker 1: neural network projects, and meanwhile, a scientist named John Hopfield
Speaker 1: submitted a research paper to the National Academy of Sciences
Speaker 1: that brought neural networks back into discussion here in the
Speaker 1: United States. And because Japan was actively investing in developing
Speaker 1: that technology, institutions in the United States began to open
Speaker 1: up the purse strings a bit because there was a
Speaker 1: concern that if there were something to this artificial neural
Speaker 1: network concept, if in fact those obstacles weren't insurmountable, as
Speaker 1: min Skin Separate had suggested, the US could potentially fall
Speaker 1: behind another country because it would fail to fund its development. So,
Speaker 1: in a desire not to have Japan take the ball
Speaker 1: and run with it, the United States began to invest
Speaker 1: again in artificial neural network research and development. In the
Speaker 1: mid nineteen eighties, computer scientists essentially rediscovered the usefulness of
Speaker 1: a process called back propagation. And I've already talked about
Speaker 1: nodes and weights and stuff, but this is going to
Speaker 1: require a little bit more explanation to under stand what
Speaker 1: back propagation is all about. So let's kind of try
Speaker 1: to visualize a neural network. So you've got your input nodes.
Speaker 1: Just think of a bunch of circles. If you were
Speaker 1: drawing it from top to bottom, this would be your
Speaker 1: top layer. This is like the funnels where you're going
Speaker 1: to feed data into the system. Now you've got a
Speaker 1: whole bunch of these at the top and they can
Speaker 1: accept the data that you're feeding in. They process that data,
Speaker 1: and then based upon some operation, they will then send
Speaker 1: an output to a node one layer down. So there's
Speaker 1: lots of other nodes in the layers below, or maybe
Speaker 1: not as many as you have initial layers. You might
Speaker 1: actually have fewer, and the layers above will send to
Speaker 1: you know, data to a specific node depending upon what
Speaker 1: the outcome is. Whatever the output is, so these nodes
Speaker 1: accept the input. These inputs have a bias and a
Speaker 1: weight to them, and this is one of the hidden layers.
Speaker 1: They will then create an output and send that on
Speaker 1: to nodes another layer down. So this goes on until
Speaker 1: you get to your output layer, where you get your
Speaker 1: final result, and then you can determine whether or not
Speaker 1: the final result matches what you were hoping for. So
Speaker 1: did your system properly identify which photos do and don't
Speaker 1: have cats in them? Now, as I mentioned earlier, you
Speaker 1: typically get results that aren't perfect, but we want to
Speaker 1: train the system to improve with every test. Back propagation
Speaker 1: is one way to do this. So with that propagation,
Speaker 1: you actually start with the final output. You've already done
Speaker 1: a test run, right, and you've got your output, and
Speaker 1: maybe your test has five possible final outcomes, but only
Speaker 1: one of those is the outcome you actually want. Okay,
Speaker 1: we'll say it's outcome number one. We're saying I want
Speaker 1: this system to more often than not come to the
Speaker 1: conclusion that's outcome number one. But you run your test.
Speaker 1: It's got you one thousand little tasks in it, and
Speaker 1: you run your test. You find out that it only
Speaker 1: arrives at outcome number one five percent of the time,
Speaker 1: which is actually worse than random chance. Right, it should
Speaker 1: be twenty percent for random chance, but it's only getting
Speaker 1: there five percent of the time. Something is going really
Speaker 1: wrong with your system for it to mistakenly go to
Speaker 1: one of the other options and very rarely go to
Speaker 1: the correct one. So let's say you also noticed the
Speaker 1: outcome number three. It goes to that one forty percent
Speaker 1: of the time. So it's making this mistake forty percent
Speaker 1: of the time and only getting it right five percent
Speaker 1: of the time. So things are seriously out of whack.
Speaker 1: You need to find which connections which would involve the
Speaker 1: biases and the weights that are within your system that
Speaker 1: are leading it to mistakenly arrive at the wrong outcome,
Speaker 1: so frequently you want to reduce those factors, and simultaneously
Speaker 1: you need to boost the ones that lead the system
Speaker 1: to arrive at outcome number one, because that's the answer
Speaker 1: you actually want the system to get to. All Right,
Speaker 1: I've been droning on for a bit. Let's take another
Speaker 1: quick break. When we come back, I'll finish up explaining
Speaker 1: this and then we'll move on to catastrophic forgetting. Okay,
Speaker 1: so we were talking about how you are looking at
Speaker 1: a system that is coming to the wrong conclusion ninety
Speaker 1: five percent of the time. It is a broken system.
Speaker 1: You have to then figure out what factors are causing
Speaker 1: this to happen, and they are numerous, right, They extend
Speaker 1: all the way up to the very top of your
Speaker 1: neural network, the other end where the input comes in.
Speaker 1: But you can't just change everything all at once. You've
Speaker 1: got to figure this out systematically, and that's what backpropagation
Speaker 1: is really all about. Which links one layer up from
Speaker 1: the output have the greatest impact on the outcome. Right,
Speaker 1: changing everything would be tedious, it would be impractical. You
Speaker 1: might even make things worse. Some of these neural networks
Speaker 1: are confoundingly complicated, so it's not really a feasible solution.
Speaker 1: So instead you look at the connections that are having
Speaker 1: the biggest impact on your outcome. So you want things
Speaker 1: where if you make a small change in either the
Speaker 1: bias or the weight, or maybe both, you'll see a
Speaker 1: larger end effect on the outcome. All the connections are
Speaker 1: arguably important, but some are more important than others. Backpropagation
Speaker 1: works backwards from the result toward the other end of
Speaker 1: the network to tweak those connections. It boosts ones that
Speaker 1: lead to the correct or desired response, and it reduces
Speaker 1: the values of those that lead to incorrect or undesired responses.
Speaker 1: If we were to think of this like the classic
Speaker 1: example and chaos theory, this could potentially involve us studying
Speaker 1: hurricane as it hits land and tracing its history back
Speaker 1: as it moved through the ocean, and we would eventually
Speaker 1: arrive at the point where it was a tropical storm,
Speaker 1: and then we would go further back and see the
Speaker 1: factors that led to the creation of that storm. And
Speaker 1: maybe if we tracked it all the way back, we
Speaker 1: would even find that one of a billion factors that
Speaker 1: made the storm was in fact, a butterfly was flapping
Speaker 1: its wings on the other side of the world and
Speaker 1: that contributed to it. Maybe we find out that butterfly
Speaker 1: flap of its wings had an impact, but it was negligible,
Speaker 1: and that if the butterfly hadn't flapped its wings, the
Speaker 1: hurricane still would have happened. That would be an example
Speaker 1: of well, we don't bother adjusting the weight of the
Speaker 1: of the impact of that butterfly flapping its wings because
Speaker 1: it doesn't matter for the end result. But what if
Speaker 1: we were to discover that that butterfly flap of its
Speaker 1: wings is the only reason the hurricane happened that, or
Speaker 1: at least was the primary reason that all the other
Speaker 1: factors pale in comparison. Well, then we'd want to make
Speaker 1: sure we boost the weight of that input, because clearly
Speaker 1: that butterfly is fundamental for hurricanes. I think hurricanes are
Speaker 1: really dangerous, and I would ask butterflies to kind of chill,
Speaker 1: all right. I mean, I don't want butterflies to go away,
Speaker 1: just you know, maybe stop flapping so much. Anyway, the
Speaker 1: formula for backpropagation gets into some calculus that is well
Speaker 1: beyond my knowledge and skill. So rather than attempt to
Speaker 1: stumble my way through an explanation that I don't actually understand,
Speaker 1: I think it's best to leave the concept at the
Speaker 1: high level that I have described right now. So just
Speaker 1: know that it gets way more granular than what I've
Speaker 1: talked about. But essentially, you're looking at those factors that
Speaker 1: led to the ultimate decision and saying which ones of
Speaker 1: these had the greatest impact, and how can I tweak
Speaker 1: them so that I can shape the outcome to one
Speaker 1: I wanted. If we were thinking about that example I
Speaker 1: gave about whether or not you go to the movies.
Speaker 1: Maybe in present day you starts thinking about past experiences
Speaker 1: where you made a decision to go out when you
Speaker 1: had a big day in the following day, and how
Speaker 1: that impacted you, perhaps negatively. Maybe you're like, man, I
Speaker 1: should have gotten a promotion by now, and then you think, well,
Speaker 1: I do go to the movies an awful lot. You
Speaker 1: might say, I need to adjust some of the factors
Speaker 1: that affect my decision making process and perhaps prioritize my career.
Speaker 1: Or if you've decided that late stage capitalism is terrible
Speaker 1: evil and that you're going to try and live a
Speaker 1: hedonistic lifestyle of a wandering soul, maybe you say, I'm
Speaker 1: going to go and see my movie with my friend,
Speaker 1: and yeah, that's just how it is, because that's the
Speaker 1: most important thing to me. You only go around this
Speaker 1: crazy world once. After all, I'm not telling you which
Speaker 1: way to go. I'm still finding my own way. But yeah,
Speaker 1: back propagation would be how you would go back and say,
Speaker 1: all right, well, because I don't like the outcome that happened,
Speaker 1: I need to change the way. These factors weigh in
Speaker 1: on the decision making process that goes through the whole system. Now,
Speaker 1: the advancements in the science of neural networks proved that
Speaker 1: the technology no longer operated under the constraints that concern
Speaker 1: Minski and support in the late sixties, so once again
Speaker 1: funding found its way to neural network research and development projects.
Speaker 1: Now let's finally talk about forgetting and what makes it catastrophic.
Speaker 1: So you could, in theory, develop an artificial neural network
Speaker 1: and have a library of training data, and the only
Speaker 1: thing you ever do with this network is you feed
Speaker 1: that same set of training data to that same neural
Speaker 1: network over and over in an effort to get performance
Speaker 1: as close to perfect as you possibly can. Just you know,
Speaker 1: it's kind of like if you have a car and
Speaker 1: you're constantly tweaking it so it will perform better, and
Speaker 1: maybe you chase one thing and it boosts performance in
Speaker 1: one area, but it kind of negatively impacts performance in
Speaker 1: another area, so then you got to tweak something else.
Speaker 1: You could be doing that with an artificial neural network
Speaker 1: forever and just be using the same set of training data.
Speaker 1: And all you're trying to do is make a system
Speaker 1: that could handle that training data better than any other
Speaker 1: system in the world, and that would be interesting, but
Speaker 1: it would be useless from a practical standpoint. You could say, like, hey,
Speaker 1: you want to see my machine that can sort through
Speaker 1: only this collection of photographs and pick out the ones
Speaker 1: that have cats in them and the ones that don't.
Speaker 1: Pretty pretty darn effectively, but not perfectly. It's not really
Speaker 1: an interesting value proposition, right, So more likely you are
Speaker 1: eventually going to start feeding lots of different kinds of
Speaker 1: data to this neural network. And know, yeah, you train
Speaker 1: the network on certain data sets, but your goal is
Speaker 1: to feed new sets of data data the system has
Speaker 1: never encountered before and rely on the system's ability to
Speaker 1: process this information correctly to get the result you want.
Speaker 1: And we might even be talking about stuff the human
Speaker 1: beings can't easily do, right, But see, the training data
Speaker 1: is going to mean that the network will start to
Speaker 1: create and reinforce certain pathways, and those pathways will over
Speaker 1: time get stronger and stronger, just as we said at
Speaker 1: the beginning of this episode. But new data is going
Speaker 1: to necessitate new pathways. Sometimes when the system begins to
Speaker 1: form these new pathways, it forgets the old pathways. So
Speaker 1: it's possible for a neural network to actually get worse
Speaker 1: at the task it had previously been trained to do
Speaker 1: with the actual training material. In fact, in a true catastrophe,
Speaker 1: the system might forget the objective and doesn't recognize what
Speaker 1: the desired outcome is meant to be, so the results
Speaker 1: can appear random and meaningless. It's as if the system
Speaker 1: has developed some form of amnesia. So this is prevalent,
Speaker 1: most prevalent anyway, in systems that rely on unguided learning.
Speaker 1: With guided learning, you have engineers who are carefully selecting
Speaker 1: the data that gets fed into a system. An unguided
Speaker 1: system would collect raw data from wherever and attempt to
Speaker 1: deliver desired results, and that those are the kinds of
Speaker 1: neural networks that are more prone to catastrophic forgetting. But
Speaker 1: as I said, machine learning systems tackle new data, maybe
Speaker 1: even new tasks, and then you get the risk of
Speaker 1: the system forgetting stuff. So I jokingly say, it's kind
Speaker 1: of like when I learned something new, it has to
Speaker 1: push out something old, like you know, my friend's phone
Speaker 1: number or something. Suddenly I can no longer remember it
Speaker 1: because I learned some new interesting fact, as if I
Speaker 1: have met my capacity for being able to know things.
Speaker 1: So learning anything new necessitates having to forget something I
Speaker 1: used to know, like gat Ye, because now gat Ye
Speaker 1: is just somebody that I used to know. But wait,
Speaker 1: there's more. Just as a system can experience catastrophic forgetting,
Speaker 1: it can also experience catastrophic remembering. This is when a
Speaker 1: system mistakenly believes it is doing one process, a task
Speaker 1: it had previously been trained to do, rather than the
Speaker 1: one it's actually trying to do. So let's say we've
Speaker 1: got an artificial neural network, and originally we taught it
Speaker 1: to recognize the photos that have cats in them versus
Speaker 1: the ones that don't. But now we have retrained the
Speaker 1: same artificial neural network to try and recognize handwritten text.
Speaker 1: Except when we feed handwritten text to the system, suddenly
Speaker 1: the system believes it's trying to determine where the cats are.
Speaker 1: This is something that can happen with machine learning systems too,
Speaker 1: and you still get bad results out of it. So
Speaker 1: this is a real problem. Now, these are not insurmountable problems.
Speaker 1: There are some solutions that are actually intuitive. For example,
Speaker 1: any gamer out there knows that it's best to save
Speaker 1: your game just before you head into a big boss battle,
Speaker 1: just in case things don't go the way you planned well.
Speaker 1: With artificial neural networks, it's maybe not a bad idea
Speaker 1: to make a copy of a network before you retrain
Speaker 1: it to do something new. Then you still have the
Speaker 1: backup if things do go pair shape. There are other
Speaker 1: approaches to decreasing the risk of catastrophic forgetting or catastrophic remembering.
Speaker 1: An article in Applied Mathematics titled Overcoming Catastrophic forgetting a
Speaker 1: neural networks describes a system in which the researchers purposefully
Speaker 1: slowed down the network's ability to change the weights involved
Speaker 1: in important tasks from previous training cycles. So this makes
Speaker 1: teaching the system to do new tasks a little more
Speaker 1: challenging because it's protecting these weights. It's preventing the system's
Speaker 1: ability to be completely plasid, which means the system has
Speaker 1: to work around these constraints and still learn how to
Speaker 1: do the new task, but in the process it means
Speaker 1: it doesn't forget how to do the previous tasks. This
Speaker 1: article is interesting because the tasks the researchers actually used
Speaker 1: the purposes of training, Like, what were they teaching the
Speaker 1: artificial neural network to do well? They were teaching it
Speaker 1: how to play atari twenty six hundred games. So they
Speaker 1: would start with one game and train the system on
Speaker 1: how to play the game. Then they would give the
Speaker 1: system a new game with different game mechanics, and the
Speaker 1: system would have to learn how to play this new game,
Speaker 1: but they wanted to see if it could still remember
Speaker 1: how to play the original game. That was kind of
Speaker 1: the system they were working on. They were tweaking things
Speaker 1: so that the machine learning artificial neural network as a
Speaker 1: whole could learn how to play multiple Atari twenty six
Speaker 1: hundred games without forgetting how to do the previous ones.
Speaker 1: This is a non trivial task. I mean, it takes
Speaker 1: a lot of work to see exactly how to preserve
Speaker 1: things so that you're not slowing down the learning process
Speaker 1: too much, but you're also not inviting the possibility of
Speaker 1: catastrophic forgetting. Now, that's just one example of how researchers
Speaker 1: are looking to mitigate the problem of catastrophic forgetting in
Speaker 1: catastrophic remembering. There are other methods as well, and maybe
Speaker 1: I'll do another episode where I'll go into more detail
Speaker 1: on some of those. They do get pretty complicated, and
Speaker 1: in fact, eventually Rerilli and I even eventually pretty early
Speaker 1: on I hit my limit for as far as I
Speaker 1: can understand the actual mechanics of the system. So rather
Speaker 1: than you know, try and punch above my weight, I
Speaker 1: think it's best to kind of be a little more general,
Speaker 1: but just to have that understanding to kind of get
Speaker 1: a better appreciation of some of the challenges relating to
Speaker 1: artificial intelligence in general and machine learning in particular. And again,
Speaker 1: like this machine learning issue, you it's really a bigger
Speaker 1: problem with more sophisticated systems that are meant to do
Speaker 1: unsupervised and unguided learning, right, those are the ones that
Speaker 1: are going to be more prone to these issues. If
Speaker 1: we're talking about supervised and guided learning, where engineers are
Speaker 1: being very careful with the data being fed to a system,
Speaker 1: it's less likely to happen. But the whole promise, or
Speaker 1: at least the you know, not the promise of the
Speaker 1: technology itself, but the promise of the people who are
Speaker 1: funding it, is that this technology is going to reach
Speaker 1: a point where it's able to learn on its own
Speaker 1: and be able to do things better than people can do,
Speaker 1: to free us up to doing, you know, stuff we
Speaker 1: want to do instead of stuff we have to do.
Speaker 1: That's like the science fiction dream version of AI. As
Speaker 1: we all know, getting there is much more painful. It's
Speaker 1: not like a simple process of Hey, we've made everything
Speaker 1: easy to do now and you don't have to work
Speaker 1: all day. You can enjoy your life and pursue your
Speaker 1: dreams and develop your hobbies and your interests, and you
Speaker 1: can have fulfillment and somehow money isn't important anymore. Like
Speaker 1: that seems to be the Star Trek version of the
Speaker 1: future that people want it to go in. But as
Speaker 1: we have seen, the process of getting there is way
Speaker 1: more painful. As you know, people face a reality of
Speaker 1: potentially being out of work because of AI, or maybe
Speaker 1: being paid way less to do work because the AI
Speaker 1: is doing most of it. These are not that's not
Speaker 1: Star Trek feature. That's getting like into Blade Runner future,
Speaker 1: So we don't want that one. By the way, the
Speaker 1: tears in the rain speech is fantastic, but you do
Speaker 1: not want to live in the Blade Runner world. Trust me.
Speaker 1: You might not want to live in the Star Trek
Speaker 1: world either, because those outfits don't look that comfortable anyway.
Speaker 1: That's my little discussion about AI, machine learning and cast
Speaker 1: trophic forgetting in castrophic. Remembering this is just one of
Speaker 1: the challenges associated with AI and machine learning. I don't
Speaker 1: mean to suggest it's the one and only, or even
Speaker 1: that it's the most important one, but it is one
Speaker 1: that I had not really heard of until I listened
Speaker 1: to that Skeptics Guide to the Universe episode over the weekend,
Speaker 1: and it was really interesting to dive into the material
Speaker 1: and read up about it and to get a better
Speaker 1: understanding of what it means and how it works. I
Speaker 1: hope you liked that episode from last year, twenty twenty three,
Speaker 1: machine Learning and Catastrophic Forgetting. I am working on other
Speaker 1: episodes that relate to AI. I also want to do
Speaker 1: an episode about companies that claim to be part of
Speaker 1: the artificial intelligence space but in fact use little if
Speaker 1: any AI technology, because that has become a thing. As
Speaker 1: we all know, when there is the combination of huge
Speaker 1: amounts of money and low amounts of understanding, you have
Speaker 1: the perfect breeding ground for scams and con artists and
Speaker 1: that kind of thing. So I do plan on doing
Speaker 1: an episode about various startups that claim at some level
Speaker 1: to be part of artificial intelligence, but when you really
Speaker 1: start to examine them, have little to no connection to
Speaker 1: that world. So being on the lookout for that, It's
Speaker 1: going to take me some time to do some research
Speaker 1: because there's lots of different sources to go through on
Speaker 1: that one. But that's what I'm working on for probably
Speaker 1: next week. I'm hoping next week. In the meantime, for
Speaker 1: those of you here in the United States, I hope
Speaker 1: you have a safe Fourth of July celebration. Make sure
Speaker 1: that you spend time with friends and loved ones, and
Speaker 1: you know, be very careful if you're going to be
Speaker 1: around fireworks. Those things are very dangerous for everyone else
Speaker 1: out there who's not celebrating a holiday and fourth of July.
Speaker 1: I hope you have an excellent Fourth of July wherever
Speaker 1: you are, and that whatever you enjoy doing, you get
Speaker 1: to do a lot of it on the Fourth of July.
Speaker 1: As long as it's you know, not hurting yourself or
Speaker 1: other people. That's it for me. I will talk to
Speaker 1: you again really soon. Tech Stuff is an iHeartRadio production.
Speaker 1: For more podcasts from iHeartRadio, visit the iHeartRadio app, Apple Podcasts,
Speaker 1: or wherever you listen to your favorite shows.