Speaker 1: Welcome to tech Stuff, a production from iHeartRadio. Hey there,
Speaker 1: and welcome to tech Stuff. I'm your host, Jonathan Strickland.
Speaker 1: I'm an executive producer with iHeartRadio and how the tech
Speaker 1: are you? So? Over this past weekend, I was listening
Speaker 1: to the podcast The Skeptics Guide to the Universe, which
Speaker 1: I have no connection to. I just listened to it,
Speaker 1: and it included a section on AI that referenced something
Speaker 1: I don't think I had heard of before, which is
Speaker 1: really talking more about my oversight than anything else. Maybe
Speaker 1: I did hear about it, but then I forgot about it,
Speaker 1: you know, catastrophically. So the thing they talked about was
Speaker 1: catastrophic forgetting in artificial intelligence, specifically in machine learning systems
Speaker 1: built on artificial neural networks. Now, before we talk about
Speaker 1: catastrophic forgetting, which as I mentioned, is related to neural
Speaker 1: networks and machine learning, we really need to do a
Speaker 1: quick reminder, not a quick reminder. We need to do
Speaker 1: a full reminder on how all this works. And that's
Speaker 1: going to require us to do a whole lot of remembering.
Speaker 1: Not a catastrophic amount, but a lot. So the history
Speaker 1: of artificial intelligence as a discipline is one of intense
Speaker 1: and important debates in fields like computer science. Now, I
Speaker 1: have often talked about how AI can be seen as
Speaker 1: the convergence of several other disciplines into its own field,
Speaker 1: and there's more than one way to approach the challenge
Speaker 1: of artificial intelligence. And in the history of AI, we
Speaker 1: actually saw that play out, and some would argue the
Speaker 1: way it played out means that we're actually just now
Speaker 1: playing catch up. So different schools of thought pushed these
Speaker 1: different approaches forward as this should be the prevailing methodology
Speaker 1: we use to devel artificial intelligence. This is important because
Speaker 1: the development of AI does not exist in a vacuum
Speaker 1: right It exists in our real world. Research requires funding,
Speaker 1: and when you've got different sides arguing that their approach
Speaker 1: to artificial intelligence is superior and that the alternatives are
Speaker 1: not just inferior, but potentially limited to the point of
Speaker 1: being useless, well you've got a metaphorical wrestling match going on.
Speaker 1: The winner takes home the big prize of getting funding
Speaker 1: for their research, and the loser has to scrabble for
Speaker 1: whatever they can find, and often they will see their
Speaker 1: work languish as a result. By the way, this is
Speaker 1: why I often bring stuff up in this podcast that
Speaker 1: is outside the realm of tech. I've received a lot
Speaker 1: of messages over the years from folks saying that I
Speaker 1: should leave out stuff like money or politics. Politics is
Speaker 1: the big one. But to me, that doesn't make sense
Speaker 1: cause tech exists within our world, a world that is
Speaker 1: largely shaped by money and politics. I don't think we
Speaker 1: can separate the tech from all of that because I
Speaker 1: believe that if you were to somehow magically remove those influences,
Speaker 1: If somehow money and politics never played a part in
Speaker 1: the development of technology, our tech would look very different
Speaker 1: from what it does today. Not necessarily better or worse,
Speaker 1: but different. I mean, think about Thomas Edison. He was
Speaker 1: very much driven by financial success, like his work in
Speaker 1: tech was really mostly about making lots of money. And
Speaker 1: without the making lots of money part, you don't really
Speaker 1: have his drive to really bring together the brightest minds
Speaker 1: of his generation and set them to work on creating
Speaker 1: incredible technology. So I think we have to take all
Speaker 1: these things into consideration. Anyway, that's a total rabbit trail,
Speaker 1: and I apologize. Let's get back to our story. It
Speaker 1: really begins around nineteen forty three when a pair of
Speaker 1: researchers at the University of Chicago first proposed the concept
Speaker 1: of the basic unit of a neural network. Those researchers
Speaker 1: were Warren McCullough and Walter Pets, And in fact, they
Speaker 1: demonstrate their idea by showing a simple electrical circuit the
Speaker 1: very basis for what would become a neural network. So
Speaker 1: their proposal was a system that would use those simple
Speaker 1: circuits to mimic the neurons that we have in our noggins.
Speaker 1: So our brain consists of a bunch of these neurons,
Speaker 1: and you might wonder how much is a bunch, Well,
Speaker 1: we're talking about on average, around one hundred billion neurons
Speaker 1: in the human brain. These neurons interconnect with each other.
Speaker 1: It's not just a one to one, right, You've got
Speaker 1: these interconnections between all these different neurons, not with every
Speaker 1: neuron connected to every other neuron, but lots of interconnections.
Speaker 1: And if we're looking at just the connection, you would
Speaker 1: count more than one hundred trillion of them in the
Speaker 1: typical human brain. And these connections in our brains make
Speaker 1: up neural circuits. Those circuits light up, and that represents
Speaker 1: us doing lots of different stuff from experiencing the world
Speaker 1: around us so perception to thinking about a past memory.
Speaker 1: You know that typically is like recreating the same pathway
Speaker 1: over and over, and sometimes we don't recreate it exactly correctly,
Speaker 1: and our memory ends up not being a perfect representation
Speaker 1: of the thing that we actually experienced. This is why
Speaker 1: things like eyewitness testimony is not always very reliable, because
Speaker 1: our memories aren't infallible. They can trick us and we
Speaker 1: can have all those pathways light up when we learn
Speaker 1: a new skill and we start forming new pathways, and
Speaker 1: then as we practice this skill, we start to reinforce
Speaker 1: those pathways. So McCulla and Pitts propose that we create
Speaker 1: machines capable of doing essentially a similar thing that our
Speaker 1: brains do, so kind of a neuromimicry, not exactly one
Speaker 1: to one the way our brains work, but inspired by
Speaker 1: the way our brains work. Now, we would be limited
Speaker 1: by what the technology of the day would be able
Speaker 1: to do, because there's no feasible way we could create
Speaker 1: a massive electrical system with one hundred billion individual simple
Speaker 1: circuits with more than one hundred trillion connections between them.
Speaker 1: That would be beyond our capability it would be beyond
Speaker 1: our resources. We could, however, create systems that used interconnected
Speaker 1: circuits to process information and to teach such a system
Speaker 1: to do specific tasks. Now, in nineteen forty nine, Donald
Speaker 1: Hebb wrote a book about biological neurons, and he titled
Speaker 1: this book the Organization of Behavior and suggested neural pathways
Speaker 1: get stronger with additional use, kind of like you know,
Speaker 1: if you exercise your muscles, you build strength over time.
Speaker 1: Will so is the same with neural pathways. And if
Speaker 1: you don't use those muscles well, then your muscles get weaker. Well,
Speaker 1: same with neural pathways. If you end up learning a skill,
Speaker 1: but then over a great amount of time you no
Speaker 1: longer practice that skill, you're gonna lose some of your ability.
Speaker 1: Maybe not all of it, but at least some of it.
Speaker 1: And you have to you know, like I think about
Speaker 1: wrestlers who come back from from retirement. Professional wrestlers, they
Speaker 1: call it ring rust. You got to knock off the
Speaker 1: ring rust and get back into step and kind of
Speaker 1: get back into your groove. And it takes a little time.
Speaker 1: Typically sometimes you know, you can get back into the
Speaker 1: game faster than others, but you get the idea, and
Speaker 1: also heb ended up proposing the concept of cells that
Speaker 1: fire together wire together, meaning that neurons that fire at
Speaker 1: the same time end up strengthening faster than other neurons do.
Speaker 1: So when you get into that system, you can actually
Speaker 1: reinforce those pathways. And for AI this would be really important.
Speaker 1: And it wasn't very long after Donald have had published
Speaker 1: this work that researchers in the field of AI tried
Speaker 1: to apply that concept that philosophy to computer science. By
Speaker 1: the mid nineteen fifties, the burgeoning computer science Lab and
Speaker 1: AI Lab at MIT was building out neural networks based
Speaker 1: on Hebb's ideas. Meanwhile, another computer scientist named Frank Rosenblatt
Speaker 1: was looking at primitive neural systems and he started with flies,
Speaker 1: like house flies. He wanted to explore systems that were
Speaker 1: involved when a fly would quickly move away after detecting
Speaker 1: a possible threat, like instantly, or at least appearing to
Speaker 1: us to instantly react to something. So, for example, a
Speaker 1: fly swater coming at it like you might be moving
Speaker 1: the fly swater very quickly, and yet the fly is
Speaker 1: able to move super fast with no perceivable delay. Right,
Speaker 1: we know that we have a delay from when we
Speaker 1: perceive something to when we can act on something. Like
Speaker 1: if you've ever been in a fender bender in a
Speaker 1: car accident, you know that that there's a delay between
Speaker 1: when you see the issue when you can hit the brake,
Speaker 1: and that can lead to accidents. Well, with flies, that
Speaker 1: delay seems to be super super small. So Rosenblatt was
Speaker 1: really interested in exploring the neurological reasons for that. How
Speaker 1: can that happen? It has to be really simple, right,
Speaker 1: There has to be a simple and more or less
Speaker 1: direct pathway that exists to allow a fly to react
Speaker 1: to detecting a potential threat like that. And if you
Speaker 1: could replicate that with electronics, you could have a very
Speaker 1: simple but potentially powerful artificial intelligence system. So he came
Speaker 1: up with this system that would be based off that
Speaker 1: very simple direct pathway that you would see in something
Speaker 1: like a fly, and he called it the perceptron. So
Speaker 1: he went back to the simple circuit design that was
Speaker 1: proposed by Pitts and McCullough and he built out the
Speaker 1: Mark one perceptron or perceptron I guess I should say,
Speaker 1: So let's talk about a perceptron like not big p
Speaker 1: but a little p perceptron. This is probably what we
Speaker 1: would call a neural node in a modern neural network.
Speaker 1: So the purpose of the perceptron was to accept inputs
Speaker 1: and produce an output based on some threshold. Like if
Speaker 1: the inputs meet a certain threshold, one output would be produced.
Speaker 1: If they failed to do so, a different output would
Speaker 1: be produced. The inputs, in turn would be assigned weights,
Speaker 1: which would factor into the output the perceptron would generate.
Speaker 1: So when we're talking weights, I mean you eights as
Speaker 1: in like how heavy something is, or in this case,
Speaker 1: how much impact that thing has. So we're talking about
Speaker 1: how much impact one input has relative to other inputs.
Speaker 1: Let me use a really mundane human example to kind
Speaker 1: of explain what this means. Let's say that your friend
Speaker 1: asks you to go see a movie with them, and
Speaker 1: it's going to be playing tonight at nine pm. But
Speaker 1: you've had a really busy day and you might not
Speaker 1: be able to even eat dinner until around nine pm.
Speaker 1: And if you go see this movie, it might mean
Speaker 1: having to skip dinner or to try and eat something
Speaker 1: really fast and unhealthy before you go to the movie.
Speaker 1: What's more, you got a really big day tomorrow and
Speaker 1: you feel like you really need to be well rested
Speaker 1: for it. However, at the same time, you haven't seen
Speaker 1: this friend in ages, and you really like this person
Speaker 1: and you've wanted to hang with them for a really
Speaker 1: long time. Plus the movie they're suggesting is one you've
Speaker 1: hell he wanted to see and you haven't gone yet. Well,
Speaker 1: you would likely assign at least unconsciously weights to each
Speaker 1: of these factors before you make your decision. You know,
Speaker 1: if getting some dinner without having to rush and also
Speaker 1: to be really well rested for tomorrow are really important
Speaker 1: to you, you'll probably reluctantly decline the offer. But if
Speaker 1: you really crave some time with your friend and you
Speaker 1: really want to see that movie before all the spoilers
Speaker 1: come out on Facebook or whatever, maybe you'll say yes.
Speaker 1: Your decision depends upon the weights you assign those factors,
Speaker 1: those inputs, even if you don't consciously think about it
Speaker 1: that way. Well. The Perceptron system worked in a similar way,
Speaker 1: produced outputs by taking the inputs into consideration, including each
Speaker 1: input's weight. Moreover, the more you submitted inputs, the more
Speaker 1: the system would quote unquote learn how to weight each
Speaker 1: of those inputs, all with the goal of bringing the
Speaker 1: actual output that the process or you know, generates closer
Speaker 1: to the one you want it to generate. Okay, I
Speaker 1: just said a lot there. We've got some more to
Speaker 1: get through. But before we get to that, let's take
Speaker 1: a quick break. All right. Before the break, we were
Speaker 1: talking about inputs and weights and the idea of getting
Speaker 1: an output that is close to what you want the
Speaker 1: system to do. That's not a guarantee, right, The system
Speaker 1: could generate an output that's quote unquote wrong, you know,
Speaker 1: depending on whatever task you've set this machine learning system
Speaker 1: to learn. And that gets a bit conceptual. So let's
Speaker 1: talk about a simple example that I love to use.
Speaker 1: If you've been listening to tech sta for a while,
Speaker 1: you've heard this before, and that's talking about pictures of cats.
Speaker 1: Because cats ruled the internet. I don't know if they
Speaker 1: still do. They won't talk to me, so they just
Speaker 1: knock things off shelves. Anyway, if your goal is to
Speaker 1: teach a computer system to differentiate photos that include a
Speaker 1: cat from photos that do not include a cat, well,
Speaker 1: you would need to train the system, and part of
Speaker 1: that includes feeding the system a whole bunch of photographs.
Speaker 1: Some of those would have cats in them, some would not,
Speaker 1: and chances are the system would misidentify photos, maybe a
Speaker 1: significant number of those photos. You would probably have false
Speaker 1: positives where the system thinks there's a cat there and
Speaker 1: there's not, and false negatives where it doesn't think there's
Speaker 1: a cat there but there is. At that point, your
Speaker 1: goal is to try and teach the system to close
Speaker 1: the gap between the actual results it produces and what
Speaker 1: you want it to produce. In some systems that means
Speaker 1: you might have to go in manually to adjust the
Speaker 1: input weights to increase the weight of one input versus
Speaker 1: another in an effort to cut down on mistakes. So
Speaker 1: the perceptron was interesting, but it was very limit in complexity.
Speaker 1: It was essentially a single layer where you'd feed a
Speaker 1: bunch of inputs in and you would get an output,
Speaker 1: So it was suitable for a subset of computational challenges,
Speaker 1: but anything beyond that was well beyond its own reach
Speaker 1: as a single layer network. By the late nineteen fifties,
Speaker 1: other researchers had created new neural networks that were multi layered,
Speaker 1: so a node or neuron didn't just accept inputs, it
Speaker 1: would generate outputs that then would become inputs for another
Speaker 1: layer down. So instead of just having one layer of nodes,
Speaker 1: you would have multiple layers of nodes. Typically you would
Speaker 1: have one at the quote unquote top of the network,
Speaker 1: and you would have outputs at the bottom, and the
Speaker 1: ones in between would be often referred to as hidden layers,
Speaker 1: and who knows how many there would be. So anyway,
Speaker 1: you would feed data to the system. The initial nodes
Speaker 1: would generate information as outputs that would become inputs for
Speaker 1: the next layer down, which would then continue the process
Speaker 1: and so on and so forth until you get to
Speaker 1: the output. So now you had artificial neural networks that
Speaker 1: could tackle more complex challenges, and you would have multiple
Speaker 1: steps in the process. Didn't necessarily mean they were automatically
Speaker 1: better than the perceptron, was just that they were able
Speaker 1: to tackle more complicated tasks. What followed is something that
Speaker 1: will probably sound really familiar to you if you ever
Speaker 1: follow technology or fads, the hype around machine learning and
Speaker 1: artificial intelligence, And keep in mind this is like the
Speaker 1: nineteen sixties. It grew beyond the technology's actual capabilities. At
Speaker 1: that time, people started to project what this technology would
Speaker 1: be able to do, and they did so thinking it
Speaker 1: was going to be in a very short turnaround, like
Speaker 1: we're right on the very precipice of a monstrous breakthrough
Speaker 1: that will bring the science fiction future into the press.
Speaker 1: So when it was realized that we weren't at that, like,
Speaker 1: that's not how progress typically works. It's usually much more
Speaker 1: gradual and humble than that, Well, then enthusiasm around AI
Speaker 1: began to take a hit. And as I mentioned already,
Speaker 1: a big part of AI research really comes down to funding,
Speaker 1: and it gets really challenging to secure funding when public
Speaker 1: opinion dims on a technology. We've seen this happen lots
Speaker 1: of times, right, Like three D television was a fad
Speaker 1: that was pushed. Now, granted, that one, you could argue
Speaker 1: was more of an example of manufacturing companies that make
Speaker 1: televisions trying to push a technology on consumers and the
Speaker 1: consumers just weren't interested. You could argue that was the
Speaker 1: case there. But virtual reality in the nineteen nineties definitely
Speaker 1: followed this pathway. There was this excitement around virtual reality.
Speaker 1: Then that excitement faded to almost nothing when people realized
Speaker 1: that the actual state of the art of the technology
Speaker 1: was far below where they expected it to be. And
Speaker 1: suddenly people who are working in VR couldn't get funding
Speaker 1: for their work, and they kind of had to scrounge
Speaker 1: around in order to keep the development going at all,
Speaker 1: and then eventually we would see that come back around again.
Speaker 1: You could argue that NFTs recently went through this too,
Speaker 1: where the hype went well beyond what NFTs could actually do.
Speaker 1: I've been really down on NFTs in general. I do
Speaker 1: think that there are potential legitimate uses for NFTs, but
Speaker 1: I think the early examples were frivolous and almost solely
Speaker 1: centered around speculation, as in like financial speculation, and as
Speaker 1: a result, there was nothing for it to do other
Speaker 1: than to create a bubble that would ultimately burst, which
Speaker 1: is what happened. And maybe NFTs will recover from that
Speaker 1: and become something that's more fundamentally useful in the Internet
Speaker 1: in the future or in digital commerce in the future,
Speaker 1: but it's going to have to get over the catastrophe
Speaker 1: that happened when the rug was pulled out from underneath
Speaker 1: n FTS, and that was all, you know, predictable and preventable.
Speaker 1: But like I've said before, like I've lifted the joke
Speaker 1: from Peter Cook, we've learned from our mistakes. We can
Speaker 1: repeat them almost exactly. Anyway, This same sort of hype
Speaker 1: cycle activity happened with neural networks and machine learning in
Speaker 1: the nineteen sixties. Then enter Marvin Minsky and Seymour Pappart
Speaker 1: of MIT's AI lab. They were leading that lab at
Speaker 1: the time. In nineteen sixty nine, they co authored a
Speaker 1: book titled Perceptrons. They were actually critical of that artificial
Speaker 1: neural network approach to AI and machine learning. They were
Speaker 1: concerned that the limitations of the technology meant that you
Speaker 1: and you need an unrealistically huge system of artificial neurons,
Speaker 1: perhaps then using that system to compute an infinite number
Speaker 1: of variations of the same process or task if you
Speaker 1: wanted to train the weights so that they were of
Speaker 1: the optimal value. So, in other words, they thought, it's
Speaker 1: too impractical, and it's going to take too much compute time,
Speaker 1: and you're never going to achieve the result you want.
Speaker 1: You're never going to get to that most perfect system.
Speaker 1: And they believed it just had fundamental inescapable flaws. They
Speaker 1: had different systems in mind. Now Minski and Separate tried
Speaker 1: to push their systems forward, and I could do a
Speaker 1: full episode about them too, and their ideas were not bad.
Speaker 1: They were different. It was a different approach. But this
Speaker 1: also meant that researchers who had been pushing the development
Speaker 1: of our artificial neural networks felt forced to move on
Speaker 1: to different projects because financial support for anything connected to
Speaker 1: the concept of neural networks effectively disappeared, right like funding
Speaker 1: just dropped for that. Because here you had these experts
Speaker 1: in computer science saying, yeah, this approach, while interesting, has
Speaker 1: already hit an insurmountable obstacle and it's not going to
Speaker 1: go any further. It's gone as far as it can go.
Speaker 1: And so a lot of computer scientists blamed Minsky and
Speaker 1: Separate for essentially demolishing funding for neural networks for more
Speaker 1: than a decade, and in fact, this would become an
Speaker 1: era that retrospectively, computer scientists would reference as the AI
Speaker 1: Winter got all Game of Thrones up in here Now.
Speaker 1: In nineteen eighty two, there was a hint of spring
Speaker 1: thawing out that AI Winter researchers in Japan were starting
Speaker 1: to resurrect work on neural network projects. And meanwhile, a
Speaker 1: scientist named John Hopfield submitted a research paper to the
Speaker 1: Neattional Academy of Sciences that brought neural networks back into
Speaker 1: discussion here in the United States, and because Japan was
Speaker 1: actively investing in developing that technology, institutions in the United
Speaker 1: States began to open up the purse strings a bit
Speaker 1: because there was a concern that if there were something
Speaker 1: to this artificial neural network concept, if in fact those
Speaker 1: obstacles weren't insurmountable, as min skin Separate had suggested, the
Speaker 1: US could potentially fall behind another country because it would
Speaker 1: fail to fund its development. So, in a desire not
Speaker 1: to have Japan take the ball and run with it,
Speaker 1: the United States began to invest again in artificial neural
Speaker 1: network research and development. In the mid nineteen eighties, computer
Speaker 1: scientists essentially rediscovered the usefulness of a process called back propagation.
Speaker 1: And I've already talked about nodes and weights and stuff,
Speaker 1: but this is going to require a little bit more
Speaker 1: explanation to understand what by propagation is all about. So
Speaker 1: let's kind of try to visualize a neural network. So
Speaker 1: you've got your input nodes. Just think of a bunch
Speaker 1: of circles. If you were drawing it from top to bottom,
Speaker 1: this would be your top layer. This is like the
Speaker 1: funnels where you're going to feed data into the system.
Speaker 1: Now you've got a whole bunch of these at the top,
Speaker 1: and they can accept the data that you're feeding in.
Speaker 1: They process that data and then based upon you some operation,
Speaker 1: they will then send an output to a node one
Speaker 1: layer down. So there's lots of other nodes in the
Speaker 1: layers below, or maybe not as many as you have
Speaker 1: initial layers. You might actually have fewer, and the layers
Speaker 1: above will send to you know, data to a specific
Speaker 1: node depending upon what the outcome is. Whatever the output is,
Speaker 1: So these nodes accept the input. These inputs have a
Speaker 1: bias and a weight to them, and this is one
Speaker 1: the hidden layers. They will then create an output and
Speaker 1: send that on to nodes another layer down. So this
Speaker 1: goes on until you get to your output layer where
Speaker 1: you get your final result, and then you can determine
Speaker 1: whether or not the final result matches what you were
Speaker 1: hoping for. So did your system properly identify which photos
Speaker 1: do and don't have cats in them? Now, as I
Speaker 1: mentioned earlier, you typically get results that aren't perfect, but
Speaker 1: we want to train the system to improve with every test.
Speaker 1: Back propagation is one way to do this. So with
Speaker 1: that propagation, you actually start with the final output. You've
Speaker 1: already done a test run, right, and you've got your output,
Speaker 1: and maybe your test has five possible final outcomes, but
Speaker 1: only one of those is the outcome you actually want. Okay,
Speaker 1: we'll say it's outcome number one. We're saying I want
Speaker 1: this system to more often than not come to the
Speaker 1: conclusion that it's outcome number one one. But you run
Speaker 1: your test. It's got one thousand little tasks in it,
Speaker 1: and you run your test, you find out that it
Speaker 1: only arrives at outcome number one five percent of the time,
Speaker 1: which is actually worse than random chance. Right, it should
Speaker 1: be twenty percent for random chance, But it's only getting
Speaker 1: there five percent of the time. Something is going really
Speaker 1: wrong with your system for it to mistakenly go to
Speaker 1: one of the other options and very rarely go to
Speaker 1: the correct one. So let's say you also noticed the
Speaker 1: outcome number three. It goes to that one forty percent
Speaker 1: of the time. So it's making this mistake forty percent
Speaker 1: of the time and only getting it right five percent
Speaker 1: of the time. So things are seriously out of whack.
Speaker 1: You need to find which connections which would involve the
Speaker 1: biases and the weights that are within your system that
Speaker 1: are leading it to mistakenly arrive at the wrong outcome.
Speaker 1: So frequently you want to reduce those factors, and simultaneously
Speaker 1: you need to boost the ones that lead the system
Speaker 1: to arrive at outcome number one, because that's the answer
Speaker 1: you actually want the system to get to. All Right,
Speaker 1: I've been droning on for a bit, Let's take another
Speaker 1: quick break. When we come back, I'll finish up explaining
Speaker 1: this and then we'll move on to catastrophic forgetting. Okay,
Speaker 1: so we were talking about how you are looking at
Speaker 1: a system that is coming to the wrong conclusion ninety
Speaker 1: five percent of the time. It is a broken system.
Speaker 1: You have to then figure out what factors are causing
Speaker 1: this to happen, and they are numerous, right, They extend
Speaker 1: all the way up to the very top of your
Speaker 1: neural network, the other end where the input comes in.
Speaker 1: But you can't just change everything all at once. You've
Speaker 1: got to figure this out systematically, and that's what backpropagation
Speaker 1: is really all about. It detects which links one layer
Speaker 1: up from the output have the greatest impact on the outcome. Right,
Speaker 1: changing everything would be tedious, It would be impractical. You
Speaker 1: might even make things worse. Some of these neural networks
Speaker 1: are confoundingly complicated, so it's not really a feasible solution.
Speaker 1: So instead you look at the connections that are having
Speaker 1: the biggest impact on your outcome. So you want things
Speaker 1: where if you make a small change in either the
Speaker 1: bias or the weight, or maybe both, you'll see a
Speaker 1: larger end effect on the outcome. All the connections are
Speaker 1: arguably important, but some are more important than others. Backpropagation
Speaker 1: works backwards from the result toward the other end of
Speaker 1: the network to tweak those connections. It boosts ones that
Speaker 1: lead to the correct or desired response, and it reduces
Speaker 1: the values of those that lead to incorrect or undesired responses.
Speaker 1: If we were to think of this like the classic
Speaker 1: example and chaos theory, this could potentially involve us studying
Speaker 1: a hurricane as it hits land and tracing its history
Speaker 1: back as it moved through the ocean, and we would
Speaker 1: eventually arrive at the point where it was a tropical storm,
Speaker 1: and then we would go further back and see the
Speaker 1: factors that led to the creation of that storm, and
Speaker 1: maybe if we tracked it all the way back we
Speaker 1: would even find that one of a billion factors that
Speaker 1: made the storm was in fact, a butterfly was flapping
Speaker 1: its wings on the other side of the world, and
Speaker 1: that contributed to it. Maybe we find out that butterfly
Speaker 1: flap of its wings had an impact, but it was negligible,
Speaker 1: and that if the butterfly hadn't flapped its wings, the
Speaker 1: hurricane still would have happened. That would be an example of, well,
Speaker 1: we don't bother adjusting the weight of the impact of
Speaker 1: that butterfly flapping its wings because it doesn't matter for
Speaker 1: the end result. But what if we were to discover
Speaker 1: that that butterfly flap of its wings is the only
Speaker 1: reason the hurricane happened that, or at least was the
Speaker 1: primary reason that all the other factors pale in comparison, Well,
Speaker 1: then we'd want to make sure we boost the weight
Speaker 1: of that input, because clearly that butterfly is fundamental for hurricanes.
Speaker 1: I think hurricanes are really dangerous, and I would ask
Speaker 1: butterflies to kind of chill, all right. I mean, I
Speaker 1: don't want butterflies to go away, just you know, maybe
Speaker 1: stop flapping so much. Anyway, the formula for back propagation
Speaker 1: gets into some calculus that is well beyond my knowledge
Speaker 1: and skill. So rather than attempt to stumble my way
Speaker 1: through an explanation that I don't actually understand, I think
Speaker 1: it's best to leave the concept at the high level
Speaker 1: that I have described right now. So just know that
Speaker 1: it gets way more granular than what I've talked about.
Speaker 1: But essentially, you're looking at those factors that led to
Speaker 1: the ultimate decision and saying which ones of these had
Speaker 1: the greatest impact, and how can I tweak them so
Speaker 1: that I can shape the outcome to one I wanted.
Speaker 1: If we were thinking about that example I gave about
Speaker 1: whether or not you go to the movies, maybe in
Speaker 1: present day you starts thinking about past experiences where you
Speaker 1: made a decision to go out when you had a
Speaker 1: big day, then the following day, and how that impacted you,
Speaker 1: perhaps negatively. Maybe you're like, man, I should have gotten
Speaker 1: a promotion by now, and then you think, well, I
Speaker 1: do go to the movies an awful lot. You might say,
Speaker 1: I need to adjust some of the factors that affect
Speaker 1: my decision making process and perhaps prioritize my career. Or
Speaker 1: if you've decided that late stage capitalism is terrible, evil,
Speaker 1: and that you're going to try and live a hedonistic
Speaker 1: lifestyle of a wandering soul. Maybe you say, I'm going
Speaker 1: to go and see my movie with my friend, and yeah,
Speaker 1: that's just how it is, because that's the most important
Speaker 1: thing to me. You only go around this crazy world once.
Speaker 1: After all, I'm not telling you which way to go.
Speaker 1: I'm still finding my own way. But yeah, backpropagation would
Speaker 1: be how you would go back and say, all right, well,
Speaker 1: because I don't like the outcome that happened, and I
Speaker 1: need to change the way. These factors weigh in on
Speaker 1: the decision making process that goes through the whole system. Now,
Speaker 1: the advancements in the science of neural networks proved that
Speaker 1: the technology no longer operated under the constraints that concern
Speaker 1: Minsky and support in the late sixties, So once again
Speaker 1: funding found its way to neural network research and development projects.
Speaker 1: Now let's finally talk about forgetting and what makes it catastrophic.
Speaker 1: So you could, in theory, develop an artificial neural network
Speaker 1: and have a library of training data, and the only
Speaker 1: thing you ever do with this network is you feed
Speaker 1: that same set of training data to that same neural
Speaker 1: network over and over. In an effort to get performance
Speaker 1: as close to perfect as you possibly can. Just you know,
Speaker 1: it's kind of like if you have a car and
Speaker 1: you're constantly tweaking it so it will perform better, and
Speaker 1: maybe you change one thing and it boosts performance in
Speaker 1: one area, but it kind of negatively impacts performance in
Speaker 1: another area, so then you got to tweak something else.
Speaker 1: You could be doing that with an artificial neural network
Speaker 1: forever and just be using the same set of training data,
Speaker 1: and all you're trying to do is make a system
Speaker 1: that could handle that training data better than any other
Speaker 1: system in the world. And that would be interesting, but
Speaker 1: it would be useless from a practical standpoint. You could say, like, hey,
Speaker 1: you want to see my machine that can sort through
Speaker 1: only this collection of photographs and pick out the ones
Speaker 1: that have cats in them and the ones that don't
Speaker 1: pretty pretty darn effectively, but not perfectly. It's not really
Speaker 1: an interesting value proposition, right, So more likely you are
Speaker 1: eventually going to start feeding lots of different kinds of
Speaker 1: data to this neural network, And yeah, you train the
Speaker 1: network on certain data sets, but your goal is to
Speaker 1: feed new sets of data data the system has never
Speaker 1: encountered before. And rely on the system's ability to process
Speaker 1: this information correctly to get the result you want. And
Speaker 1: we might even be talking about stuff the human beings
Speaker 1: can't easily do. Right. But see, the training data is
Speaker 1: going to mean that the network will start to create
Speaker 1: and reinforce certain pathways, and those pathways will over time
Speaker 1: get stronger and stronger, just as we said at the
Speaker 1: beginning of this episode. But new data is going to
Speaker 1: necessitate new pathways. Sometimes, when the system begins to form
Speaker 1: these new pathways, it forgets the old pathways. So it's
Speaker 1: possible for a neural network to actually get worse at
Speaker 1: the task it had previously been trained to do with
Speaker 1: the actual training material. In fact, in a true catastrophe,
Speaker 1: the system might forget the objective and doesn't recognize what
Speaker 1: the desired outcome is meant to be, so the results
Speaker 1: can appear random and meaningless. It's as if the system
Speaker 1: has developed some form of amnesia. So this is prevalent,
Speaker 1: most prevalent anyway, in systems that rely on unguided learning.
Speaker 1: With guided learning, you have engineers who are carefully selecting
Speaker 1: the data that gets fed into a system. An unguided
Speaker 1: system would collect raw data from wherever and attempt to
Speaker 1: deliver desired results, and that those are the kinds of
Speaker 1: neural networks that are more prone to catastrophic forgetting. But
Speaker 1: as I said, machine learning systems tackle new data, maybe
Speaker 1: even new tasks, and then you get the risk of
Speaker 1: the system forgetting stuff. So I jokingly say, it's kind
Speaker 1: of like when I learned something new, it has to
Speaker 1: push out something old, like you know, my friend's phone
Speaker 1: number or something. Suddenly I can no longer remember it
Speaker 1: because I learned some new interesting fact, as if I
Speaker 1: have met my capacity for being able to know things.
Speaker 1: So learning anything new necessitates having to forget something I
Speaker 1: used to know, like gat Ye, because now gat Ye
Speaker 1: is just somebody that I used to know. But wait,
Speaker 1: there's more. Just as a system can experience catastrophic forgetting,
Speaker 1: it can also experience catastrophic remembering. This is when a
Speaker 1: system mistakenly believes it is doing one process a task
Speaker 1: it had previously been trained to do, rather than the
Speaker 1: one it's actually trying to do. So let's say we've
Speaker 1: got an artificial neural network, and originally we taught it
Speaker 1: to recognize the photos that have cats in them versus
Speaker 1: the ones that don't. But now we have retrained the
Speaker 1: same artificial neural network to try and recognize handwritten text,
Speaker 1: except when we feed handwritten text to the system, suddenly
Speaker 1: the system believes it's trying to determine where the cats are.
Speaker 1: This is something that can happen with machine learning systems too,
Speaker 1: and you still get bad results out of it. So
Speaker 1: this is a real problem. Now, these are not insurmountable problems.
Speaker 1: There are some solutions that are actually intuitive. For example,
Speaker 1: any game out their nose that it's best to save
Speaker 1: your game just before you head into a big boss battle,
Speaker 1: just in case things don't go the way you planned. Well.
Speaker 1: With artificial neural networks, it's maybe not a bad idea
Speaker 1: to make a copy of a network before you retrain
Speaker 1: it to do something new. Then you still have the
Speaker 1: backup if things do go pair shape. There are other
Speaker 1: approaches to decreasing the risk of catastrophic forgetting or catastrophic remembering.
Speaker 1: An article in applied Mathematics titled Overcoming catastrophic forgetting in
Speaker 1: neural networks describes a system in which the researchers purposefully
Speaker 1: slowed down the network's ability to change the weights involved
Speaker 1: in important tasks. From previous training cycles. So this makes
Speaker 1: teaching the system to do new tasks a little more
Speaker 1: challenging because it's protecting these weights. It's preventing the system's
Speaker 1: ability to be completely plastic, which means the system has
Speaker 1: to work around these constraints and still learn how to
Speaker 1: do the new task, but in the process it means
Speaker 1: it doesn't forget how to do the previous tasks. This
Speaker 1: article is interesting because the tasks the researchers actually used
Speaker 1: the purposes of training, Like, what were they teaching the
Speaker 1: artificial neural network to do well. They were teaching it
Speaker 1: how to play Atari twenty six hundred games. So they
Speaker 1: would start with one game and train the system on
Speaker 1: how to play the game. Then they would give the
Speaker 1: system a new game with different game mechanics, and the
Speaker 1: system would have to learn how to play this new game,
Speaker 1: but they wanted to see if it could still remember
Speaker 1: how to play the original game. That was kind of
Speaker 1: the system they were working on. They were tweaking things
Speaker 1: so that the machine learning artificial neural network as a
Speaker 1: whole could learn how to play multiple Atari twenty six
Speaker 1: hundred games without forgetting how to do the previous ones.
Speaker 1: This is a non trivial task. I mean, it takes
Speaker 1: a lot of work to see exactly how to preserve
Speaker 1: things so that you're not slowing down the learning process
Speaker 1: too much, but you're also not inviting the possibility of
Speaker 1: catastrophic forgetting. Now that's just one example of how researchers
Speaker 1: are looking to mitigate the problem of catastrophic forgetting in
Speaker 1: catastrophic remembering, there are other methods as well, and maybe
Speaker 1: I'll do another episode where I'll go into more detail
Speaker 1: on some of those. They do get pretty complicated, and
Speaker 1: in fact, eventually Rerilli and I even eventually pretty early on,
Speaker 1: I hit my limit for as far as I can
Speaker 1: understand the actual mechanics of the system. So rather than
Speaker 1: try and punch above my weight, I think it's best
Speaker 1: to kind of be a little more general, but just
Speaker 1: to have that understanding to kind of get a better
Speaker 1: appreciation of some of the challenges relating to artificial intelligence
Speaker 1: in general and machine learning in particular. And again, like
Speaker 1: this machine learning issue, it's a bigger problem with more
Speaker 1: sophisticated systems that are meant to do unsupervised and unguided learning, right,
Speaker 1: those are the ones that are going to be more
Speaker 1: prone to these issues. If we're talking about supervised and
Speaker 1: guided learning, where engineers are being very careful with the
Speaker 1: data being fed to a system, it's less likely to happen.
Speaker 1: But the whole promise, or at least the you know,
Speaker 1: not the promise of the technology itself, but the promise
Speaker 1: of the people who are funding it, is that this
Speaker 1: technology is going to reach a point where it's able
Speaker 1: to learn on its own and be able to do
Speaker 1: things better than people can do, to free us up
Speaker 1: to doing, you know, stuff we want to do instead
Speaker 1: of stuff we have to do. That's like the science
Speaker 1: fiction dream version of AI. As we all know, getting
Speaker 1: there is much more painful. It's not like a simple
Speaker 1: process of Hey, we've made everything easy to do now,
Speaker 1: and you don't have to worry called day. You can
Speaker 1: enjoy your life and pursue your dreams and develop your
Speaker 1: hobbies and your interests, and you can have fulfillment and
Speaker 1: somehow money isn't important anymore. Like that seems to be
Speaker 1: the Star Trek version of the future that people want
Speaker 1: it to go in. But as we have seen, the
Speaker 1: process of getting there is way more painful. As you know,
Speaker 1: people face a reality of potentially being out of work
Speaker 1: because of AI, or maybe being paid way less to
Speaker 1: do work because the AI is doing most of it.
Speaker 1: These are not That's not Star Trek future. That's getting
Speaker 1: like into Blade Runner future, So we don't want that one.
Speaker 1: By the way, the tears in the Rain speech is fantastic,
Speaker 1: but you do not want to live in the Blade
Speaker 1: Runner world. Trust me, you might not want to live
Speaker 1: in the Star Trek world either, because those outfits don't
Speaker 1: look that comfortable. Anyway. That's my little discussion about AI,
Speaker 1: machine learning and castrophic forgetting in cast trophic. Remembering again,
Speaker 1: this is just one of the challenges associated with AI
Speaker 1: and machine learning. I don't mean to suggest it's the
Speaker 1: one and only, or even that it's the most important one,
Speaker 1: but it is one that I had not really heard
Speaker 1: of until I listened to that Skeptics Guide to the
Speaker 1: Universe episode over the weekend, and it was really interesting
Speaker 1: to dive into the material and read up about it
Speaker 1: and to get a better understanding of what it means
Speaker 1: and how it works. And as I said, we'll probably
Speaker 1: revisit this topic in the future, especially since AI is
Speaker 1: such a big deal these days. Okay, but that's it
Speaker 1: for this episode. Of tech Stuff. I hope you are
Speaker 1: all well, and I will talk to you again really soon.
Speaker 1: Tech Stuff is an iHeartRadio production. For more podcasts from iHeartRadio,
Speaker 1: visit the iHeartRadio app, Apple Podcasts, or wherever you listen
Speaker 1: to your favorite shows.