Speaker 1: Welcome to Tech Stuff, a production from I Heart Radio.
Speaker 1: Hey there, and welcome to tech Stuff. I'm your host,
Speaker 1: Jonathan Strickland. I'm an executive producer with iHeart Radio. And
Speaker 1: how the tech are you? Alright? Well, I'm still on vacation.
Speaker 1: I'll be coming back soon, so tomorrow you should expect
Speaker 1: a brand new episode unless something goes wrong while I'm
Speaker 1: trying to get back. Hopefully nothing like that happens, And
Speaker 1: so we thought we'd have a little rerun. This episode
Speaker 1: originally published in April one, so just last year. It
Speaker 1: is titled machine Learning one oh one. And I wanted
Speaker 1: to do this one because, as always, we hear a
Speaker 1: lot about artificial intelligence and machine learning in the news
Speaker 1: and in media, and often those topics get a little confusing.
Speaker 1: They can come across more broad than some people intend,
Speaker 1: or or they can be somewhat misguided in their interpretations.
Speaker 1: So I thought it would be useful to have a
Speaker 1: little refresher course on machine learning and artificial intelligence to
Speaker 1: hope you enjoy, uh and I will be back at
Speaker 1: the end. Back in nineteen eighties, six comedy science fiction
Speaker 1: film that I saw in the theater about a robot,
Speaker 1: the game sentience and becomes a total goofball what it will.
Speaker 1: It hit theaters in eighties six and it was called
Speaker 1: Short Circuit. The movie starred Steve Gutenberg, Ali Sheety, and
Speaker 1: lamentably a white actor named Fisher Stevens playing a non
Speaker 1: white character, someone who is Indian. I should add that's
Speaker 1: not Steven's fault. I mean, he auditioned to be in
Speaker 1: a movie and he got a gig. He didn't cast
Speaker 1: himself in the film, and he has since talked about
Speaker 1: his experiences, realizing the problems with a white man playing
Speaker 1: a non white character, but setting aside all the problematic
Speaker 1: white washing, the movie showed this robot, who in the
Speaker 1: course of the film names itself Johnny five learning. It
Speaker 1: learns about the world around it, it learns about people,
Speaker 1: It learns about human concepts like humor and emotion, and
Speaker 1: the general idea was pretty cute. Now, the nifty thing
Speaker 1: is machines actually can learn. In fact, machine learning is
Speaker 1: a really important field of study these days, complete with
Speaker 1: its own challenges and risks. I've talked about machine learning
Speaker 1: a few times in the past, but I figured we
Speaker 1: could do a deeper dive to understand what machine learning
Speaker 1: is what it isn't how people are leveraging machine learning
Speaker 1: and why I said that it does come with risks,
Speaker 1: So let's learn about machines learning. It will be impossible
Speaker 1: to talk about machine learning without also talking about artificial
Speaker 1: intelligence or AI. And this term artificial intelligence is a
Speaker 1: real doozy. It trips people up, even people who have
Speaker 1: dedicated their lives to researching and developing artificial intelligence. You
Speaker 1: can get two experts in AI talking about AI and
Speaker 1: find out that because they have slightly different takes on
Speaker 1: what AI is, there are some communication issues. It's not
Speaker 1: as simple as red versus blue would have you think
Speaker 1: what does the A stand for? So when you really
Speaker 1: boil it down, it comes out as as no big
Speaker 1: surprise that there's a lot of ambiguity here. After all,
Speaker 1: how would you define intelligence just intelligence, not artificial intelligence,
Speaker 1: just intelligence? Well? Would it be the ability to learn,
Speaker 1: that is, to acquire skills and knowledge? Or is it
Speaker 1: the application of learning? Is it problems solving? Is it
Speaker 1: being able to think ahead and make plans in order
Speaker 1: to achieve a specific goal? Is it the ability to
Speaker 1: examine a problem and deconstructed in order to figure out
Speaker 1: the best solution. A more specific version of problem solving.
Speaker 1: Is it the ability to recognize, understand, and navigate emotional scenarios? Now,
Speaker 1: arguably it's all of these things and more. We all
Speaker 1: have kind of an intuitive grasp on what intelligence is,
Speaker 1: but defining it in a simple way tends to feel
Speaker 1: reductive and it leaves out a lot of important details.
Speaker 1: So if defining just general intelligence is hard, it stands
Speaker 1: for a reason that defining artificial intelligence is also a
Speaker 1: tough job. Heck, even coming up with a number of
Speaker 1: different types of a I is tricky. And if you
Speaker 1: don't believe me, just google the phrase different types of
Speaker 1: artificial intelligence. Never mind, you don't. You don't really actually
Speaker 1: have to do that. I already did it, though, Feel
Speaker 1: free to do it yourself and check my work if
Speaker 1: you like. When I googled that phrase different types of AI,
Speaker 1: some of the top results included a blog post on
Speaker 1: BMC software titled four types of artificial Intelligence. But then
Speaker 1: there was also an article on code bots that was
Speaker 1: titled what are the three types of AI? And then
Speaker 1: there was an article from Forbes titled seven types of
Speaker 1: artificial intelligence. See, we can't even agree on how many
Speaker 1: versions of a EI there are because defining a I
Speaker 1: is really hard. It largely depends upon how you view
Speaker 1: AI and then how you break it down into different
Speaker 1: realms of intelligence. Now we could go super high level,
Speaker 1: because a classic way to look at AI is strong
Speaker 1: versus weak Artificial intelligence stro on AI UH sometimes called
Speaker 1: artificial general intelligence, would be a machine that processes information
Speaker 1: and at least appears to have some form of consciousness
Speaker 1: and self awareness and the ability to both have experiences
Speaker 1: and to be aware that it is having experiences. It
Speaker 1: might even feel emotion, though maybe not emotions that we
Speaker 1: could easily identify or sympathize with. So this would be
Speaker 1: the kind of machine that would think in a way
Speaker 1: similar to humans. It would be able to sense its
Speaker 1: environment and not just react, but really process what is
Speaker 1: going on and build and understanding. It's the type of
Speaker 1: AI that we see a lot in science fiction. A's
Speaker 1: the type of AI of Johnny five from Short Circuit
Speaker 1: or how from two thousand one, or the droids in
Speaker 1: Star Wars. It's also a type of artificial intelligence that
Speaker 1: we have yet to actually achieve in the real world.
Speaker 1: So then what is week AI. Well, you could say
Speaker 1: it's everything else, or you could say it's the building
Speaker 1: blocks that maybe collectively will lead to strong AI week.
Speaker 1: AI involves processes that allow machines to complete tasks, So,
Speaker 1: for example, image recognition software could fall into this category.
Speaker 1: Once upon a time, in order to search photos effectively,
Speaker 1: you needed to actually add meta data like tags to
Speaker 1: those photos. So, for example, I might tag pictures of
Speaker 1: my dog with the meta tag dog, and then if
Speaker 1: I wanted to see photos of my pooch, then I
Speaker 1: would pull up my photo app and search the term dog,
Speaker 1: and all the photos that I had tagged with the
Speaker 1: word dog would show up. But if I had failed
Speaker 1: to tag some pictures of my dog, those pictures wouldn't
Speaker 1: pop up in search because the computer program wasn't actually
Speaker 1: looking for dogs in my photos. It was just looking
Speaker 1: for photos that had that particular meta tag attached to it.
Speaker 1: But now we've reached a point where at least some
Speaker 1: photo apps are using image recognition to analyze photos, and
Speaker 1: these will return results that the algorithm has identified as
Speaker 1: having a reasonable chance of meeting your search query. So
Speaker 1: if I used an app like that and I put
Speaker 1: in dog as my search term, it could pull up
Speaker 1: photos that had no meta tags attached to them at all.
Speaker 1: Because the search is relying on image recognition. Now, this
Speaker 1: also means that if the image recognition algorithm isn't very good,
Speaker 1: I could get some images that don't have a dog
Speaker 1: in them at all, or it might miss other images
Speaker 1: that have my dog in them. But my point is
Speaker 1: that the ability to identify whether or not a dog
Speaker 1: is in a particular photo represents a kind of weak
Speaker 1: artificial intelligence. You wouldn't say that the photo search tool
Speaker 1: possesses humanlike intelligence, because really it only does one thing.
Speaker 1: It's analyzing photos and looks for matches to specific search queries,
Speaker 1: but it can't do anything outside of that use case. However,
Speaker 1: that's just one little example. There are all sorts of
Speaker 1: other ones, like voice recognition, environmental sensing, course plotting, that
Speaker 1: kind of thing, and in some circles, as we get
Speaker 1: better at making machines and systems that can do these things,
Speaker 1: those elements seem to kind of drift away from the
Speaker 1: ongoing conversation about artificial intelligence. A guy named Larry Tessler,
Speaker 1: who was a computer scientist who worked at lots of
Speaker 1: really important places like Xerox, Park and Amazon and Apple,
Speaker 1: he once observed, quote, intelligence is whatever machines haven't done yet.
Speaker 1: End quote. So his point was that the reason that
Speaker 1: AI is really hard to talk about is that the
Speaker 1: goal post for why actually is artificial intelligence is constantly moving.
Speaker 1: Now this pretty much mirrors how we think about things
Speaker 1: like consciousness. Lots of people study consciousness, and the general
Speaker 1: sense I get is that it's a lot easier for
Speaker 1: people to talk about what isn't consciousness rather than what
Speaker 1: consciousness actually is. And it seems like artificial intelligence is
Speaker 1: in a similar place, which really isn't that big of
Speaker 1: a surprise as we closely associate intelligence with consciousness. Now
Speaker 1: this leads us to why there are so many different
Speaker 1: takes on how many types of AI there are. It
Speaker 1: all depends on how you classify different disciplines in artificial intelligence,
Speaker 1: and over time, a lot of disciplines that were previously
Speaker 1: distinct from AI have sort of converged into becoming part
Speaker 1: of the AI discussion. Machine learning, as it turns out,
Speaker 1: was part of the AI discussion, branch off from it,
Speaker 1: and then rejoined the AI discussion years later. So I
Speaker 1: am not going to go down all the different approaches
Speaker 1: to classification because I don't know that they would be
Speaker 1: that valuable to us. They would really just illustrate that
Speaker 1: there are a lot of different ways to look at
Speaker 1: the subject. So if you ever find yourself in a
Speaker 1: conversation about AI, it might be a good idea to
Speaker 1: set a few ground rules as to what everyone means
Speaker 1: when they use the term artificial intelligence. That can help
Speaker 1: with expectations and understanding. Or you could just run for
Speaker 1: the nearest exit, which is what people tend to do
Speaker 1: whenever I start talking about it anyway. What about machine learning, Well,
Speaker 1: from one perspective, you could say machine learning is a
Speaker 1: sub discipline of artificial intelligence, although like I said, it
Speaker 1: hasn't always been viewed as such. I think most people
Speaker 1: would say that the ability to learn that is to
Speaker 1: take information and experience and then have some form of
Speaker 1: understanding of those things so that you can apply that
Speaker 1: to future tasks, potentially getting better over time. I would
Speaker 1: say most people would call that part of intelligence. But
Speaker 1: you could also be a bit more wishy washy and
Speaker 1: say it's related to, you know, artificial intelligence, as opposed
Speaker 1: to being part of AI, since the definition of AI
Speaker 1: is let's say, fluid. Either way of classifying machine learning works.
Speaker 1: As far as I'm concerned, machine learning boils down to
Speaker 1: the idea of creating a system that can learn as
Speaker 1: it performs a task. It can learn what works and
Speaker 1: more importantly, what does not work. You may have heard
Speaker 1: that we learn a lot more from our mistakes than
Speaker 1: we do from our successes, which there's pretty much true
Speaker 1: in my experience. When something goes wrong, it's usually, but
Speaker 1: not always, possible to trace the event or events that
Speaker 1: led to the failure. You can identify decisions that we're
Speaker 1: probably the wrong ones or that led to a bad outcome,
Speaker 1: But if you have a success, it's hard to figure
Speaker 1: out which decisions were key to that successful outcome. Did
Speaker 1: your decision at step two set you on the right path,
Speaker 1: or was your choice at step three so good that
Speaker 1: it helped correct a mistake that you made it step two.
Speaker 1: But a good approach to machine learning involves a system
Speaker 1: that can adjust things on its own to reduce mistakes
Speaker 1: and increase the success rate. And another way of putting
Speaker 1: it is that instead of programming a system to arrive
Speaker 1: at a specific outcome, you are training the system to
Speaker 1: learn how to do it by itself. And that sounds
Speaker 1: a bit magical when you put it that way, doesn't it?
Speaker 1: It sounds like someone just took a computer and showed
Speaker 1: it pictures of cat and then expected the computer to
Speaker 1: know what a cat was. And this actually does mirror
Speaker 1: an actual project that really did do that, But I'm
Speaker 1: leaving out some big important information in the middle. Now,
Speaker 1: one big step is that computers and machines can't just
Speaker 1: magically learn by default. People first had to come up
Speaker 1: with a methodology that allows machines to go through the
Speaker 1: process of completing a task, then making adjustments to the
Speaker 1: process of doing that task, which would then improve future results.
Speaker 1: We have to lay the groundwork in architecture and theory
Speaker 1: and algorithms. We have to build the logical pathways that
Speaker 1: computers can follow in order for them to learn. A
Speaker 1: lot of machine learning revolves around patterns and pattern recognition.
Speaker 1: So what do I mean by patterns? Well, I mean
Speaker 1: some form of regularity and predictability. Machine learning models analyze
Speaker 1: patterns and attempt to draw conclusions based on those patterns.
Speaker 1: This in itself is tricky stuff. So why is that? Well,
Speaker 1: it's because sometimes we might think there's a pattern when
Speaker 1: in reality there is not. We humans are pretty good
Speaker 1: at recognizing patterns, which makes sense. It's a survival mechanism.
Speaker 1: If you were to look at tall grass and you
Speaker 1: see patterns that suggest the presence of a predator like
Speaker 1: a tiger, well you would know that danger is nearby,
Speaker 1: and you would have the opportunity to do something about
Speaker 1: that to help your chances of survival. If, however, you
Speaker 1: remained blissfully unaware of the danger, you'd be far more
Speaker 1: likely to fall prey to that hungry tiger. So recognizing
Speaker 1: patterns is one of the abilities that gave humans a
Speaker 1: chance to live another day, and, from an evolutionary standpoint,
Speaker 1: a chance to make more humans. But sometimes we wins
Speaker 1: will perceive a pattern where none actually exists. A simple
Speaker 1: example of this is the fun exercise of laying on
Speaker 1: your back outside, looking up at the clouds and saying,
Speaker 1: what does that cloud remind you? Of? The shapes of clouds,
Speaker 1: which have no significance and are the product of environmental factors,
Speaker 1: can seem to suggest patterns to us. We might see
Speaker 1: a dog, or a car or a face, but we
Speaker 1: know that what we're really seeing with just the appearance
Speaker 1: of a pattern, it's it's not evidence of a pattern
Speaker 1: actually being there. It's noise, not signal. But it could
Speaker 1: be misinterpreted as signal. Well, it turns out that in
Speaker 1: machine learning applications this is also an issue. I'll talk
Speaker 1: about it more towards the end of this episode. Computers
Speaker 1: can sometimes misinterpret data and determine something represents a pattern
Speaker 1: when it really doesn't. When that happens, a system relying
Speaker 1: on machine learning can whose false positives, and the consequences
Speaker 1: can sometimes be funny, like hey, this image recognition software
Speaker 1: thinks this coffee mug is actually a kidney cat. Or
Speaker 1: they can be really serious and potentially harmful. Hey, this
Speaker 1: facial recognition software has misidentified a person, marking them as, say,
Speaker 1: a person of interest in a criminal case. And it's
Speaker 1: all because this facial recognition software isn't very good at
Speaker 1: differentiating people of color. That's a real problem that really happens. Now,
Speaker 1: when we come back, I'll give a little overview of
Speaker 1: the evolution of machine learning. But before we do that,
Speaker 1: let's take a quick break to talk about the history
Speaker 1: of machine learning. We first have to look back much
Speaker 1: much earlier, long before the era of computers, and talk
Speaker 1: about how thinkers like Thomas Bayes thought about the act
Speaker 1: of problem solving. Bays was born way back in two,
Speaker 1: so quite a bit before we were thinking about machine learning,
Speaker 1: but he was interested in problem solving for problems involving probabilities,
Speaker 1: and specifically the relationship between different probabilities. I think it's
Speaker 1: easier to talk about if I give you an example.
Speaker 1: So let's make a silly one, all right, So let's
Speaker 1: say we got ourselves a plucky podcaster. Hey there, everybody,
Speaker 1: It's Jonathan Strickland, and it's Tuesday as I record this,
Speaker 1: And because of who I am, you know who this
Speaker 1: podcaster is. And because it's Tuesday, there is a chance
Speaker 1: I am wearing a they might be Giants T shirt.
Speaker 1: And we also know that if this podcaster is wearing
Speaker 1: a they might be Giants T shirt on a Tuesday,
Speaker 1: there's a sixty chance that I'm going to end up
Speaker 1: wearing pajamas on Wednesday. But we also know that if
Speaker 1: I did not where they might be Giant's shirt on Tuesday,
Speaker 1: and remember there's a six chance I didn't, then we
Speaker 1: know there's an eighty percent chance I'm going to be
Speaker 1: wearing pajamas on Wednesday. Will Bays worked out a way
Speaker 1: that described the sort of probability relationship between different discrete
Speaker 1: events and using his reasoning, you can work forward or
Speaker 1: backward based on probabilities. Theys would describe wearing a they
Speaker 1: Might be Giant shirt on Tuesday as one event and
Speaker 1: wearing pajamas on Wednesday as a separate event, and then
Speaker 1: describe the two not only determining how likely it is
Speaker 1: I'll wear pajamas on Wednesday, but if we start with
Speaker 1: the later event, in other words, that we start with
Speaker 1: the fact that it's Wednesday and I'm wearing pajamas, we
Speaker 1: could work out how likely it was that yesterday, on Tuesday,
Speaker 1: I was wearing they Might be Giants shirt. That was
Speaker 1: his his contribution, that you can work this in either
Speaker 1: direction if you know these different variables. Now, Bay has
Speaker 1: never published his thoughts, but rather send an essay explaining
Speaker 1: it to a friend of his, who then made sure
Speaker 1: that the work was published. After Bays had passed away,
Speaker 1: and a few decades later, Pierre Simon Laplace would take
Speaker 1: this work that Bays had done and flesh it out
Speaker 1: into an actual formal theorem. It's an important example of
Speaker 1: conditional probability, and a lot of what machine learning is
Speaker 1: really boiled down to is dealing with different probabilities, not certainties, which,
Speaker 1: when you get down to it, is what most of
Speaker 1: us are doing most of the time. Right. We make
Speaker 1: decisions based on at least perceived probabilities. Sometimes these decisions
Speaker 1: might feel like they're a coin flip situation, that any
Speaker 1: choice is equally likely to precipitate a good outcome or
Speaker 1: a bad outcome. Other Times we might make a choice
Speaker 1: because we feel the probabilities are stacked favorably one way
Speaker 1: over another. Sometimes we will make a choice to back
Speaker 1: the least probable outcome, because well, humans are not always superrational.
Speaker 1: In hex sometimes the long shot does pay off, so
Speaker 1: that keeps Vegas in business. Bayes' theorem is just one
Speaker 1: example of ways that mathematicians and philosophers figured out ways
Speaker 1: to mathematically express problem solving and decision making, And a
Speaker 1: lot of this was figuring out if there were a
Speaker 1: way to boil down things that most of us approached
Speaker 1: through intuition and experience. So it's kind of neat, and
Speaker 1: also the more you look into it, the more likely
Speaker 1: you might find it's little spooky, because it's weird to
Speaker 1: consider that our approaches to making choices and solving problems
Speaker 1: can be reduced down to mathematical expressions. But let's leave
Speaker 1: the potential existential crises alone for now, shall we. So
Speaker 1: moving on, we have another smarty pants we need to
Speaker 1: talk about Andre Markov, mathematician. In the early twentie century.
Speaker 1: He began studying the nature of certain random processes that
Speaker 1: follow a particular type of rule, which we now call
Speaker 1: the Markov property. That rule says that for this particular process,
Speaker 1: the next stage of the process only depends upon the
Speaker 1: current stage, but not any stages that came before then.
Speaker 1: So let's take my ridiculous T shirt example and let's
Speaker 1: build it out a little bit further. Let's say that
Speaker 1: I've got three T shirts to my name. One of
Speaker 1: them is that they might be Giant's shirt. One is
Speaker 1: a plain blue T shirt, and the third is a
Speaker 1: shirt that has the tech Stuff logo on it. And
Speaker 1: it's based off of long observation that you've determined these
Speaker 1: following facts. If I am wearing that they might be
Speaker 1: Giant's shirt today, I definitely will not wear it tomorrow.
Speaker 1: But there's a fifty fifty shot I'll wear either the
Speaker 1: blue shirt or the tech Stuff shirt. Now, if I'm
Speaker 1: wearing the blue shirt today, there's a ten chance I'm
Speaker 1: going to wear the same blue shirt tomorrow. Don't worry,
Speaker 1: I'll wash it first. There's a sixty chance that I'll
Speaker 1: wear the tech Stuff shirt, and there's a thirty percent
Speaker 1: chance I'll wear the they Might Be Giant shirt. But
Speaker 1: if I'm wearing the tech stuff shirt today, there's a
Speaker 1: seventy chance I'll wear it again tomorrow because I like
Speaker 1: to promote myself. But there's a thirty percent chance I'll
Speaker 1: wear the they Might be Giant shirt, and there is
Speaker 1: no chance that I'm going to wear the blue one
Speaker 1: in this case. So those are our various scenarios. Right
Speaker 1: which shirt I will wear tomorrow depends only upon which
Speaker 1: shirt I am wearing today. What I wore yesterday has
Speaker 1: no bearing on the outcome for tomorrow, So today is
Speaker 1: all that matters. And depending on which shirt I wear,
Speaker 1: you can make some probability predictions for tomorrow. So we
Speaker 1: can actually use this approach to figure out the probability
Speaker 1: that I might wear the tech Stuff shirts, say ten
Speaker 1: days in a row, since there's a better than even
Speaker 1: chance that if I'm wearing tech Stuff today, I'll end
Speaker 1: up wearing it again tomorrow, and if I wear it tomorrow,
Speaker 1: then there's a better than fift chance that I'm going
Speaker 1: to wear it the following day. But at some point
Speaker 1: you're going to see that the odds are starting to
Speaker 1: be against you, for you know, increasingly long strings of
Speaker 1: wearing the tech stuff shirt. Anyway, Markov chains would become
Speaker 1: one of the types of processes that machine learning models
Speaker 1: would incorporate, with some models looking at the current state
Speaker 1: of a given process and then make predictions on what
Speaker 1: the next state will be with no need to look
Speaker 1: back at the previous decisions. The Markov chain is memory less.
Speaker 1: Now that's just a couple of the mathematicians whose work
Speaker 1: underlies elements of machine learning. There's also structure we need
Speaker 1: to talk about. In a man named Donald Hebb wrote
Speaker 1: a book titled The Organization of Behavior, and in that book,
Speaker 1: Hebb gave hypothesis on how neurons, that is, how how
Speaker 1: brain cells interact with one another. His ideas included the
Speaker 1: notion that if two neurons interact with one another regularly,
Speaker 1: that is, if one fires, that the second one is
Speaker 1: also likely to fire. They end up forming a tighter
Speaker 1: communicative relationship with each other. Not long after his expression
Speaker 1: of this hypothesis. Computer scientists began to think of a
Speaker 1: potential way to do this artificially, with machines creating the
Speaker 1: equivalent of artificial neurons. The relative strength in relationship between
Speaker 1: artificial neurons is something we describe by Wait, that's going
Speaker 1: to be an important part of machine learning. WIT. By
Speaker 1: the way, is W E I G H T, as
Speaker 1: in this relationship is weighted more heavily than that relationship.
Speaker 1: In the early nineteen fifties, an IBM researcher named Arthur
Speaker 1: Samuel created a program designed to win at checkers. The
Speaker 1: program would do a quick analysis of where pieces were
Speaker 1: on a checkerboard and whose move it was, and then
Speaker 1: calculate the chances of each side winning the game based
Speaker 1: on those positions. And it did this with a mini
Speaker 1: max approach. Alright, so checkers is a two player turn
Speaker 1: based game. Player one makes a move, then player two
Speaker 1: can make a move. There are a finite number of
Speaker 1: moves that can be made, a finite number of possibilities,
Speaker 1: though admittedly it's a pretty good number of possibilities. But
Speaker 1: let's say a game has been going on for a
Speaker 1: few moves, and you've got your two sides you've got
Speaker 1: the red checkers over on player one side and the
Speaker 1: black checkers for a player to Let's say it's player
Speaker 1: one's move. For the purposes of this example, will say
Speaker 1: that player one really just has one piece that they
Speaker 1: can actually move on this turn, and it can move
Speaker 1: into one of two open spaces. So player one has
Speaker 1: to make a choice. After that choice, it's going to
Speaker 1: be player two's turn, so we can create a decision
Speaker 1: treat illustrating the possible choices and the possible outcomes of
Speaker 1: those choices. These choices are the children of the starting
Speaker 1: position for player one, so player one's starting position has
Speaker 1: two children. Player too will have their own choices to
Speaker 1: make after that decision has been made, but those choices
Speaker 1: are going to depend upon whatever move player one ultimately takes.
Speaker 1: So we can extend out our decision treat showing the
Speaker 1: branching possible moves that player Too might make, And these
Speaker 1: are the children of the two possible outcomes of our choice.
Speaker 1: After player two's turn, it's player ones turn again, which
Speaker 1: means we need to branch those decisions out even further.
Speaker 1: And this is all before player one has even made
Speaker 1: that first choice. We're just evaluating possibilities. At some point,
Speaker 1: either when we have plotted far enough out that we
Speaker 1: know all possible outcomes of the game, or we're just
Speaker 1: reaching a point where it would be unmanageable for us
Speaker 1: to go any further, we need to actually analyze what
Speaker 1: our options are. The endpoints represent either a win, a loss,
Speaker 1: or a draw for player one, or, if we haven't
Speaker 1: extended out the tree all the way to the end
Speaker 1: of the game, at least a change in advantage, whether
Speaker 1: it would be in player one's advantage to make that
Speaker 1: move or disadvantage. We could actually assign numerical values to
Speaker 1: each end point, with positive values representing an advantage for
Speaker 1: player one and a negative value representing an advantage for
Speaker 1: a player too, and once we do that, we can
Speaker 1: see which pathways tend to lead to better outcomes for
Speaker 1: player one. We work backward through the decision tree, so
Speaker 1: on all the decisions that end in an advantage for
Speaker 1: player one, we can say this is the choice that
Speaker 1: player one would take. But then we know that a
Speaker 1: player to player two is always going to choose whichever
Speaker 1: choice has the greatest advantage for that player, so we
Speaker 1: have to actually take that into account as we're working backward,
Speaker 1: and this is how we can finally get to the
Speaker 1: point where we decide which move we're going to make.
Speaker 1: Because these decisions as you go backward up the tree,
Speaker 1: they ultimately inform you which of those two choices is
Speaker 1: going to give you the best result. Those values, well,
Speaker 1: those are weights. So for player one, the goal is
Speaker 1: to pick the path that has the highest positive value.
Speaker 1: For player too, it's to pick the path that has
Speaker 1: the lowest possible value or the highest negative value if
Speaker 1: you prefer so. In other words, player one might be
Speaker 1: thinking something like, if I move to Spot A, my
Speaker 1: chance of winning this game, But if I moved to
Speaker 1: Spot B, it's only so. Of course, those percentages will
Speaker 1: also depend on what player two is going to do
Speaker 1: in response. Some moves that player two might do could
Speaker 1: end up guaranteeing a win for player one. This is
Speaker 1: the mini max approach, and there's an algorithm that guides it.
Speaker 1: It depends upon the current position within a game and
Speaker 1: how many moves or how much depth it has to
Speaker 1: take into account, and for which player is it actually
Speaker 1: helping out. What happens is if player one does this
Speaker 1: evaluation and finds that both options are negative, well, then
Speaker 1: this is something that happens in games, right, Sometimes you
Speaker 1: find out there is no good move, like any move
Speaker 1: you make is going to be a losing move. Well,
Speaker 1: the only option at that point is to choose the
Speaker 1: least bad had one, so it would be whatever the
Speaker 1: smallest negative value choice was. Our Next big development that
Speaker 1: I need to mention is Frank Rosenblatt's artificial neural network
Speaker 1: called Perceptron. Its purpose was to recognize shapes and patterns,
Speaker 1: and it was originally going to be its own machine
Speaker 1: like actual hardware, but the first incarnation of Perceptron would
Speaker 1: actually be in the form of software rather than hardware.
Speaker 1: There was a purpose built Perceptron later, but the original
Speaker 1: one was software. Despite some early excitement, the Perceptron proved
Speaker 1: to be somewhat limited in its capabilities, and interest in
Speaker 1: artificial neural networks died down for a while as a result.
Speaker 1: In a way, you could kind of compare this to
Speaker 1: some other technologies that got a big hype cycle and
Speaker 1: then later deflated. Virtual reality is the one I always
Speaker 1: go with. Back in the nineteen nineties, the world was
Speaker 1: really hyped for virtual reality. People had incredibly unrealistic x
Speaker 1: spectations for what VR actually meant and what it could do,
Speaker 1: and when it turned out the VR wasn't nearly as
Speaker 1: sophisticated as people were imagining, a lot of enthusiasm dropped
Speaker 1: out for the entire field, and with that dropped funding
Speaker 1: and support, and as a result, development and VR hit
Speaker 1: a real wall, with only a fraction of the people
Speaker 1: who had been working in the field sticking around, and
Speaker 1: they had to scramble just to find funding to keep
Speaker 1: their projects going. So VR was effectively put on the
Speaker 1: shelf and wouldn't make much progress for nearly twenty years. Well.
Speaker 1: Artificial neural networks had a very similar issue, but other
Speaker 1: computer scientists eventually found ways to design artificial neural networks.
Speaker 1: They could do some pretty amazing things if they had
Speaker 1: access to enough data. When we come back, i'll talk
Speaker 1: a little bit more about that and what it all means,
Speaker 1: but first let's take another quick break. So we left
Speaker 1: off with the AI field going into hibernation for a
Speaker 1: little bit. Theory and mathematics were bumping up against the
Speaker 1: limitations of technology, which wasn't quite at the level to
Speaker 1: put all that theory to the test. Plus there needed
Speaker 1: to be some tweaks to the approaches, but those came
Speaker 1: with time and more mathematicians found new ways to create
Speaker 1: artificial neural networks capable of stuff like pattern recognition and learning.
Speaker 1: So let's imagine another decision tree. We've got our starting position.
Speaker 1: This is probably where we put some input. We would
Speaker 1: feed data into a system, and let's say from that
Speaker 1: starting position, we have a process that's going to transform
Speaker 1: that input into one of two possible ways. So we've
Speaker 1: got two potential outputs for that first step. Like our
Speaker 1: mini max example, we can go down several layers of
Speaker 1: possible choices, and we can wait the relationships between these
Speaker 1: different choices. So if the incoming value is higher than
Speaker 1: a certain amount, maybe the node sends it down one pathway,
Speaker 1: But if the value is lower than that arbitrary amount,
Speaker 1: the node will send it down a different pathway. This
Speaker 1: is drastically oversimplifying, but I hope you kind of get
Speaker 1: the idea. It's like a big sorting system, and the
Speaker 1: goal is that at the very end whatever comes out
Speaker 1: as output is correct or true. Ideally, you've got a
Speaker 1: system that is self improving. It trains itself to be better.
Speaker 1: But how the heck does that happen? Well, let's consider
Speaker 1: cats for a bit, not the musical and good Heaven's
Speaker 1: definitely not the movie musical. That is a subject that
Speaker 1: deserves its own episode. Maybe one day I'll figure out
Speaker 1: a way to tell a cackled that film with some
Speaker 1: sort of tech capacity, But honestly, I'm just not ready
Speaker 1: to do that yet. From like an emotional standpoint as
Speaker 1: well as a research one. No, Let's say you're teaching
Speaker 1: a computer system to recognize cats pictures of cats, and
Speaker 1: the system has an artificial neural network that accepts input
Speaker 1: pictures of cats and then filters that input through the
Speaker 1: network to make the determination does this picture include a
Speaker 1: cat in it? And you start feeding it lots of images.
Speaker 1: The neural network acts on the data according to the
Speaker 1: weighted relationship between the artificial neurons, and it produces an output.
Speaker 1: Now here's the thing. We already know what we want
Speaker 1: the output to be, because we can recognize if a
Speaker 1: picture has a cat inet or not. Maybe we've got
Speaker 1: one thousand pictures. This is the training data we're going
Speaker 1: to use for this machine learning process. We also know
Speaker 1: that eight hundred of those pictures have a cat in
Speaker 1: the and two don't, so we know what we want
Speaker 1: the results to be. We've got an artificial neural network
Speaker 1: in which some neurons or nodes will accept input and
Speaker 1: perform a function based on that input, and then the
Speaker 1: weighted connections that neuron has to other neurons will determine
Speaker 1: where it passes the information down until we get to
Speaker 1: an output. And this happens until we get that conclusion.
Speaker 1: So what happens if the computer's answer is wrong? What
Speaker 1: if we feed those one thousand photos to it and
Speaker 1: says only three hundred of them have cats in them?
Speaker 1: While we have to go back and adjust those weighted connections,
Speaker 1: because clearly something didn't go right, the connections within the
Speaker 1: network need to be readjusted. We would likely start closest
Speaker 1: to our output and see which neurons seem to contribute
Speaker 1: to the mistake, which which neurons were responsible, In other words,
Speaker 1: for it to say, oh, only three these pictures had
Speaker 1: cats in them, and then we would adjust the weights,
Speaker 1: the incoming weights of connections to those neurons in order
Speaker 1: to try and favor pathways that lead to correct answers.
Speaker 1: Then we feed it the one thousand pictures again and
Speaker 1: we look at those results. Then we do this again
Speaker 1: and again and again, every time, tweaking the network a
Speaker 1: little bit so that it gets a bit better. Eventually,
Speaker 1: when we have trained the system, we can start to
Speaker 1: feed brand new data to the network, not the stuff
Speaker 1: we've trained it on, but pictures that we and the
Speaker 1: system have never seen before. And if our network is
Speaker 1: a good one, if we have trained it well, it
Speaker 1: will sort through these new photos and it will count
Speaker 1: up the ones that have the cat pictures lickety split.
Speaker 1: This approach is called supervised learning because it involves kind
Speaker 1: of grading the network on its homework and then working
Speaker 1: with it to get better. Heck, with the right algorithm,
Speaker 1: a neural network can learn to recognize and differentiate patterns
Speaker 1: even if we never explicitly told the system what it
Speaker 1: was looking for. Google discovered this several years ago when
Speaker 1: it fed several thousand YouTube videos to an enormous artificial
Speaker 1: neural network. The system analyzed the videos that were fed
Speaker 1: to it and gradually recognized patterns that represented different types
Speaker 1: of stuff, like people or like cats, because there are
Speaker 1: a lot of cat videos on YouTube, and the network
Speaker 1: got to the point where it could identify an image
Speaker 1: of a cat fairly reliably better than seventy of the time,
Speaker 1: even though it was never told how to do that,
Speaker 1: or it was never even told what a cat was. So,
Speaker 1: as Google representatives put it, they said, it had to
Speaker 1: invent the concept of a cat. It had to recognize
Speaker 1: that cats are not the same as people, which I
Speaker 1: think is a big slap in the face to some cats. Really,
Speaker 1: what it said was that I recognized this particular pattern
Speaker 1: of features, and I recognized that these other instances of
Speaker 1: creatures that have a similar pattern seemed to match that,
Speaker 1: and so I draw the conclusion that this instance of
Speaker 1: a thing belongs with all these other instances of things
Speaker 1: that are similar in characteristics. So this was more of
Speaker 1: an example of unsupervised learning, and that the system, when
Speaker 1: fed enough data, began to categorize stuff all on its
Speaker 1: own through its own parameters. Now, one neat way that
Speaker 1: computer scientists will train up systems for certain types of
Speaker 1: applications is through a generative adversarial network, which I admit
Speaker 1: sounds kind of sinister, doesn't it, And I mean it
Speaker 1: can be, but it doesn't have to be essentially near
Speaker 1: Using two different artificial neural networks. One of the networks
Speaker 1: has a specific job. It's to fool the other network.
Speaker 1: So the other network's job is to detect attempts to
Speaker 1: fool it versus legitimate data. So let's use an example.
Speaker 1: Let's say you're trying to create a system that can
Speaker 1: make realistic but entirely computer generated, that is, fabricated photographs
Speaker 1: of people. So, in other words, these are computer generated
Speaker 1: images that don't actually represent a real person at all.
Speaker 1: We've got one artificial neural network, the generator, and its
Speaker 1: job is to create images of people that can pass
Speaker 1: as real photographs. Then we've got our other network, which
Speaker 1: is the discriminator. This is trying to sort out real
Speaker 1: photos of actual people from pictures that have been generated
Speaker 1: by the generative system. And we pick these two networks
Speaker 1: against each other. The idea here is that both systems
Speaker 1: get better as they test one another out. If the
Speaker 1: generator network is falling behind because the discriminator can suss
Speaker 1: out the fakes too easily, well, then it's time to
Speaker 1: tweak some weights in that neural network that are leading
Speaker 1: to dissatisfactory computer generated images and try it again. But then,
Speaker 1: if the discriminator is starting to miss fakes while, it's
Speaker 1: time to tweak the discriminator network. So it's better at
Speaker 1: spotting the false pictures. Now along the way, some pretty
Speaker 1: extraordinary stuff can happen. There are photos of computer generated faces,
Speaker 1: not altered pictures, not ones created by a human artist,
Speaker 1: but entirely composed via a computer, and they can look
Speaker 1: absolutely realistic, complete with consistent lighting and shadows. This is
Speaker 1: only after lots of training sessions the networks learn what
Speaker 1: the giveaways are, like, what is it that leads the
Speaker 1: discriminator to say, no, this is a fake photo, and
Speaker 1: how can you fix that? It reminds me a bit
Speaker 1: of how photo experts used to point out really bad
Speaker 1: photoshop jobs and explaining how certain elements like shadows or
Speaker 1: edges or whatever, we're a dead giveaway that someone had
Speaker 1: altered an image. Well, similar rules exist for generated images,
Speaker 1: and through training, the generator gets better at making really
Speaker 1: convincing examples that don't fall into the traps that would
Speaker 1: reveal it as a fake. Over time, generative networks can
Speaker 1: get good enough to produce stuff that would be very
Speaker 1: difficult for a human to tell apart from the quote
Speaker 1: unquote real thing, and discriminators can get good enough to
Speaker 1: detect fakes that would otherwise pass human inspection. So an
Speaker 1: example of This is the current ongoing battle with deep fakes.
Speaker 1: These are computer generated videos that appear to be legit.
Speaker 1: If they're done well enough, they can have famous people
Speaker 1: in them. Doesn't have to be a famous person, but
Speaker 1: it can show a video of someone doing something that
Speaker 1: they absolutely never did, but according to the video, they did,
Speaker 1: and it can be really convincing if it's done well.
Speaker 1: A good deep fake can fool people if you aren't
Speaker 1: paying too much attention. Some of the really good ones
Speaker 1: can pass pretty deep scrutiny. So this requires researchers to
Speaker 1: come up with solutions that are pretty subtle and beyond
Speaker 1: the average person's ability to replicate, like looking at the
Speaker 1: reflections in the person's eyes and whether or not they
Speaker 1: seem realistic or a computer generated. But that really just
Speaker 1: represents another hurdle for the generative side. So in other words,
Speaker 1: this is a seesaw approach, right. It's creating fakes on
Speaker 1: one side and detecting them on the other side. It's
Speaker 1: something we see in artificial intelligence in general. A similar
Speaker 1: our story played out with the old capture systems, where
Speaker 1: you know, we saw back and forth between methods to
Speaker 1: try and weed out bots by using capture images that
Speaker 1: only humans could really parse, and then we saw improved
Speaker 1: bots that could analyze these images and return correct results,
Speaker 1: which meant it was necessary to create more difficult captures.
Speaker 1: Eventually get to a point where the captures are difficult
Speaker 1: enough where the average person can't even pass them, and
Speaker 1: then you have to go to a different method. We
Speaker 1: also see this play out in the cyber security realm,
Speaker 1: where you might say the thieves get better at lock picking,
Speaker 1: and then security experts make better locks, and the cycle
Speaker 1: just repeats endlessly. One thing that has really fueled machine
Speaker 1: learning recently is the era of big data. Being able
Speaker 1: to harvest information on a truly massive scale provides the
Speaker 1: opportunity to feed that data into various machine learning systems
Speaker 1: to search for meaning within that data. These systems might
Speaker 1: scour the information to look for stuff like criminal activity
Speaker 1: like financial crimes or the attempt to move some money
Speaker 1: around from various criminal exploits. Or it could be used
Speaker 1: to look for trends like market trends, or it might
Speaker 1: be used to plot possible spikes in COVID nineteen transmission
Speaker 1: where those might occur where people should really be focusing
Speaker 1: their attention. But now we got to think back on
Speaker 1: what I said earlier about looking up at the sky
Speaker 1: and seeing shapes in the clouds. There's a risk that
Speaker 1: comes along with machine learning. Actually, technically there are a
Speaker 1: lot of risks, but this one is a biggie. It
Speaker 1: is possible for machines like humans, to detect a pattern
Speaker 1: where there really isn't a pattern. Systems might interpret noise
Speaker 1: to be signal, and depending on what you're using the
Speaker 1: system to do, that could lead you to some seriously dangerous,
Speaker 1: incorrect conclusions. In some cases, you could just be inconvenient,
Speaker 1: but depending on what you're working toward, it could be catastrophic.
Speaker 1: And so computer scientists know they have to do a
Speaker 1: lot of analysis to make sure that patterns that are
Speaker 1: identified through machine learning processes are actually real before acting
Speaker 1: on that information. Likewise, bias is something that we humans have, well,
Speaker 1: it's also something that machine learning systems have too. Now,
Speaker 1: sometimes bias is intentional. It can take the form of
Speaker 1: those weighted relationships between artificial neurons. Other times, a systems architects,
Speaker 1: you know, the people who put it together, They might
Speaker 1: have introduced bias, not through conscious effort, but merely through
Speaker 1: the approach they took and that approach might have been
Speaker 1: too narrow. We've seen this pop up a lot again
Speaker 1: with facial recognition technologies, many of which have a sliding
Speaker 1: scale of efficacy. They might be more reliable with certain
Speaker 1: ethnicities like white people, over others. That points that a
Speaker 1: likely problem with the way those systems were trained. This
Speaker 1: is one of the reasons why many companies have made
Speaker 1: a choice to stop supplying certain parties like police forces
Speaker 1: and military branches with facial recognition systems. The systems aren't
Speaker 1: reliable for all demographic groups and thus could cause disproportionate
Speaker 1: harm to certain populations. It would be a technological approach
Speaker 1: to systemic racism, and this stuff is already out there
Speaker 1: in the wild. You might think a computer system can't
Speaker 1: be biased or prejudiced or racist, and sure, we're still
Speaker 1: not at the point where these systems are thinking in
Speaker 1: the way that humans do, but the outcome is still
Speaker 1: disproportionately harmful to some groups. That's not to say that
Speaker 1: machine learning itself is bad. It's not bad. It's a tool,
Speaker 1: just as all technology is a tool used properly with
Speaker 1: a careful hand to make sure that biases understood and
Speaker 1: where needed mitigated and where work can be double or
Speaker 1: triple checked before acted upon. It is a remarkably useful tool,
Speaker 1: one that will power and design and improve elements in
Speaker 1: our lives if it's under the correct stewardship. But it
Speaker 1: does require a bit more hands on work. We can't
Speaker 1: just leave it to the machines just yet. Well, that
Speaker 1: wraps up this look at the concept of machine learning
Speaker 1: and some of the thought that underlies it. This really
Speaker 1: is a very high level treatment of machine learning. There
Speaker 1: are plenty of resources online if you want to dive
Speaker 1: in and learn more. A lot of them get very
Speaker 1: heavy into the math, so if that's not your bag,
Speaker 1: it might be a little challenging to navigate. It certainly
Speaker 1: is for me. I love learning about the stuff, but
Speaker 1: a lot of it requires me to look up a term,
Speaker 1: then look up a term that explains that term, and
Speaker 1: so on, and I go down a rabbit hole. I
Speaker 1: hope you enjoyed that classic episode. I guess not classic,
Speaker 1: that rerun episode of tech stuff. You can't call it
Speaker 1: a classic if it's just a year old, right, So anyway,
Speaker 1: I will be back again tomorrow hopefully, and we will
Speaker 1: have a new episode, y'all If you want to get
Speaker 1: in touch with me and let me know what you
Speaker 1: would like me to cover in future episodes. There are
Speaker 1: a couple of ways of doing that. You can drop
Speaker 1: a note on Twitter. Several of you have been doing
Speaker 1: that recently and I've got I've got a list of topics.
Speaker 1: So thank you so much. That's fantastic. I really appreciate it.
Speaker 1: Keep them coming. The The handle for the podcast Twitter
Speaker 1: feed is text Stuff hs W. If, however, you would
Speaker 1: like to leave me a voice message, you can go
Speaker 1: to the I Heart Radio app go to the tech
Speaker 1: stuff page. There's a little microphone icon you click on
Speaker 1: that you can leave a message of up to thirty
Speaker 1: seconds and if you like me to include that message
Speaker 1: in an upcoming episode, just let me know in the message.
Speaker 1: Because I'm an opt in kind of guy. That's it.
Speaker 1: Hope you all are doing well and I'll talk to
Speaker 1: you again really soon. Y text Stuff is an I
Speaker 1: Heart Radio production. For more podcasts from my Heart Radio,
Speaker 1: visit the i Heart Radio app, Apple Podcasts, or wherever
Speaker 1: you listen to your favorite shows.