Speaker 1: Welcome to tech Stuff, a production from I Heart Radio.
Speaker 1: Hey there, and welcome to tech Stuff. This is your host,
Speaker 1: Jonathan Strickland. I'm an executive producer with I Heart Radio
Speaker 1: and I love all things sex. You know, folks, Back
Speaker 1: in nineteen eighties, six comedy science fiction film that I
Speaker 1: saw in the theater about a robot the games sentience
Speaker 1: and becomes a total goofball what it will? It hit
Speaker 1: theaters in eight six and it was called Short Circuit.
Speaker 1: The movie starred Steve Gutenberg, Ali Sheety, and lamentably a
Speaker 1: white actor named Fisher Stevens playing a non white character,
Speaker 1: someone who is Indian. I should add that's not Steven's fault.
Speaker 1: I mean, he auditioned to be in a movie and
Speaker 1: he got a gig. He didn't cast himself in the film,
Speaker 1: and he has since talked about his experiences realizing the
Speaker 1: problems with a white man playing a non white character,
Speaker 1: but setting aside all the problematic whitewashing, the movie showed
Speaker 1: this robot, who in the course of the film names
Speaker 1: itself Johnny five. Learning. It learns about the world around it,
Speaker 1: It learns about people, It learns about human concepts like
Speaker 1: humor and emotion, and the general idea was pretty cute.
Speaker 1: Now the nifty thing is machines actually can learn. In fact,
Speaker 1: machine learning is a really important field of study these days,
Speaker 1: complete with its own challenges and risks. I've talked about
Speaker 1: machine learning a few times in the past, but I
Speaker 1: figured we could do a deeper dive to understand what
Speaker 1: machine learning is, what it isn't, how people are leveraging
Speaker 1: machine learning, and why. I said that it does come
Speaker 1: with risks, so let's learn about machines learning. It will
Speaker 1: be impossible to talk about machine learning without also talking
Speaker 1: about artificial intelligence or AI. And this term artificial intelligence
Speaker 1: is a real doozy. It trips people up, even people
Speaker 1: who have dedicated their lives to researching and developing artificial intelligence.
Speaker 1: You can get two experts in AI talking about AI
Speaker 1: and find out that because they have slightly different takes
Speaker 1: on what AI is, there are some communication issues. It's
Speaker 1: not as simple as red versus blue would have you think,
Speaker 1: what does the A stand for? So when you really
Speaker 1: boil it down, it comes out as as no big
Speaker 1: surprise that there's a lot of ambiguity here. After all,
Speaker 1: how would you define intelligence just intelligence, not artificial intelligence
Speaker 1: just intelligence? Well, would it be the ability to learn,
Speaker 1: that is, to acquire skills and knowledge? Or is it
Speaker 1: the application of learning? Is it problems solving? Is it
Speaker 1: being able to think ahead and make plans in order
Speaker 1: to achieve a specific goal? Is it the ability to
Speaker 1: examine a problem and deconstructed in order to figure out
Speaker 1: the best solution a more specific version of problem solving?
Speaker 1: Is it the ability to recognize, understand, and navigate emotional scenarios? Now,
Speaker 1: arguably it's all of these things and more. We all
Speaker 1: have kind of an intuitive grasp on what intelligence is,
Speaker 1: but defining it in a simple way tends to feel
Speaker 1: reductive and it leaves out a lot of important details.
Speaker 1: So if defining just general intelligence is hard, it stands
Speaker 1: for reason that defining artificial intelligence is also a tough job. Heck,
Speaker 1: even coming up with a number of different types of
Speaker 1: AI is tricky. And if you don't believe me, just
Speaker 1: google the phrase different types of artificial intelligence. Never mind,
Speaker 1: you don't. You don't really actually have to do that.
Speaker 1: I already did it, though, Feel free to do it
Speaker 1: yourself and check my work if you like. When I
Speaker 1: googled that phrase different types of AI, some of The
Speaker 1: top results included a blog post on BMC Software titled
Speaker 1: four types of Artificial Intelligence. But then there was also
Speaker 1: an article on code bots that was titled what are
Speaker 1: the three types of AI? And then there was an
Speaker 1: article from Forbes titled seven types of Artificial Intelligence. See,
Speaker 1: we can't even agree on how many versions of A
Speaker 1: EITHER are because defining a I is really hard. It
Speaker 1: largely depends upon how you view AI and then how
Speaker 1: you break it down into different realms of intelligence. Now
Speaker 1: we could go super high level because a classic way
Speaker 1: to look at AI is strong versus weak artificial intelligence.
Speaker 1: Strong AI UH sometimes called artificial general intelligence, would be
Speaker 1: a machine that processes information and at least appears to
Speaker 1: have some form of consciousness and self awareness and the
Speaker 1: ability to both have experiences and to be aware that
Speaker 1: it is having experiences. It might even feel emotion, though
Speaker 1: maybe not emotions that we could easily identify or sympathize with.
Speaker 1: So this would be the kind of machine that would
Speaker 1: think in a way similar to humans. It would be
Speaker 1: able to sense its environment and not just react, but
Speaker 1: really process what is going on and build and understanding.
Speaker 1: It's the type of AI that we see a lot
Speaker 1: in science fiction. That's the type of AI of Johnny
Speaker 1: five from Short Circuit, or how from two thousand one,
Speaker 1: or the draw aids in Star Wars. It's also a
Speaker 1: type of artificial intelligence that we have yet to actually
Speaker 1: achieve in the real world. So then what is weak AI. Well,
Speaker 1: you could say it's everything else, or you could say
Speaker 1: it's the building blocks that maybe collectively will lead to
Speaker 1: strong AI. Week AI involves processes that allow machines to
Speaker 1: complete tasks. So, for example, image recognition software could fall
Speaker 1: into this category. Once upon a time, in order to
Speaker 1: search photos effectively, you needed to actually add meta data
Speaker 1: like tags to those photos. So, for example, I might
Speaker 1: tag pictures of my dog with the meta tag dog,
Speaker 1: and then if I wanted to see photos of my pooch,
Speaker 1: then I would pull up my photo app and search
Speaker 1: the term dog and all the photos that I had
Speaker 1: tagged with the word dog would show up. But if
Speaker 1: I had failed to tag some pictures of my dog,
Speaker 1: those pictures wouldn't pop up in search because the computer
Speaker 1: program wasn't actually looking for dogs and my photos, it
Speaker 1: was just looking for photos that had that particular meta
Speaker 1: tag attached to it. But now we've reached a point
Speaker 1: where at least some photo apps are using image recognition
Speaker 1: to analyze photos, and these will return results that the
Speaker 1: algorithm has identified as having a reasonable chance of meeting
Speaker 1: your search query. So if I used an app like
Speaker 1: that and I put in dog as my search term,
Speaker 1: it could pull up photos that had no meta tags
Speaker 1: attached to them at all, because the search is relying
Speaker 1: on image recognition. Now, this also means that if the
Speaker 1: image recognition algorithm isn't very good, I could get some
Speaker 1: images that don't have a dog in them at all,
Speaker 1: or it might miss other images that have my dog
Speaker 1: in them. But my point is that the ability to
Speaker 1: identify whether or not a dog is in a particular
Speaker 1: photo represents a kind of weak artificial intelligence. You wouldn't
Speaker 1: say that the photo search tool possesses humanlike intelligence, because
Speaker 1: really it only does one thing. It's analyzing photos and
Speaker 1: looks for matches to specific search queries, but it can't
Speaker 1: do anything outside of that use case. However, that's just
Speaker 1: one little example. There are all sorts of other ones,
Speaker 1: like voice recognition, environmental sensing, course plotting, that kind of thing,
Speaker 1: and in some circles, as we get better at making machines,
Speaker 1: and systems that can do these things. Those elements seem
Speaker 1: to kind of drift away from the ongoing conversation about
Speaker 1: artificial intelligence. A guy named Larry Tesler, who was a
Speaker 1: computer scientist who worked at lots of really important places
Speaker 1: like Xerox Park and Amazon and Apple. He once observed,
Speaker 1: quote intelligence is whatever machines haven't done yet, end quote.
Speaker 1: So his point was that the reason that AI is
Speaker 1: really hard to talk about is that the goal post
Speaker 1: for what actually is artificial intelligence is constantly moving. Now,
Speaker 1: this pretty much mirrors how we think about things like consciousness.
Speaker 1: Lots of people study consciousness, and the general sense I
Speaker 1: get is that it's a lot easier for people to
Speaker 1: talk about what isn't consciousness rather than what consciousness actually is.
Speaker 1: And it seems like artificial intelligence is in a similar place,
Speaker 1: which really isn't that big of a surprise as we
Speaker 1: closely associate intelligence with consciousness. Now this leads us to
Speaker 1: why there are so many different takes on how many
Speaker 1: types of AI there are. It all depends on how
Speaker 1: you classify different disciplines in artificial intelligence, and over time,
Speaker 1: a lot of disciplines that were previously distinct from AI
Speaker 1: have sort of converged into becoming heart of the AI discussion.
Speaker 1: Machine learning, as it turns out, was part of the
Speaker 1: AI discussion, branched off from it, and then rejoined the
Speaker 1: AI discussion years later. So I am not going to
Speaker 1: go down all the different approaches to classification because I
Speaker 1: don't know that they would be that valuable to us.
Speaker 1: They would really just illustrate that there are a lot
Speaker 1: of different ways to look at the subject. So if
Speaker 1: you ever find yourself in a conversation about AI, it
Speaker 1: might be a good idea to set a few ground
Speaker 1: rules as to what everyone means when they use the
Speaker 1: term artificial intelligence. That can help with expectations and understanding.
Speaker 1: Or you could just run for the nearest exit, which
Speaker 1: is what people tend to do whenever I start talking
Speaker 1: about it anyway. What about machine learning, Well, from one perspective,
Speaker 1: you could say machine learning is a sub discipline of
Speaker 1: artificial and eligence, although like I said, it hasn't always
Speaker 1: been viewed as such. I think most people would say
Speaker 1: that the ability to learn that is to take information
Speaker 1: and experience and then have some form of understanding of
Speaker 1: those things so that you can apply that to future tasks.
Speaker 1: Potentially getting better over time. I would say most people
Speaker 1: would call that part of intelligence, but you could also
Speaker 1: be a bit more wishy washy and say it's related to,
Speaker 1: you know, artificial intelligence, as opposed to being part of AI.
Speaker 1: Since the definition of AI is let's say, fluid, either
Speaker 1: way of classifying. Machine learning works. As far as I'm concerned,
Speaker 1: machine learning boils down to the idea of creating a
Speaker 1: system that can learn as it performs a task. It
Speaker 1: can learn what works and more importantly, what does not work.
Speaker 1: You may have heard that we learn a lot more
Speaker 1: from our mistakes than we do from our successes, which
Speaker 1: there's pretty much true in my experience. When something goes wrong,
Speaker 1: it's usually, but not always, possible to trace the event
Speaker 1: or events that led to the failure. You can identify
Speaker 1: decisions that we're probably the wrong ones or that led
Speaker 1: to a bad outcome, But if you have a success,
Speaker 1: it's hard to figure out which decisions were key to
Speaker 1: that successful outcome. Did your decision at step two set
Speaker 1: you on the right path, or was your choice at
Speaker 1: step three so good that it helped correct a mistake
Speaker 1: that you made it step two. But a good approach
Speaker 1: to machine learning involves a system that can adjust things
Speaker 1: on its own to reduce mistakes and increase the success rate.
Speaker 1: And another way of putting it is that instead of
Speaker 1: programming a system to arrive at a specific outcome, you
Speaker 1: are training the system to learn how to do it
Speaker 1: by itself. And that sounds a bit magical when you
Speaker 1: put it that way, doesn't it. It sounds like someone
Speaker 1: just took a computer and showed it pictures of cats
Speaker 1: and then expected the computer to know what a cat was.
Speaker 1: And this actually does mirror an actual project that really
Speaker 1: did do that, But I'm leaving out some big important
Speaker 1: information in the middle. Now. One big step is that
Speaker 1: computers and machines can't just magically learn by default. People
Speaker 1: first had to come up with a methodology that allows
Speaker 1: machines to go through the process of completing a task,
Speaker 1: then making adjustments to the process of doing that task,
Speaker 1: which would then improve future results. We have to lay
Speaker 1: the groundwork in architecture and theory and algorithms. We have
Speaker 1: to build the logical pathways that computers can follow in
Speaker 1: order for them to learn. A lot of machine learning
Speaker 1: revolves around patterns and pattern recognition. So what do I
Speaker 1: mean by patterns? Well, I mean some form of regularity
Speaker 1: and predictability. Machine learning models analyze patterns and attempt to
Speaker 1: draw conclusions based on those patterns. This in itself is
Speaker 1: tricky stuff. So why is that, Well, it's because sometimes
Speaker 1: we might think there's a pattern, when in reality there
Speaker 1: is not. We humans are pretty good at recognizing patterns,
Speaker 1: which makes sense. It's a survival mechanism. If you were
Speaker 1: to look at tall grass and you see patterns that
Speaker 1: suggest the presence of a predator like a tiger, well
Speaker 1: you would know that danger is nearby, and you would
Speaker 1: have the opportunity to do something about that to help
Speaker 1: your chances of survival. If, however, you remained blissfully unaware
Speaker 1: of the danger, you'd be far more likely to fall
Speaker 1: prey to that hungry tiger. So recognizing patterns is one
Speaker 1: of the abilities that gave humans a chance to live
Speaker 1: another day, and, from an evolutionary standpoint, a chance to
Speaker 1: make more humans. But sometimes we humans will perceive a
Speaker 1: pattern where none actually exists. A simple example of this
Speaker 1: is the fun exercise of laying on your back outside,
Speaker 1: looking up at the clouds and saying, what does that
Speaker 1: cloud remind you of? The shapes of clouds? Which have
Speaker 1: no significance and are the product of environmental factors, can
Speaker 1: seem to suggest patterns to us. We might see a dog,
Speaker 1: or a car or a face, but we know that
Speaker 1: what we're really seeing with just the appearance of a pattern,
Speaker 1: it's it's not evidence of a pattern actually being there.
Speaker 1: It's noise, not signal, but it could be misinterpreted as signal. Well,
Speaker 1: it turns out that in machine learning applications this is
Speaker 1: also an issue. I'll talk about it more towards the
Speaker 1: end of this episode. Computers can sometimes misinterpret data and
Speaker 1: termines something represents a pattern when it really doesn't. When
Speaker 1: that happens, a system relying on machine learning can produce
Speaker 1: false positives, and the consequences can sometimes be funny, like hey,
Speaker 1: this image recognition software thinks this coffee mug is actually
Speaker 1: a kiddie cat, or they can be really serious and
Speaker 1: potentially harmful. Hey, this facial recognition software has misidentified a person,
Speaker 1: marking them as, say, a person of interest in a
Speaker 1: criminal case. And it's all because this facial recognition software
Speaker 1: isn't very good at differentiating people of color. That's a
Speaker 1: real problem that really happens. Now, when we come back
Speaker 1: I'll give a little overview of the evolution of machine learning,
Speaker 1: but before we do that, let's take a quick break
Speaker 1: to talk about the history of machine learning. We first
Speaker 1: have to look back much much earlier, long before the
Speaker 1: era of computers, and talk about how thinkers like Thomas
Speaker 1: Bayes thought about the act of problem solving. Bays was
Speaker 1: born way back in two so quite a bit before
Speaker 1: we were thinking about machine learning, but he was interested
Speaker 1: in problem solving for problems involving probabilities, and specifically the
Speaker 1: relationship between different probabilities. I think it's easier to talk
Speaker 1: about if I give you an example. So let's make
Speaker 1: a silly one, all right, So let's say we got
Speaker 1: ourselves a plucky podcaster. Hey there, everybody, It's Jonathan Strickland,
Speaker 1: and it's Tuesday as I record this, And because of
Speaker 1: who I am, you know who this podcaster is. And
Speaker 1: because it's Tuesday, there is a chance I am wearing
Speaker 1: a they might Be Giants T shirt. And we also
Speaker 1: know that if this podcaster is wearing a they might
Speaker 1: be Giants T shirt on a Tuesday, there's a sixty
Speaker 1: percent chance that I'm going to end up wearing pajamas
Speaker 1: on Wednesday. But we also know that if I did
Speaker 1: not where they Might be Giants shirt on Tuesday, and
Speaker 1: remember there's a sixty chance I didn't, then we know
Speaker 1: there's an eighty percent chance I'm going to be wearing
Speaker 1: pajamas on Wednesday. Well, Bays worked out a way that
Speaker 1: described this sort of probability relationship between different discrete events,
Speaker 1: and using his reasoning, you can work forward or backward
Speaker 1: based on probabilities. Bays would describe wearing a they Might
Speaker 1: be Giant shirt on Tuesday as one event and wearing
Speaker 1: pajamas on Wednesday as a separate event, and then describe
Speaker 1: the two not only determining how likely it is I'll
Speaker 1: wear pajamas on Wednesday, but if we start with the
Speaker 1: later event. In other words, if we start with the
Speaker 1: fact that it's Wednesday and I'm wearing pajamas, we could
Speaker 1: work out how likely it was that yesterday, on Tuesday,
Speaker 1: I was wearing they Might Be Giants shirt. That was
Speaker 1: his his contribution, that you can work this in either
Speaker 1: direction if you know these different variables. Now, Bay has
Speaker 1: never published his thoughts, but rather send an essay explaining
Speaker 1: it to a friend of his, who then made sure
Speaker 1: that The work was published after Bays had passed away,
Speaker 1: and a few decades later Pierre Simon Laplace would take
Speaker 1: this work that Bays had done and flesh it out
Speaker 1: into an actual formal theorem. It's an important example of
Speaker 1: conditional probability, and a lot of what machine learning is
Speaker 1: really boiled down to is dealing with different probabilities, not certainties, which,
Speaker 1: when you get down to it, is what most of
Speaker 1: us are doing most of the time. Right We make
Speaker 1: decisions based on at least perceived probabilities. Sometimes these decisions
Speaker 1: might feel like they're a coin flip situation that any
Speaker 1: choice is equally likely to precipitate a good outcome or
Speaker 1: a bad outcome. Other times we might make a choice
Speaker 1: because we feel the probabilities are stacked favorably one way
Speaker 1: over another. Sometimes we will make a choice to back
Speaker 1: the least probable outcome because well, humans are not always
Speaker 1: superrational and hex sometimes the long shot does pay off,
Speaker 1: so that keeps Vegas in business. Bayes' theorem is just
Speaker 1: one example of ways that mathematicians and philosophers figured out
Speaker 1: ways to mathematically express problem solving and decision making, and
Speaker 1: a lot of this was figuring out if there were
Speaker 1: a way to boil down things that most of us
Speaker 1: approached through intuition and experience. So it's kind of neat,
Speaker 1: and also the more you look into it, the more
Speaker 1: likely you might find it's a little spooky, because it's
Speaker 1: weird to consider that our approaches to making choices and
Speaker 1: solving problems can be reduced down to mathematical expressions. But
Speaker 1: let's leave the potential existential crises alone for now, shall we.
Speaker 1: So moving on, we have another smarty pants we need
Speaker 1: to talk about Andre Markov, a Russian mathematician. In the
Speaker 1: early twentieth century. He began studying the nature of certain
Speaker 1: random processes that follow a particular type of rule, which
Speaker 1: we now call the Markov property. That rule says that
Speaker 1: for this particular process, the next stage of the process
Speaker 1: only depends upon the current stage, but not any stages
Speaker 1: that came before then. So let's take my ridiculous T
Speaker 1: shirt example, and let's build it out a little bit further.
Speaker 1: Let's say that I've got three T shirts to my name.
Speaker 1: One of them is that they might be Giant's shirt,
Speaker 1: one is a plain blue T shirt, and the third
Speaker 1: is a shirt that has the tech stuff logo on it,
Speaker 1: and it's based off of long observation that you've determined
Speaker 1: these following facts. If I am wearing that they Might
Speaker 1: Be shirt today, I definitely will not wear it tomorrow.
Speaker 1: But there's a fifty shot I'll wear either the blue
Speaker 1: shirt or the tech Stuff shirt. Now, if I'm wearing
Speaker 1: the blue shirt today, there's a ten percent chance I'm
Speaker 1: going to wear the same blue shirt tomorrow. Don't worry,
Speaker 1: I'll wash it first. There's a sixty chance that I'll
Speaker 1: wear the tech Stuff shirt, and there's a thirty percent
Speaker 1: chance I'll wear the they Might Be Giant shirt. But
Speaker 1: if I'm wearing the tech Stuff shirt today, there's a
Speaker 1: seventy chance I'll wear it again tomorrow because I like
Speaker 1: to promote myself. But there's a thirty percent chance I'll
Speaker 1: wear the they Might Be Giant shirt, and there is
Speaker 1: no chance that I'm going to wear the blue one
Speaker 1: in this case. So those are our various scenarios. Right
Speaker 1: which shirt I will wear tomorrow depends only upon which
Speaker 1: shirt I am wearing today. What I wore yesterday has
Speaker 1: no bearing on the outcome for tomorrow, So today is
Speaker 1: all that matters. And depending on which shirt I wear,
Speaker 1: you can make some probability predictions for tomorrow. So we
Speaker 1: can actually use this approach to figure out the probability
Speaker 1: that I might wear the tech Stuff shirts, say ten
Speaker 1: days in a row, since there's a better than even
Speaker 1: chance that if I'm wearing tech stuff today, I'll end
Speaker 1: up wearing it again tomorrow. And if I wear it tomorrow,
Speaker 1: then there's a better than fifty chance that I'm going
Speaker 1: to wear it the following day. But at some point
Speaker 1: you're going to see that the odds are starting to
Speaker 1: be against you, for you know, increasingly long strings of
Speaker 1: wearing the tech Stuff shirt. Anyway, Markov chains would become
Speaker 1: one of the types of processes that machine learning models
Speaker 1: would incorporate, with some models looking at the current state
Speaker 1: of a given process and then make predictions on what
Speaker 1: the next state will be with no need to look
Speaker 1: back at the previous decision. The Markov chain is memory less.
Speaker 1: Now that's just a couple of the mathematicians whose work
Speaker 1: underlies elements of machine learning. There's also structure we need
Speaker 1: to talk about. In ninety nine, a man named Donald
Speaker 1: Hebb wrote a book titled The Organization of Behavior, and
Speaker 1: in that book, Hebb gave a hypothesis on how neurons,
Speaker 1: that is, how how brain cells interact with one another.
Speaker 1: His ideas included the notion that if two neurons interact
Speaker 1: with one another regularly, that is, if one fires, that
Speaker 1: the second one is also likely to fire. They end
Speaker 1: up forming a tighter communicative relationship with each other. Not
Speaker 1: long after his expression of this hypothesis, computer scientists began
Speaker 1: to think of a potential way to do this artificially,
Speaker 1: with machines creating the equivalent of artificial neurons. The relative
Speaker 1: strength and relationship between artificial neurons is something we described
Speaker 1: by wait that's going to be an important part of
Speaker 1: machine learning. WIT. By the way, is W E I
Speaker 1: G H T, as in this relationship is weighted more
Speaker 1: heavily than that relationship. In the early nineteen fifties, and
Speaker 1: IBM researcher named Arthur Samuel created a program designed to
Speaker 1: win at checkers. The program would do a quick analysis
Speaker 1: of where pieces were on a checkerboard and whose move
Speaker 1: it was, and then calculate the chances of each side
Speaker 1: winning the game based on those positions, and it did
Speaker 1: this with a mini max approach. Alright, so checkers is
Speaker 1: a two player turn based game. Player one makes a move,
Speaker 1: then player two can make a move. There are a
Speaker 1: finite number of moves that can be made, a finite
Speaker 1: number of possibilities, though admittedly it's a pretty good number
Speaker 1: of possibilities. But let's say a game has been going
Speaker 1: on for a few moves, and you've got your two sides.
Speaker 1: You've got the red checkers over on player one side
Speaker 1: and the black checkers for a player to Let's say
Speaker 1: it's player one's move. For the purposes of this example,
Speaker 1: will say that player one really just has one piece
Speaker 1: that they can actually move on this turn, and it
Speaker 1: can move into one of two open spaces. So player
Speaker 1: one has to make a choice. After that choice, it's
Speaker 1: going to be player two's turn. So we can create
Speaker 1: a decision tree illustrating the possible choices and the possible
Speaker 1: outcomes of those choices. These choices are the children of
Speaker 1: the starting position for player one, so player one's starting
Speaker 1: position has two children. Player too will have their own
Speaker 1: choices to make after that decision has been made, but
Speaker 1: those choices are going to depend upon whatever move player
Speaker 1: one ultimately takes. So we can extend out our decision
Speaker 1: tree showing the branching possible move that Player Too might make,
Speaker 1: and these are the children of the two possible outcomes
Speaker 1: of our first choice. After player two's turn, it's player
Speaker 1: ones turn again, which means we need to branch those
Speaker 1: decisions out even further. And this is all before player
Speaker 1: one has even made that first choice. We're just evaluating possibilities.
Speaker 1: At some point, either when we have plotted far enough
Speaker 1: out that we know all possible outcomes of the game,
Speaker 1: or we're just reaching a point where it would be
Speaker 1: unmanageable for us to go any further. We need to
Speaker 1: actually analyze what our options are. The endpoints represent either
Speaker 1: a win, a loss, or a draw for player one, or,
Speaker 1: if we haven't extended out the tree all the way
Speaker 1: to the end of the game, at least a change
Speaker 1: in advantage, whether it would be in player one's advantage
Speaker 1: to make that move or disadvantage. We could actually assign
Speaker 1: numerical values to each in the point, with positive values
Speaker 1: representing an advantage for player one and a negative value
Speaker 1: representing an advantage for Player Too. And once we do that,
Speaker 1: we can see which pathways tend to lead to better
Speaker 1: outcomes for Player one. We work backward through the decision tree.
Speaker 1: So on all the decisions that end in an advantage
Speaker 1: for player one, we can say this is the choice
Speaker 1: that player one would take. But then we know that
Speaker 1: for player to player two is always going to choose
Speaker 1: whichever choice has the grace advantage for that player. So
Speaker 1: we have to actually take that into account as we're
Speaker 1: working backward, and this is how we can finally get
Speaker 1: to the point where we decide which move we're going
Speaker 1: to make, because these decisions, as you go backward up
Speaker 1: the tree, they ultimately inform you which of those two
Speaker 1: choices is going to give you the best result. Those values, well,
Speaker 1: those are weights. So for player one, the goal is
Speaker 1: to pick the path that has the highest positive value.
Speaker 1: For player too, it's to pick the path that has
Speaker 1: the lowest possible value or the highest negative value, if
Speaker 1: you prefer. So. In other words, player one might be
Speaker 1: thinking something like, if I moved to Spot A, my
Speaker 1: chance of winning this game is but if I moved
Speaker 1: to Spot B, it's only so. Of course, those percentages
Speaker 1: will also depend on what player two is going to
Speaker 1: do in response. Some moves that Player Too might do
Speaker 1: could end up guaranteeing a win for player one. This
Speaker 1: is the mini max approach, and there's an algorithm that
Speaker 1: guides it. It depends upon the current position within a
Speaker 1: game and how many moves or how much depth it
Speaker 1: has to take into account, and for which player is
Speaker 1: it actually helping out. What happens is if player one
Speaker 1: does this evaluation and finds that both options are negative, well,
Speaker 1: then this is something that happens in games. Right. Sometimes
Speaker 1: you find out there is no good move, like any
Speaker 1: move you make is going to be a losing move. Well,
Speaker 1: the only option at that point is to choose the
Speaker 1: least bad one, so it would be whatever the smallest
Speaker 1: negative value choice was. Our next big development that I
Speaker 1: need to mention is Frank Rosenblatt's artificial neural network called Perceptron.
Speaker 1: Its purpose was to recognize shapes and patterns, and it
Speaker 1: was originally going to be its own machine like actual hardware,
Speaker 1: but the first incarnation of Perceptron would actually be in
Speaker 1: the form of software rather than hardware. There was a
Speaker 1: purpose built Perceptron later, but the original one was software.
Speaker 1: Despite some early excitement, the Perceptron proved to be somewhat
Speaker 1: limited in its capabilities and interest in artificial neural networks
Speaker 1: died down for a while as a result. In a way,
Speaker 1: you could kind of compare this to some other technologies
Speaker 1: that got a big hype cycle and then later deflated.
Speaker 1: Virtual reality is the one I always go with. Back
Speaker 1: in the nineteen nineties, the world was really hyped for
Speaker 1: virtual reality. People had incredibly unrealistic expectations for what VR
Speaker 1: actually meant and what it could do. And when it
Speaker 1: turned out the VR wasn't nearly as sophisticated as people
Speaker 1: were imagining, a lot of enthusiasm dropped out for the
Speaker 1: entire field, and with that dropped funding and support, and
Speaker 1: as a result, development and VR hit a real wall,
Speaker 1: with only a fraction of the people who had been
Speaker 1: working in the field sticking around, and they had to
Speaker 1: scramble just to find funding to keep their projects going.
Speaker 1: So VR was effectively put on the shelf and wouldn't
Speaker 1: make much progress for nearly twenty years. Well. Artificial neural
Speaker 1: networks had a very similar issue, but other computer scientists
Speaker 1: eventually found ways to design artificial neural networks. They could
Speaker 1: do some pretty amazing things if they had access to
Speaker 1: enough data. When we come back, i'll talk a little
Speaker 1: bit more about that and what it all means. But
Speaker 1: first let's take another quick break. So we left off
Speaker 1: with the AI field going into hibernation for a little bit.
Speaker 1: Theory and mathematics were bumping up against the limitations of technology,
Speaker 1: which wasn't quite at the level to put all that
Speaker 1: theory to the test. Plus there needed to be some
Speaker 1: tweaks to the approaches, but those came with time and
Speaker 1: more mathematicians found new ways to create artificial neural networks
Speaker 1: capable of stuff like pattern recognition and learning. So let's
Speaker 1: imagine another decision tree. We've got our starting position. This
Speaker 1: is probably where we put some input. We would feed
Speaker 1: data into a system, and let's say from that starting position,
Speaker 1: we have a process that's going to transform that input
Speaker 1: into one of two possible ways. So we've got two
Speaker 1: potential outputs for that first step. Like our mini max example,
Speaker 1: we can go down several layers of possible choices, and
Speaker 1: we can wait the relationships between these different choices. So
Speaker 1: if the incoming value is higher than a certain amount,
Speaker 1: maybe the node sends it down one pathway, but if
Speaker 1: the value is lower than that arbitrary amount, the node
Speaker 1: will send it down a different pathway. This is drastically oversimplifying,
Speaker 1: but I hope you kind of get the idea. It's
Speaker 1: like a big sorting system and the goal is that
Speaker 1: at the very end, whatever comes out as output is
Speaker 1: correct or true. Ideally, you've got a system that is
Speaker 1: self improving. It trains itself to be better. But how
Speaker 1: the heck does that happen? Well, let's consider cats for
Speaker 1: a bit, not the musical and could heavens definitely not
Speaker 1: the movie music a coal that is a subject that
Speaker 1: deserves its own episode. Maybe one day I'll figure out
Speaker 1: a way to tackle that film with some sort of
Speaker 1: tech capacity, but honestly, I'm just not ready to do
Speaker 1: that yet, from like an emotional standpoint as well as
Speaker 1: a research one. No, let's say you're teaching a computer
Speaker 1: system to recognize cats pictures of cats, and the system
Speaker 1: has an artificial neural network that accepts input pictures of
Speaker 1: cats and then filters that input through the network to
Speaker 1: make the determination does this picture include a cat in it?
Speaker 1: And you start feeding it lots of images. The neural
Speaker 1: network acts on the data according to the weighted relationship
Speaker 1: between the artificial neurons, and it produces an output. Now
Speaker 1: here's the thing we already know what we want the
Speaker 1: output to be because we can recognize of a picture
Speaker 1: has a cat inet or not. Maybe we've got one
Speaker 1: thousand pictures. This is the training data we're going to
Speaker 1: use for this machine learning process. We also know that
Speaker 1: eight hundred of those pictures have a cat in them
Speaker 1: and two hundred don't, so we know what we want
Speaker 1: the results to be. We've got an artificial neural network
Speaker 1: in which some neurons or nodes will accept input and
Speaker 1: perform a function based on that input, and then the
Speaker 1: weighted connections that neuron has to other neurons will determine
Speaker 1: where it passes the information down until we get to
Speaker 1: an output. And this happens until we get that conclusion.
Speaker 1: So what happens if the computer's answer is wrong. One
Speaker 1: if we feed those one thousand photos to it and
Speaker 1: says only three hundred of them have cats in them.
Speaker 1: While we have to go back and adjust those weighted
Speaker 1: connections because clearly something didn't go right, the connections within
Speaker 1: the network need to be readjusted. We would likely start
Speaker 1: closest to our output and see which neurons seem to
Speaker 1: contribute to the mistake, which which neurons were responsible, in
Speaker 1: other words, for it to say, oh, only three these
Speaker 1: pictures had cats in them. And then we would adjust
Speaker 1: the weights, the incoming weights of connections to those neurons
Speaker 1: in order to try and favor pathways that lead to
Speaker 1: correct answers. Then we feed it the one thousand pictures
Speaker 1: again and we look at those results. Then we do
Speaker 1: this again and again and again, every time tweaking the
Speaker 1: network a little bit so that it gets a bit better. Eventually,
Speaker 1: when we have trained the system, we can start to
Speaker 1: feed brand new data to the network, not the stuff
Speaker 1: we've trained it on, but pictures that we and the
Speaker 1: system have never seen before. And if our network is
Speaker 1: a good one, if we have trained it well, it
Speaker 1: will sort through these new photos and it will count
Speaker 1: up the ones that have the cat pictures lickety split.
Speaker 1: This approach is called supervised learning because it involves kind
Speaker 1: of grading the network on its homework and then working
Speaker 1: with it to get better. Heck, with the right algorithm,
Speaker 1: a neural network can learn to recognize and differentiate patterns
Speaker 1: even if we never explicitly told the system what it
Speaker 1: was looking for. Google discovered this several years ago when
Speaker 1: it fed several thousand YouTube videos to an enormous artificial
Speaker 1: neural network. The system analyzed the videos that were fed
Speaker 1: to it and gradually recognized patterns that represented different types
Speaker 1: of stuff, like people or like cats, because there are
Speaker 1: a lot of cat videos on YouTube, and the network
Speaker 1: got to the point where it could identify an image
Speaker 1: of a cat fairly reliably better than seventy of the time,
Speaker 1: even though it was never told how to do that,
Speaker 1: or it was never even told what a cat was. So,
Speaker 1: as Google representatives put it, they said, it had to
Speaker 1: invent the concept of a cat. It had to recognize
Speaker 1: that cats are not the same as people, which I
Speaker 1: think is a big slap in the face to some cats. Really,
Speaker 1: what it said was that I recognized this particular pattern
Speaker 1: of features, and I recognized that these other instances of
Speaker 1: creatures that have a similar pattern seemed to match that,
Speaker 1: and so I draw the conclusion that this instance of
Speaker 1: a thing belongs with all these other instances of things
Speaker 1: that are similar in characteristics. So this was more of
Speaker 1: an example of unsupervised learning, and that the system, when
Speaker 1: fed enough data, began to categorize stuff all on its
Speaker 1: own through its own parameters. Now, one neat way that
Speaker 1: computer scientists will train up systems for certain types of applications.
Speaker 1: Is through a generative adversarial network, which I admit sounds
Speaker 1: kind of sinister, doesn't it, And I mean it can be,
Speaker 1: but it doesn't have to be. Essentially, you're using two
Speaker 1: different artificial neural networks. One of the networks has a
Speaker 1: specific job, it's to fool the other network. So the
Speaker 1: other network's job is to detect attempts to fool it
Speaker 1: versus legitimate data. So let's use an example. Let's say
Speaker 1: you're trying to create a system that can make realistic
Speaker 1: but entirely computer generated, that is, fabricated photographs of people. So,
Speaker 1: in other words, these are computer generated images that don't
Speaker 1: actually represent a real person at all. We've got one
Speaker 1: artificial neural network, the generator, and its job is to
Speaker 1: create images of people that can pass as real photographs.
Speaker 1: Then we've got our other network, which is the discriminator.
Speaker 1: This is trying to sort out real photos of actual
Speaker 1: people from pictures that have been generated by the generative system.
Speaker 1: And we put these two networks against each other. The
Speaker 1: idea here is that both systems get better as they
Speaker 1: test one another out. If the generator network is falling
Speaker 1: behind because the discriminator can suss out the fakes too easily. Well,
Speaker 1: then it's time to tweak some weights in that neural
Speaker 1: network that are leading to dissatisfactory computer generated images and
Speaker 1: try it again. But then, if the discriminator is starting
Speaker 1: to miss fakes while, it's time to tweak the discriminator
Speaker 1: network so it's better at spotting the false pictures. Not
Speaker 1: Along the way, some pretty extraordinary stuff can happen. There
Speaker 1: are photos of computer generated faces, not altered pictures, not
Speaker 1: ones created by a human artist, but entirely composed by
Speaker 1: a computer, and they can look absolutely realistic, complete with
Speaker 1: consistent lighting and shadows. This is only after lots of
Speaker 1: training sessions the networks learn what the giveaways are, like,
Speaker 1: what is it that leads the discriminator to say, no,
Speaker 1: this is a fake photo, and how can you fix that?
Speaker 1: It reminds me a bit of how photo experts used
Speaker 1: to point out really bad photoshop jobs and explaining how
Speaker 1: certain elements like shadows or edges or whatever, we're a
Speaker 1: dead giveaway that someone had altered an image. Well, similar
Speaker 1: rules exist for generated images, and through training, the generator
Speaker 1: gets better at making really convincing examples that don't fall
Speaker 1: into the traps that would reveal it as a fake.
Speaker 1: Over time, generative networks can get good enough to produce
Speaker 1: stuff that would be very difficult for a human to
Speaker 1: tell apart from the quote unquote real thing, and discriminators
Speaker 1: can get good enough to detect fakes that would otherwise
Speaker 1: pass human inspection. So an example of this is the
Speaker 1: current ongoing battle with deep fakes. These are computer generated
Speaker 1: videos that appear to be legit. If they're done well enough,
Speaker 1: they can have famous people in them. Doesn't have to
Speaker 1: be a famous person, but it can show a video
Speaker 1: of someone doing something that they absolutely never did, but
Speaker 1: according to the video, they did, and it can be
Speaker 1: really convincing if it's done well. A good deep fake
Speaker 1: can fool people if you aren't paying too much attention.
Speaker 1: Some of the really good ones can pass pretty deep scrutiny.
Speaker 1: So this requires researchers to come up with solutions that
Speaker 1: are pretty subtle and beyond the average person's ability to replicate,
Speaker 1: like looking at the reflections in the person's eyes and
Speaker 1: whether or not they seem realistic or a computer generated.
Speaker 1: But that really just represents another hurdle for the generative side.
Speaker 1: So in other words, this is a seesaw approach, right.
Speaker 1: It's creating face as on one side and detecting them
Speaker 1: on the other side. It's something we see an artificial
Speaker 1: intelligence in general. A similar story played out with the
Speaker 1: old capture systems, where you know, we saw back and
Speaker 1: forth between methods to try and weed out bots by
Speaker 1: using capture images that only humans could really parse, and
Speaker 1: then we saw improved bots that could analyze these images
Speaker 1: and return correct results, which men it was necessary to
Speaker 1: create more difficult captures. Eventually get to a point where
Speaker 1: the captures are difficult enough where the average person can't
Speaker 1: even pass them, and then you have to go to
Speaker 1: a different method. We also see this play out in
Speaker 1: the cyber security realm, where you might say, the thieves
Speaker 1: get better at lock picking, and then security experts make
Speaker 1: better locks, and the cycle just repeats endlessly. One thing
Speaker 1: that has really fueled machine learning recently is the era
Speaker 1: of big data. Being able to harvest information on a
Speaker 1: truly massive scale provides the opportunity to feed that data
Speaker 1: into various machine learning systems to search for meaning within
Speaker 1: that data. These systems might scour the information to look
Speaker 1: for stuff like criminal activity like financial crimes or the
Speaker 1: attempt to move some money around from various criminal exploits.
Speaker 1: Or it could be used to look for trends like
Speaker 1: market trends, or it might be used to plot possible
Speaker 1: spikes in COVID nineteen transmission where those might occur where
Speaker 1: people should really be focusing their attention. But now we
Speaker 1: got to think back on what I said earlier about
Speaker 1: looking up at the sky and seeing shapes in the clouds.
Speaker 1: There's a risk that comes along with machine learning. Actually,
Speaker 1: technically there are a lot of risks, but this one
Speaker 1: is a biggie. It is possible for machines like humans
Speaker 1: to detect a pattern where there really isn't a pattern.
Speaker 1: Systems might interpret noise to be signal, and depending on
Speaker 1: what you're using the system to do, that could lead
Speaker 1: you to some seriously dangerous incorrect conclusions. In some cases,
Speaker 1: you could just be inconvenient, but depending on what you're
Speaker 1: working toward, it could be catastrophic. And so computer scientists
Speaker 1: know they have to do a lot of analysis to
Speaker 1: make sure that patterns that are identified through machine learning
Speaker 1: processes are actually real before acting on that information. Likewise,
Speaker 1: bias is something that we humans have. Well, it's also
Speaker 1: something that machine learning systems have too. Now, sometimes bias
Speaker 1: is intentional. It can take the form of those weighted
Speaker 1: relationships between artificial neurons. Other times, a systems architects, you know,
Speaker 1: the people who put it together. They might have introduced bias,
Speaker 1: not through conscious effort, but merely through the approach they took,
Speaker 1: and that approach might have been too narrow. Ow we've
Speaker 1: seen this pop up a lot again with facial recognition technologies,
Speaker 1: many of which have a sliding scale of efficacy. They
Speaker 1: might be more reliable with certain ethnicities like white people,
Speaker 1: over others. That points a a likely problem with the
Speaker 1: way those systems were trained. This is one of the
Speaker 1: reasons why many companies have made a choice to stop
Speaker 1: supplying certain parties like police forces and military branches with
Speaker 1: facial recognition systems. The systems aren't reliable for all demographic
Speaker 1: groups and thus could cause disproportionate harm to certain populations.
Speaker 1: It would be a technological approach to systemic racism, and
Speaker 1: this stuff is already out there in the wild. You
Speaker 1: might think a computer system can't be biased or prejudiced
Speaker 1: or racist, and sure, we're still not. At the point
Speaker 1: where these systems are thinking in the way that humans do,
Speaker 1: but the outcome is still disproportionately harmful to some groups.
Speaker 1: Now that not to say that machine learning itself is bad.
Speaker 1: It's not bad. It's a tool, just as all technology
Speaker 1: is a tool used properly with a careful hand to
Speaker 1: make sure that biases understood and where needed mitigated, and
Speaker 1: where work can be double or triple checked before acted upon.
Speaker 1: It is a remarkably useful tool, one that will power
Speaker 1: and design and improve elements in our lives if it's
Speaker 1: under the correct stewardship. But it does require a bit
Speaker 1: more hands on work. We can't just leave it to
Speaker 1: the machines just yet. Well, that wraps up this look
Speaker 1: at the concept of machine learning and some of the
Speaker 1: thought that underlies it. This really is a very high
Speaker 1: level treatment of machine learning. There are plenty of resources
Speaker 1: online if you want to dive in and learn more.
Speaker 1: A lot of them get very heavy into the math.
Speaker 1: So if that's not your bag, Uh, it might be
Speaker 1: a little challenging to navigate. It certainly is for me.
Speaker 1: I love learning about the stuff, but um, a lot
Speaker 1: of it requires me to look up a term, then
Speaker 1: look up a term that explains that term, and so on,
Speaker 1: and I go down a rabbit hole. But hopefully you
Speaker 1: have a better appreciation for what machine learning is at
Speaker 1: this point. If you have suggestions for topics I should
Speaker 1: cover in future text Stuff episodes, let me know. The
Speaker 1: best way to get in touch with me is through
Speaker 1: Twitter and the handle is text stuff H s W,
Speaker 1: and I'll talk to you again really soon. Text Stuff
Speaker 1: is an I Heart Radio production. For more podcasts from
Speaker 1: my heart Radio, visit the i heart Radio app, Apple Podcasts,
Speaker 1: or wherever you listen to your favorite shows.