Speaker 1: Get in touch with technology with tech Stuff from how
Speaker 1: stuff works dot com. Hey there, and welcome to tech Stuff.
Speaker 1: My name is Jonathan Strickland. I happen to be the
Speaker 1: host of this show. I'm also an executive producer at
Speaker 1: how stuff Works. And hey, I love all things tech,
Speaker 1: and today we're doing a little listener mail request. Dan
Speaker 1: wrote in and asked if I might do an episode
Speaker 1: about Apple's so called neural engine in its more recent iPhones.
Speaker 1: So today we are going to learn what a neural
Speaker 1: engine is and what it does. And if you guys,
Speaker 1: by the way, have any requests for topics you've always thought, Hey,
Speaker 1: I want to have an episode about this particular tech topic. Remember,
Speaker 1: you can send those to me by sending an email
Speaker 1: to tex Stuff at how stuff works dot com. And
Speaker 1: now let's talk about this neural engine. Well, the general
Speaker 1: public for heard about this topic back in September two
Speaker 1: thousand seventeen, when Apple CEO Tim Cook presented at what
Speaker 1: has become an annual tradition for Apple at around that
Speaker 1: time of year, pretty much every September is when Apple
Speaker 1: will come out and unveil the latest in its line
Speaker 1: of iPhone smartphones, and in that would have been the
Speaker 1: Iconic iPhone X, the tenth anniversary edition of the iPhone
Speaker 1: also the one that's been discontinued now. Cook listed off
Speaker 1: a lot of features when he went to that presentation,
Speaker 1: but the one we're really interested in today is part
Speaker 1: of the phones A eleven micro processor, also called the
Speaker 1: A eleven Bionic CPU. The most recent iPhones as of
Speaker 1: the recording of this podcast now have the next generation
Speaker 1: of that chip, the A twelve, But in both cases,
Speaker 1: the neural engine is one of the elements that gets
Speaker 1: a lot of coverage. So let's go to the A
Speaker 1: eleven since that was the first one to have it.
Speaker 1: It's more than just a CPU. It's technically a system
Speaker 1: on a chip or s O a C. It's an
Speaker 1: ARM sixty four bit chip. But that doesn't really tell
Speaker 1: you anything if you're not, you know, deep into the
Speaker 1: world of micro processors. So what does that actually mean. Well,
Speaker 1: the ARM based part means that it's it's based on
Speaker 1: the ARM micro architecture in chip design. So for our
Speaker 1: purposes we can simplify this to say, the chips components,
Speaker 1: the stuff that's actually on the microprocessor are laid out
Speaker 1: in a way that was developed by ARM Holdings, that's
Speaker 1: the company behind ARM processors. Now that is different from
Speaker 1: the layout you would find in a chip that was
Speaker 1: made by Intel, for example. So the architecture part literally
Speaker 1: refers to the layout of components in the microprocessor and
Speaker 1: how they interact with each other. And generally speaking, companies
Speaker 1: that make microprocessors develop an architecture. They do so in
Speaker 1: a way that is supposed to maximize the efficiency of
Speaker 1: the chips. So if you get the most power for
Speaker 1: the least amount of energy input you can with the
Speaker 1: least amount of waste, really is the best way of
Speaker 1: putting it. You don't want to waste too much and
Speaker 1: produce too much heat. And then you typically would then
Speaker 1: reduce the size of the various components. And then after
Speaker 1: you reduce the size of the components, you might figure
Speaker 1: out a new architecture that makes better use of these
Speaker 1: smaller components. And this process goes on and on. Intel
Speaker 1: calls this the TIC talk methodology. So that's what the
Speaker 1: ARM based part means. It's from this particular company following
Speaker 1: this particular layout. As for that sixty four bit part,
Speaker 1: what does that mean, Well, that refers to the data
Speaker 1: width of the arithmetic logic unit or a LU. The
Speaker 1: says the part of the processor that actually carries out
Speaker 1: those operations on data from computer instructions. So data with
Speaker 1: essentially tells you how much information the a L you
Speaker 1: can accept or handle at a given time, and it
Speaker 1: tells you this in bits. Now, a bit, just to
Speaker 1: remind you, is a single unit of computational information, and
Speaker 1: it is binary, meaning has two states, which we designate
Speaker 1: as being either a zero or a one. Some people
Speaker 1: say often on or false and true, but it's zero
Speaker 1: and one. The number of bits tells you how big
Speaker 1: these actual numbers can get. Before the a L you
Speaker 1: can't handle them anymore. So let's say you have an
Speaker 1: eight bit chip, because that's a lot easier to talk about.
Speaker 1: You would be able to add, subtract, multiply, divide, you know,
Speaker 1: the basic arithmetic lot logical operation to eight bit numbers.
Speaker 1: With an eight bit chip, Now, a single bit is
Speaker 1: a zero or a one, and eight bit number you
Speaker 1: can represent as a string of eight eight numbers, either
Speaker 1: zeros or ones, So you could have eight zeros in
Speaker 1: a row, up to eight ones in a row and
Speaker 1: everything in between. So it could be seven zeros and
Speaker 1: then a one or it could be six zeros and
Speaker 1: then a one and then another zero. You get the point.
Speaker 1: With that many combinations, that means you would be able
Speaker 1: to go from the typical numbers of zero to two
Speaker 1: hundred fifty five. That's with eight bit. However, we're not
Speaker 1: talking eight bit. We're talking about a sixty four bit chips.
Speaker 1: So now you have sixty four digits in a row
Speaker 1: that can be either a zero or one. That provides
Speaker 1: you a lot more combinations, which means you could range
Speaker 1: in number from zero row to nine quintillion, two hundred
Speaker 1: twenty three quadrillion, three hundred seventy two trillion, thirty six billion,
Speaker 1: eight hundred fifty four million, seven hundred seventy five thousand,
Speaker 1: eight hundred seven. That's a pretty big range. It can
Speaker 1: handle way way larger numbers than an eight bit chip.
Speaker 1: So that tells you the type of architecture this chip
Speaker 1: has and the amount of data it can handle at
Speaker 1: a time. The A eleven has six cores, so processors
Speaker 1: with multiple cores can work on parts of a problem simultaneously.
Speaker 1: If you have something that's called a parallel problem, you
Speaker 1: can divide that problem up into different segments and have
Speaker 1: different cores tackle it. Two of those six cores are
Speaker 1: what Apple calls high performance cores. They have a clock
Speaker 1: speed of two point three thirty nine giga hurts uh
Speaker 1: in the A eleven, So the clock speed tells you
Speaker 1: how many clock cycles a CPU can perform per second.
Speaker 1: Two point three nine gigga hurts means that these cores
Speaker 1: can each perform two point thirty nine billion clock cycles
Speaker 1: per second. Now, clock cycles do not easily translate over
Speaker 1: into actions. It's not necessarily one clock cycle per action.
Speaker 1: But generally these numbers tell you how much a core
Speaker 1: of the processor is able to handle per second, how
Speaker 1: many tasks it can do per second, assuming a certain
Speaker 1: number of clock cycles per task. Now, these two cores
Speaker 1: are referred to as Monsoon. The other four cores are
Speaker 1: what Apple refers to as energy efficient cores. They are
Speaker 1: not at that same high clock speed. They are meant
Speaker 1: to handle more routine tasks. They are called mistral. So
Speaker 1: you have Monsoon and Mistral, two Monsoon cores for mistral cores.
Speaker 1: But the A eleven is not just a CPU. Also
Speaker 1: has a three core graphics processing unit or GPU incorporated
Speaker 1: into this chip. And then there are the two processing
Speaker 1: cores dedicated specifically to handling tasks related to machine learning algorithms.
Speaker 1: This pair of processors are the neural engine. They are
Speaker 1: essentially an artificial neural network. And I've talked a little
Speaker 1: bit about artificial neural networks before, but we're really going
Speaker 1: to try and get an understanding of what makes them
Speaker 1: special today, because that's really why neural engine means anything
Speaker 1: in the first place. So this means we get to
Speaker 1: do a quick history lesson because this is tech stuff,
Speaker 1: and of course we have to go into the history.
Speaker 1: So here we go back in the nineteen forties and
Speaker 1: the nineteen fifties, there were some smart guys named Warren
Speaker 1: McCullough who was a neurophysiologist, and another guy named Walter
Speaker 1: Pitts who was a computer scientist and a logician, and
Speaker 1: they began developing theories that brought together computational science and neuroscience,
Speaker 1: in other words, the way machines process information and the
Speaker 1: way brains process information, which is different. McCullough wrote a
Speaker 1: couple of papers about this, and he asserted that the
Speaker 1: basic unit of logic in the brain is the neuron.
Speaker 1: So the nerve cell, the brain cell, is your your
Speaker 1: basic unit of logic in a brain, so it would
Speaker 1: act kind of like a gate or a transistor in
Speaker 1: a circuit. And so you might have a transistor being
Speaker 1: the smallest unit, not not metric of logic, but the
Speaker 1: smallest unit to allow this to happen in a circuit
Speaker 1: neurons in the brain. Pets and McCullough began developing computer
Speaker 1: algorithms that attempted to guide machines to process information in
Speaker 1: a way that was at least conceptually similar to the
Speaker 1: way our brains process information. McCullough had proposed that by
Speaker 1: doing this, you could train a machine to wreck niye
Speaker 1: handwritten characters like numbers or letters, even if those representations
Speaker 1: varied in size or style. And I've talked about this
Speaker 1: being a challenge in the past as well, that training
Speaker 1: a computer to recognize a specific type of image or
Speaker 1: a specific thing in an image is challenging. So I
Speaker 1: always use coffee mugs as an example. I don't know why,
Speaker 1: but I like that that particular one. So we're gonna
Speaker 1: go with it again. If you were to create a
Speaker 1: computer program where you feed an image of a coffee
Speaker 1: mug to the computer program, and you tell the computer
Speaker 1: program this image corresponds with this concept called coffee mug.
Speaker 1: And the image shows a blue coffee mug and its
Speaker 1: handle is pointed toward the right of the perspective of
Speaker 1: the viewer. And then you were to feed a different image,
Speaker 1: maybe of that same coffee mug, but now at a
Speaker 1: different angle. Well, the machine as looking at this as
Speaker 1: if it's a totally new thing. It cannot just uh
Speaker 1: extricate that information and say, oh, this is also a
Speaker 1: coffee mug, or maybe it's a different coffee mug. It's
Speaker 1: a different color or a different size or different shape.
Speaker 1: The computer doesn't understand the concept of coffee mug. So
Speaker 1: how can you teach it this concept? How can you
Speaker 1: train it so it recognizes coffee mugs? That was what
Speaker 1: McCulloch was looking at. Then you have another guy who
Speaker 1: came along, Frank Rosenblat, very smart man, who built on
Speaker 1: this work. He developed an artificial neuron called the perceptron. Now,
Speaker 1: a perceptron's job is, from a very high level, pretty simple.
Speaker 1: It accepts multiple binary inputs. So it accepts inputs that
Speaker 1: are either zeros or ones, and then it produces a
Speaker 1: single binary output either a zero or a one based
Speaker 1: upon processing that information. So let's say you want to
Speaker 1: create a program that can help you decide which restaurant
Speaker 1: you want to go to, and you've come up with
Speaker 1: three criteria that you think are really important in order
Speaker 1: for you to make this decision. And the three criteria
Speaker 1: you have are is the restaurant within a twenty minute
Speaker 1: drive or less? So, is it relatively close? Will a
Speaker 1: meal cost less than fifty dollars for two people to
Speaker 1: have dinner there? And does the restaurant serve tacos? Those
Speaker 1: are your three points of criteria, and you can represent
Speaker 1: each of those variables with a binary figure. So, for example,
Speaker 1: you could say that if the restaurant is closer than
Speaker 1: a twenty minute drive, if it is nearby, you represent
Speaker 1: that variable with a one. If it is further away
Speaker 1: than that, it's a zero. If the dinner for two
Speaker 1: is cheaper than fifty dollars, that's a one. If it's
Speaker 1: more expensive, it's a zero. And if it serves tacos,
Speaker 1: it's a one. And if it does not serve tacos,
Speaker 1: it's a big fat zero. Then you have a list
Speaker 1: of various restaurants you could feed each restaurant through your
Speaker 1: criteria and see how they do. Uh, And then you
Speaker 1: could narrow your choices this way, and perhaps there is
Speaker 1: no single restaurant that meets all those criteria, so you
Speaker 1: really should take another step. And that's where Rosenblatt introduces
Speaker 1: the concept of weights, where you you change how important
Speaker 1: each of the criteria are in relation to each other.
Speaker 1: Weights are real numbers that indicate the importance of particular criterion.
Speaker 1: So you want, let's say all those three criteria you've identified,
Speaker 1: the distance, the cost, and whether or not they have tacos.
Speaker 1: You have decided the most critical piece of information is
Speaker 1: whether or not the restaurant serves tacos. So you could
Speaker 1: then assign a greater weight to that criterion, saying this
Speaker 1: is more important to me, and that will influence the
Speaker 1: output of the neuron. You must also determine a threshold
Speaker 1: value for the decision. In other words, you say, in
Speaker 1: order to produce a positive result to tell me, yes,
Speaker 1: this is a restaurant you should go to, you must
Speaker 1: at least meet this threshold. That's the minimum value the
Speaker 1: calculation has to meet or exceed in order to produce
Speaker 1: a go to this restaurant result. I'll explain a bit
Speaker 1: more about this in just a second, But first I'm
Speaker 1: going to take a quick break and thank our sponsors.
Speaker 1: That threshold value that I mentioned before the break is
Speaker 1: really important because it tells your model what sort of
Speaker 1: results count as valid versus not valid. So let's say
Speaker 1: I've waited the criteria so that the distance to the
Speaker 1: restaurant and the expense of the meal each have a
Speaker 1: weight of two, but the presence of tacos is a six.
Speaker 1: That's how important I think tacos are. And I've said
Speaker 1: a threshold of four. Well, that means that if the
Speaker 1: restaurant is relatively close and it's relatively inexpensive, it's going
Speaker 1: to pass my criteria because I've given a weight of
Speaker 1: two for both of those and added together that's four.
Speaker 1: It equals the threshold. Good to go. But even if
Speaker 1: the restaurant is far away and even if it's expensive,
Speaker 1: if it serves tacos, it still passes my criteria because
Speaker 1: I gave the tacos a weight of six. Raising the
Speaker 1: threshold value reduces the number of valid restaurants. So if
Speaker 1: I make the threshold eight instead of four, now the
Speaker 1: only way I can get a valid result a result
Speaker 1: of yes, go to this restaurant is if the restaurant
Speaker 1: has tacos and it's either close by, or it's inexpensive,
Speaker 1: or both. And if I said the threshold were ten,
Speaker 1: all three criteria would need to be met for this
Speaker 1: option to be valid. Now, an artificial intelligence for the
Speaker 1: purposes of notation, many people will move the threshold value
Speaker 1: to the other side of the equation, and in this
Speaker 1: case we now call it a bias, and a bias
Speaker 1: essentially is a measurement to tell you how easy or
Speaker 1: difficult it is to get the perceptron to fire off
Speaker 1: a positive value. If you have a big positive bias,
Speaker 1: that means it's easier for the perceptron to produce a
Speaker 1: positive result a one. A large negative bias does the opposite,
Speaker 1: and thus you would get a zero. So we can
Speaker 1: write out the perceptron's rules like this. Take the value
Speaker 1: of a variable which is either going to be a
Speaker 1: zero or a one. It will be binary. You multiply
Speaker 1: the value of this variable by the weight of that variable,
Speaker 1: and weights can be different values. Let's say that the
Speaker 1: distance and expense are both weighted at two. Tacos gets
Speaker 1: a big hefty six. You're going to add your various
Speaker 1: weighted variable results together, and then you add the I
Speaker 1: s for the perceptron. And in our example, the bias
Speaker 1: is a minus six. That's to tell us that in
Speaker 1: order for this perceptron to fire, you have to you
Speaker 1: have to be able to factor in that minus six
Speaker 1: and beat it. So if after adding these elements together,
Speaker 1: you get a result that is zero or lower, the
Speaker 1: output is a zero or a negative, saying, don't go
Speaker 1: to this restaurant. So after adding that negative six, if
Speaker 1: you have a zero or less, you don't go. If
Speaker 1: you get a result that's greater than zero, it's a
Speaker 1: positive result, it says, go to that restaurant. So here
Speaker 1: in our hypothetical perceptron, we've decided on a bias of
Speaker 1: minus six, and we take our three variables as we
Speaker 1: examine a single restaurant. So this restaurant is twenty five
Speaker 1: minutes away. So that means for our first variable, which
Speaker 1: is all about distance, it gets a zero because it
Speaker 1: is further than twenty minutes away. So that variable is
Speaker 1: a zero. And we multiply the variable times the weight.
Speaker 1: The weight is too for that particular variable two time
Speaker 1: zero is zero. Then I look and I see that
Speaker 1: dinner for two of that restaurant's gonna set me back
Speaker 1: thirty dollars, but that's below the limit we had set
Speaker 1: of fifty dollars. So that means the value of the
Speaker 1: variable is one. It is cheaper than fifty dollars, so
Speaker 1: that gets a one. The weight for this variable is
Speaker 1: to so multiply the weight times of variable two times
Speaker 1: one is two. Then we have the question does the
Speaker 1: restaurants serve tacos? And I know you're dying to know this.
Speaker 1: I'm glad to report the restaurant does in fact serve tacos,
Speaker 1: And that means that the variable is a one. It's positive,
Speaker 1: and we waited this variable very heavily with a six,
Speaker 1: So six times one is six. Now we have to
Speaker 1: add all of those results together, so we have zero
Speaker 1: from the first one, too, from the second one, six
Speaker 1: from the third one. Add that together you get eight.
Speaker 1: Now we have to add in the bias, and the
Speaker 1: bias for this perceptron is a minus six. Eight plus
Speaker 1: minus six gives us a final value of two. Two
Speaker 1: is greater than zero. So by the rules we have established,
Speaker 1: the perceptron says this is a positive result and fires
Speaker 1: off a one. So the restaurant we fed to the
Speaker 1: perceptron met the criteria based on that bias. Now, if
Speaker 1: our bias had been minus ten or minus nine, we
Speaker 1: would have not produced this positive result. We have gotten
Speaker 1: a zero or negative number and it would have said no.
Speaker 1: So that bias is very important, as is the weight
Speaker 1: of the various variables. And that is one neuron. Now
Speaker 1: you can actually create layers of neurons. That's why we
Speaker 1: call it an artificial neural network, not just an artificial neuron.
Speaker 1: And by doing that you can have results from one
Speaker 1: neuron's decisions feed directly into another neuron. Also, a perceptron
Speaker 1: can perform as a type of logical gait called a
Speaker 1: name end gate in a n D that stands for
Speaker 1: not and it's a type of logical gate that can
Speaker 1: produce a false or negative output if all its inputs
Speaker 1: are true or positive. So, in other words, with the
Speaker 1: right weights and biases, a perceptron will produce an output
Speaker 1: of zero if all of its inputs are ones. The
Speaker 1: nand gate in computer science is a universal gate because
Speaker 1: you can use different creations and combinations of nand gates
Speaker 1: and build any kind of computation. You just have to
Speaker 1: link them together properly in order to do it. It's
Speaker 1: not always the most efficient way to do this, but
Speaker 1: it does work. So if you had perceptrons that accepted
Speaker 1: two variables, each with a weight of minus two, and
Speaker 1: the perceptron had a bias of three, it would act
Speaker 1: like a nandgate. That's because if both variables are one,
Speaker 1: then the final equation you'd get to determine the output
Speaker 1: would be minus two because you multiply the weight of
Speaker 1: minus two times the variable of one, and then you
Speaker 1: have to add a second minus two because the second
Speaker 1: variable is the same way. And then you would add
Speaker 1: the bias, which is three. But minus two plus minus
Speaker 1: two is minus four. You add in plus three, you
Speaker 1: get a minus one is the result minus one is
Speaker 1: less than zero, which means they output for the perceptron
Speaker 1: must be zero as opposed to one. You get a
Speaker 1: false or an off or a zero result. Two positive
Speaker 1: inputs create a negative output when a few times you
Speaker 1: can say two positives make a negative. Now that means
Speaker 1: we can ask progressively more complicated questions, with each perceptron
Speaker 1: handling one aspect of that question and feeding into another
Speaker 1: layer of perceptrons. Each perceptron will produce either a positive
Speaker 1: or a negative result, so you either get a one
Speaker 1: or a zero, and these results will feed into other
Speaker 1: neurons in the network, which will use them to perform
Speaker 1: their own calculations of their own weights and their own biases.
Speaker 1: All of this is to feed those questions through a
Speaker 1: network to produce a result, and I should be clear
Speaker 1: the weights for each variable along this path can change
Speaker 1: from one part of the decision making process to the next.
Speaker 1: We're not just talking about identical perceptrons all through the network,
Speaker 1: and that last bit is the most important part, because
Speaker 1: if this were just a matter of setting biases and
Speaker 1: weights and building out a network of perceptrons, there'd be
Speaker 1: nothing special about it, because we already have nannd gates.
Speaker 1: They existed before perceptrons. It would just mean that we
Speaker 1: have a different way to implement something we could already do,
Speaker 1: and finding a new way to do something you were
Speaker 1: already doing is rarely super transformative. You might be able
Speaker 1: to make it a better way of doing the same thing,
Speaker 1: but in this case it might be less efficient than
Speaker 1: the old way. However, there is something else that makes
Speaker 1: these perceptrons special, and that's by pairing them with those
Speaker 1: special algorithms that Cola and Pets were proposing back in
Speaker 1: the forties and fifties. These would be learning algorithms. These
Speaker 1: algorithms are instructions that can, based upon external stimuli, dynamically
Speaker 1: and automatically tune the weights and biases of perceptrons in
Speaker 1: a neural network. In other words, a program can guide
Speaker 1: the network so that it learns how to solve problems.
Speaker 1: But how well. It all comes down to making small
Speaker 1: changes in those weights and biases in order to fine
Speaker 1: tune outputs. So let's say we're working on an image
Speaker 1: recognition algorithm. That's one of the big things that the
Speaker 1: neural engine and Apple's iPhones do. They that's one of
Speaker 1: their main purposes. So in our example, let's say we're
Speaker 1: training the neural network to recognize handwritten printed lowercase letters.
Speaker 1: It's very similar to what McCulla was talking about. But
Speaker 1: let's say our model is having trouble differentiating a lowercase
Speaker 1: L with a lowercase I. It was just having issues
Speaker 1: being able to tell those two apart in particular. Now
Speaker 1: we've got a specific example in which our model is
Speaker 1: misidentifying an L as an eye. Let's say, in the
Speaker 1: hypothetical situation, and so we decide we're gonna make some
Speaker 1: minor tweaks in the weights and biases earlier on in
Speaker 1: the artificial neural network to guide our network so that
Speaker 1: it can more readily tell the difference between l lower
Speaker 1: case ls and lower case eyes. And we get our
Speaker 1: model closer to being able to tell that difference. We
Speaker 1: keep making these small adjustments until we get more consistent output.
Speaker 1: The network as a whole is said to quote unquote
Speaker 1: learn through this process. It's getting better and creating an
Speaker 1: output there's more reflective reality. But there's a bit of
Speaker 1: a problem, and anyone who has worked in QA has
Speaker 1: probably already spotted what it is. For everybody else. I'm
Speaker 1: gonna explain it in just a minute, but first let's
Speaker 1: take another quick break to thank our sponsor. So what
Speaker 1: was that problem I was talking about before the break? Well,
Speaker 1: if you've ever worked in any sort of programming environment,
Speaker 1: you know that when you introduce changes in code, you
Speaker 1: might fix whatever problem you're focusing on at the moment,
Speaker 1: but you might also break something else that's already in
Speaker 1: the code. With perceptrons. That happens when you start tweaking
Speaker 1: weights and biases, because a small change in one spot
Speaker 1: in a network can have sort of a ripple effect
Speaker 1: with unintended consequences. So, for example, in our little hypothetical situation,
Speaker 1: maybe your new model can better tell the difference between
Speaker 1: a lower case L and a lower case I, but
Speaker 1: now the lowercase J is giving it problems the way
Speaker 1: perceptron's work. Small changes in the network can reduce much
Speaker 1: larger variations and output, so it's sort of like the
Speaker 1: butterfly effect in action. Computer scientists created a different type
Speaker 1: of artificial neuron network that addresses this problem, and this
Speaker 1: type is called a sigmoid neuron. Really, I should say
Speaker 1: they created a different type of artificial neuron, So the
Speaker 1: sigmoid neuron. What the heck is this? Well, from a
Speaker 1: high level, sigmoid neurons look kind of like perceptrons, but
Speaker 1: while you'd either use either a zero or a one
Speaker 1: as the value for an input into a perceptron, a
Speaker 1: sigmoid neuron can accept a zero, a one, or any
Speaker 1: number in between zero and one. The output a sigmoid
Speaker 1: neuron produces is called the logistic function or sigmoid function.
Speaker 1: This gets a bit complicated on a surface level, particularly
Speaker 1: if like me, you're a little rusty on your algebra
Speaker 1: and calculus, but generally speaking, the end result is that
Speaker 1: using this type of artificial neuron, you can make small
Speaker 1: changes to weights and biases and not create a larger
Speaker 1: effect on the ultimate output. You'll still make small adjustments
Speaker 1: to the output. There are a lot of resources online
Speaker 1: that go into greater detail about sigma neurons. I'm not
Speaker 1: going to go into more detail here because without visual
Speaker 1: aids and being able to go into algebraic functions, it
Speaker 1: gets a little hard for me to explain. But in
Speaker 1: your typical neural network, you would have an input layer
Speaker 1: and you would have an output layer, So you have
Speaker 1: a layer where information comes in, and you would have
Speaker 1: the output layer where new information comes out. But between
Speaker 1: those two you would have what are called hidden layers.
Speaker 1: Then just really means that they're not input or output
Speaker 1: there in the middle. Hidden makes it sound like they're
Speaker 1: super clandestine and spy worthy and cool, but really they're
Speaker 1: just in between input and output. They perform processes on
Speaker 1: the inputs they receive, and they passed them on as
Speaker 1: outputs to other neurons to have more processes put on
Speaker 1: them until you finally get the output. The sort of
Speaker 1: networks I've described so far are called feed forward networks,
Speaker 1: and that means pretty much what sounds like. You plug
Speaker 1: in and puts the information passes one way through the network,
Speaker 1: and you eventually get output as the information continues to move,
Speaker 1: and we typically visualize this in a left to right
Speaker 1: kind of of display, so you imagine input coming in
Speaker 1: from the left side, passing through this network, having various
Speaker 1: processes put on it as each of these neurons UH
Speaker 1: decides if it counts as a positive or a negative response,
Speaker 1: or with sigmoid neurons, some degree in between, and then
Speaker 1: plugging that into the next neuron until you finally get
Speaker 1: to the output. It always gets fed forward. But that's
Speaker 1: not the only type of artificial neural network. There are
Speaker 1: also things called recurrent neural networks, in which neurons fire
Speaker 1: at some predetermined amount of time. Then they typically settle
Speaker 1: down they're not firing at all, but the next group
Speaker 1: of neurons start to fire. This creates kind of a
Speaker 1: cascade effect through the network, and occasionally there it could
Speaker 1: be neurons that feed back into previous neurons. There's a
Speaker 1: feedback loop. It's more challenging to make a powerful learning
Speaker 1: algorithm with recurrent neural networks because it gets super duper complicated.
Speaker 1: But recurrent neural networks pose potentially huge utility in the future.
Speaker 1: So an artificial neural network can be made up as
Speaker 1: of as few as a few dozen artificial neurons all
Speaker 1: the way up to millions of artificial neurons, and we
Speaker 1: trained them through various processes such as back propagation. Now
Speaker 1: that's when you take the actual output of the process
Speaker 1: and you compare it to what you wanted it to produce,
Speaker 1: and then you use the difference between those two results
Speaker 1: to make changes to the weights and biases in the network.
Speaker 1: So here's an example where training a our network to
Speaker 1: recognize pictures of cats, because this has actually been done.
Speaker 1: Google famously did this. So you're training your network to
Speaker 1: recognize what a cat is based upon a picture, and
Speaker 1: you use a picture that you know is a picture
Speaker 1: of a cat, so you already know the answer to this.
Speaker 1: You're teaching the computer to learn the answer to this.
Speaker 1: You know that the answer is cat, and you feed
Speaker 1: the image through this system, It analyzes the data, it
Speaker 1: gives you an output, and you see how well it did.
Speaker 1: Did it correctly identify the image as a cat, did
Speaker 1: it assign a certain level of of certainty to its conclusion,
Speaker 1: and if it's far off, you could start making changes
Speaker 1: to those weights and biases to help guide the system
Speaker 1: into determining, oh, yes, that is a cat. Training a
Speaker 1: network multiple times refines this process to the point where
Speaker 1: you can start to introduce brand new inputs to the system,
Speaker 1: inputs that the system has never encountered before, and get
Speaker 1: a reliable result. So with Google's example, you might feed
Speaker 1: it thousands or tens of thousands, or hundreds of thousands
Speaker 1: or more images of cats, and each time the system
Speaker 1: is told now that there is a cat in this image,
Speaker 1: and it begins to refine its approach, figuring out which
Speaker 1: weights and biases it needs to tweak in order to
Speaker 1: get to that result. And then you feed it a
Speaker 1: whole group of new images and you don't tell it
Speaker 1: if there are cats in those images or not. Then
Speaker 1: you leave it to the system to determine are there
Speaker 1: cats in these pictures? And if you have trained it properly,
Speaker 1: if those weights and biases are actually well tweaked, then
Speaker 1: the system should be able to reliably pick out the
Speaker 1: pictures that have cats in them. That's the idea. Now,
Speaker 1: there's tons more to be said about artificial neural networks,
Speaker 1: but i'll give you I've given a quick overview. Let's
Speaker 1: let's jump back over to Apple for a second, because
Speaker 1: that was the whole purpose of this episode. So what
Speaker 1: is a neural engine actually used for. Well, for the iPhone,
Speaker 1: it's used mainly in processing speech and image data. It's
Speaker 1: the neural engine that can analyze your face, for example,
Speaker 1: and then translate your expressions into animated form. You can
Speaker 1: create animated emoji this way, So you could use the
Speaker 1: little application and create a customized surprise emoji that copies
Speaker 1: the way you look when you make a sort of
Speaker 1: an exaggerated surprise face. You could do that. The neural
Speaker 1: engine takes the incoming data the images it's pulling from
Speaker 1: the camera, analyze it, and then helps create an animated
Speaker 1: image that mirrors what you did. The neural engine also
Speaker 1: analyzes visual data for the purposes of augmented reality. That's
Speaker 1: when you overlay digital information on top of a view
Speaker 1: of the physical world around you. So with smartphones like
Speaker 1: the iPhone, it means holding your phone up and looking
Speaker 1: at the world through your phone screen. So the camera
Speaker 1: on your phone is giving you a live video feed
Speaker 1: of whatever you're pointing the phone at it, and then
Speaker 1: you use an augmented reality app, and on top of
Speaker 1: that video image your phone will overlay some sort of
Speaker 1: digital information. Could be a game, it could be information
Speaker 1: about your surroundings. The digital information can appear to be
Speaker 1: anchored to the physical space itself. So you could have
Speaker 1: an augmented reality application that let's you view a virtual
Speaker 1: piece of furniture in your house. And so when you
Speaker 1: hold up the phone, you use the app to place
Speaker 1: a virtual chair, let's say, in a specific location in
Speaker 1: a room, and you can walk around this virtual chair
Speaker 1: holding your phone up, and it looks like the chair
Speaker 1: is actually there, even as your perspective changes. You can
Speaker 1: circle around it and view the chair from all the
Speaker 1: different angles as if it were actually sitting there in
Speaker 1: the room. It's anchored to its place that you've put
Speaker 1: it within the view of the room. The neural engine
Speaker 1: is analyzing all this information that's coming in from the
Speaker 1: camera and helping the app create the image of the chair,
Speaker 1: keeping it the appropriate size and orientation with respect your
Speaker 1: viewing angle. And the neural engine can use this ability
Speaker 1: to help you go through stuff like your photos. Let's
Speaker 1: say you've got an adorable pet, like my doggie Timbolt.
Speaker 1: He is adorable. The iPhone can use its neural engine
Speaker 1: and image recognition algorithms to return the pictures of your
Speaker 1: pet in response to a search query. So my wife,
Speaker 1: who has an iPhone, could do this with our dogs.
Speaker 1: She could search for the word dog in her photo
Speaker 1: app and then she would get countless images of Tibolt.
Speaker 1: And I know it works because she's done it. Apple
Speaker 1: has included access to the neural engine so that app
Speaker 1: developers can actually take advantage of that technology as well.
Speaker 1: They'll doubtlessly create new ways to leverage this tech, so
Speaker 1: we'll have to keep our eyes open to see what
Speaker 1: comes out of it. Neural networks in general are becoming
Speaker 1: increasingly important in machine learning and artificial intelligence, so it's
Speaker 1: likely to grow as a branch of computer science for
Speaker 1: the next several years. And that wraps up this episode.
Speaker 1: If you have suggestions for future episodes of tech Stuff,
Speaker 1: maybe it's a technology, a person in tech company, anything
Speaker 1: like that, Send me an email the addresses tech Stuff
Speaker 1: at how stuff works dot com. You can draw me
Speaker 1: a line on Facebook or Twitter. The handle for both
Speaker 1: of those is tech Stuff H s W. Don't forget,
Speaker 1: we have a merchandise store over at t public dot
Speaker 1: com slash tech stuff. That's T E E public dot
Speaker 1: com slash tech stuff. You can go and get your uh,
Speaker 1: your caption test, the prove you're not a robot sticker
Speaker 1: or T shirt or tote bag or whatever type of
Speaker 1: thing you would like that on. It's pretty cool, So
Speaker 1: go check that out, and don't forget to follow us
Speaker 1: on Instagram and I'll talk to you again really soon.
Speaker 1: For more on this and thousands of other topics, visit
Speaker 1: how staff works dot com ye