Speaker 1: Welcome to Tech Stuff, a production from I Heart Radio.
Speaker 1: Hey there, and welcome to tech Stuff. I'm your host,
Speaker 1: Jonathan Strickland. I'm an executive producer with I Heart Radio,
Speaker 1: and I love all things tech, And uh, you know
Speaker 1: what the today's episode I was gonna I was gonna
Speaker 1: make it a one partner, but it turns out there's
Speaker 1: just way too much stuff, not just about the topic
Speaker 1: at hand, but the various components that make up this
Speaker 1: topic that require me to do more than one. So
Speaker 1: this is gonna likely be a two parter. But today
Speaker 1: I thought we could look back at the development and
Speaker 1: evolution of a famous AI personality. This virtual assistant celebrated
Speaker 1: an anniversary recently, and I must apologize for being a
Speaker 1: couple of days late with this, but this particular servant
Speaker 1: debuted on October fourth, two thousand eleven, technically for the
Speaker 1: second time, but the history of the actual technology dates
Speaker 1: back much further. And of course, I'm talking about Sirie,
Speaker 1: Apple's virtual assistant that can interpret voice commands and return
Speaker 1: results based on them. This is not just some dull
Speaker 1: history lesson, however, Sirie really has an incredible backstory, ranging
Speaker 1: from a science fiction vision of the future to a
Speaker 1: secret project intended to augment the decision making capabilities of
Speaker 1: the United States military. Yeah, Siri had a pretty tough background.
Speaker 1: The story of Sirie is complicated, and not just because
Speaker 1: of the internal history of developing the technology, but also
Speaker 1: because the tool relies on a lot of converging technological
Speaker 1: trend There are elements of voice recognition, UH, speech to text,
Speaker 1: natural language interpretation, and other technologies that fall under the
Speaker 1: very broad umbrella of artificial intelligence. So get settled, it's
Speaker 1: time to talk about Siri. Also, if you're listening to
Speaker 1: this near Apple devices, I apologize because there's a good
Speaker 1: chance those devices might start talking back at me. But
Speaker 1: I refuse to do an episode where I just refer
Speaker 1: to the subject as you know who. You could argue
Speaker 1: that the origins of Siri can be found in a
Speaker 1: promotional video that Apple produced back in nineteen seven to
Speaker 1: show off a concept of an artificially intelligent smart assistant.
Speaker 1: Now that alone is interesting, but what really is amazing
Speaker 1: is that the arbitrary date they chose as the setting
Speaker 1: for this video was two thousand evan, probably September. We
Speaker 1: know that because there is a part within the video
Speaker 1: where a character asks for information that had been published
Speaker 1: five years previously, and the published information had a publication
Speaker 1: date of two thousand six. Now this means that the
Speaker 1: actual debut of Syrie as an Apple product was just
Speaker 1: one month after the fictional events in that video from nine.
Speaker 1: That's just a coincidence, but it's a cool one. The
Speaker 1: Knowledge Navigator video shows a man walking into a study,
Speaker 1: really nice one, and unfolding a tablet style computer device.
Speaker 1: Then he walks off away to stare at stuff as
Speaker 1: a virtual assistant reads off his messages and meetings on
Speaker 1: his calendar. The virtual assistant appears as a video and
Speaker 1: a little window on the screen of the tablet, and
Speaker 1: it's you know, like shot from the shoulders up, kind
Speaker 1: of a the bust of a young man, and the
Speaker 1: video takes up that one little corner of the tablet device.
Speaker 1: So in this visualization, the virtual assistant isn't just a
Speaker 1: disembodied voice. It also has a face. Also, everyone in
Speaker 1: this video is extremely white, which I guess is kind
Speaker 1: of a given for the time period and the people involved,
Speaker 1: but it just comes across as so white. I mean,
Speaker 1: we're doing this with the benefit of the glasses of
Speaker 1: twenty I just wanted to throw that out there anyway.
Speaker 1: The video goes on to have the real life man
Speaker 1: who is a professor in this video, ask his virtual
Speaker 1: assistant to pull up lecture notes uh and unread articles
Speaker 1: that relate back to the lecture he's He's asking for
Speaker 1: a lecture notes of a lecture he gave a year ago.
Speaker 1: He's giving essentially the same lecture now, but he wants
Speaker 1: to update it with the latest information, and he even
Speaker 1: asks the virtual assistant to summarize those unread articles that
Speaker 1: had been published in the year since his last lecture.
Speaker 1: The virtual assistant is thus aggregating information, analyzing that information
Speaker 1: for context, and then delivering summaries, which is that's a
Speaker 1: pretty sophisticated set of artificially intelligent tasks. He also, the
Speaker 1: professor uses the device and virtual assistant to call and
Speaker 1: collaborate with a peer in real time. Now, this was
Speaker 1: not the only video that Apple would produce to showcase
Speaker 1: this kind of general idea, however, arguably it is the
Speaker 1: most famous of those videos. Now, as I said, Knowledge
Speaker 1: Navigator came out of Apple, and Steve Jobs would later
Speaker 1: play a pivotal role in how the company would introduce Sirie,
Speaker 1: but This was not a Steve Jobs project because Jobs
Speaker 1: had been ousted from the company Apple, or he had
Speaker 1: quit in disgust, depending upon which version of the story
Speaker 1: you're listening to. Anyway, he had left a couple of
Speaker 1: years before this video was produced. The Knowledge Navigator was
Speaker 1: something that Apple CEO John Scully had described in a
Speaker 1: book titled Odyssey. Now, of course, in science fiction stories
Speaker 1: we have no shortage of instances where a human is
Speaker 1: interacting with a computer or otherwise artificially intelligent device like
Speaker 1: a robot, but the Knowledge Navigator seemed to lay down
Speaker 1: the foundations toward future products like Siri and the iPad,
Speaker 1: not to mention the potential uses of the Internet, which
Speaker 1: inn was definitely a thing. It existed, but most of
Speaker 1: the mainstream public remained unaware of it because the Worldwide
Speaker 1: Web wouldn't even come along for another few years. However,
Speaker 1: while you can look at this video and say, ah,
Speaker 1: this must be where Apple got that idea, they probably
Speaker 1: got to work right away on Siri, well you'd be
Speaker 1: wrong because the early work, in fact, the vast bulk
Speaker 1: of the work on Syrie to bring it to life,
Speaker 1: didn't start at Apple at all. It didn't involve the company.
Speaker 1: So our story now turns to a very different organization,
Speaker 1: the Defense Advanced Research Projects Agency, better known as DARPA.
Speaker 1: Now this is part of the United States Department of Defense.
Speaker 1: Back in nineteen fifty eight, the then President of the
Speaker 1: United States, Dwight D. Eisenhower, authorized the foundation of this agency,
Speaker 1: though at the time it was called the Advanced Research
Speaker 1: Project Agency or ARPA. Defense would be added later. This
Speaker 1: agency would play a critical role in the evolution of
Speaker 1: technologies in the United States, and the mission of DARPA
Speaker 1: and ARPA before it is quote to make pivotal investments
Speaker 1: and breakthrough technology is for national security end quote, and
Speaker 1: that wording is really precise. It's easy to imagine DARPA
Speaker 1: as being housed in some enormous underground bunker filled with
Speaker 1: scientists who are building out crazy devices like robo scorpions
Speaker 1: or a blender that can also teleport or something. But
Speaker 1: in reality, DARPA is more about funding research than conducting research. Now,
Speaker 1: don't get me wrong, the agency relies heavily on experts
Speaker 1: to evaluate proposals and consider to whom the agency should
Speaker 1: send money. But the purpose of DARPA is to enable
Speaker 1: others to do important work. DARPA has played a huge
Speaker 1: role in countless technological breakthroughs. This way. Much of the
Speaker 1: technologies that would go on to power the Internet started
Speaker 1: with ARPA net, a kind of precursor network to the
Speaker 1: Internet and one that was funded by ARPA. Thus the
Speaker 1: name the DARPA Grand Challenge just helped get self driving
Speaker 1: cars into gear. You know, pun intended. They also created
Speaker 1: difficult scenarios for humanoid robots to go through. That was
Speaker 1: a few years ago and was really cool. The competitions
Speaker 1: DARPA hosts have specific goals and metrics, and that guides
Speaker 1: the designers and engineers who are working on them as
Speaker 1: they build out technologies. It's good to define your goal.
Speaker 1: It really gives you focus when you're trying to develop
Speaker 1: the technology to meet that goal. Winning a challenge is
Speaker 1: a big deal, though the cash prize may not even
Speaker 1: cover the amount of money participants have spent through the
Speaker 1: development of those technologies, and there are entire businesses, or
Speaker 1: at least divisions within businesses that can be borne out
Speaker 1: of these challenges. The Grand Challenges are just one way
Speaker 1: DARPA encourages technological development. Often, the agency will create a
Speaker 1: specific goal such as the design of a robotic exoskeleton
Speaker 1: that can help you know, US soldiers carry heavy loads
Speaker 1: while they are on foot for longer distances, and then
Speaker 1: they'll send out an RFP, which is a request for proposal.
Speaker 1: The agency considers the proposals that it receives from this
Speaker 1: RFP and then decides which, if any, they will accept
Speaker 1: and then fund. Then after a given amount of time.
Speaker 1: You know, it's dependent upon the specific project, we find
Speaker 1: out if anything comes out of it. Sometimes nothing does,
Speaker 1: as some technological problems may prove more challenging than others
Speaker 1: and may require more time to evolve the various technologies
Speaker 1: to make it possible. So it might push the field,
Speaker 1: but you might not have a finished product at the
Speaker 1: end of it. Other times you do get a finished
Speaker 1: product anyway. In two thousand three, a decade and a
Speaker 1: half after the Knowledge Navigator videos came out of Apple,
Speaker 1: DARPA identified a new opportunity, and this was one that
Speaker 1: was borne out of necessity. The challenge was that we
Speaker 1: have access to way more information today than we did
Speaker 1: in the past. So decades ago, military commanders had to
Speaker 1: make decisions based on limited information. They'd rely a great
Speaker 1: deal on their own expertise and experience in order to
Speaker 1: make up for the fact that they only had part
Speaker 1: of the picture. And while a great commander has a
Speaker 1: better chance of making the right call than an inexperienced
Speaker 1: commander would, the limited amount of information could still contribute
Speaker 1: to disaster. You might be the greatest commander of all time,
Speaker 1: but if you're lacking a key part of information, you
Speaker 1: might make a decision that is terrible. So flash forward
Speaker 1: to two thousand three, and now the story had kind
Speaker 1: of flip flopped. Now military commanders would receive more information
Speaker 1: than they could reasonably handle. The challenge now wasn't to
Speaker 1: use intuition to make up for blind spots, but rather,
Speaker 1: how do you synthesize all this information so that you
Speaker 1: can make the right decision. Too much information was proving
Speaker 1: to be kind of as big a problem as too
Speaker 1: little information, at least in some cases, and so DARPA
Speaker 1: wished to fund the development of a smart system that
Speaker 1: could help commanders make sense of all the data coming
Speaker 1: in from day to day. Now, DARPA projects tend to
Speaker 1: be labyrinthian, with lots of bits and pieces and a
Speaker 1: lot of different companies and research labs and more organizations
Speaker 1: might tackle all or part of one of these projects.
Speaker 1: The cognitive computing section of DARPA had a program called
Speaker 1: Perceptive Assistance that Learn or PAL, which seems nice. It
Speaker 1: was this part of the program that would fund the
Speaker 1: development of a virtual cognitive assistant. The amount of funding
Speaker 1: was twenty two million dollars. What a great PAL. The
Speaker 1: organization that landed this deal was s r I International,
Speaker 1: itself an incredibly influential organization. It's a nonprofit scientific research institution.
Speaker 1: Originally it was called the Stanford Research Institute because it
Speaker 1: was established by the trustees of Stanford University back in
Speaker 1: nineteen forty six, though the organization would separate from the
Speaker 1: university formally in the nineteen seventies and become a standalone,
Speaker 1: nonprofit scientific research lab. The organization has played a role
Speaker 1: in advancing materials science, developing liquid crystal displays or l
Speaker 1: c d s, creating telesurgery implementations, and more. And now
Speaker 1: it was going to tackle DARPA's request for a cognitive
Speaker 1: computer assistant. S r I International created a project called
Speaker 1: the Cognitive Assistant that Learns and Organizes or KALO or
Speaker 1: CALO if you prefer. And this appears to be another
Speaker 1: case where they landed upon that acronym first and then
Speaker 1: worked backward, as klo seems to come from the Latin
Speaker 1: word colognists, which means soldiers servant, and I probably mispronounced
Speaker 1: that because even though I was a medievalist, it's almost
Speaker 1: criminal I never took Latin. The concept, however, hearkens back
Speaker 1: to some of what we would see in that Knowledge
Speaker 1: Navigator video from a system that would be able to
Speaker 1: receive and interpret information, presumably from multiple sources, and provide
Speaker 1: a meaningful presentation or even interpretation of that data to humans,
Speaker 1: which is a pretty tall order, and let's break down
Speaker 1: a bit of what an assistant would need to do
Speaker 1: in order to accomplish this. We'll leave help the voice
Speaker 1: activation parts for now, as that would not be absolutely
Speaker 1: critical to make this work. You know, you might have
Speaker 1: a system that gives daily briefings on its own, or
Speaker 1: you might have one that you activate through text commands
Speaker 1: or some other user interface. It wouldn't necessarily have to
Speaker 1: be voice activated. But on the back end, what has
Speaker 1: to happen for this to work well? Presumably such a
Speaker 1: system would need to pull in data from a number
Speaker 1: of disparate sources, so the assistant wouldn't just be reciting
Speaker 1: facts and figures that we're coming from a centralized data server. Instead,
Speaker 1: it might be assimilating data from numerous sources into a
Speaker 1: cohesive presentation. On top of that, the data might be
Speaker 1: in different formats, meaning the system would need to be
Speaker 1: able to analyze the information inside different types of files.
Speaker 1: This isn't an easy thing to do. There's a reason
Speaker 1: we have a lot of specialized programs for working with
Speaker 1: specific types of files. When I put together these podcasts,
Speaker 1: I use a word processor for my notes, and I
Speaker 1: use an audio editing piece of software to record and
Speaker 1: edit the podcasts. Now I need both of those programs
Speaker 1: because neither of them can do the job that the
Speaker 1: other one does. I don't have like a all purpose
Speaker 1: program that does everything. Accessing different file formats, even in
Speaker 1: the same general family of applications is tricky. Beyond that,
Speaker 1: the way information can be presented within each file could
Speaker 1: be very different. It's very possible for us to open
Speaker 1: up multiple spreadsheets and even using the same basic spreadsheet
Speaker 1: program let's just say Excel, It's possible for us to
Speaker 1: open up half a dozen Excel spreadsheets that are all
Speaker 1: presenting the same information but doing so in different ways,
Speaker 1: and that might not be obvious at casual glance. You
Speaker 1: might look at one and the other and not immediately realize, oh,
Speaker 1: these are both saying the same thing. Just think about
Speaker 1: how information could be presented as a table or a
Speaker 1: graph or a chart. The AI assistant would ideally be
Speaker 1: able to access information no matter what format it was in.
Speaker 1: Nomatter are what a version of that format it was in,
Speaker 1: be able to interpret it and then be able to
Speaker 1: deliver a meaningful analysis to the user. Now, as data
Speaker 1: sets grow, this becomes increasingly difficult, which I should point
Speaker 1: out is the whole reason DARPA wanted to fund research
Speaker 1: into this in the first place. Military commanders were faced
Speaker 1: with a growing mountain of information that was increasingly difficult
Speaker 1: to parse. The analysis might also need to incorporate natural
Speaker 1: language recognition features. And I've talked about natural language a
Speaker 1: lot in previous episodes, but if we boil it down,
Speaker 1: it's the language that we humans use to communicate with
Speaker 1: one another. It's our natural way of expressing our thoughts.
Speaker 1: But the way we humans process and communicate information is
Speaker 1: different from how machines do it. We can be subtle.
Speaker 1: We can use stuff like metaphors and allegories and just
Speaker 1: different phrasing. Computers are, you know, a lot more literal. Hey,
Speaker 1: if you break it down to the most basic unit
Speaker 1: of machine information, you know, the bit. You see how
Speaker 1: literal computers are. A bit is either a zero or
Speaker 1: a one, or if you prefer, it's either off and
Speaker 1: on or no and yes. But using lots of bits,
Speaker 1: we can describe information in a way that provides more
Speaker 1: subtlety than just nowhere. Yes. But my point is that
Speaker 1: computers don't naturally process information the way we do, and
Speaker 1: so an entire branch of artificial intelligence called natural language
Speaker 1: processing evolved to create ways for computers to interpret what
Speaker 1: we mean when we express things within natural language. Making
Speaker 1: this more complicated is that, of course, there's no one
Speaker 1: way to say any given thing. We've got lots of
Speaker 1: ways to express the same general thought. And added to that,
Speaker 1: we have lots of different languages. There are around seven
Speaker 1: thousand different langue whig is spoken in the world today,
Speaker 1: though you could probably get away with a couple of
Speaker 1: dozen and cover the vast majority of the world's population
Speaker 1: that way. But these languages have their own vocabularies, their
Speaker 1: own syntaxes, their own expressions. So not only do we
Speaker 1: have multiple ways of saying things within one language, we
Speaker 1: have all these different languages to worry about. If you
Speaker 1: were to send ten people into a room with an
Speaker 1: AI assistant, and those ten people have a task they're
Speaker 1: supposed to perform with the help of this AI assistant,
Speaker 1: odds are no two people are going to go about
Speaker 1: it exactly the same way. And yet a working virtual
Speaker 1: assistant needs to be able to interpret and respond to
Speaker 1: every case and do so reliably on the back end,
Speaker 1: and AI system needs to be able to interpret data
Speaker 1: coming from different sources that may have very different ways
Speaker 1: of expressing similar ideas. This is an enormous task. Now,
Speaker 1: when we come back, I'll talk more about what s
Speaker 1: R I was doing and how the military project would
Speaker 1: evolve ultimately into Apple's Personal Assistant. But first let's take
Speaker 1: a quick break. Now I've only scratched the surface of
Speaker 1: what makes the creation of an AI assistant capable of
Speaker 1: accessing information from numerous sources and making that information useful
Speaker 1: really required. Let's talk a bit about the parameters of
Speaker 1: this project itself. So if you remember I said that
Speaker 1: the deal was initially for twenty two million dollars, and
Speaker 1: that would end up funding the creation of a five
Speaker 1: hundred person project, and the project spanned five years initially
Speaker 1: to investigate the possibility of building out such an AI system.
Speaker 1: Over time, more money would end up going into the
Speaker 1: research system, and it totaled around a hundred fifty million
Speaker 1: dollars by the end of the produc inject. The lab
Speaker 1: where it all went down would receive the charming nickname
Speaker 1: nerd City. A large part of the project focused on
Speaker 1: creating a program that could learn a user's behaviors. So
Speaker 1: not only could this personal assistant respond to what you
Speaker 1: were asking, it would gradually learn the way you behaved
Speaker 1: and it would adapt to you to work more effectively.
Speaker 1: Now this comes into the arena of pattern recognition. We
Speaker 1: humans are pretty darn good at recognizing patterns. In fact,
Speaker 1: we're so good that sometimes we will quote unquote recognize
Speaker 1: a pattern even when there isn't a pattern there. In
Speaker 1: some cases, this can come across as charming, such as
Speaker 1: when we see a face in a cloud, right, that's
Speaker 1: not really a pattern there. We're recognizing a pattern where
Speaker 1: none really exists. It's all based on our perspective in
Speaker 1: our imaginations. Now, in other cases, it's not so charming.
Speaker 1: It can actually lead to faulty reasoning. So I'm going
Speaker 1: to give you a very basic example that I hear
Speaker 1: all the time, particularly now that we're in October and
Speaker 1: there's some full moon weirdness going on. So there's a
Speaker 1: fairly widespread belief that there's a connection between full moons
Speaker 1: and an increase in the number of medical emergencies that happened.
Speaker 1: Generally speaking, that people act irresponsibly during a full moon,
Speaker 1: and that often results in injury, which means greater activity
Speaker 1: at hospitals. Now, this belief is most likely due to
Speaker 1: confirmation bias. That is, we already have a belief in place,
Speaker 1: and the belief is that full moons lead to more
Speaker 1: accidents because of people acting irresponsibly. That is what we believe.
Speaker 1: It doesn't have evidence yet, and then when things do
Speaker 1: get busy at a hospital and there happens to be
Speaker 1: a full moon, we register that as evidence for our belief. Aha,
Speaker 1: says the mistaken person. The full moon explains it. However,
Speaker 1: on nights when it is busy but there is no
Speaker 1: full moon, there's no hit, no one, no one takes
Speaker 1: notice of how odd you know, it's crazy busy, but
Speaker 1: there's no full moon tonight. We don't do that. Likewise,
Speaker 1: if it happens to not be busy but there's a
Speaker 1: full moon, you're also not likely to notice. You're not
Speaker 1: likely to say, like hunt, it's not very busy tonight,
Speaker 1: but there's a full moon out. So it's only when
Speaker 1: you have the full moon and the busy hospital where
Speaker 1: the evidence appears to support your belief and confirm your bias.
Speaker 1: But in truth, when you take a step back and
Speaker 1: you do an objective study and you look at the
Speaker 1: times when a hospital is busy, and you look at
Speaker 1: when there was a full moon, and you look to
Speaker 1: see if there's any correlation, it falls apart. Now I
Speaker 1: got a little off track there, But the point I
Speaker 1: wanted to make is that we humans are biologically attuned
Speaker 1: to recognizing patterns. It's very likely that pattern recognition is
Speaker 1: one of the traits that really helped us survive thousands
Speaker 1: of years ago, which is why it's so intrinsic in
Speaker 1: the human experience. But building programs, computer systems that are
Speaker 1: capable of identifying patterns and separating out what is signal
Speaker 1: versus what is noise is its own really big challenge.
Speaker 1: S r I was hoping to create a program that
Speaker 1: could look for patterns and user behavior in order to
Speaker 1: respond with greater precision and accuracy to user requests and
Speaker 1: ultimately to anticipate future requests. Now we see the sort
Speaker 1: of pattern recognition and response in lots of technology today.
Speaker 1: There are several smart thermostats on the market right now,
Speaker 1: for example, that can track when you tend to raise
Speaker 1: or lower the temperature in your home, and after a while,
Speaker 1: the thermostat learns that, hey, maybe you like it nice
Speaker 1: and chilly at night, but you want it to be
Speaker 1: warm and toasty in the morning, and so the thermostat
Speaker 1: begins to adjust itself in preparation for that based on
Speaker 1: your previous behaviors. Now that is a very simple example.
Speaker 1: Extrapolate that out and you begin to imagine a technology
Speaker 1: that is anticipating what you need or want, perhaps before
Speaker 1: you're even aware of it yourself, which can get kind
Speaker 1: of creepy but also sort of magical. But in truth,
Speaker 1: it's because this system is detecting patterns that we aren't
Speaker 1: even able to recognize ourselves. The danger there, of course,
Speaker 1: is that the systems can sometimes mistakenly identify a pattern
Speaker 1: when in fact there's not really a pattern there. Very
Speaker 1: similar to the case I was explaining about with the
Speaker 1: full moon and the busy hospital. Even computer systems can
Speaker 1: make those sort of mistakes, and depending upon the implementation,
Speaker 1: that can be a real problem. But that's a that's
Speaker 1: an issue for a different podcast. Now. When it comes
Speaker 1: to humans, pattern recognition is so ingrained in most of
Speaker 1: us that it can actually be kind of hard to explain.
Speaker 1: You notice, when something happens, and if that same thing
Speaker 1: happens later with the same general results as the first time,
Speaker 1: it reinforces your first perception of that thing, and if
Speaker 1: it happens over and over, their brain essentially comes to
Speaker 1: understand that when I see X happen, I can expect
Speaker 1: why to follow, and from that you might eventually realize
Speaker 1: that there are other correlating factors that may or may
Speaker 1: not be present. When this goes on. With computers, the
Speaker 1: goal is to create systems that can analyze input, whether
Speaker 1: that input is an image file or typed text or
Speaker 1: spoken words or whatever, and it first has to interpret
Speaker 1: that input, has to identify it and figure out the
Speaker 1: defining features and attributes of that input, then compare that
Speaker 1: against known patterns to see if the input matches or
Speaker 1: doesn't match those patterns. And in a way, you can
Speaker 1: think of this as a computer system receiving input and
Speaker 1: asking the question have I seen this before? And if so,
Speaker 1: what is the correct response? If the input matches no pattern,
Speaker 1: the system then has to have the correct response for that.
Speaker 1: So a very simple example might just be a failed state,
Speaker 1: in which case the virtual assistant might reply with something
Speaker 1: like I'm sorry, I don't know how to do that yet,
Speaker 1: or something along those lines. Now, remember earlier I mentioned
Speaker 1: that we humans have a lot of different ways to
Speaker 1: say the same general thing. For example, with my smart speaker,
Speaker 1: I might ask it to turn the lights on full,
Speaker 1: meaning I want them to be all the way up.
Speaker 1: I might say make the lights. I might just say
Speaker 1: make it brighter. And the system has to take this input,
Speaker 1: analyze it, and make a statistical determination to guess at
Speaker 1: what is that I actually want to have happen. I
Speaker 1: say guess because in each case we're really looking at
Speaker 1: a system that has multiple options when it comes to
Speaker 1: a response, and each option gets a probability assigned to
Speaker 1: it based on how closely that option matches with the input,
Speaker 1: So I might say make it brighter, and the underlying
Speaker 1: system recognizes that there's a n chance I mean, increase
Speaker 1: the brightness of the lights of the room, my men,
Speaker 1: and the system has determined that that's the most probable answer. Right,
Speaker 1: it's probably correct, so it goes with that, but still
Speaker 1: kind of a guess. Now, there are a lot of
Speaker 1: different ways to go about doing this, but the one
Speaker 1: you hear about a lot would be artificial neural networks.
Speaker 1: I've talked a lot about these in recent episodes, so
Speaker 1: we'll just give kind of the quick overview. So you've
Speaker 1: got a computer system has artificial neurons. These are called nodes,
Speaker 1: and the job of a node is to accept incoming
Speaker 1: input from two or more sources. The node is then
Speaker 1: to perform an operation on those inputs, and then it
Speaker 1: generates an output, which it then passes on to other
Speaker 1: nodes further in the system. You can think of the
Speaker 1: nodes as existing in a series of levels, with the
Speaker 1: top level being where input comes in and the bottom
Speaker 1: level being where the ultimate output comes out. So the
Speaker 1: nodes are level down except incoming inputs then perform other
Speaker 1: operations on them and pass it further down the chain
Speaker 1: and so on until ultimately you get an output or response.
Speaker 1: Now that's a gross oversimplification of what's going on, but
Speaker 1: generally you get the idea of the process. Now, let's
Speaker 1: complicate things a little bit to get these sort of
Speaker 1: neural networks to generate the results you want. One thing
Speaker 1: you can do is mess with how each node values
Speaker 1: or ways each of the inputs coming into that node.
Speaker 1: So I'm going to use some names human names for
Speaker 1: nodes here just to make things easier to understand. Let's
Speaker 1: say we've got a node named Billy. Billy is on
Speaker 1: the second layer of nodes, so it's one layer down
Speaker 1: from where direct input comes into the system. So there
Speaker 1: are nodes above Billy that are sending information to Billy.
Speaker 1: We'll say that the two nodes that give Billy information
Speaker 1: are named Sue and Jim Bob. Sue and Jim Bob
Speaker 1: send Billy information, and it's Billy's job to determine what
Speaker 1: further information to send down the pipeline. Like I need
Speaker 1: to do an operation based on this bits of these
Speaker 1: bits of information that are coming to me, and then
Speaker 1: I have to come up with a result. Only Billy
Speaker 1: has been told that Sue's information tends to be a
Speaker 1: little more important than Jimbob's information is, and so if
Speaker 1: there's a question as to what to do, it's better
Speaker 1: to lean more on sue use information than on Jimbob's information.
Speaker 1: We would call this waiting as n W E I
Speaker 1: G H T I n G. Computer scientists wait the
Speaker 1: inputs going into nodes in order to train a system
Speaker 1: to generate the results to the scientists want. One way
Speaker 1: to do this is through a process called back propagation.
Speaker 1: Back propagation is when you know what result you want
Speaker 1: the system to arrive at. So let's use the classic
Speaker 1: example of identifying pictures that have cats in them. As
Speaker 1: a human, you can quickly determine if a photo has
Speaker 1: a cat in it or not. You'll spot it right away.
Speaker 1: So you feed a picture through this system and you
Speaker 1: wait for the system to tell you if yes, there's
Speaker 1: a kitty cat in the picture or no. The images
Speaker 1: cat free. And let's say that the picture you fed
Speaker 1: to the system in fact does have a cat in it.
Speaker 1: You can see it, but when you feed it through
Speaker 1: the system, the system fail is to find the cat
Speaker 1: and says nope, there's no cat here. Well, you know
Speaker 1: that the system got it wrong. So what you might
Speaker 1: do as a computer scientist is you look at that
Speaker 1: final level of nodes right at the output level to
Speaker 1: see which factors led those nodes to come to the
Speaker 1: conclusion that there was no cat in the photo. You
Speaker 1: then look at the inputs that are coming into those
Speaker 1: nodes and you see how they are weighted, and you
Speaker 1: change the weights of those inputs in order to force
Speaker 1: that last level of nodes to say, oh, no, there
Speaker 1: definitely is a cat here. And so on. You move
Speaker 1: up from the output level and you go up level
Speaker 1: by level, tweaking the waitings of incoming data so that
Speaker 1: the system is tweaked to more accurately determined if a
Speaker 1: photo has a cat in it. Now, this takes a
Speaker 1: lot of work, and it also means using huge data sets.
Speaker 1: You know, you're feeding hundreds of thousands or millions of images,
Speaker 1: so of them with cats, some of them without, and
Speaker 1: training the system over and over again to train it
Speaker 1: before you start feeding it brand new images to see
Speaker 1: if it still works. And this can be a laborious
Speaker 1: process to train a machine learning system, but the result
Speaker 1: is that you end up with a system that hopefully
Speaker 1: is pretty accurate a doing whatever it was you were
Speaker 1: training it to do, you know, like recognized cats. But
Speaker 1: that's just one approach to machine learning. There are others.
Speaker 1: Some like the version I just described, fall into a
Speaker 1: broad category called supervised learning. Others are in unsupervised learning.
Speaker 1: In fact, Kalo was largely built through unsupervised learning, meaning
Speaker 1: the machine had to train itself as it performed tasks
Speaker 1: using inputs that hadn't been curated specifically for training purposes.
Speaker 1: It's just an enormous amount of information coming in that
Speaker 1: the system has to process. So, in other words, for Kalo,
Speaker 1: the system wasn't dealing with like a stack of a
Speaker 1: million photos, seventy of which had cats and which didn't.
Speaker 1: Kayla was working with real world information and attempting to
Speaker 1: suss out what to do with it in real time. Now,
Speaker 1: to go into how unsupervised machine learning works would require
Speaker 1: a full episode on its own, but it is a
Speaker 1: fascinating and complicated subject, so I probably will tackle it
Speaker 1: at some point. I'm just gonna spare you guys for
Speaker 1: right now. The real point I'm making is that s
Speaker 1: RI I International spent years building out systems that could
Speaker 1: do a wide range of tasks based on inputs. Pattern
Speaker 1: recognition was actually just one relatively small piece of that.
Speaker 1: Creating an ability to pull data from different sources in
Speaker 1: a meaningful way is its own incredibly challenging problem, as
Speaker 1: I alluded to earlier, particularly as the number of sources
Speaker 1: you're pulling from and the variety of formats the data
Speaker 1: is in begins to increase, it becomes easier for the
Speaker 1: system to make mistakes as you throw more variety at it,
Speaker 1: and it requires a lot of refinement. Frankly, it's actually
Speaker 1: a task that's so big I have trouble grasping it.
Speaker 1: The Kalo project became the largest AI program in history
Speaker 1: up to that point. It was an incredible achievement. It
Speaker 1: brought together different disciplines of artificial intelligence into a cohesive
Speaker 1: project with a solid goal. By the two thousand's, artificial
Speaker 1: intelligence was a sprawling collection of computer science disciplines, each
Speaker 1: with incredible depth to them. So you might find an
Speaker 1: expert in one field of AI who would have little
Speaker 1: to no experience with another branch under the same general
Speaker 1: discipline of artificial intelligence. There was a prevailing feeling that
Speaker 1: the various branches of AI had each become so complex
Speaker 1: they would never work together. The Kalo project proved that wrong.
Speaker 1: When we come back, i'll explain how part of this
Speaker 1: military project would break away to become the virtual assistant,
Speaker 1: ultimately finding its way onto iOS devices. But first let's
Speaker 1: take another quick break. Adam Chair, whose name I'm likely mispronouncing,
Speaker 1: and I apologize, but he was an engineer at s
Speaker 1: r I working on Kalo, and he worked with a
Speaker 1: team that had the daunting task of assimilating the work
Speaker 1: that was being done by twenties seven different engineering teams
Speaker 1: into a cohesive virtual assistant. So, as I mentioned just
Speaker 1: before the break, the disciplines of AI had each gotten
Speaker 1: very deep, very broad, and required a lot of specialization.
Speaker 1: So you have these different engineering teams working within various disciplines,
Speaker 1: and it was chairs team that needed to bring all
Speaker 1: these together and make it into a working, coherent hole.
Speaker 1: The results were really phenomenal. Now I'll give you a
Speaker 1: hypothetical use for Kalo. Let's say that you've got a
Speaker 1: project team and there are ten people on your team,
Speaker 1: including you, and let's say there's a meeting that's on
Speaker 1: the books for tomorrow morning at a particular conference room,
Speaker 1: and it's supposed to be a status update meeting for
Speaker 1: the project. It turns out that two out of the
Speaker 1: ten people on your team are no longer able to
Speaker 1: make the meeting due to last minute high priority conflicts,
Speaker 1: so they've had to cancel out of the meeting. KALO
Speaker 1: would be able to detect the change in status of
Speaker 1: those two people and say, all right, these two are
Speaker 1: no longer going to the meeting. Then KALO could determine
Speaker 1: how important those two people were to the overall team,
Speaker 1: essentially saying what are their roles? What what role are
Speaker 1: they performing within the context of this team, and is
Speaker 1: it a critical role for this meeting. It can also
Speaker 1: look at the importance of the meeting itself, like, oh, well,
Speaker 1: this is a status update, so it's really just to
Speaker 1: keep the team, you know, informed of what's going on.
Speaker 1: It's not a mission critical type of meeting. It could
Speaker 1: take all that into account. Then KALO can make a
Speaker 1: determination on its own whether or not it should keep
Speaker 1: the meeting in place and go ahead just without those
Speaker 1: two people and maybe just send updates to those two people,
Speaker 1: or to cancel the meeting entirely notifying all the participants
Speaker 1: about it. Then look at the different calendars of those participants,
Speaker 1: book a new meeting, including securing a space for that
Speaker 1: meeting and sending out new invites. It would even be
Speaker 1: able to look at the purpose of the meeting and
Speaker 1: flag information that's relevant to that meeting, essentially creating a
Speaker 1: sort of meeting dossier on demand. So it's really, you know,
Speaker 1: incredible sophisticated stuff. Now, that was the fully fledged Kalo,
Speaker 1: but an offshoot of this project, or maybe it's it's
Speaker 1: better to say it was a smaller sister project that
Speaker 1: existed at the same time it launched in two thousand three.
Speaker 1: Along with Kalo. This other one was called Vanguard, at
Speaker 1: least within s r I, and it was taking a
Speaker 1: more scaled down approach of building out an assistant and
Speaker 1: looking at how it could be useful on mobile devices. Now, again,
Speaker 1: this was in two thousand three, before smartphones would really
Speaker 1: become a mainstream product because Apple wouldn't even introduce the
Speaker 1: iPhone until two thousand seven. But s r I was
Speaker 1: working on implementations of a more limited virtual assistant and
Speaker 1: then showing it off to companies like Motorola. One person
Speaker 1: at Motorola who was really impressed with this work was
Speaker 1: a guy named Dog Kittlaus. Kittlaus attempted to convince his
Speaker 1: superiors that Motorola that Vanguard was a really important piece
Speaker 1: of work, but he didn't find any real interest over
Speaker 1: at Motorola, so he did something fairly brazen. In two
Speaker 1: thousand seven, he quit his job at Motorole and he
Speaker 1: joined SRI International with the intent of exploring ways to
Speaker 1: spin off a new business that would develop an implementation
Speaker 1: of the Kalo Vanguard virtual assistant, but for the consumer market.
Speaker 1: The result would be a new company called Sirie s
Speaker 1: I r I, which is kind of the way you
Speaker 1: would say s r I if you were trying to
Speaker 1: pronounce it as if it were an acronym as opposed
Speaker 1: to an initialism. Adam Chair, after some convincing from Kittlaus,
Speaker 1: joined the venture as the vice president of Engineering. Kit
Speaker 1: Loss would be the CEO. Tom Gruber, who had studied
Speaker 1: computer science at Stanford and then pioneered work in various
Speaker 1: fields of artificial intelligence, would become the chief technology officer
Speaker 1: for the company. Interestingly, the Serie team didn't initially call
Speaker 1: their own virtual assistant project SIRIE. Instead, the new spinoff company,
Speaker 1: SIRI would call their virtual Assistant how H a l
Speaker 1: after the AI system in the book and film two
Speaker 1: thousand one. They did take an extra step to reassure
Speaker 1: people that this time HOW would behave itself. So, if
Speaker 1: you're not familiar with the story of two thousand one,
Speaker 1: the artificially Intelligent computer system HOW begins to malfunction and
Speaker 1: begins to interpret its mission in such a way that
Speaker 1: it compels it to start killing off the crew inside
Speaker 1: a spacecraft, kind of a worst case scenario with AI.
Speaker 1: While SIRIE began to get off the ground, it was
Speaker 1: licensing technologies from s r I to power the virtual assistant,
Speaker 1: and it also began to hire the talent needed to
Speaker 1: bring this idea to life. At the same time, Apple
Speaker 1: was pushing the smartphone industry into the limelight with the
Speaker 1: introduction of the first iPhone. This was all happening at
Speaker 1: two thousand seven. It was clear that the push for
Speaker 1: a virtual assistant was coming at just the right time,
Speaker 1: as Apple's implementation of smartphone technology was a grand slam
Speaker 1: home run. To use a sports analogy, it soon became
Speaker 1: obvious that the future of computing was going to be,
Speaker 1: at least in large part mobile That in turn opened
Speaker 1: up opportunities to create new ways to interact with mobile
Speaker 1: devices in order to do the stuff we needed to
Speaker 1: do now. It's obvious to say this, but mobile devices
Speaker 1: have a very different user interface from your typical computer.
Speaker 1: Interacting with a handheld computer by tapping on a screen
Speaker 1: or talking to it creates different opportunities for crafting experiences
Speaker 1: than someone sitting down to a computer with a keyboard
Speaker 1: and mouse. There's a potential need for a voice activated
Speaker 1: personal assistant that could help you carry out your tasks,
Speaker 1: particularly ones that might need multiple steps. Sirie the Company
Speaker 1: came along just as the need for Sirie the App
Speaker 1: was beginning to take shape, so it was the right
Speaker 1: place at the right time. In two thousand seven, Apple
Speaker 1: had not yet opened up the opportunity for independent app
Speaker 1: developers to submit apps for the iPhone. That wouldn't actually
Speaker 1: happen until July tenth, two thou eight, essentially a year
Speaker 1: after the iPhone had debuted. The Serie team was still
Speaker 1: hard at work building out the virtual assistant app they
Speaker 1: had in mind in two thousand and eight, while they
Speaker 1: were licensing technology from s r I International, you know,
Speaker 1: from the Vanguard and the the Kalo projects, they still
Speaker 1: had to build out the systems that would actually power
Speaker 1: Syria on the back end. Generally speaking, their approach was
Speaker 1: to create an app where a person could ask Syria
Speaker 1: question and the app would record that request as a
Speaker 1: little audio file, send that audio file to a server
Speaker 1: and a data center, and the first step then would
Speaker 1: be to transcribe the audio file into text, so we're
Speaker 1: talking about speech to text here. Then the system would
Speaker 1: need to parse the request. What is actually being asked here?
Speaker 1: What is the command or request saying. Now, in some systems,
Speaker 1: a computer will break down a sentence into its various components,
Speaker 1: you know, a subject, verb, and object, and then try
Speaker 1: to figure out what is actually being set. Adam Chair
Speaker 1: took a different approach with his team. They taught their
Speaker 1: system the meaning of real world objects. So, rather than
Speaker 1: trying to parse out what a sentence meant by first
Speaker 1: figuring out what's the subject, what's the verb, and what's
Speaker 1: the object that the subject is acting upon, Siri started
Speaker 1: off by looking at real world concepts within the request.
Speaker 1: Siri would then map the request against a list of
Speaker 1: possible responses and then employ that statistical probability model that
Speaker 1: I mentioned earlier. What are the odds that someone was
Speaker 1: asking for dire actions to an Italian restaurant versus asking
Speaker 1: Siri to provide a recipe for an Italian dish, for example.
Speaker 1: So if I activate my virtual assistant and say I
Speaker 1: want linguini, that's a pretty broad thing to say, right.
Speaker 1: The app has to guess at whether I mean I
Speaker 1: want to go someplace that serves linguini or I want
Speaker 1: to make it myself. Now, my personal app would have
Speaker 1: learned by my behaviors that I am very lazy and
Speaker 1: would realize that I am actually asking for someone to
Speaker 1: bring me linguini. So there's no doubt Siri would return
Speaker 1: results of Italian restaurants that deliver as a result from
Speaker 1: my request. And keep in mind, Sirie was intended to
Speaker 1: learn from user behaviors and a tune itself to those
Speaker 1: behaviors over time. Beyond that, Siri would pull information from
Speaker 1: multiple sources to provide results. So if I asked about
Speaker 1: a restaurant, Siri would provide all sorts of data about
Speaker 1: the restaurant, from user reviews, to directions to the restaurant,
Speaker 1: to menu items to what price range I might expect
Speaker 1: at that place. Syria could also tap into other stuff
Speaker 1: like the phone's location, and thus give relevant answers based
Speaker 1: on my location, so I wouldn't have to worry about
Speaker 1: getting irrelevant search results if I happened to be far
Speaker 1: from home, right Siri wouldn't suggest that I go and
Speaker 1: get food from a place that's right down the street
Speaker 1: from my house in Atlanta while I happen to be
Speaker 1: in New York City, for example. The team also gave
Speaker 1: Sirie a bit of an attitude. Siri could be sassy
Speaker 1: and had a bit of a potty mouth. In fact,
Speaker 1: Siri would occasionally drop an F bomb here or there now.
Speaker 1: According to Kittlaus, the goal was eventually to offer extensions
Speaker 1: to Siri so that end users could kind of pick
Speaker 1: the apps personality. Maybe you want a no nonsense virtual
Speaker 1: assistant that just provides the information you need and that's it.
Speaker 1: Maybe you wanted more of a good fee sidekick, or
Speaker 1: maybe you wanted a virtual assistant who could give you
Speaker 1: some serious attitude on occasion. The goal down the line
Speaker 1: was to create options for people to kind of shape
Speaker 1: their experience, but that would end up on the cutting
Speaker 1: room floor due to a very big reason. The serie
Speaker 1: app made its debut in the iPhone app store. In January,
Speaker 1: three weeks after it debuted, Kit Loss received a phone
Speaker 1: call from an unlisted number, a call that he almost
Speaker 1: didn't even answer, but when he did answer, the person
Speaker 1: on the other end of the call happened to be
Speaker 1: Steve Jobs, the CEO of Apple. Jobs was over the
Speaker 1: moon about Sirie and wanted to meet with kit Lost
Speaker 1: to discover some pretty enormous options, the biggest one being
Speaker 1: that Apple itself would acquire Sirie. Now. At the time Sirie,
Speaker 1: the company was working on developing a version of the
Speaker 1: app for Android phones, having reached a deal with varies
Speaker 1: in to create a version of Sirie that could be
Speaker 1: the default app on all Verizon Android phones moving forward.
Speaker 1: The Apple deal would ultimately derail that agreement, as Jobs
Speaker 1: was insistent that Sirie be an Apple exclusive. In fact,
Speaker 1: when Apple would introduce Sirie on October fourth, two thousand eleven,
Speaker 1: it seemed like it was being presented as a purely
Speaker 1: Apple product, that it didn't have a life outside of
Speaker 1: Apple at all. It came across as it just being
Speaker 1: Apple all along. And of course, the day after Apple
Speaker 1: would introduce SyRI to the public, Steve Jobs himself passed away.
Speaker 1: October five, two thousand eleven. But that part of the
Speaker 1: story will have to wait for part two because, as
Speaker 1: I said, this is going longer than I anticipated. So
Speaker 1: in our next episode we'll pick up probably actually a
Speaker 1: little earlier than where I'm leaving off here, actually, because
Speaker 1: there's still some other details we should talk about as
Speaker 1: far as how Siri works and the actual arrangement of
Speaker 1: Apple's acquisition, and then we'll talk about how the app
Speaker 1: has evolved and changed under Apple's ownership, and will also explore,
Speaker 1: you know, a little bit about series distant cousins like
Speaker 1: Alexa and Google Assistant and others, because all of these
Speaker 1: work in similar ways, though they have their own specific
Speaker 1: processes to handle requests, and so if you do an
Speaker 1: Apples to Apples comparison, it does break down ultimately once
Speaker 1: you start getting down to how things are working in
Speaker 1: detail on the back end. So I won't go into
Speaker 1: full mode on those because it would require multiple episodes
Speaker 1: on that. But we will talk more about Siri and
Speaker 1: what has happened in the years since its acquisition in
Speaker 1: our next episode. If you guys have suggestions for future
Speaker 1: topics I should tackle on tech stuff, let me know
Speaker 1: the best way to do that is to reach out
Speaker 1: on Twitter. The handle we use is text stuff H
Speaker 1: s W and I'll talk to you again really soon.
Speaker 1: Text Stuff is an I Heart Radio production. For more
Speaker 1: podcasts from my heart Radio, visit the i heart Radio app,
Speaker 1: Apple Podcasts, or wherever you listen to your favorite shows.