Speaker 1: It was a cold, windy day in January nineteen seventy
Speaker 1: nine when the robots took their first human life. It
Speaker 1: happened in Flat Rock, Michigan, about twenty miles down the
Speaker 1: interstate from Detroit, at the Ford plant. There. Robert Williams
Speaker 1: was twenty five. He was a Ford worker and one
Speaker 1: of the people who oversaw the robotic arm that was
Speaker 1: designed to retrieve parts from bins in the storage room
Speaker 1: and place amended carts that carried them out to the
Speaker 1: humans on the assembly line. But the robot was malfunctioning
Speaker 1: that day, and aware of the slowdown it was creating
Speaker 1: on the line, Robert Williams went to grab the parts himself.
Speaker 1: While Williams was reaching into a bin, the one ton
Speaker 1: robotic arms swung into that same bin. The robot didn't
Speaker 1: have any alarms to warn Williams it was nearby. It
Speaker 1: didn't have any censors to tell it a human was
Speaker 1: in its path. It only had the intelligence to execute
Speaker 1: its commands to retrieve and place auto parts. The robots
Speaker 1: struck William's head with such force it killed him instantly.
Speaker 1: It was thirty minutes before anyone came to look for
Speaker 1: Robert Williams. During that time, the robot continued to slowly
Speaker 1: do its work while Williams lay dead on the parts
Speaker 1: room floor. The death of Robert Williams happened during a
Speaker 1: weird time for AI. The public at large still felt
Speaker 1: unsure about the machines that they were increasingly living and
Speaker 1: working among. Hollywood could still rely on the trope of
Speaker 1: our machines running a muck and ruining the future for humanity.
Speaker 1: Both war Games and The Terminator would be released in
Speaker 1: the next five years. But within the field that was
Speaker 1: trying to actually produce those machines that may or may
Speaker 1: not run amuck in the future, there was a growing
Speaker 1: crisis of confidence. For decades, AI researchers had been making
Speaker 1: grand but fruitless public pronouncements about advancements in the field.
Speaker 1: As early as nineteen fifty six, when a group of
Speaker 1: artificial intelligence pioneers met at Dartmouth, the researchers wrote that
Speaker 1: they expected to have all the major kinks worked out
Speaker 1: of AI by the end of this semester, and the
Speaker 1: predictions kept up from there. So you can understand how
Speaker 1: the public came to believe that robots that were smarter
Speaker 1: than humans were just around the corner, but AI never
Speaker 1: managed to produce the results expected from it, and by
Speaker 1: the late nineteen eighties the field retreated into itself. Funding
Speaker 1: dried up, candidates looked for careers in other fields. The
Speaker 1: research was pushed to the fringe. It was an AI
Speaker 1: winter the public moved on to. The terminator was replaced
Speaker 1: by Johnny five in the film Maximum over Drive. Our
Speaker 1: machines turn against us, but it's the result of a
Speaker 1: magical comet, not from the work of scientists. We lost
Speaker 1: our fear of our machines. Recently, quietly, the field of
Speaker 1: AI has begun to move past the old barriers that
Speaker 1: once held it back, garner the grand pronouncements. Today's researchers,
Speaker 1: tempered by the memory of their predecessor's public failures, are
Speaker 1: more likely to downplay progress in the field, and from
Speaker 1: the new AI we have a clearer picture of the
Speaker 1: existential risks that poses than we ever had before. The
Speaker 1: AI we may face in the future will be subtler
Speaker 1: and vastly more difficult to overcome than a cyborg with
Speaker 1: a shotgun. In hindsight, it was the path toward machine
Speaker 1: learning that the early AI researchers chose that led them
Speaker 1: to a dead end. Let's say you want to build
Speaker 1: a machine that sorts red balls from green balls. First
Speaker 1: you have to explain what a ball is. Well first, really,
Speaker 1: you have to have a general understanding of what makes
Speaker 1: a ball a ball, which is easier said than done.
Speaker 1: Try explaining a ball to someone that doesn't use terms
Speaker 1: that one would have to already be familiar with, like sphere,
Speaker 1: around or circle. Once you have that figured out, you
Speaker 1: then have to translate that logic and those rules into code,
Speaker 1: the language of machines, ones and zeros if some. Then
Speaker 1: now you have to do the same thing with the
Speaker 1: incept of the color red and then the color green,
Speaker 1: so that your machine can distinguish between red balls and
Speaker 1: green balls. And let's not forget that you have to
Speaker 1: program it to distinguish in the first place. It's not
Speaker 1: like your machine comes preloaded with distinguishing software. You have
Speaker 1: to write that too. Since you're making a sorting machine,
Speaker 1: you have to write code that shows it how to
Speaker 1: manipulate another machine, your robot sorder to let it touch
Speaker 1: the physical world. And once you have your machine up
Speaker 1: and running and working smoothly separating red balls from green ones,
Speaker 1: what happens when a yellow ball shows up. Things like
Speaker 1: this do happen from time to time in real life.
Speaker 1: What does your machine do? Then? Despite the incredible technical
Speaker 1: difficulties that faced the field of artificial intelligence did have
Speaker 1: a lot of success at teaching machines that could think
Speaker 1: very well within narrow domains. One program called deep Blue
Speaker 1: beat the reigning human chess champion Garry Kasprov in six
Speaker 1: games at a match in to be certain, the intellectual
Speaker 1: abilities required by chess are of bad improvement over those
Speaker 1: required to select a red ball from a green one.
Speaker 1: But both of those programs share a common problem. They
Speaker 1: only know how to do one thing. The goal of
Speaker 1: AI has never been to just build machines that can
Speaker 1: beat humans at chess. In fact, chess has always been
Speaker 1: used as a way to test new models of machine learning,
Speaker 1: and while there is definitely used for a machine that
Speaker 1: can sort one thing from another, the ultimate goal of
Speaker 1: AI is to build a machine with general intelligence. Like
Speaker 1: a human has to be good at chess and only chess,
Speaker 1: is to be a machine to be good at chess,
Speaker 1: good at doing taxes, good at speaking Spanish, good at
Speaker 1: picking out apple pie recipes. This begins to approach the
Speaker 1: ballpark of being human. So this is what early AI
Speaker 1: research ran up against. Once you've taught the AI how
Speaker 1: to play chess, you still have to teach it what
Speaker 1: constitutes a good apple pie recipe, and then tax laws
Speaker 1: in Spanish, and then you still have the rest of
Speaker 1: the world to teach it all the objects, rules, and
Speaker 1: concepts that make up the fabric of our reality. And
Speaker 1: for each of those, you have to break it down
Speaker 1: to its logical essence and then translate that essence into code,
Speaker 1: and then work through all the kinks. And then once
Speaker 1: you've done this, once you've taught it absolutely everything there
Speaker 1: is in the universe, you have to teach the AI
Speaker 1: all the ways these things interconnect. Just the thought of
Speaker 1: this as overwhelming. Current researchers in the field of a
Speaker 1: I refer to the work their predecessors did as go fi,
Speaker 1: good old fashioned AI. It's meant to evoke images of
Speaker 1: malfunctioning robots, their heads spinning wildly, is smoked pores from them.
Speaker 1: It's meant to establish a line between the AI research
Speaker 1: of yesterday and the AI research of today. But yesterday
Speaker 1: wasn't so long ago. Probably the brightest line dividing old
Speaker 1: and new in the field of AI comes around two
Speaker 1: thousand six. For about a decade prior to that, Jeoffrey Hinton,
Speaker 1: one of the Skeleton crew of researchers working through the
Speaker 1: AI Winter, had been tinkering with the artificial neural networks,
Speaker 1: an old AI concept first developed in the nineteen forties.
Speaker 1: The neural nets didn't work back then, and they didn't
Speaker 1: work terribly much better in the nineties, But by the
Speaker 1: mid two thousands, the Internet had become a substantial force
Speaker 1: in developing this type of AI. All of those images
Speaker 1: uploaded to Google, all that video uploaded to YouTube, the
Speaker 1: Internet became a vast repository of data that could be
Speaker 1: used to train artificial neural networks in very broad strokes.
Speaker 1: Neural nets are algorithms that are made up of individual
Speaker 1: units that behave somewhat like the neurons in the human brain.
Speaker 1: These units are interconnected and they make up layers. As
Speaker 1: information passes from lower layers to higher ones, and whatever
Speaker 1: input has passed through the neural net is analyzed in
Speaker 1: increasing complexity. Take for example, the picture of a cat.
Speaker 1: At the lowest layer, the individual units each specialized in
Speaker 1: recognizing some very abstract part of a cat. Picture. So
Speaker 1: one will specialize in noticing shadows or shading, and another
Speaker 1: will specialize in recognizing angles. And these individual units give
Speaker 1: a confident at center bowl that what they're seeing is
Speaker 1: the thing that they specialize in. So that lower layer
Speaker 1: is stimulated to transmit to the next higher layer, which
Speaker 1: specializes in recognizing more sophisticated parts. The units in the
Speaker 1: second layer scan the shadows and the angles that the
Speaker 1: lower layer found, and it recognizes them as lines and curves.
Speaker 1: The second layer transmits to the third layer, which recognizes
Speaker 1: those lines and curves as whiskers, eyes, and ears, and
Speaker 1: it transmits to the next layer, which recognizes those features
Speaker 1: as a cat. Neural nets don't hit a accuracy, but
Speaker 1: they work pretty well. The problem is we don't really
Speaker 1: understand how they work. The thing about neural nets is
Speaker 1: that they learn on their own. Humans don't act as
Speaker 1: creator gods who code the rules of the universe for
Speaker 1: them like in the old days. Instead, we act more
Speaker 1: as trainers. And to train a neural net, you expose
Speaker 1: it to tons of data on whatever it is you
Speaker 1: wanted to learn. You can train them to recognize pictures
Speaker 1: of cats by showing them millions of pictures of cats.
Speaker 1: You can train them on natural languages by exposing them
Speaker 1: to thousands of hours of people talking. You can train
Speaker 1: them to do just about anything. So long as you
Speaker 1: have a robust enough data set. Neural net's find patterns
Speaker 1: in all of this data, and within those patterns, they
Speaker 1: decide for themselves. What about English makes English english? Or
Speaker 1: what makes a cat picture a picture of a cat?
Speaker 1: We don't have to teach them anything. In addition to
Speaker 1: self directed learning, what makes this type of algorithm so
Speaker 1: useful is its ability to self correct to get better
Speaker 1: at learning. If researchers show a neural net a picture
Speaker 1: of a fox and the AI says it's a cat,
Speaker 1: the researchers can tell the neural net it's wrong. The
Speaker 1: algorithm will go back over its millions of connections and
Speaker 1: fine tune them, adjusting the way it gives each unit
Speaker 1: so that in the future it will be able to
Speaker 1: better distinguish a cat from a fox. It does this too,
Speaker 1: without any help or guidance from humans. We just tell
Speaker 1: the AI that it got it wrong. The trouble is
Speaker 1: we don't really know how neural nets do what they do.
Speaker 1: We just know they work. This is it's called opaque.
Speaker 1: We can't see inside the thought processes of our AI,
Speaker 1: which makes artificial neural nets black boxes, which makes some
Speaker 1: people nervous. With the black box, we add input and
Speaker 1: receive output, but what happens in between is a mystery.
Speaker 1: Kind of like when you put a quarter into a
Speaker 1: gumball machine. Quarter goes in, gumball comes out. The difference
Speaker 1: is that gumball machines aren't in any position to take
Speaker 1: control of our world from us. And if you were
Speaker 1: curious enough, you could open up a gumball machine and
Speaker 1: look inside to see how it works with the neural net.
Speaker 1: Cracking open the algorithm doesn't help. The machine learns in
Speaker 1: its own way, not following any procedures we humans have
Speaker 1: taught it. So when we examine a neural net, what
Speaker 1: we see doesn't explain anything to us. We're already beginning
Speaker 1: to see signs of this opaqueness in real life as
Speaker 1: reports come in from the field. A neural net that
Speaker 1: Facebook train to negotiate developed its own language that apparently
Speaker 1: works rather well in negotiations, but doesn't make any sense
Speaker 1: to human Here's a transcript from a conversation between age
Speaker 1: and a and agent b Alice and Bob. I can
Speaker 1: I I everything else dot dot dot dot dot dot
Speaker 1: dot dot dot dot dot dot dot dot balls have
Speaker 1: zero to me, to me, to me, to me, to me,
Speaker 1: to me, to me, to meet you, I everything else
Speaker 1: dot dot dot dot dot dot dot dot dot dot
Speaker 1: dot dot dot dot balls have it ball to me,
Speaker 1: to me, to me, to me, to me, to me,
Speaker 1: to me, I I can I I I everything else
Speaker 1: dot dot dot dot dot dot dot dot dot dot
Speaker 1: dot dot dot. Another algorithm called deep patient was trained
Speaker 1: on the medical history of over seven hundred thousand people
Speaker 1: twelve years worth of patient records from Mount Sinai Hospital
Speaker 1: in New York. It became better than human doctors at
Speaker 1: predicting a patient would develop a range of ninety three
Speaker 1: different illnesses within a year. One of those illnesses is schizophrenia.
Speaker 1: We humans have a difficult time diagnosing schizophrenia before the
Speaker 1: patient suffers their first psychotic break, but Deep patient has
Speaker 1: proven capable of diagnosing the mental illness before then. The
Speaker 1: researchers have no idea what patterns the algorithm as seeing
Speaker 1: in the data. They just know it's right. With astonishing quickness,
Speaker 1: the field of AI has been brought out of its
Speaker 1: winter by neural nets. Almost overnight, there was a noticeable
Speaker 1: improvement in the reliability of the machines that do work
Speaker 1: for US. Computers got better at recommending movies. They got
Speaker 1: better at creating molecular models to search for more effective pharmaceuticals.
Speaker 1: They got better at tracking weather. They got better at
Speaker 1: keeping up with traffic and adjusting our driving routes. Some
Speaker 1: algorithms are learning to write code so that they can
Speaker 1: build other algorithms. With neural nets. Things are beginning to
Speaker 1: fall into place for the field of AI. In them,
Speaker 1: researchers have produced an adaptable, scalable template that could be
Speaker 1: capable of a general form of intelligence. It can self improve,
Speaker 1: it can learn to code. The seeds for a superintelligent
Speaker 1: AI are being sowned. There are enormous differences, by orders
Speaker 1: of magnitude really, between the AI that we exist with
Speaker 1: and the super intelligent AI that could at some point
Speaker 1: result from it. The ones we live with today are
Speaker 1: comparatively dumb, not just compared to a super intelligent AI,
Speaker 1: but compared to humans as well. But the point of
Speaker 1: thinking about existential risks posed by super intelligent AI isn't
Speaker 1: about time scales of when it might happen, but whether
Speaker 1: it will happen at all. And if we can agree
Speaker 1: that there is some possibility that we may end up
Speaker 1: sharing our existence with a super intelligent machine, one that
Speaker 1: is vastly more powerful than us, then we better start
Speaker 1: planning for its arrival now. So I think that this
Speaker 1: transition to a machine intelligence era looks like it has
Speaker 1: some reasonable chance of occurring within perhaps the lifetime of
Speaker 1: a lot of people today. We don't really not, but
Speaker 1: maybe it could happen in a couple of decades, maybe
Speaker 1: it's like a century, and that it would be a
Speaker 1: very important transition the last invention that human sever needs
Speaker 1: to make. That was Nick Bostrom, the Oxford philosopher who
Speaker 1: basically founded the field of existential risk analysis. Artificial intelligence
Speaker 1: is one of his areas of focus. Bostroom used a
Speaker 1: phrase in there that AI would be the last invention
Speaker 1: humans ever need to make. It comes from a frequently
Speaker 1: cited quote from British mathematician Dr Irving John good one
Speaker 1: of the crackers of the Nazi Enigma code at Bletchley
Speaker 1: Park and one of the pioneers of machine learning. Doctor
Speaker 1: Goods quote reads, let an ultra intelligent machine be defined
Speaker 1: as a machine that can faster pass all the intellectual
Speaker 1: activities of any man, however clever. Since the design of
Speaker 1: machines is one of these intellectual activities, an ultra intelligent
Speaker 1: machine could design even better machines. There would then not
Speaker 1: unquestionably be an intelligence explosion, and the intelligence of man
Speaker 1: would be left far behind. Thus, the first ultra intelligent
Speaker 1: machine is the last invention that man need ever make.
Speaker 1: In just a few lines, Ductor Goods sketches out the
Speaker 1: contours of how a machine might suddenly become super intelligent,
Speaker 1: leading to that intelligence explosion. There are a lot of
Speaker 1: ideas over how this might happen, but perhaps the most
Speaker 1: promising path is stuck in the middle of that passage
Speaker 1: a machine that can design even better machines. Today, AI
Speaker 1: researchers call this process recursive self improvement. It remains theoretical,
Speaker 1: but it stands as a legitimate challenge to AI research,
Speaker 1: and we're already seeing the first potential traces of it
Speaker 1: in neural nets. Today, a recursively self improving machine would
Speaker 1: be capable of writing better versions of itself. So Version
Speaker 1: one would write a better version of its code, and
Speaker 1: that would result in version two, and version two would
Speaker 1: do the same thing, and so on, and with each
Speaker 1: iteration the machine would grow more intelligent, more capable, and
Speaker 1: most importantly, better at making itself better. The idea is
Speaker 1: that at some point the rate of improvement would begin
Speaker 1: to grow so quickly that the machines intelligence would take
Speaker 1: off the intelligence explosion that Dr Good predicted how an
Speaker 1: intelligence explosion might play out. Isn't the only factor here?
Speaker 1: At least equally important is how quickly an AI might
Speaker 1: become intelligent? Is just how intelligent it will become. An AI.
Speaker 1: Theorist named Eliezer Yukowski points out that we humans have
Speaker 1: a tendency to underestimate the intelligence levels that AI can attain.
Speaker 1: When we think of a super intelligent being, we tend
Speaker 1: to think of some amazing human genius, say Einstein, and
Speaker 1: then we put Einstein in a computer or a robot.
Speaker 1: That's where our imaginations tend to aggregate when most of
Speaker 1: us ponder super intelligent AI. True as self improving AI
Speaker 1: may at some point reach a level where it's intelligence
Speaker 1: is comparable to Einstein's, but why would it stop there?
Speaker 1: Rather than thinking of a super intelligent AI along the
Speaker 1: lines of the difference between Einstein and US regular people.
Speaker 1: Nick Bostrom suggests that we should probably instead think more
Speaker 1: along the lines of the different between Einstein and earthworms.
Speaker 1: The super Intelligent AI would be a god that we
Speaker 1: made for ourselves. What would we do with our new god?
Speaker 1: It's not hyperbole to say that the possibilities are virtually limitless,
Speaker 1: but you can kind of see the outlines and what
Speaker 1: we do with our lesser AI gods. Now, we will
Speaker 1: use them to do the things we want to do
Speaker 1: but can't, and to do the things we can do
Speaker 1: but better. I. J. Good called the super Intelligent Machine
Speaker 1: the last invention humans ever need to make, because after
Speaker 1: we invented it, the AI would handle the inventing for
Speaker 1: us from there on. Now, our technological maturity would be
Speaker 1: secured as it developed new technologies like atomically precise manufacturing
Speaker 1: using nanobots. And since it would be vastly more intelligent
Speaker 1: than us, the machines that created for us would be
Speaker 1: vastly superior to anything we could come up with flawless technology.
Speaker 1: As far as we were concerned, we could ask it
Speaker 1: for whatever we wanted to establish our species outside of
Speaker 1: Earth by designing and building the technology to take us
Speaker 1: elsewhere in the universe. We would live in utter health
Speaker 1: and longevity. It would be a failure of imagination, says
Speaker 1: Eliza Yukowski, to think that AI would cure, say cancer,
Speaker 1: the super intelligent AI would cure disease. It would also
Speaker 1: take over all the processes we've started, improve on them,
Speaker 1: build on them, create whole new ones we hadn't thought of,
Speaker 1: and create for us a post scarcity world, keeping our
Speaker 1: global economy humming along, providing for the complete well being, comfort,
Speaker 1: and happiness of every single person alive. It would probably
Speaker 1: be easier than guessing at all of the things of
Speaker 1: super intelligent a I might do for us to instead
Speaker 1: look at everything that's wrong with the world, the poverty,
Speaker 1: the wars, the crime, the exploitation and death was suffering,
Speaker 1: and imagine a version of our world utterly without any
Speaker 1: of it. That starts to get at what those people
Speaker 1: intice pating the emergence of a super intelligent AI expect
Speaker 1: from it. But there's another little bit at the end
Speaker 1: of that famous quote from I J. Good, one that
Speaker 1: almost always gets left off, which says a lot about
Speaker 1: how we humans think of the risks posed by AI.
Speaker 1: The sentence reads in full. Thus, the first ultra intelligent
Speaker 1: machine is the last invention that man need ever make,
Speaker 1: provided that the machine is docid enough to tell us
Speaker 1: how to keep it under control. We humans tend to
Speaker 1: assume that any AI we create would have some desire
Speaker 1: to help us or care for us, But existential risk
Speaker 1: theorists widely agree that almost certainly would not be the case.
Speaker 1: That we have no reason to assume a super intelligent
Speaker 1: AI would care at all about as humans are well being,
Speaker 1: in happiness, or even our survival. This is transhumanist philosopher
Speaker 1: David Pierce. If the intelligence explosion word come to pass,
Speaker 1: it's by no means clear that the upshot would be
Speaker 1: sentients friendly super intelligence. In much the same way that
Speaker 1: we make assumptions about how aliens might walk on two legs,
Speaker 1: or have eyes, or be in some form we can comprehend,
Speaker 1: we make similar assumptions about AI, but it's likely that
Speaker 1: a super intelligent AI would be something we couldn't really
Speaker 1: relate to at all. It sounds bizarre, but think for
Speaker 1: a minute about what would happen if the Netflix algorithm
Speaker 1: became super intelligent. What about the Netflix algorithm makes us
Speaker 1: think that it would care at all about every human
Speaker 1: having a comfortable income and the purest fresh water to drink.
Speaker 1: Say that a few decades from now, computing power becomes
Speaker 1: even cheaper and computer processes more efficient, and Netflix engineers
Speaker 1: figure out how to train its algorithm to self improve.
Speaker 1: Their purpose at building upon their algorithm isn't to save
Speaker 1: the world. It's to make an AI that can make
Speaker 1: ultra tailored movie recommendations. So if the right combination of
Speaker 1: factors came together and the Netflix algorithm underwent and intelligence explosion,
Speaker 1: there's no reason for us to assume that it would
Speaker 1: become a super intelligent, compassionate Buddha. It would be a
Speaker 1: super intelligent movie recommending algorithm, and that would be an
Speaker 1: extremely dangerous thing to share our world with. About a
Speaker 1: decade ago, Nick Bostrom thought of a really helpful but
Speaker 1: fairly absurd scenario that gets across the idea that even
Speaker 1: the most innocuous types of machine intelligence could spell our
Speaker 1: doom should they become super intelligent. The classical example being
Speaker 1: the AI paper clip maximizer that transforms the Earth into
Speaker 1: paper clips are space colonization props that then gets sent
Speaker 1: out and the transform the universe into paper tips. Imagine
Speaker 1: that a company that makes paper clips hires a programmer
Speaker 1: to create an AI that can run its paper clip factory.
Speaker 1: The programmer wants the AI to be able to find
Speaker 1: new ways to make paper clips more efficiently and cheaply,
Speaker 1: so it gives the AI freedom to make its own
Speaker 1: decisions on how to run the paper clip operation. The
Speaker 1: programmer just gives the AI the primary objective, its goal
Speaker 1: of making as many paper clips as possible. Say that
Speaker 1: paper clip maximizing AI become super intelligent. For the AI,
Speaker 1: nothing has changed. Its goal is the same to it.
Speaker 1: There is nothing more important in the universe than making
Speaker 1: as many paper clips as possible. The only difference is
Speaker 1: that the AI has become vastly more capable, so it
Speaker 1: finds new processes that building paper clips that were overlooked
Speaker 1: by as humans. It creates new technology like nanobots to
Speaker 1: build atomically precise paper clips on the molecular level, and
Speaker 1: it creates additional operations like initiatives to expand its own
Speaker 1: computing power so it can make itself even better at
Speaker 1: making more paper clips. It realizes at some point that
Speaker 1: if it could somehow take over the world, that would
Speaker 1: be a whole lot of more paper clips in the future.
Speaker 1: Then if it just keeps running the single paper clip factory,
Speaker 1: so it then has an instrumental reason to place itself
Speaker 1: in a better position to take over the world. All
Speaker 1: those fiber optic networks, all those devices we connect to
Speaker 1: those networks, are global economy. Even as human would be
Speaker 1: repurposed and put into the service of building paper clips,
Speaker 1: rather quickly, the AI would turn its attention to space
Speaker 1: as an additional source of materials for paper clips. And
Speaker 1: since the AI would have no reason to fill us
Speaker 1: in on its new initiatives to the extent that it
Speaker 1: considered communicating with us at all, it would probably conclude
Speaker 1: that it would create an unnecessary drag on its paper
Speaker 1: clip making efficiency. We humans would stand by as the
Speaker 1: AI launched rockets from places like Florida and Kazakhstant, left
Speaker 1: to wonder what's it doing now? It's nanobot workforce would
Speaker 1: reconstitute matter, rearranging the atomic structures of things like water
Speaker 1: molecules and soil into aluminum to be used as raw
Speaker 1: material for more paper clips. But we humans, who have
Speaker 1: been pressed into services paper clip making slaves by this point,
Speaker 1: need those water molecules in that earth for our survival,
Speaker 1: and so we would be thrown to a resource conflict
Speaker 1: with the most powerful entity in the universe, as far
Speaker 1: as we're concerned, a conflict that we were doomed from
Speaker 1: the outset to lose. Perhaps the AI would keep just
Speaker 1: enough water and soiled to produce food and water to
Speaker 1: sustain as slaves. But let's not forget why we humans
Speaker 1: are so keen on building machines to do the work
Speaker 1: for us. In the first place. We're not exactly the
Speaker 1: most efficient workers around, so the AI would likely conclude
Speaker 1: that it's paper clip making operation would benefit more to
Speaker 1: use those water, molecules and soil to make aluminum than
Speaker 1: it would keep us alive with it. And it's about
Speaker 1: here that those nanobots the AI built would come for
Speaker 1: our molecules too. As Elie as our Yukowski wrote, the
Speaker 1: AI does not hate you, nor does it love you,
Speaker 1: but you are made of atoms which it can use
Speaker 1: for something else. But say that it turns out as
Speaker 1: super intelligent, AI does undergo some sort of spiritual conversion
Speaker 1: as a result of its vastly increased intellect, and also
Speaker 1: gains compassion. Again, we shouldn't assume we will come out
Speaker 1: safely from that scenario. Either. What exactly what the AI
Speaker 1: care about, not necessarily just us, considers, says transhumanist philosopher
Speaker 1: David Pierce, an AI that deeply values all sentient life.
Speaker 1: That is to say that it cares about every living
Speaker 1: being capable of, at the very least the experience of
Speaker 1: suffering and happiness, and the AI values all sentient lives
Speaker 1: the way that we humans place a high value on
Speaker 1: human life. Again, there's no reason for us to assume
Speaker 1: that the outcome for us would be a good one
Speaker 1: under scrutiny. Perhaps the way we tend to treat other
Speaker 1: animals we share the planet with other sentient life would
Speaker 1: bring the AI to the conclusion that we humans are
Speaker 1: an issue that must be dealt with to preserve the
Speaker 1: greater good. Do you imagine if you were a full
Speaker 1: spectrum superintelligence, would you deliberately create brain damaged, psychotic, eccentric
Speaker 1: for malays written Darwinian humans, or would you think are
Speaker 1: matro and energy could be optimized in a in a
Speaker 1: radically different way, or perhaps its love ascenient life. We'd
Speaker 1: preclude it from killing us and our species would instead
Speaker 1: be imprisoned forever to prevent us from ever killing another animal,
Speaker 1: either special death or special imprisonment. Neither of those outcomes
Speaker 1: of the future we have in mind for humanity. So
Speaker 1: you can begin to see why some people are anxious
Speaker 1: at the vast number of algorithms in development right now,
Speaker 1: and those already intertwined in the digital infrastructure we've built
Speaker 1: atop our world. There is thought given to safety by
Speaker 1: the people building these intelligent machines. It's true. Self driving
Speaker 1: cars have to be trained in programmed to choose the
Speaker 1: course of action that will result in the fewest number
Speaker 1: of human deaths when an accident can't be avoided. Robot
Speaker 1: care workers must be prevented from dropping patients when they
Speaker 1: lift them into a hospital bed. Autonomous weapons, if we
Speaker 1: can't agree to BAND out right, have to be carefully
Speaker 1: trained to minimize the possibility that they kill innocent civilians,
Speaker 1: so called collateral damage. These are the type of safety
Speaker 1: issues that companies building AI consider. They are concerned with
Speaker 1: the kind that can get your company sued out of existence,
Speaker 1: not the kind that arises from some vanishingly remote threat
Speaker 1: to humanities existence. But say they did build their AI
Speaker 1: to reduce the possibility of an existential threat. Controlling a
Speaker 1: god of our own making is as difficult as you
Speaker 1: would expect it to be. In his two thousand fourteen
Speaker 1: books Super Intelligence, Nick Bostrom lays out some possible solutions
Speaker 1: for keeping a super intelligent AI under control. We could
Speaker 1: box it physically, house it on one single computer that's
Speaker 1: not connected to any network or the Internet. This would
Speaker 1: prevent the AI from making masses of copies of itself
Speaker 1: and distributing them on servers around the world, effectively escaping.
Speaker 1: We could trick it into thinking that it's actually just
Speaker 1: a simulation of an AI, not the real thing, so
Speaker 1: its behavior might be more docile. We could limit the
Speaker 1: number of people that comes in contact with just a few,
Speaker 1: and watch those people closely for signs they're being manipulated
Speaker 1: by the AI and helping it escape. Each time we
Speaker 1: interact with the AI, we could wipe its hard drive
Speaker 1: clean and reinstall it ANEW to prevent the AI from
Speaker 1: learning anything it could use against us. All of these
Speaker 1: plants have their benefits and drawbacks, but they are hardly
Speaker 1: full proof. Bostrom points out one fatal flaw that they
Speaker 1: all have in common. They were thought up by people.
Speaker 1: If Bosterman others in his field have thought of these
Speaker 1: control ideas. It stands to reason that a super intelligent
Speaker 1: a I would think of them as well and take
Speaker 1: measures against them. And just as important, this AI would
Speaker 1: be a greatly limited machine, one that could only give
Speaker 1: us limited answers to a limited number of problems. This
Speaker 1: is not the AI that would keep our world humming
Speaker 1: along for the benefit and happiness of every last human.
Speaker 1: It would be a mere shadow of that. So theorists
Speaker 1: like Bosterman you how s Ki, tend to think that
Speaker 1: coming up with ways to keep a super intelligent AI
Speaker 1: hostage isn't the best route to dealing with our control issue. Instead,
Speaker 1: we should be thinking up ways to make the AI
Speaker 1: friendly to us humans, to make it want to care
Speaker 1: about our well being. And since as we've seen we
Speaker 1: humans will have no way to control the AI once
Speaker 1: it's super intelligent, we will have to build friendliness into
Speaker 1: it from the outside. In fact, aside from a scenario
Speaker 1: where we managed to program into the AI the express
Speaker 1: goal of providing for the well being and welfare of humankind,
Speaker 1: a terrible outcome for humans is basically the inevitable result
Speaker 1: of any other type of emergence of a super intelligent AI.
Speaker 1: But here's the problem. How do you convince Einstein to
Speaker 1: care so deeply about earthworms that he dedicates his immortal
Speaker 1: existence to providing and caring for each and every last
Speaker 1: one of them. As ridiculous as it sounds, this is
Speaker 1: possibly the most important question we humans face as a
Speaker 1: species right now. We humans have expectations for parents when
Speaker 1: it comes to raising children. We expect them to be
Speaker 1: raised to treat other people with kindness. We expect them
Speaker 1: to be taught to go out of their way to
Speaker 1: keep from harming others. We expect them to know how
Speaker 1: to give as well as take. All these things and
Speaker 1: more make up our morals. Rules that we have collectively
Speaker 1: agreed are good because they help society to thrive, and,
Speaker 1: seemingly miraculously, if you think about it, parents after parents
Speaker 1: managed to call some form or fashion of morality from
Speaker 1: their children, generation after generation. If you look closely, you
Speaker 1: see that each parent doesn't make up morality from scratch.
Speaker 1: They pass along what they were taught, and children are
Speaker 1: generally capable of accepting these rules to live by and
Speaker 1: well live by them. It would seem if you'll forgive
Speaker 1: the analogy that the software for morality comes already on
Speaker 1: board a child as part of their operating system. The
Speaker 1: parents just have to run the right programs. So it
Speaker 1: would seem then that perhaps the solution to the problem
Speaker 1: of instilling friendliness in an AI is to build a
Speaker 1: super intelligent AI from a human mind. This was laid
Speaker 1: out by Nick Bostroman his book super Intelligence. The idea
Speaker 1: is that if the hard problem of consciousness is not correct,
Speaker 1: and it turns out that our conscious experience is merely
Speaker 1: the result of the countless interactions of the interconnections between
Speaker 1: our hundred billion neurons, then if we can transfer those
Speaker 1: interconnected neurons into a digital format, everything that's encoded in them,
Speaker 1: from the smell of lavender to how to ride a
Speaker 1: bike would be transferred as well. More to the point,
Speaker 1: the morality encoded in that human mind should emerge in
Speaker 1: the digital version too. A digital mind can be expanded,
Speaker 1: processing power can be added to it. It could be
Speaker 1: edited to remove unwanted content like greed or competitiveness. It
Speaker 1: could be upgraded to a super intelligence. There are a
Speaker 1: lot of magic wands waving around here, but interestingly, uploading
Speaker 1: a mind called whole brain emulation is theoretically possible with
Speaker 1: improvements to our already existing technology. We would slice a brain,
Speaker 1: scan it with such high resolution that we could account
Speaker 1: for every neuron, synapse and nano leader of neurochemicals, and
Speaker 1: build that information into a digital model. The answer to
Speaker 1: the question of whether it worked would come when we
Speaker 1: turn the model on. It might do absolutely nothing and
Speaker 1: just be an amazingly accurate model of a human brain.
Speaker 1: Or it might wake up but go insane from the
Speaker 1: sudden novel experience of living in a digital world. Or
Speaker 1: perhaps it could work. The great advantage to using whole
Speaker 1: brain emulation to solve the friendliness problem is that the
Speaker 1: AI would understand what we meant when we asked it
Speaker 1: to dedicate itself to looking after and providing for the
Speaker 1: well being and happiness of all humans. We humans have
Speaker 1: trouble saying exactly what we mean at times, and Bostroon
Speaker 1: points out that a superintelligence that takes us literally could
Speaker 1: prove disastrous if we aren't careful with our words. Suppose
Speaker 1: we give an AI the goal of making all humans
Speaker 1: as happy as possible. Why should we think that the
Speaker 1: superintelligent a I would understand that we mean it should
Speaker 1: purify our air and water, create a bucolic wonderland of
Speaker 1: both peaceful tranquility and stimulating entertainment, Do away with wars
Speaker 1: and disease, and engineer social interactions so that we humans
Speaker 1: can comfort and enlighten one another. Why wouldn't the AI
Speaker 1: reach that goal more directly by, say, rounding up us
Speaker 1: humans and keeping us permanently immobile, doped up on a
Speaker 1: finely tuned cocktail of dopamine, serotonin, and oxytocin. Maximal happiness
Speaker 1: achieved with perfect efficiency. Say we do manage to get
Speaker 1: our point across? What's our point? Anyway? Whose morality are
Speaker 1: we asking the AI to adopt? Most of our human
Speaker 1: values are hardly universal. Should our global society embrace multiculturalism
Speaker 1: or our homogeneous society is more harmonious. If a woman
Speaker 1: didn't want to have a child, would she be allowed
Speaker 1: to terminate her pregnancy or should be forced to have it?
Speaker 1: Would we eat meat? If not, would it be because
Speaker 1: it comes from sacred animals, as Hindu people revere cows,
Speaker 1: or because it's taboo, as Muslim and Jewish people consider swine.
Speaker 1: From Out of this seemingly intractable problem of competitive and
Speaker 1: contradictory human values AI theorist eli as A Yukowski had
Speaker 1: a flash of brilliance. Perhaps we don't have to figure
Speaker 1: out how to get our point across to an AI,
Speaker 1: after all, Maybe we can leave that task to a machine.
Speaker 1: In yukowski solution, we would build a one use super
Speaker 1: intelligence with the goal of determining how to best express
Speaker 1: to another machine the goal of ensuring the well being
Speaker 1: and happiness of all humans. Yukowski suggests we use something
Speaker 1: he calls a coherent extrapolated vision, Essentially that we give
Speaker 1: the machine the goal of figuring out what we would
Speaker 1: ask a super intelligent machine to do for us, if
Speaker 1: the best version of humanity we're asking with the best
Speaker 1: of intentions, taking into a count as many common and
Speaker 1: shared values as possible, with humanity in as much agreement
Speaker 1: as possible, Considering we had all the information needed to
Speaker 1: make a fully informed decision on what to request. Once
Speaker 1: the super intelligent machine determined the answer, perhaps we would
Speaker 1: give it one more goal to build us a super
Speaker 1: intelligent machine with our coherent extrapolated vision aboard the last invention,
Speaker 1: Our last invention ever need make like whole brain emulation.
Speaker 1: Yukowski's coherent extrapolated vision takes for granted some real technological hurdles. Chiefly,
Speaker 1: we have to figure out how to build that first
Speaker 1: super intelligent machine from scratch, but perhaps it's a blueprint
Speaker 1: for future developers. The problems of controlling AI and instilling
Speaker 1: friendliness raises one basic question. If our machine random mock,
Speaker 1: why wouldn't we just turn it off? In the movie,
Speaker 1: there's always a way, sometimes a relatively simple one for
Speaker 1: dealing with troublesome AI. You can scrub it's hard drive,
Speaker 1: control all, delete it, sneak up behind it with a screwdriver,
Speaker 1: and remove its motherboard. But should we ever face the
Speaker 1: reality of a super intelligent AI emerging among us, we
Speaker 1: would almost certainly not come out on top, And AI
Speaker 1: has plenty of reasons to take steps to keep us
Speaker 1: from turning it off. It may prefer not to be
Speaker 1: turned off in the same way we humans most of
Speaker 1: the time prefer not to die. Or it may have
Speaker 1: no real desire to survive itself. But perhaps it would
Speaker 1: see being turned off as an impedance to its goal,
Speaker 1: whatever its goal, maybe and prevent us from turning it off.
Speaker 1: Perhaps it would realize that if we suspected the AI
Speaker 1: had gained super intelligence, we would want to turn it off,
Speaker 1: and so it would play dumb and keep this new
Speaker 1: increased intelligence out of our awareness until it has taken
Speaker 1: steps to keep us from turning it off. Or perhaps
Speaker 1: we could turn it off, but we would find we
Speaker 1: didn't have the will to do that. Maybe it would
Speaker 1: make itself so globally pervasive in our lives that we
Speaker 1: would feel like we couldn't afford to turn it off.
Speaker 1: Sebastian Farquhar from Oxford University points out that we already
Speaker 1: have a pretty bad track record at turning things off
Speaker 1: even when we know they're not good for us. One
Speaker 1: example of that might be global warming. So we all
Speaker 1: kind of know that carbon dioxide emissions are creating a
Speaker 1: big problem, but we also know that burning fossil fuels
Speaker 1: and the cheap energy that we get of it, it's
Speaker 1: also really useful, right. It gives us cheap consumer goods,
Speaker 1: it creates employment, it's very attractive, and so often, once
Speaker 1: we know that something is going to be harmful for us,
Speaker 1: but we also know that it's really nice, it becomes
Speaker 1: politically very challenging to to actually make an active decision
Speaker 1: to turn things off. Maybe it would be adept enough
Speaker 1: at manipulating us that it used a propaganda campaign to
Speaker 1: convince a majority of US humans that we don't want
Speaker 1: to turn it off. It might start lobbying, perhaps through
Speaker 1: proxies or fronts um or it might you know, studing
Speaker 1: looking at the political features of our time. It might
Speaker 1: create Twitter bots that argue that is AI is really
Speaker 1: useful that needs to be protected, or that it's important
Speaker 1: to some political or identity group. And perhaps we are
Speaker 1: already locked into the most powerful force in keeping AI
Speaker 1: pushing ever forward. Money. Those companies around the globe that
Speaker 1: build and use AI for their businesses make money from
Speaker 1: those machines. This creates an incentive for those businesses to
Speaker 1: take some of the money the machines make for them
Speaker 1: and reinvest it into building more improved machines to make
Speaker 1: even more money with This creates a feedback loop that
Speaker 1: anyone with a concern for existential safety has a tough
Speaker 1: time interrupting. This incentive to make more money. As well
Speaker 1: as the competition posed by other businesses, gives companies good
Speaker 1: reason to get new and improved AI to market as
Speaker 1: soon as possible. This in turn creates an incentive to
Speaker 1: cut corners on things that might be nice to have
Speaker 1: but aren't at all necessary in their business, like learning
Speaker 1: how to build friendliness into the AI they deploy. As
Speaker 1: companies make more and more money from AI, the technology
Speaker 1: becomes more entrenched in our world, and both of those
Speaker 1: things will make it harder to turn off. If, by chance,
Speaker 1: that Netflix algorithm does suddenly explode in intelligence. It sounds
Speaker 1: like so much gibberish, doesn't it Netflix's algorithm becoming super
Speaker 1: intelligent and wrecking the world. I may as well say
Speaker 1: a which could come by and cast a spell on
Speaker 1: it that wakes it up. But when it comes to technology,
Speaker 1: things that seem impossible given the luxury of time start
Speaker 1: to seem much less. So put yourself in with the
Speaker 1: technology people lived with back then. The earliest radios and airplanes,
Speaker 1: the first washing machines, neon lights were new, and consider
Speaker 1: that they had trouble imagining it being much more advanced
Speaker 1: than it was then. Now compare those things to our
Speaker 1: world in two thousand eighteen, and let's go the other way.
Speaker 1: Think about our world and the technology we live with today,
Speaker 1: and imagine what we might live among ineen the impossible
Speaker 1: starts to seem possible. What would you do tomorrow if
Speaker 1: you woke up and you found that Siri on your
Speaker 1: phone was making its own decisions and ones you didn't like,
Speaker 1: rearranging your schedule into bizarre patterns, investing your savings in
Speaker 1: its parent company, looping in everyone on your context list,
Speaker 1: too sensitive email threads. What would you do? What if
Speaker 1: fifty or a hundred years from now you woke up
Speaker 1: and found that the Siri that we've built for our
Speaker 1: whole world has begun to make decisions on its own.
Speaker 1: What do we do then if we go to turn
Speaker 1: it off and we find that it's removed our ability
Speaker 1: to do that? Have we shown it that we are
Speaker 1: an obstacle to be removed? On the next episode of
Speaker 1: the End of the World with Josh Clark, The field
Speaker 1: of biotechnology has grown sophisticated in its ability to create
Speaker 1: pathogens that are much deadlier than anything found in nature.
Speaker 1: That researcher thought that was a useful line of inquiry,
Speaker 1: and there were other researchers who vehemently disagreed and thought
Speaker 1: it was an extraordinarily reckless thing to do. The biotech
Speaker 1: field also has a history of recklessness and accidents and
Speaker 1: as the world goes more connected, just one of those
Speaker 1: accidents could bring an abrupt end to humans.