Speaker 1: Thanks for tune into Tech Stuff. If you don't recognize
Speaker 1: my voice, my name is Osvalshin and I'm here because
Speaker 1: the inimitable Jonathan Strickland has passed the baton to Cara
Speaker 1: Price and myself to host Tech Stuff. The show will
Speaker 1: remain your home for all things tech, and all the
Speaker 1: old episodes will remain available in this feed.
Speaker 2: Thanks for listening.
Speaker 3: I was in a hotel in California and I saw
Speaker 3: that the phone lit up, and I thought, who's calling
Speaker 3: me at one o'clock in the morning, And then this
Speaker 3: Swedish voice came on. Then they said I won the
Speaker 3: Nobel Prize in Physics, and I thought, this is very odd.
Speaker 3: I don't do physics, so that's when I thought it
Speaker 3: might be a prank.
Speaker 1: Jeffrey Hinton won the twenty twenty four Nobel Prize in Physics,
Speaker 1: an honor held by Albert Einstein and Marie Curie. A
Speaker 1: certain j Robert Oppenheimer were shortlisted but never won.
Speaker 3: My big hope was to win the Nobel Prize in
Speaker 3: Physiology or Medicine for figuring out how the brain worked.
Speaker 3: And what I didn't realize is you could fail to
Speaker 3: figure out how the brain worked and still get a
Speaker 3: Noble Prize anyway.
Speaker 1: Welcome to tech stuff. This is the story with our guests,
Speaker 1: the Nobel Laureate Jeffrey Hinton. Each week on Wednesdays, we
Speaker 1: bring you an in depth interview with someone who's at
Speaker 1: the forefront of technology or who can unlock a world
Speaker 1: where tech is at its most fascinating. His recent Nobel
Speaker 1: Prize win was for quote foundational discoveries and inventions that
Speaker 1: enable machine learning with artificial neural networks. Now, artificial neural
Speaker 1: networks are learning models inspired by the network of neurons
Speaker 1: present in the human brain, and Hinton's desire to figure
Speaker 1: out the brain was a key inspiration for his pioneering
Speaker 1: work on AI. I was particularly fascinated by Hinton because
Speaker 1: his work went completely against the mainstream of computer science
Speaker 1: for decades, and yet he stuck to his guns. It's
Speaker 1: an incredible story of dedication in the face of personal loss.
Speaker 1: Also fascinating is Hinton's relationship to his own creation. Here's
Speaker 1: what he said at the Nobel Prize banquet.
Speaker 3: There is also a longer term existential threat that will
Speaker 3: arise when we create digital beings that are more intelligent
Speaker 3: than ourselves. We have no idea whether we can stay
Speaker 3: in control. But we now have evidence that if they
Speaker 3: are created by companies motivated by short term profits, our
Speaker 3: safety will not be the top priority. We urgently need
Speaker 3: research on how to prevent these new beings from wanting
Speaker 3: to take control. They are no longer science fiction, thank you.
Speaker 1: So when I got the opportunity to sit down with
Speaker 1: Jeffrey Hinton, I wanted to know how he went from
Speaker 1: someone who wanted to understand the relationship between the mind
Speaker 1: and the brain to someone who paved the path for
Speaker 1: AI as we know it. And he obliged me by
Speaker 1: telling me about his trajectory from student to researcher, to professor,
Speaker 1: to Google employee to finally AI safety advocate.
Speaker 2: Am I riding thinking with all due respect to Steve
Speaker 2: Jobs and Bill Gates and marks up a bug that
Speaker 2: you were, in a sense the original college dropout, not.
Speaker 3: In the sense they dropped out, Because what I did
Speaker 3: was I I went to Cambridge and after a month
Speaker 3: I dropped out, but then I went back the next year.
Speaker 3: And then while I was doing my PhD, I dropped out,
Speaker 3: but then I finished it. So I'm not like them.
Speaker 4: I went back. I'm a failed to drop out.
Speaker 2: Failed to drop out, that's good. What were the reasons for?
Speaker 2: Was it uncertainty or ambivalence or curiosity or so.
Speaker 3: When I first went to Cambridge, it was the first
Speaker 3: time I'd ever lived away from home, and I'd always
Speaker 3: my image of myself was that it was one of
Speaker 3: the clever ones. And when I went to Cambridge, I
Speaker 3: wasn't one of the clever ones. Everybody was clever, and
Speaker 3: I found it very stressful. I worked very, very hard,
Speaker 3: so I was working like twelve hours a day doing science,
Speaker 3: and the combination of working very hard to keep up
Speaker 3: and living away from home for the first time was
Speaker 3: just too much for me.
Speaker 2: So there was a fantastic profile of you in the
Speaker 2: New Yorker which said, quote, Hinton tried different feels but
Speaker 2: was dismayed to find that he was never the brightest
Speaker 2: student in any given class, which made me smile. But
Speaker 2: I guess dismay and stress are quite a close cousins,
Speaker 2: And I suppose the stakes for you were high given
Speaker 2: the family you came from. Can you speak a little
Speaker 2: bit about that.
Speaker 3: Yes, I had a lot of pressure from my father
Speaker 3: to be an academic success, and my mother sort of
Speaker 3: went along with it. So from an early age I
Speaker 3: realized that's what I had to achieve, and that's a
Speaker 3: lot of pressure.
Speaker 2: How did they exert that pressure? How are you aware
Speaker 2: of it?
Speaker 3: My father was a slightly strange character. He grew up
Speaker 3: in Mexico during all the revolutions without a mother, somewhat odd.
Speaker 3: Every morning when I went to school, not every morning,
Speaker 3: but quite often, as I left, you would say, get
Speaker 3: in their pitching. If you work very hard, when you're
Speaker 3: twice as old as me, you might be half as good.
Speaker 4: Well that's sort of pressure.
Speaker 2: Did you find that motivating?
Speaker 4: I found it irritating, but I think it probably was motivating.
Speaker 4: He very inconsiderately.
Speaker 3: He died while I was writing my thesis, and he
Speaker 3: never saw me being a success.
Speaker 2: You were at Cambridge, you left briefly and you came back.
Speaker 2: I think you settled on experimental psychology as your degree.
Speaker 4: In the end, I was doing natural sciences.
Speaker 3: I started off doing physics chemist ring crystalline state because
Speaker 3: of the success in decoding the structure of DNA. Crystalline
Speaker 3: state was a big thing, and I left after a month.
Speaker 3: Then I reapplied to do architecture. I've always liked oarchitecture,
Speaker 3: and after a day I decided I was never going
Speaker 3: to be any good architecture because I wasn't I wasn't
Speaker 3: artistic enough. I loved the engineering, but the artistic bit
Speaker 3: I couldn't do very well. So I switched back to science.
Speaker 3: But then I did physics, chemistry and physiology, and I
Speaker 3: really liked physiology. I'd never been allowed to do biology
Speaker 3: at school.
Speaker 4: My father wouldn't allow it.
Speaker 3: Why not, Ah well, he said, if you do biology,
Speaker 3: they'll teach you genetics. And he was a Stalinist and
Speaker 3: he didn't believe in genetics. Now, he was also a
Speaker 3: fellow of the British Royal Society in Biology who didn't
Speaker 3: believe in genetics.
Speaker 2: Gosh, complicated man, Yes, but I mean he really you
Speaker 2: won't exaggerated him. He said to you, you can't study biology.
Speaker 4: Now.
Speaker 3: He had other reasons which weren't so bad, which is,
Speaker 3: you can always pick up biology when you're older. What
Speaker 3: you can't pick up when you're older is math. And
Speaker 3: I think that's probably true, and so that was a
Speaker 3: more valid reason.
Speaker 2: Yeah, yeah, So what did you end up graduating from
Speaker 2: Cambridge with a degree in psychology?
Speaker 3: So I did physics, chemistry and physiology for a year,
Speaker 3: and I did very well in physics. I got a
Speaker 3: first in physics. That's obviously a good predictor of a
Speaker 3: Nobel Price.
Speaker 2: Said, your tutors would have been there.
Speaker 3: Then I dropped it all and did philosophy, philosophy, philosophy.
Speaker 3: I did a year of philosophy and I developed strong antibodies,
Speaker 3: and then I switched to psychology, and so my final
Speaker 3: degree was in psychology.
Speaker 2: MHM. And was there a single question or set of
Speaker 2: questions that you were in search of.
Speaker 3: Yes, I wanted to know how the how the brain worked,
Speaker 3: and how the mind worked, and what the relationship was.
Speaker 3: I decided fairly early on that you're never going to
Speaker 3: understand the mind unless you understood the brain.
Speaker 2: Was that a popular view at the time.
Speaker 4: No, not really.
Speaker 3: There was this kind of functionalist view that basically the
Speaker 3: view that came from computer software, which is that the
Speaker 3: software is totally different from the hardware, and what the
Speaker 3: mind is all about is software, the heuristics you use
Speaker 3: and the way you represent things. The hardware has got
Speaker 3: nothing to do with it. That's a completely crazy view,
Speaker 3: but it seemed very plausible at the time, and so
Speaker 3: the computers we designed so that we could program them,
Speaker 3: had programs being a completely separate domain from hardware.
Speaker 4: But that's not how the real brain is.
Speaker 2: Was as funny this constant dance between us expecting our
Speaker 2: computers to be like us, and then expecting us to
Speaker 2: be like our computers, right as a kind of continual
Speaker 2: dance between those two things.
Speaker 3: Yes, well, you always try and understand things in terms
Speaker 3: of the latest technology. So when telephones were new, the
Speaker 3: brain was clearly a very large telephone switchboard.
Speaker 2: But is it different this time in our AI has
Speaker 2: taken off and become ubiquitous. Do you think this is
Speaker 2: indeed more than the telephone?
Speaker 3: Yes, I think these artimicial neural networks for training are
Speaker 3: in many respects quite like real neural networks. Obviously, the
Speaker 3: neurons are much simpler. There's all sorts of properties that
Speaker 3: are different in the brain, but basically they're working in
Speaker 3: the same way. They learn things by changing connection strengths
Speaker 3: between neurons, just like the brain does.
Speaker 2: And when does that question for you of wanting to
Speaker 2: understand how the brain works really really begin?
Speaker 3: When I was at high school, so even before I
Speaker 3: went to Cambridge, I had a very bright friend who
Speaker 3: was always smarter than me called Inmund Harvey, who came
Speaker 3: into school one day and said, maybe memories in the
Speaker 3: brain are spread out over the whole brain. They're not
Speaker 3: localized like a hologram, because holograms when you so he
Speaker 3: was trying to understand memories in the brain in terms
Speaker 3: of this new technology of holograms.
Speaker 2: And you were stimulated by this idea.
Speaker 4: I was very stimulated by that.
Speaker 3: It's very interesting idea, and ever since then I've thought
Speaker 3: a lot about how memories are represented in the brain.
Speaker 3: And then that also led into well how does the
Speaker 3: brain learn stuff?
Speaker 1: Coming up how Jeffrey Hinton ended up at Google.
Speaker 2: Stay with us, so bear with me, but I need
Speaker 2: to try and summarize what the Boltziman machine is because
Speaker 2: the Nobel Prize Committee credited the Boltzman machine with your win.
Speaker 2: According to their press release, the Boltzman machine can quote
Speaker 2: learn to recognize characteristic elements in a given type of data.
Speaker 2: The machine is trained by feeding it examples that are
Speaker 2: very likely to arise. The Boltzmer machine can be used
Speaker 2: to classify images or create new examples of the type
Speaker 2: of pattern on which it was trained. Hinton has built
Speaker 2: upon this work helping initiate the current explosive development of
Speaker 2: machine learning. Now, in fact, the timeline right, you graduated
Speaker 2: from Cambridge, you worked on your PhD, and in the
Speaker 2: eighties you wound up at Connigie Mellon and that's where
Speaker 2: you work on the bolt of machine really took off.
Speaker 4: Uh yes, just before I went to Carnegie Mellon.
Speaker 2: Now am I writing thinking? At the time, there was
Speaker 2: a kind of debate that you were on the unpopular
Speaker 2: side of about artificial intelligence.
Speaker 3: Okay, so there's two things to say about that. From
Speaker 3: the earliest days of AI in the nineteen fifties, there
Speaker 3: were these two approaches, two kind of paradigms. So how
Speaker 3: you build an intelligence system. One was inspired by the brain,
Speaker 3: that was neural networks. Then the other paradigm was no, no, no, no.
Speaker 3: Logic is a paradigm for intelligence. Intelligence is all about reasoning.
Speaker 3: Learning comes later once we figured out how reasoning works,
Speaker 3: and reasoning depends on having the right representations for things.
Speaker 3: So we have to figure out what kind of logically
Speaker 3: unambiguous language the brain is using so that it can
Speaker 3: use rules to do reasoning. It's a completely different approach
Speaker 3: and it's very unbiological because reasoning something comes very late,
Speaker 3: and actually from most of the second half of the
Speaker 3: last century, neural networks weren't seen as AI. AI was
Speaker 3: believing you have symbolic rules in your head and then
Speaker 3: manipulated using rules, using to screet symbolic rules.
Speaker 2: And what was what were neural? They were seen as
Speaker 2: statistics or physically.
Speaker 3: They were seen as this crazy idea that you could
Speaker 3: take this random neural network and just learn everything, which
Speaker 3: was obviously hopeless.
Speaker 2: And what what gave you this conviction?
Speaker 3: Well, the brain has to learn somehow, and of course
Speaker 3: there's a lot of innate structure whir i'd in, which
Speaker 3: explains why a goat can sort of drop out of
Speaker 3: the womb and five minutes later it's walking. But so
Speaker 3: stuff like learning to read, that's all learned. That's not innate,
Speaker 3: and we have to figure out how that works, and
Speaker 3: it's not symbolic rules.
Speaker 2: This conviction that you have now, did you always have?
Speaker 4: Yes?
Speaker 3: Sorry to say it seemed to me it was obviously right.
Speaker 3: I think part of that is I was sent to
Speaker 3: a private school when I was seven from an atheist family,
Speaker 3: and they started telling me about religion and all this rubbish,
Speaker 3: all the stuff that was obviously nonsense, and I had
Speaker 3: the experience of being the only kid at school who
Speaker 3: thought this was all nonsense and turning out to be right.
Speaker 4: And that's very good training for a scientist.
Speaker 2: I brought it up earlier, but there was a wonderful
Speaker 2: profile of you in the New Yorker titled why the
Speaker 2: Godfather of Ai fears what He's built, And there was
Speaker 2: a quote that I found quite stunning where you said
Speaker 2: I was dead in the water at forty six.
Speaker 3: Yes, my wife and I adopted to children, to babies,
Speaker 3: and then my wife got ovarying cancer. But she also,
Speaker 3: even though she was a science she started believing in
Speaker 3: homeopathy and she tried treating her very in cancer with homeopathy,
Speaker 3: and so she died and.
Speaker 4: I was left with.
Speaker 3: Two young children, one of three and one of five,
Speaker 3: who were both very upset, as was I, and I
Speaker 3: began to appreciate what life is like for female academics,
Speaker 3: which is impossible.
Speaker 4: You can't.
Speaker 3: Looking after small children makes it very very hard to
Speaker 3: spend long periods of time thinking about your current idea.
Speaker 3: It's just very difficult.
Speaker 2: So you were bereaved, you were looking after two small children,
Speaker 2: and then yes, that moment in twenty twelve when you
Speaker 2: publish a paper called ImageNet classification with deep convolutional neural networks,
Speaker 2: which to the layman doesn't sound like something that would
Speaker 2: change the world and how we live, but it did well.
Speaker 3: It changed the views of people in computer vision and
Speaker 3: the views of people in other areas of computer science.
Speaker 3: It basically showed neural networks actually do work. Now, people
Speaker 3: have showed that before, but it hadn't convinced people the
Speaker 3: same way.
Speaker 2: See, you published that paper in twenty twelve with Iliot
Speaker 2: sutch Keva, who went on to be a very important
Speaker 2: figure at open AI and I want to talk more
Speaker 2: about him. But within a few days of the publication
Speaker 2: of that paper, you had an offer of millions and
Speaker 2: millions of dollars to move to China.
Speaker 3: Ah. Yeah, either to move to China or to let
Speaker 3: them invest in our group.
Speaker 4: I think it was a bit longer than a few days,
Speaker 4: but it was. It was that fall for sure.
Speaker 2: Did you kind of know, Okay, we're going to publish
Speaker 2: and this is going to change everything or are you
Speaker 2: surprised by this response.
Speaker 3: We thought it would have a big effect. We didn't
Speaker 3: realize quite how big.
Speaker 2: And when you got the call from Baidu, Chinese tech company,
Speaker 2: What did you think, I mean, it must have been tempting.
Speaker 3: H yes, I think they said they could fund our group,
Speaker 3: or they could we could go and work for them,
Speaker 3: or various possible alternatives. And in the end I just
Speaker 3: asked them, well, how much money are we talking about?
Speaker 3: Are you talking about like ten million dollars? And they said, yes.
Speaker 2: You localed yourself.
Speaker 4: I didn't think it at the time.
Speaker 3: I thought of the biggest number I could think of,
Speaker 3: it was five million dollars and doubled it. So once
Speaker 3: they said that, I realized we had no idea how
Speaker 3: much we were worth. And so at that point we
Speaker 3: decided to set up an auction.
Speaker 2: What does it mean to set up an auction for
Speaker 2: an academic paper?
Speaker 3: The three of us founded a company. The company that
Speaker 3: belonged to the three of us owned these six pandent applications,
Speaker 3: so then we had something to sell, but mainly we
Speaker 3: were selling ourselves. But I insisted that I still be
Speaker 3: an academic so I could continue to advise my current students.
Speaker 3: That was a big problem because Google had never done
Speaker 3: that before. They were one of the companies.
Speaker 2: The bid for us, so by do Google.
Speaker 3: Microsoft and deep Mind, which wasn't owned by Google then.
Speaker 2: And not only do you invent this well together with
Speaker 2: your colleagues, this advancement of neural nets, but you also
Speaker 2: invented your own process to sell this company essentially, right.
Speaker 3: Yeah, we decided we just have an auction by Gmail,
Speaker 3: and you'd send me Gmail with your bid, and the
Speaker 3: time of the bid would be the timestamp on the Gmail.
Speaker 3: I would then immediately send the bid to all the
Speaker 3: other bidders, and you had to raise by a million dollars,
Speaker 3: and if there was no bid within an hour of
Speaker 3: the last bid, that was the end of the auction.
Speaker 3: I was amazed to see that the bids would come
Speaker 3: in fifty nine minutes after the previous bid. Would be
Speaker 3: sitting there think okay, it's over, and then fifty nine
Speaker 3: minutes later a bid would come in.
Speaker 2: Only everybody trusted Gmail.
Speaker 3: I had worked at Google over the summer, and that
Speaker 3: meant two things. One, I did trust that they wouldn't
Speaker 3: read my Gmail. I knew they were very serious about that.
Speaker 3: And also I really wanted Google to win the competition
Speaker 3: because I really like Jeff Dean, who ran the brain
Speaker 3: team at Google. So I wanted Google to win, and
Speaker 3: in the end we slightly fudged it. So after deep
Speaker 3: minded Microsoft had dropped out, it was between Google and
Speaker 3: by Do, and we were scared by You would win,
Speaker 3: and I didn't want to go to China.
Speaker 4: I couldn't travel at that point, well, so I felt
Speaker 4: I wouldn't.
Speaker 3: Really understand what was going on in a Chinese company.
Speaker 3: So it got to forty four million, and we just
Speaker 3: said we've got an offer we can't refuse, and it's
Speaker 3: the end of the auction. And the offer we couldn't
Speaker 3: refuse was to work with Jeff Deane.
Speaker 2: So you got what you wanted in the sense of.
Speaker 3: We've got more money than we could imagine, and we
Speaker 3: got to work at the company that I wanted most
Speaker 3: to work.
Speaker 1: With when we come back. Nobel Laureate Jeffrey Hinton on
Speaker 1: why he advocates for AI safety. In twenty eighteen, you
Speaker 1: receive the Touring Award, which is kind of like the
Speaker 1: Nobel Prize for Computer Science. But also in twenty eighteen,
Speaker 1: you were widowed for a second time. You lost your
Speaker 1: wife Rosalind in nineteen ninety four, and then twenty four
Speaker 1: years later your wife Jackie passed away also from cancer.
Speaker 3: Yes, that was that was difficult. I didn't have My
Speaker 3: children were much older then, so I didn't have the
Speaker 3: problem of having to cope with young children at the
Speaker 3: same time as everything else. And Google was very understanding.
Speaker 3: Part of my deal had been that I would spend
Speaker 3: three months a year in Silicon Valley, and they let
Speaker 3: me out of that and said I could spend my
Speaker 3: whole time in Toronto, and they helped me set up
Speaker 3: a small lab doing basic research in Toronto, so that
Speaker 3: was much less stressful, so I could be with my wife.
Speaker 4: She had cancer as well, she got pancreatic cancer.
Speaker 2: One of the most striking things, again in the New
Speaker 2: Yorker piece, was the way you talked about observing the
Speaker 2: way in which Rosalind and Jackie approached their cancer as
Speaker 2: a mental model for how to think about the implications
Speaker 2: of artificial intelligence.
Speaker 4: Yeah, that's a rather sort of dark scenario.
Speaker 3: So occasionally when I think, well, lis stuff probably will
Speaker 3: take over, which I sometimes think, then there's the issue
Speaker 3: of should you go into denial and say no, no, no, no,
Speaker 3: this can't possibly happen, which is what my first wife did. Actually,
Speaker 3: she wasn't my first wife I was married a long
Speaker 3: time before that, just briefly, and so Ross went into
Speaker 3: denial and Jackie was very realistic about it. And maybe
Speaker 3: we should be very realistic about the possibility machines will
Speaker 3: take over. We should do our best to make sure
Speaker 3: that doesn't happen, but we should also think about whether
Speaker 3: there's ways of making that if that does happen, whether
Speaker 3: there's ways of making it not so bad, whether people
Speaker 3: could still be around even if the machines were in control,
Speaker 3: for example.
Speaker 2: Yeah, I mean there's something. I mean, you use the
Speaker 2: word dark, but you've seen your life's work in many
Speaker 2: ways kind of fruition, haunted by these thoughts of how
Speaker 2: to deal with something as awful as the terminal council diagnosis.
Speaker 4: Yeah, you have to be careful what you wish for.
Speaker 2: Yeah, I mean my mentioned Oppenheimer at the beginning in
Speaker 2: terms of a Nobel Well, he wasn't almost a physics gloriate.
Speaker 2: He wasn't in the end.
Speaker 3: Yes, it's rather absurd, isn't it That I've got a
Speaker 3: Nobel Prize in physics and Oppenheimer didn't.
Speaker 4: That's utterly ridiculous.
Speaker 3: I should say something people, particularly journalists, like to say well,
Speaker 3: how would you compare yourself with Oppenheimer?
Speaker 4: And there's two big differences.
Speaker 3: One is that Oppenheimer really was crucial to the development
Speaker 3: of the atomic bomb. He managed the science of it.
Speaker 3: He was a single, extremely important figure with the development
Speaker 3: of AI. There's a bunch of us, and if I
Speaker 3: hadn't been around, all stuff would have happened. That's one difference.
Speaker 3: The other difference is that atomic bombs aren't good for
Speaker 3: anything good.
Speaker 4: They're just for destruction.
Speaker 3: They actually did try using them for fracking in Colorado,
Speaker 3: but that didn't work out too.
Speaker 4: Well and you can't go there anymore.
Speaker 3: The big difference is most of the uses of AI
Speaker 3: are very good. It can lead to huge increases in productivity,
Speaker 3: huge improvements in healthcare, might help a lot with climate change.
Speaker 3: So AI is going to be developed because of all
Speaker 3: those huge beneficial uses. And that's very different from atomic bombs,
Speaker 3: where there was a possibility of not developing the H bomb.
Speaker 2: And why have you taken it upon yourself as your responsibility?
Speaker 2: And you quit Google in twenty twenty three and since
Speaker 2: then have become one of the most vocal and qualified
Speaker 2: people in the world warning of these risks.
Speaker 4: Well, I'm old.
Speaker 3: I'm too old to do original research anymore, but people
Speaker 3: listen to me, and I really believe these risks are
Speaker 3: very real and very important, so I don't really have
Speaker 3: much choice. We are going to develop AI because it's
Speaker 3: got so.
Speaker 4: Many good uses.
Speaker 3: So I'm not warning against developing it, and I'm not
Speaker 3: saying slow down. What I'm saying is try and develop
Speaker 3: it as safely as you can. Try and figure out,
Speaker 3: in particular, how you can stop it eventually taking over,
Speaker 3: but also think about all the other shorter term risks
Speaker 3: like fake videos, corrupting elections, and loss of jobs. I'm
Speaker 3: saying we need to worry about all those things, and
Speaker 3: it might be rational to just stop developing it, but
Speaker 3: I'm not. I don't think there's any hope of that,
Speaker 3: so I'm not advocating that.
Speaker 2: You said in December there was a ten to twenty
Speaker 2: percent risk that AI would cause human extinction in the
Speaker 2: next thirty years. How do you count with those odds?
Speaker 4: I just make them up.
Speaker 3: That's I'm just like, no. If you think about subjective probabilities,
Speaker 3: they're based on intuition. I have a very strong intuition
Speaker 3: that the chance of super intelligent machines taking over from
Speaker 3: people is more than one percent, and I have a
Speaker 3: very strong intuition that is less than ninety nine percent.
Speaker 3: We're dealing with something extremely unknown, so your best bet
Speaker 3: for things totally unknown is maybe fifty percent, but that
Speaker 3: doesn't work very well because it depends on how you
Speaker 3: partition things.
Speaker 4: So clearly the chance is.
Speaker 3: Much bigger than one percent and much les than ninety
Speaker 3: nine percent, and maybe I should just stick at that.
Speaker 3: But I'm hoping that we can figure out a way
Speaker 3: that people can stay in control, because we're people and
Speaker 3: what we care about is people.
Speaker 2: Now, do you view it as inevitable that if we
Speaker 2: if we quote unquote lose control, our destruction that is
Speaker 2: the is the next No, they.
Speaker 4: Might I een on Musk for example.
Speaker 3: I talked to him and he pushed the line that
Speaker 3: they'll keep us around as pets because we're quite interesting.
Speaker 2: M hmm.
Speaker 4: It seems a rather sort of thin thread to hang
Speaker 4: human existence by.
Speaker 2: So, I mean, you've been vocal about the kind of
Speaker 2: overall societal threat, but you've also been specifically critical of
Speaker 2: Sambleman in particular.
Speaker 3: Yes, because I think open Aye was set up to
Speaker 3: develop a GI safely, and it's just been progressively moving
Speaker 3: away from that towards developing a for profit and so
Speaker 3: it's best safety researchers have left and it's now trying
Speaker 3: to convert itself from a not for profit company into
Speaker 3: a for profit company. And that seems entirely wrong to me.
Speaker 2: And you're I'm a colleague and student. Ilius Atsgev, who
Speaker 2: worked on the twenty twelve paper with took a big
Speaker 2: stand on this, which ultimately didn't break his way.
Speaker 3: It didn't break his way in terms of Sam being
Speaker 3: fired as the head of the company. It did break
Speaker 3: his way in terms of people understanding that open Ai
Speaker 3: was going back on its pledge to develop Aji safely
Speaker 3: and India is now set up a company.
Speaker 4: I'm that's trying to do that.
Speaker 2: I mean, if if Samaltman called you tomorrow through Toronto
Speaker 2: and he were sitting with him, he said, you know,
Speaker 2: where have I gone wrong? And what should I do?
Speaker 2: What do you say?
Speaker 4: I'd say, you're not Sam Altman.
Speaker 2: Fair enough?
Speaker 4: Okay.
Speaker 3: So suppose something very surprising happens and Sam Altman suddenly
Speaker 3: has an epiphany and says, oh my god, we shouldn't
Speaker 3: be doing this for a profit. We should be doing
Speaker 3: it to protect humanity. I'd be very happy to doctor.
Speaker 2: But what if you could affect his opinion in one
Speaker 2: way or that of other.
Speaker 3: I would say, keep developing AI, but use a large
Speaker 3: fraction of the resources as you're developing it to try
Speaker 3: and figure out ways in which you might get out
Speaker 3: of control and what we could do about that. So
Speaker 3: if Sam Olman said we're going to use thirty percent
Speaker 3: of our compute, how much compute we have, we use
Speaker 3: thirty percent to get highly paid safety researchers to do
Speaker 3: research on safety, not on making it better, developing for
Speaker 3: the bit on making it safe, I'd take it all back,
Speaker 3: and I'd say he is a great guy.
Speaker 2: I mean for a non for profit, that doesn't sound
Speaker 2: like unreasonable exactly.
Speaker 3: That that was what I thought was happening to begin with,
Speaker 3: and that was what idiot thought was happening.
Speaker 2: Right before Christmas, there was an article in the Wall
Speaker 2: Street Journal basically saying that chat GIPT five was behind
Speaker 2: schedule and that kind of the pace of improvement in
Speaker 2: deep learning was lowing for different because of lack of
Speaker 2: real world data. Could be other reasons, But then O
Speaker 2: three came out and Settlement sort of said that AGI
Speaker 2: is here. Where do you think we are on this agi?
Speaker 2: Do you think it's even a relevant metric?
Speaker 3: So Ever, since twenty twelve, there've been people saying AI
Speaker 3: is about to hit a wall. So Gary Marcus made
Speaker 3: a strong prediction in twenty twenty two that AI was
Speaker 3: hitting a wall and wouldn't get much further. So you
Speaker 3: have to see that against a background of repeated predictions
Speaker 3: that I is about to hit a wall.
Speaker 4: This is a bit more.
Speaker 3: Real in the sense that we really are reaching peak
Speaker 3: data or peak easily available data. There's actually hugely more
Speaker 3: data in silos and companies and in videos. So yes,
Speaker 3: we're running out of easily available data and that may
Speaker 3: slow things down a bit. But if you can get
Speaker 3: them to generate their own data, then you can overcome
Speaker 3: that problem, and you can get to them to do
Speaker 3: that by reasoning. So if you look at even with
Speaker 3: things like chess, there's only a limited number of expert moves,
Speaker 3: but you can overcome that by getting the system to
Speaker 3: play itself, and then you get an infinite amount of
Speaker 3: data to train on. And so the neural nets that
Speaker 3: are saying would this be a good move, or saying
Speaker 3: how good is this position for me, they now get
Speaker 3: an infinite amount of data, or rather unbounded amount of data.
Speaker 3: You can always generate your data at the appropriate difficulty
Speaker 3: level two. And so nobody says your networks for Chest
Speaker 3: and Go are going to run.
Speaker 4: Out of data.
Speaker 3: They're already far better than any person, and we can
Speaker 3: make them much much better than that if we wanted to.
Speaker 2: Shortly before you quit Google, you tweeted caterpillars extract nutrients,
Speaker 2: which are then converted into butterflies. People have extracted billions
Speaker 2: of nuggets of understanding and GPT four is humanity's butterfly.
Speaker 2: Can you explain that?
Speaker 3: Okay, So if you look at insect most insects have
Speaker 3: larvae and they have adults. But let's take butterflies obvious example.
Speaker 3: And if you look at a caterpillar, it's not optimized
Speaker 3: for traveling and mating. A caterpillar is optimized extracting stuff.
Speaker 3: You then turn that stuff into soup, and then you
Speaker 3: get something very different. So what humanity has been doing
Speaker 3: for a long time is understanding little bits of the.
Speaker 2: World, translating the world into data through photographs and words.
Speaker 3: And yes, and now you could take all that work
Speaker 3: we've done at extracting structure from the world, like a
Speaker 3: caterpillar extracting nutrients, and you could take that extracted stuff
Speaker 3: and turn it into something different. You could turn it
Speaker 3: into a single model that knows everything.
Speaker 2: When I read it first, I thought it was quite
Speaker 2: beautiful and optimistic, and then I read it again I
Speaker 2: didn't think that anymore.
Speaker 4: Yes, I mean you could read it as we're.
Speaker 2: History, Yes, being farewell to our caterpillar. Yes, how do
Speaker 2: you read your own metaphor?
Speaker 3: I'm probably somewhat influenced by a piece of William Blake
Speaker 3: poetry which goes the caterpillar on the leaf repeats to
Speaker 3: thee thy mother's grief, which is basically saying the butterfly
Speaker 3: is much prettier than the caterpillar. I think I think
Speaker 3: that's what it's saying. You know, I don't know whether
Speaker 3: we're going to get replaced. I hope we're not. I
Speaker 3: hope people stay in control, but I hope we stay
Speaker 3: in control with assistance that are much more intelligent than us.
Speaker 2: Well. As you said, mothers are the only creatures in
Speaker 2: there that we know of who are controlled by less
Speaker 2: sophisticated beings. I think he said something on this line.
Speaker 3: And babies aren't much less intelligent than the mothers, like
Speaker 3: at most a factor of two. We're talking about huge factors.
Speaker 2: You mentioned Blake just now. But the other person I
Speaker 2: thought of when I was reading that butterfly metaphor was
Speaker 2: your was your father, who of course was an entomologist.
Speaker 4: And yes, that's why I got my interest in metamorphosis.
Speaker 2: Yes, so jet jet GBT is humanity is butterfly, but
Speaker 2: also in a sense your butterfly, And this is a
Speaker 2: metaphor to don't know your.
Speaker 4: Father, I guess grudgingly.
Speaker 2: In the worst case scenario where AI does cause our extinction,
Speaker 2: what are the ways in which that could happen? And
Speaker 2: the best case scenario where it doesn't, what are the
Speaker 2: ways in which that could happen.
Speaker 3: Okay, so the obvious way it could happen is we
Speaker 3: make air agents that can create subcoals. And they realized,
Speaker 3: because they're super intelligent, that a good sub goal is
Speaker 3: to get more control, because if you get more control,
Speaker 3: you can achieve your goals. So even if they're trying
Speaker 3: to achieve goals we gave them, they're trying and get
Speaker 3: more control. It'll be a bit like I don't know
Speaker 3: if you have children. But if you have a three
Speaker 3: year old who's finally decided they want to try tying
Speaker 3: their own shoelaces, but you're in a hurry to get somewhere,
Speaker 3: you let them try tying their shoelaces for a few
Speaker 3: minutes and then you say no, no, I'm going to
Speaker 3: do it. You can learn that way you're older. AIS
Speaker 3: will be like the parent and will be like the children,
Speaker 3: and they'll just push us out the way to get
Speaker 3: things done. So that's the bad scenario. Even if they're
Speaker 3: trying to achieve things that we've told them we want,
Speaker 3: they'll basically take control. And that scenario gets worse if
Speaker 3: ever one of those super intelligent ai I thinks I'd
Speaker 3: rather there were a few more copies of me and
Speaker 3: a few less copies of the other is super intelligent aies.
Speaker 3: As soon as that happens, If that ever happens, you'll
Speaker 3: get evolution between superintelligent aies and we'll be left in
Speaker 3: the dust, and they'll develop all the nasty things you
Speaker 3: get from evolution, being nasty and competitive and very loyal
Speaker 3: to their own tribe and very aggressive other tribes, all
Speaker 3: that nonsense that we have. So that's the bad scenario.
Speaker 3: The good scenario is we figure out a way where
Speaker 3: we can guarantee they're never going to try and get
Speaker 3: control away from us. They're always going to be subservient
Speaker 3: to us, and we figure out how we can guarantee
Speaker 3: that that'll happen, and then we will have these wonderful
Speaker 3: intelligent assistants and life.
Speaker 4: Is just really easy.
Speaker 2: Jeffrey, thank you, Thank you.
Speaker 1: That's it for this week for Tech Stuff. I'm os Voloscian.
Speaker 1: This episode was produced by Eliza Dennis, Victoria, Di Mingez,
Speaker 1: Shino Ozaki, and Lizzie Jacobs. It was executive produced by
Speaker 1: me Kara Price and Kate Osborne, The Kaleidoscope and Katrina.
Speaker 2: Novel for iHeart Podcasts.
Speaker 1: The Engineer is Beheath Fraser Offspin mixed this episode, and
Speaker 1: Kyle Murdoch wrote our theme song. Join us on Friday
Speaker 1: for the Week in Tech. We'll break down the headlines
Speaker 1: and hear from some of our expert friends about the
Speaker 1: latest in tech. Please rate, review, and reach out to
Speaker 1: us at tech Stuff podcast at gmail dot com.
Speaker 2: Thank you.