Speaker 1: Pushkin. Artificial intelligence is this weird, big phrase that suddenly
Speaker 1: seems to be everywhere, and it can be hard to
Speaker 1: know exactly what it means. But when businesses say they're
Speaker 1: using artificial intelligence, they usually mean one particular thing. They
Speaker 1: mean automated systems that can take in lots and lots
Speaker 1: of data and use the data to make predictions. This
Speaker 1: is called machine learning, and it's spreading everywhere. Drug companies
Speaker 1: use it to predict which molecules are likely to work
Speaker 1: as medicines. Hedge funds use it to predict which stocks
Speaker 1: are going to go up or down. Instagram uses it
Speaker 1: to predict which adds I'm most likely to click on
Speaker 1: For the record, the machine has learned that I will
Speaker 1: often click on ads for overpriced workout clothes. Anyway, if
Speaker 1: you want to understand what's happening with business and technology today,
Speaker 1: you really have to understand machine learning. I'm Jacob Goldstein.
Speaker 1: This is What's Your Problem, the show where entrepreneurs and
Speaker 1: engineers talk about how they're going to change the world
Speaker 1: once they've solved a few problems. My guest today is
Speaker 1: Luis Vonon, the founder and CEO of Duolingo. Duolingo is
Speaker 1: both a wildly popular language app and also a hardcore
Speaker 1: tech company built on machine learning. Luis used to be
Speaker 1: a professor of computer science at Carnegie Mellon, and in
Speaker 1: our conversation he was really candid about the technical limits
Speaker 1: of what Duolingo can do today. The app is good
Speaker 1: at teaching people to read and to understand, he said,
Speaker 1: but Duolingo is not as good at teaching people to
Speaker 1: speak a new language. And solving that problem turns out
Speaker 1: to be part of this great, big, interesting frontier problem
Speaker 1: that is relevant not just for Duolingo, but for the
Speaker 1: whole field of artificial intelligence. We started out talking about
Speaker 1: the origins of Duolingo, which go back to a different problem,
Speaker 1: one that Louis discovered before he'd ever heard of machine learning.
Speaker 1: It was a problem he saw all around him when
Speaker 1: he was growing up in Guatemala. I was fortunate that
Speaker 1: my mother basically spent essentially her entire net worth on
Speaker 1: my education, and so I was fortunately that I got
Speaker 1: a good education. But then I could see the people
Speaker 1: who would get public education barely learn how to read
Speaker 1: and write. This is just what would happen, And you
Speaker 1: cannot expect that these people are going to become, you know,
Speaker 1: the CEO of a public company or anything like that,
Speaker 1: because they kind of wont. Often people talk about education
Speaker 1: as an engine for reducing inequality, but what you're describing
Speaker 1: is the exact opposite, and it's education, when you have
Speaker 1: to pay for it, is a mechanism for perpetuating inequality.
Speaker 1: And I really believe that, and I believe that's true
Speaker 1: in most countries in the world. There may be some countries,
Speaker 1: you know, like the Scandinavian countries, where pretty much everybody
Speaker 1: gets the same education everything, right, Yeah, there may be
Speaker 1: some countries like that, But in the vast majority of countries,
Speaker 1: if you have money, you can get a much better education.
Speaker 1: So I wanted to do something that would give equal
Speaker 1: access to education to everybody, and so we started with that.
Speaker 1: But then we started thinking, Okay, if education is pretty general,
Speaker 1: let's start by teaching one thing. Eventually we settled on
Speaker 1: teaching languages for a number of reasons, the biggest one
Speaker 1: of which is that learning English in particular can completely
Speaker 1: change people's lives. If you know English, you can double
Speaker 1: your incompotential in most countries. Wow, it's just as simple
Speaker 1: as that. And so it's it's why why is that.
Speaker 1: Basically it opens up for almost any job. You can
Speaker 1: get a better version of that job. For example, you
Speaker 1: could be a waiter, or you could be a waiter
Speaker 1: at the five star hotel. You could be an executive assistant,
Speaker 1: or you could be an executive assistant for a multinational
Speaker 1: ceoyeah yeah, okay, So really teaching English the core, the
Speaker 1: core sort of reducing inequality dream of a language is
Speaker 1: really teaching English to people in largely in poorer countries. Yes,
Speaker 1: so what we wanted to do was teach teach English.
Speaker 1: But you know, if you're going to teach English, we
Speaker 1: may as well teach other languages, So teach those and
Speaker 1: do so for free. So that's what Luis did and
Speaker 1: it worked. Today, tens of millions of people use Dual
Speaker 1: Lingo every month to learn English and dozens of other
Speaker 1: languages for free. The company makes money by selling ads
Speaker 1: and premium subscriptions. It went public in twenty twenty one
Speaker 1: and is currently worth billions of dollars. And the company
Speaker 1: really is built on machine learning. Luis gave me a
Speaker 1: few key examples of the way the company uses the technology.
Speaker 1: So let me tell you a few of the things
Speaker 1: that we do. One of the things that we do
Speaker 1: we like very much is we have data on whenever
Speaker 1: people use dual Lingo. We record every exercise that they
Speaker 1: do and whether they got it right or wrong, and
Speaker 1: if they got it wrong, why they got it wrong.
Speaker 1: With all of this data, we're able to do certain
Speaker 1: things with artificial intelligence. For examples, for every exercise that
Speaker 1: we're about to give you able to predict what is
Speaker 1: the probability that you're going to get this exercise right
Speaker 1: or wrong. So, in a sense, that is a thing
Speaker 1: that a teacher in a classroom could do fairly easily,
Speaker 1: right a teacher with twenty students, But you're able to
Speaker 1: do it with whatever how many people use your app
Speaker 1: actively forty two million per month, so the machine can
Speaker 1: do that for all forty two million people at the
Speaker 1: same time. More or less, yes, and very accurately. Part
Speaker 1: of the secret source of Duelingo is that we realized
Speaker 1: if we were to only give you things that you're
Speaker 1: not very good at, we'd basically be giving you lessons
Speaker 1: from hell every time. So we can't do that because
Speaker 1: that frustrates users. So what we do is, whenever you
Speaker 1: start a listening and doing we're actually trying to optimize
Speaker 1: for two things at the same time. We're trying to
Speaker 1: teach you things you know that you don't not very
Speaker 1: good at, but also we're trying to keep you motivated
Speaker 1: and engaged. Yea. And the way we do that is
Speaker 1: we try to give you exercises for which we know
Speaker 1: you have about an eighty percent chance of getting them right. Huh.
Speaker 1: And have you found that to be the sweet spot?
Speaker 1: I mean, have you done like experiments and sort of
Speaker 1: turn the dial. We've done that, and we're you know,
Speaker 1: we're not the first to figure this out. I mean,
Speaker 1: there's a lot of literature and psychology, etc. Just and
Speaker 1: the number is not exactly eighty percent. It's a little
Speaker 1: higher than that. It's like eighty three percent or something.
Speaker 1: But there's there's a number, and it really is the
Speaker 1: case that if that number is higher, that means these
Speaker 1: things are a little easier for you. Then you get
Speaker 1: a little bored. You feel like you're not learning right.
Speaker 1: If I'm getting ninety five percent right, I'm like, what,
Speaker 1: I'm just wasting my time? And you feel bored because
Speaker 1: it's like it's like a game that you always win.
Speaker 1: I mean, that's that's nice. At the very beginning, but
Speaker 1: then you're just not going to play it, and then
Speaker 1: if it's lower than that, that means that things are
Speaker 1: too hard for you. You get very frustrated and you
Speaker 1: go away. And there's a lot of tricks that you know,
Speaker 1: certainly app developers play, and you know we play as well.
Speaker 1: So I'll tell you another kind of similar trick. You know,
Speaker 1: we end up applying it to language. But the easiest
Speaker 1: way to understand this trick is with a slot machine.
Speaker 1: When you get two out of three, it's you almost
Speaker 1: got it, you gotta do one more, you gotta do
Speaker 1: one more. You just gotta do one more because you
Speaker 1: got it. So there's this, there's this you're so close
Speaker 1: to psychological trick that we played. It's like, oh, there's
Speaker 1: two out of three, almost got it, But you knew
Speaker 1: I was going to get two out of three. Yes, sure,
Speaker 1: you gave me two easy ones and when there was
Speaker 1: super hard that's exactly right. So so we we played
Speaker 1: this type of trick where just people are like, almost
Speaker 1: got it, and that gets them to do another one.
Speaker 1: So you know, in our case, we just we basically
Speaker 1: spend a lot of time training computers to figure out
Speaker 1: what it is that makes people use Duolingo for longer,
Speaker 1: and also that we teach them more so that that's
Speaker 1: a major use for artificial intelligence. Main use is just
Speaker 1: in teaching better. After the break a big problem, Louise
Speaker 1: and dual Lingo are still trying to solve a problem
Speaker 1: that turns out to be a big frontier problem for
Speaker 1: all of artificial intelligence. That's the end of the ads.
Speaker 1: Now we're going back to the show. So let's talk
Speaker 1: now about problems you haven't solved yet. You know, like,
Speaker 1: what are you what are you trying to figure out?
Speaker 1: What are you working on that that isn't quite working yet.
Speaker 1: So dual linguals is very effective at teaching you all
Speaker 1: kinds of things. But if you go look under the
Speaker 1: hood or you know, what is it that you're learning.
Speaker 1: You're learning reading really well. You're learning writing pretty well,
Speaker 1: but not as well as reading. You're learning listening pretty well,
Speaker 1: but you're not learning spontaneous speaking very well. This, by
Speaker 1: the way, is also you're not something you're not learning
Speaker 1: very well in university semesters, Like you're basically not learning
Speaker 1: that well either in due lingo or in university semesters. Okay,
Speaker 1: it's just harder to teach in a sort of classroom.
Speaker 1: What you need to do to teach that is basically,
Speaker 1: have you really interact with wealth? For now another human
Speaker 1: and just you just practice that a lot. Now, here's
Speaker 1: the here's the thing about that. I know how to
Speaker 1: get you to interact with another human. Just put another
Speaker 1: human there. The problem is about eighty percent of our
Speaker 1: users just does not want to talk to a stranger
Speaker 1: in a language that they're not very good. So the
Speaker 1: problem that we're trying to solve here is how do
Speaker 1: we practice kind of spontaneous conversation but without having a
Speaker 1: human on the other side. And we've been working on that,
Speaker 1: and you know we're not there yet. Can I just interrupt?
Speaker 1: Because I mean we were talking about artificial intelligence? Right?
Speaker 1: The most famous test that I know of, the most
Speaker 1: famous idea I know of of artificial intelligence in a
Speaker 1: computer is can a computer hold one end of a conversation? Right? Like?
Speaker 1: That's the classic touring test is like you're going to
Speaker 1: have this like chat conversation and can you tell if
Speaker 1: the person on the other end is a person or
Speaker 1: a machine? Like? That's the og artificial intelligence idea, right,
Speaker 1: I mean, are you telling me, that's what you're trying
Speaker 1: to solve. Not quite. I mean, it would be awesome
Speaker 1: if we would solve that. I mean, but that's the dream,
Speaker 1: right solution to that, that is the dream. But notice
Speaker 1: in our case, we don't actually care if the human
Speaker 1: can tell that there's a computer on the other side. Okay,
Speaker 1: it's okay. As long as it practices thing and as
Speaker 1: long as you're able to carry on a conversation in
Speaker 1: a way that seems a little natural or something, it's okay.
Speaker 1: If if it, you know, goes off the rails every
Speaker 1: now and then, So tell me, what is it that
Speaker 1: you're trying to build. This is exciting, Like what are
Speaker 1: you trying to do? We're starting with text, by the way,
Speaker 1: so either just basically a texting conversation. So think of
Speaker 1: it as like a chat bot in uh, you know,
Speaker 1: in Spanish, where it just you're just having a real
Speaker 1: a little conversation. You've had lots of people have had
Speaker 1: experience with chat bots. Yes, they're right, Like you go
Speaker 1: to whatever, cancel your cable and they want you to text,
Speaker 1: and then you realize you're texting with the machine. So
Speaker 1: like that's the that's step one. So that's the idea.
Speaker 1: That's step one. We of course, I mean a lot
Speaker 1: of those experiences with chatbots are are very um, they're
Speaker 1: just very geared at whatever it is you're trying to do. So,
Speaker 1: for example, that chapel maybe very good at at canceling
Speaker 1: your cable, but only that in my experience, they're not
Speaker 1: even good at that. No, they're not that great. So
Speaker 1: we're trying to do that, and you know, we're not
Speaker 1: there yet. I don't think so you're like out on
Speaker 1: the frontier. We are. We are like you're trying to, yes,
Speaker 1: and we're not there yet. I mean, this is something
Speaker 1: that's going to take us, not just us, I mean
Speaker 1: the whole academic community and technology just a few more years.
Speaker 1: But so let me ask you this. Can we talk
Speaker 1: about that in a way that would be like, can
Speaker 1: we try and just go one level into sort of
Speaker 1: what you're trying to do and like what works and
Speaker 1: what doesn't work, and like why it's hard? Yeah, I mean,
Speaker 1: you know, the first, by the ways, the first way
Speaker 1: you think of if you're trying to make a chap
Speaker 1: the first thing you think of is, okay, I'm just
Speaker 1: going to program the computer. Forget about artificial intelligence. I'm
Speaker 1: just going to program the computer to respond to specific questions,
Speaker 1: and how many possible questions could there be? You start thinking, okay, well,
Speaker 1: when the person says high, we're going to program a
Speaker 1: think to say hi back. When the person says, how
Speaker 1: are you doing, We're going to program I think to
Speaker 1: say I'm doing pretty well? How about you? Yeah? V
Speaker 1: zero of a chatbot. And this, you know, this comes
Speaker 1: from you know, fifty years ago. This is what you
Speaker 1: start doing. The problem is there's billions of things that
Speaker 1: people can say, and so we may have programmed the
Speaker 1: thing of what to say, but you know how you're doing,
Speaker 1: and we can respond. But if instead of asking that,
Speaker 1: they may ask like, hey, did you watch the game
Speaker 1: last night? And we just have no idea how to
Speaker 1: respond to that. About a decade ago, Louis says Ai,
Speaker 1: researchers started trying a really different approach. Rather than trying
Speaker 1: to teach computers every rule, they started throwing massive amounts
Speaker 1: of documents and texts at computers and essentially telling the
Speaker 1: computers figure out the patterns in all these documents. So
Speaker 1: when somebody writes something like did you watch the game
Speaker 1: last night? The computers should be able to predict what
Speaker 1: kinds of answers might follow. This strategy clearly has not
Speaker 1: entirely worked yet. That's why it's still a problem solving.
Speaker 1: It will take both more text and more clever algorithms
Speaker 1: to help computers make sense of that text. But Louis says,
Speaker 1: you can see progress every time you open your Gmail
Speaker 1: or a Google Doc. And I don't know if you've used,
Speaker 1: for example, you use Google Docs lately or Gmail like
Speaker 1: it finishes off your sentences now. And basically the way
Speaker 1: this works is, you know, this system has looked at
Speaker 1: a ton ton of text that has been written by
Speaker 1: a lot of people. In the case of Google Docs,
Speaker 1: I actually don't know what they look at, but I
Speaker 1: wouldn't be surprised if they look at everything that has
Speaker 1: ever been written in Google Docs. I'm going to tell
Speaker 1: you one that happened to me in Google Docs today
Speaker 1: when I was typing notes for this interview, I typed
Speaker 1: zone of pr and then you know what, you know
Speaker 1: how it completed it proximal development. Yes, it knew I
Speaker 1: was going to write zone of proximal development. Yep. No,
Speaker 1: this is amazing, And they just see that if you'd
Speaker 1: write the zone of per there's like a ninety five
Speaker 1: percent chance that it ends in proximal development. What is
Speaker 1: the zone of proximal development? You know? In teaching, you know,
Speaker 1: there's this concept of just keeping you at this zone
Speaker 1: of proximal development, which is always kind of challenging you,
Speaker 1: giving you things that you don't know. But but there
Speaker 1: are all things that are fair to give you. Proximal
Speaker 1: means like close to or next two. Right. So it's
Speaker 1: the idea is like you know a thing, yes, then
Speaker 1: like what you want to teach the person is the
Speaker 1: very next thing, right, It's like that's right, It's like
Speaker 1: the frontier of your knowledge. I like it because it
Speaker 1: applies to like the way you teach, but also to
Speaker 1: your work, right, And like it's just a nice life idea, right,
Speaker 1: It's like the next thing you want, the next thing.
Speaker 1: And I feel like the chat bot is maybe a
Speaker 1: version of that at the level of your company. Yeah, yeah,
Speaker 1: it really is. It really is a nice idea, and
Speaker 1: it is I mean, and if you think about it,
Speaker 1: this is what a great teacher does. You know. I've
Speaker 1: said this inside the company at due Lingo. UM. All
Speaker 1: we need to do is first figure out what you know,
Speaker 1: by the way, not that easy to figure out what
Speaker 1: you know. But let's first figure out what you know
Speaker 1: and then just take you to that zone of proximal development.
Speaker 1: Because now we know what you know, just take you
Speaker 1: to the frontier and then just keep expanding it as
Speaker 1: fast as possible. That's all we need to do. Of course,
Speaker 1: this is easily said, hard to do. Yeah. Um, And
Speaker 1: is there a limit to what you can do with
Speaker 1: a computer? Is there anything a teacher can do? Is
Speaker 1: that a computer will never be able to do? You know?
Speaker 1: Of course I do lingual We love teachers. If they
Speaker 1: are a good teacher and also have the time, they
Speaker 1: are much more able to adapt to their students than
Speaker 1: a computer is. Um. But I don't believe that will
Speaker 1: always be the case. I mean, I think at some
Speaker 1: point it's not just teachers. I mean teachers, this is
Speaker 1: one thing. I mean, at some point my belief and
Speaker 1: this is of course just my belief. People, not everybody agrees.
Speaker 1: They believe that computers will be able to do every
Speaker 1: single thing that humans can. Now you may start asking
Speaker 1: really tough questions like can they love? Yeah, I don't know,
Speaker 1: I don't know what they can love or not but
Speaker 1: from the outside it will look just as if they
Speaker 1: love so who knows who knows what's going on inside?
Speaker 1: Who knows that they that's like a big yeah, we're
Speaker 1: big philosophical questions that I'm not here today, and that's right,
Speaker 1: nor am I. But I do think from input output behavior,
Speaker 1: I don't see why. I don't see any reason why
Speaker 1: computers won't be able to do everything that humans can.
Speaker 1: So they can teach, but they can also write a
Speaker 1: computer code. They can also run companies, they can also
Speaker 1: make podcasts, they can do everything. Should be able to
Speaker 1: do that. I think they should be able to do that.
Speaker 1: I don't know when that'll happen, but they should be
Speaker 1: able to do that. In a minute, the lightning round, well,
Speaker 1: hear what job Luise would love to do but thinks
Speaker 1: he wouldn't be very good at. And the real reason
Speaker 1: treasure Chess keep showing up in duo lingo. And now
Speaker 1: back to the show. We're going to finish with a
Speaker 1: lightning round, not counting duo Lingo. What's your favorite app
Speaker 1: on your phone? Spotify? What have you been listening to
Speaker 1: on Spotify? I'm always a huge fan of the band
Speaker 1: called Churches with a v to Virchase. So that's what
Speaker 1: I was listening to this morning on my work at work.
Speaker 1: If you have a ten minute break in the middle
Speaker 1: of the day, what do you do to relax? Played
Speaker 1: this game called Class Royale. We are a lot of
Speaker 1: the gaming mechanics that we use for duel and will
Speaker 1: come from gaming companies, like the treasure chests, exactly right,
Speaker 1: the treasure chests. If you ever played Class Royale, they
Speaker 1: have the treasure chests. If somebody's going to go to
Speaker 1: visit Guatemala for the first time, what's one thing they
Speaker 1: should definitely do? Oh? Um, Decal is the Mayan ruins. Um.
Speaker 1: You know, if I feel very strong, I've been to
Speaker 1: southern Mexico where they have chi Chenitsa. It's a joke
Speaker 1: compared to the Mayan ruins in Guada. There's there's one
Speaker 1: pyramid in Chichenitsa. There are four hundred in Guatemala in Decal,
Speaker 1: So yeah, they should like I like that. Not only
Speaker 1: are you recommending Tikal, you're also taking as I have
Speaker 1: no trouble with Chichenitsa. It's just they are very good
Speaker 1: at marketing. Amazing. What would you do if you couldn't
Speaker 1: do the job you do. Now, well there's what what
Speaker 1: would I actually do? On? What would I'd like to do?
Speaker 1: I would love to be a writer. I don't think
Speaker 1: i'd be a very good one. Um So if I
Speaker 1: if I wasn't doing the job that I'm doing right now,
Speaker 1: you know, I'd probably be back to being a professor.
Speaker 1: How will you know when it's time to retire? I'm
Speaker 1: never retiring, That's what everybody says. Well, maybe I will,
Speaker 1: but I mean right now, I don't. I don't want
Speaker 1: to do that. Luis Vaughan is the founder and CEO
Speaker 1: of Duelingo. Today's show was produced by Edith Russelo. It
Speaker 1: was edited by Kate Parkinson Morgan and Robert Smith, and
Speaker 1: it was engineered by Amanda kay Wong. Theme music by
Speaker 1: Louis Kara. Our development team is Lee, Tom Mulad and
Speaker 1: Justine Lang. A huge team of people makes What's Your
Speaker 1: Problem possible. That team includes, but is not limited to,
Speaker 1: Jacob Weisberg, Mia Lobel, Heather Fain, John Schnars, Kerry Brodie,
Speaker 1: Carli mcgleory, Christina Sullivan, Jason Gambrell, Brand Hayes, Eric Sandler,
Speaker 1: Maggie Taylor, Morgan Ratner, Nicolemrano, Mary Beth Smith, Royston Deserve,
Speaker 1: Maya Kanig, Daniello, Lakhan, Kazia Tan and David Clever. What's
Speaker 1: Your Problem is a co production of Pushkin Industries and iHeartMedia.
Speaker 1: To find more Pushkin podcast Us, listen on the iHeartRadio app,
Speaker 1: Apple Podcasts, or wherever. I'm Jacob Goldstein and I'll be
Speaker 1: back next week with another episode of What's Your Problem.