WEBVTT - Ep96 "What’s the future of education in an AI world?" (Part 2: Sal Khan) 0:00:05.200 --> 0:00:07.840 What are the most important things for our students to 0:00:07.960 --> 0:00:10.719 learn now that we're in a world where AI is 0:00:10.840 --> 0:00:15.160 so much smarter than any human, Instead of learning skills 0:00:15.200 --> 0:00:18.120 that may not matter as much in the future, can 0:00:18.160 --> 0:00:21.000 AI help us to learn the important things? 0:00:21.320 --> 0:00:24.239 And what are the important things? Do we give up on. 0:00:24.360 --> 0:00:28.320 Education or are the very clever ways that we can 0:00:28.400 --> 0:00:31.160 direct our teaching to get the best out of our schools. 0:00:31.480 --> 0:00:34.760 What would Saul Khan, founder of the Kahn Academy say 0:00:34.800 --> 0:00:37.479 about all this? Well, you're lucky because you're about to 0:00:37.520 --> 0:00:43.720 find out. Welcome to inner Cosmos with me David Eagelman. 0:00:44.040 --> 0:00:47.200 I'm a neuroscientist and author at Stanford and in these 0:00:47.240 --> 0:00:51.040 episodes we dive into our three pound universe to uncover 0:00:51.200 --> 0:00:54.440 some of the most surprising aspects of our lives and 0:00:54.520 --> 0:01:07.160 our times. Today's episode is part two about the future 0:01:07.520 --> 0:01:11.399 of education. In part one last week, I covered the 0:01:11.560 --> 0:01:17.080 long road of how technology changes schooling. For example, when 0:01:17.080 --> 0:01:20.520 the printing press was invented in fourteen forty, some thinkers 0:01:20.520 --> 0:01:23.800 of the time were sure that invention would dumb us 0:01:23.800 --> 0:01:26.440 all down, because now if you ask a student for 0:01:26.480 --> 0:01:29.440 an answer, the answer is right there, It's written down. 0:01:29.480 --> 0:01:31.959 You just pull the book off the shelf and wham, 0:01:32.000 --> 0:01:35.440 it's as though you can pretend you know something. And 0:01:35.520 --> 0:01:39.959 we went through similar debates when desk calculators were introduced, 0:01:40.200 --> 0:01:42.800 will the kids become stupid? And then there was the 0:01:42.880 --> 0:01:45.360 invention of the World Wide Web, And then when they 0:01:45.800 --> 0:01:48.640 Google burst onto the scene, and all you had to 0:01:48.680 --> 0:01:52.000 do was type your question into this little box and 0:01:52.080 --> 0:01:54.480 it had indexed all the information in the world and 0:01:54.560 --> 0:01:57.480 it could point you precisely where you needed. 0:01:57.200 --> 0:01:58.760 To go to find the answer. 0:01:59.400 --> 0:02:03.120 So suddenly it didn't matter what question the teacher asked you. 0:02:03.520 --> 0:02:05.440 As long as somebody somewhere. 0:02:05.080 --> 0:02:07.240 In the world knew the answer and had typed it 0:02:07.280 --> 0:02:09.919 on the web page, it was just one click away 0:02:09.919 --> 0:02:13.920 from you. In all these cases, educators were forced to 0:02:14.120 --> 0:02:18.840 change the way they asked questions and assessed a student's knowledge. 0:02:19.200 --> 0:02:21.080 And you know what, it wasn't that hard. 0:02:21.880 --> 0:02:24.880 So where are we with technology in general and AI 0:02:24.960 --> 0:02:31.280 in particular right now? I made three main arguments in 0:02:31.360 --> 0:02:36.160 last week's episode. I argued that the extant technology wires 0:02:36.240 --> 0:02:39.880 the brain of each generation differently, as in how they're 0:02:39.919 --> 0:02:42.960 wired to best learn and the ways they seek information. 0:02:43.520 --> 0:02:46.400 Then I gave many examples of the ways in which 0:02:46.440 --> 0:02:50.239 AI can be used to massively improve the way we teach, 0:02:50.680 --> 0:02:54.600 from individualized tutoring to reducing the workload on the teachers, 0:02:54.840 --> 0:02:59.400 to training critical thinking and magnifying creativity. If you want 0:02:59.440 --> 0:03:02.559 to hear more about that, please check out that last episode. 0:03:02.639 --> 0:03:05.440 And finally, I made the argument that the purpose for 0:03:05.560 --> 0:03:10.960 schooling for education is to give the students roots and wings, 0:03:11.320 --> 0:03:14.600 as Gerta phrased it, which just means giving them the 0:03:14.680 --> 0:03:18.480 foundations that they need for critical thinking and the ability 0:03:18.520 --> 0:03:21.880 to create new things and go off in novel directions. 0:03:22.760 --> 0:03:23.760 So today we're going to. 0:03:23.680 --> 0:03:25.800 Get someone else's take on the state of things, and 0:03:25.840 --> 0:03:29.600 not just anyone, but someone who has single handedly had 0:03:29.639 --> 0:03:33.320 perhaps the largest impact on the planet's education. 0:03:33.440 --> 0:03:35.640 That I know. That's all Con. 0:03:35.920 --> 0:03:37.640 And if you don't know his name, you surely know 0:03:37.720 --> 0:03:42.280 his creation, the Con Academy, which is an educational platform 0:03:42.640 --> 0:03:47.280 that provides free and very high quality online courses. It 0:03:47.320 --> 0:03:50.760 does this across math and science, and history and economics. 0:03:50.440 --> 0:03:51.680 Anything you want to learn. 0:03:51.840 --> 0:03:56.440 It has video lessons and interactive exercises and learning tools, 0:03:56.840 --> 0:03:59.760 and the key is, unlike a normal school system, this 0:04:00.080 --> 0:04:04.000 allows users to progress at their own pace, so Saul 0:04:04.080 --> 0:04:07.280 has a unique vantage point to view the world's education 0:04:07.840 --> 0:04:10.520 and as my neighbor here in Silicon Valley, he has 0:04:10.560 --> 0:04:14.600 the opportunity to see the latest greatest tech and understand 0:04:14.640 --> 0:04:16.120 where things are going. 0:04:16.720 --> 0:04:20.599 So, without further ado, here's my interview with Saul con. 0:04:24.800 --> 0:04:28.640 Okay, sal So, you're famous for having started the con Academy. 0:04:28.720 --> 0:04:30.160 Tell us about the history of that. 0:04:30.640 --> 0:04:32.480 Oh hi, you know, I guess you go back about 0:04:32.520 --> 0:04:35.560 twenty years ago. A little over twenty years ago. My 0:04:35.640 --> 0:04:38.080 original background was in technology, and I go to business 0:04:38.080 --> 0:04:39.720 school and I find myself. I'm working at a hedge 0:04:39.720 --> 0:04:42.000 fund in Boston. Two thousand and four. I'm a year 0:04:42.000 --> 0:04:45.040 out of business school. I get married. My wife grew 0:04:45.120 --> 0:04:47.080 up in New Jersey, so all of my family from 0:04:47.120 --> 0:04:49.320 New Orleans, where I was born and raised, comes to 0:04:49.360 --> 0:04:51.839 New Jersey for the wedding. While they were staying with me. 0:04:52.520 --> 0:04:54.599 My cousin, it just came out of conversation was having 0:04:54.600 --> 0:04:57.080 trouble with math Navia. She was twelve years old, and 0:04:57.120 --> 0:04:58.840 when I asked her about it is unit conversion, etc. 0:04:59.040 --> 0:05:00.839 I told Navy, I'm a hundred percent sure you can. 0:05:01.040 --> 0:05:03.240 You can learn this material if you're up for it. 0:05:03.279 --> 0:05:05.040 I'm happy to tutor you remotely when you go back 0:05:05.080 --> 0:05:07.920 to New Orleans. She agrees, she goes back. We start 0:05:07.960 --> 0:05:11.080 working together, slowly, but surely she gets unit conversion. She 0:05:11.080 --> 0:05:12.960 gets caught up with her class, littlehead of her class. 0:05:13.400 --> 0:05:15.560 Then I'd become what I call a tiger cousin. And 0:05:15.640 --> 0:05:19.440 I call up her school and they. 0:05:19.320 --> 0:05:21.000 Said, who are you? I said, I'm her cousin. 0:05:21.480 --> 0:05:24.320 And I actually convinced them to let her retake a 0:05:24.360 --> 0:05:26.280 placement exam because I thought, like, well, she knows her 0:05:26.279 --> 0:05:29.600 material now, and they did, and she went from being 0:05:29.680 --> 0:05:31.560 like a for lack of better word, to remut, like 0:05:31.800 --> 0:05:34.440 in a slower track, to not being in a faster track. 0:05:34.520 --> 0:05:35.840 So I was like, well, this is cool. Like I'm 0:05:35.880 --> 0:05:37.200 able to connect with a young cousin. 0:05:38.000 --> 0:05:40.359 I start tutoring her younger brother's words preadient, the family 0:05:40.400 --> 0:05:42.680 free tutoring is going on. Before I know it, I'm 0:05:42.720 --> 0:05:45.440 tutoring ten to fifteen cousins, family friends. You know, I 0:05:45.440 --> 0:05:49.080 saw a common pattern that they were struggling, not because 0:05:49.080 --> 0:05:51.040 they weren't going to good schools, not because they didn't 0:05:51.040 --> 0:05:53.719 have good teachers, not because they weren't hard working or bright. 0:05:54.040 --> 0:05:56.159 It's because they had gaps in their learning, especially in 0:05:56.160 --> 0:05:57.920 a topic like math. But even you know, part of 0:05:57.960 --> 0:05:59.880 the reason we were having trouble in science was gaps 0:06:00.040 --> 0:06:02.840 sometimes in their math. So I started writing software for 0:06:02.880 --> 0:06:06.680 them to kind of give them practice them being your cousins, 0:06:06.680 --> 0:06:08.760 my cousins. Yeah, this was back in the day. If 0:06:08.800 --> 0:06:12.599 you had to look for practice problems on factoring polynomials, 0:06:12.880 --> 0:06:14.640 there were stuff on the internet, but they weren't very 0:06:14.680 --> 0:06:16.080 good and they were very inconsistent. It's like, well, I 0:06:16.080 --> 0:06:17.680 think I could write a little piece of software that'll 0:06:17.880 --> 0:06:20.640 generate problems for them, and then I want to keep 0:06:20.680 --> 0:06:22.680 track of how they're doing. And then that was the 0:06:22.720 --> 0:06:25.120 first conducaut I mean, nothing to do with videos. Then 0:06:25.480 --> 0:06:26.400 this was two thousand and five. 0:06:26.400 --> 0:06:27.680 Then how many people were using it? 0:06:28.279 --> 0:06:31.039 Initially just my cousins, and I was seeing benefit and 0:06:31.040 --> 0:06:33.600 I could keeping track of them. Yeah, ten to fifteen cousins, 0:06:34.520 --> 0:06:37.280 and then words started to spread. Their friends started to 0:06:37.279 --> 0:06:38.680 sign up to it because they're like, oh, this is 0:06:38.680 --> 0:06:41.440 really helping me get good at math, and so their 0:06:41.480 --> 0:06:43.200 friends and at some point had to shut it down. 0:06:43.200 --> 0:06:44.920 There got to about ten thousand folks who were trying 0:06:44.920 --> 0:06:47.000 to use just the software, and it was taking down 0:06:47.040 --> 0:06:50.280 my thirty dollars a month web hosting, and that wasn't 0:06:50.320 --> 0:06:53.360 engineered perfectly either to scale to that. So I actually 0:06:53.360 --> 0:06:55.159 just didn't allow anyone else to sign up, which is 0:06:55.160 --> 0:06:57.920 like the opposite of Silicon Valley, Like, no, I don't 0:06:58.000 --> 0:07:00.080 like it, there's too many people here. And then and 0:07:00.640 --> 0:07:02.800 I was showing this software off at a dinner party. 0:07:02.800 --> 0:07:04.880 And by this point we had moved out to northern California, 0:07:05.360 --> 0:07:07.120 and I was at a dinner party in San Mateo 0:07:07.600 --> 0:07:10.080 and the host and I was showing off this stuff 0:07:10.120 --> 0:07:11.600 I was making, and she said, well, how are you 0:07:11.640 --> 0:07:14.040 scaling your lessons. I'm like, I'm not. He's like, whight 0:07:14.040 --> 0:07:16.480 you record some videos on YouTube for your family. I said, 0:07:16.480 --> 0:07:19.280 that's a dumb idea. You know that's cat's playing piano. 0:07:19.400 --> 0:07:22.720 That's not serious. But I did it anyway. Two thousand 0:07:22.720 --> 0:07:26.720 and six, Okay, and yeah, I just started making mini lessons. 0:07:26.720 --> 0:07:28.560 Actually YouTube forced them to be mini lessons at a 0:07:28.560 --> 0:07:30.560 ten minute limit at the time, on things that I 0:07:30.600 --> 0:07:32.600 found myself having to re explain to my cousins a lot, 0:07:32.640 --> 0:07:34.520 Like I'd explain least common multiple to one of my 0:07:34.560 --> 0:07:36.840 cousins one week and then the next week, I'm re 0:07:36.880 --> 0:07:39.440 explaining it to another cousin, and so I said, well, okay, 0:07:39.520 --> 0:07:41.400 I know least common multiples shows up a lot, you know, 0:07:41.400 --> 0:07:42.120 negative exposed. 0:07:42.120 --> 0:07:42.920 So I just kept doing it. 0:07:42.960 --> 0:07:45.800 I started emailing them and saying, any question you have, 0:07:45.920 --> 0:07:48.560 I'll do it asynchronously. You email me the question, I'll 0:07:48.560 --> 0:07:50.000 give you a video explanation of it. 0:07:50.240 --> 0:07:50.680 Wow. 0:07:50.720 --> 0:07:53.080 And so we started building up a library of that. 0:07:53.200 --> 0:07:56.560 And you know, my cousins really appreciated it. I sometimes joke, 0:07:56.640 --> 0:07:58.280 but it's true. They said they liked me better on 0:07:58.320 --> 0:08:02.000 YouTube than in person. And what they were saying is 0:08:02.040 --> 0:08:05.040 that they like the asynchronous, the on demand, the no shame. 0:08:05.440 --> 0:08:06.800 But we still were getting on the phone and they 0:08:06.800 --> 0:08:10.600 did appreciate me as a human being being in their life, 0:08:11.360 --> 0:08:12.880 and we were able to go deeper on the phone. 0:08:13.040 --> 0:08:15.440 But and yeah, and then one thing led to another. 0:08:15.520 --> 0:08:17.600 Two thousand and seven, two thousand and eight, fifteen to 0:08:17.640 --> 0:08:18.480 one hundred thousand folks. 0:08:18.760 --> 0:08:19.680 I was still at the Hedge Fund. 0:08:20.080 --> 0:08:21.960 I set up as a nonprofit in two thousand and eight, 0:08:22.040 --> 0:08:24.520 the mission Free World Class Education for Anyone, Anywhere. Two 0:08:24.560 --> 0:08:26.800 thousand and nine, quit the day job. Two thousand and 0:08:26.840 --> 0:08:28.720 nine was a hard year, but twenty ten, we got 0:08:28.720 --> 0:08:30.680 our first real funding to become an organization. 0:08:30.880 --> 0:08:31.160 Wow. 0:08:31.200 --> 0:08:34.280 And how many people use con Academy now, Oh, you know, 0:08:34.280 --> 0:08:36.760 it depends on how you account for it. I think 0:08:36.800 --> 0:08:39.719 we're one hundred and seventy million registered users, but. 0:08:39.760 --> 0:08:41.520 You know, not everyone uses it all the time. 0:08:42.880 --> 0:08:44.520 You know, in a given year, it's probably closer to 0:08:44.520 --> 0:08:46.320 about one hundred million folks are using it. 0:08:46.440 --> 0:08:46.960 Incredible. 0:08:47.000 --> 0:08:49.960 I mean, that must make you the biggest single educator 0:08:50.520 --> 0:08:51.719 and now I know you have a whole team of 0:08:51.760 --> 0:08:54.960 people doing it now, but the biggest educational organization in 0:08:55.000 --> 0:08:55.280 the world. 0:08:55.320 --> 0:08:57.920 I would assume. Yes, it depends how you account for it. 0:08:58.160 --> 0:09:00.079 Yeah, I mean, but in some ways yeah, yeah, I 0:09:00.120 --> 0:09:01.960 mean some people can make an argument that you know, 0:09:02.040 --> 0:09:06.079 YouTube amongst other things, does some stuff. But yeah, I look, 0:09:06.120 --> 0:09:07.959 but I always point out to the team that's nice. 0:09:08.000 --> 0:09:09.880 But we have a long way to go because even 0:09:10.080 --> 0:09:13.560 of that one hundred million, there's probably about three million 0:09:13.640 --> 0:09:15.640 that are like really getting a lot out of it. 0:09:15.720 --> 0:09:18.640 They're using it almost like you know, regularly, and those 0:09:18.679 --> 0:09:20.040 people are really getting an education. 0:09:20.280 --> 0:09:20.920 Incredible. 0:09:21.120 --> 0:09:23.600 Okay, So now we find ourselves in twenty twenty five 0:09:23.960 --> 0:09:26.640 and AI is on the scene, and so how are 0:09:26.679 --> 0:09:29.680 you using AI in the con academy. 0:09:29.800 --> 0:09:32.120 Yeah, And I remind my because it's very easy to 0:09:32.120 --> 0:09:35.280 get nambored with technology, especially things as cool as generative AI. 0:09:35.760 --> 0:09:38.840 We had a preview if you rewind up two two 0:09:38.880 --> 0:09:42.000 and a half years summer of twenty twenty two. Sam 0:09:42.000 --> 0:09:44.600 Altman and Greg Brockman reach out to me at the 0:09:44.600 --> 0:09:48.439 time and it's just this kind of cryptic email saying 0:09:48.520 --> 0:09:50.040 and I didn't know them that well. I'd run into 0:09:50.080 --> 0:09:52.600 them at some conferences and this and that, and they said, Hey, 0:09:52.800 --> 0:09:55.040 we're training our new model. We think this is the 0:09:55.080 --> 0:09:57.559 one that's going to really get people interested. We want 0:09:57.559 --> 0:10:00.000 to launch with some social positive use cases that you 0:10:00.000 --> 0:10:02.800 GPT four it ended up being GPT four And this 0:10:02.920 --> 0:10:06.920 was this was about five months before chat GPT came out, 0:10:07.240 --> 0:10:09.440 which actually was a bit of an accident. Chat which 0:10:09.440 --> 0:10:11.440 and Chat GPT wasn't even built on GPT four. It 0:10:11.440 --> 0:10:14.240 was built on GPT three point five. But when they 0:10:14.880 --> 0:10:16.880 emailed me, I was like, I was curious. I had 0:10:16.880 --> 0:10:19.120 seen GPT two and GPT three. I thought they were 0:10:19.200 --> 0:10:21.560 very cool, but I didn't think there were anywhere close 0:10:21.840 --> 0:10:23.040 to ready for prime time. 0:10:23.240 --> 0:10:24.640 But as a nerd, I just wanted to see what 0:10:24.679 --> 0:10:27.360 they were going to do. So two weeks later they 0:10:27.400 --> 0:10:30.640 had finished the training and they had myself and our 0:10:30.679 --> 0:10:34.000 chief Learning officer, Kristin. We were on a zoom with 0:10:34.040 --> 0:10:36.599 them and they had a little chat interface and this 0:10:36.880 --> 0:10:37.280 was new too. 0:10:37.320 --> 0:10:39.199 People don't remember these, you know, just even have a 0:10:39.280 --> 0:10:41.640 chat interface with an AI. And they had an ap 0:10:41.760 --> 0:10:45.240 biology question and there's actually context why they picked ap biology. 0:10:45.559 --> 0:10:47.719 But they said, so, what's the answer and I said, oh, 0:10:48.400 --> 0:10:51.680 c it's osmosis. And then they clicked enter and the 0:10:51.760 --> 0:10:54.320 AI said, oh the answer ce. I'm like, okay, ask 0:10:54.360 --> 0:10:57.200 it why. It explained it. And people are used to 0:10:57.200 --> 0:10:59.760 this now, but in the summer of twenty twenty two, 0:11:00.360 --> 0:11:03.480 this was different than anything anyone had ever seen before. 0:11:03.480 --> 0:11:06.160 And I'm like, okay, this is weird. Ask it why 0:11:06.200 --> 0:11:09.520 the other choices are incorrect? It explained it beautifully, I said, 0:11:09.800 --> 0:11:11.280 ask it tried another question like that. 0:11:11.800 --> 0:11:14.080 It did a great job. And that's when I was like, oh, 0:11:14.120 --> 0:11:15.680 what is this like? Am I being punked? 0:11:15.760 --> 0:11:15.800 Like? 0:11:16.800 --> 0:11:18.920 But we saw it before, not all of open AI 0:11:19.040 --> 0:11:22.240 even saw it. But then they gave Kristen myself access 0:11:22.280 --> 0:11:25.080 to it over the weekend and we couldn't sleep, and 0:11:25.120 --> 0:11:27.280 we couldn't tell anyone about it, Like, because this was 0:11:27.280 --> 0:11:29.480 the biggest secret, I couldn't tell my wife about it. 0:11:30.720 --> 0:11:32.520 But I was like middle of the night, I was 0:11:32.520 --> 0:11:34.000 slacking christ and I was like, look, I just got 0:11:34.080 --> 0:11:35.760 it to write the Decoration of Independence and the voice 0:11:35.760 --> 0:11:36.360 of Donald Trump. 0:11:36.360 --> 0:11:37.079 Isn't this hilarious? 0:11:37.080 --> 0:11:39.160 And I'm writing poems and I'm doing this and look, 0:11:39.160 --> 0:11:40.520 if you prompt it the right way, can. 0:11:40.400 --> 0:11:41.160 Act as a tutor. 0:11:41.679 --> 0:11:45.120 And that's when going back to what Kin Academy was 0:11:45.120 --> 0:11:47.959 always about. As I mentioned, I started tutoring my cousins 0:11:48.240 --> 0:11:50.680 and everything that we've done over the last fifteen twenty years, 0:11:50.679 --> 0:11:53.040 it's really about how do we use technology to approximate 0:11:53.040 --> 0:11:55.840 elements of a tutor on demand bite sized video kind 0:11:55.880 --> 0:11:59.000 of approximates an on demand explanation from a tutor. These 0:11:59.000 --> 0:12:02.600 practice problems that you're doing, the problems that you need 0:12:02.640 --> 0:12:04.360 to do as opposed to what everyone else is doing. 0:12:04.679 --> 0:12:06.000 That's what you would do with a little bit of 0:12:06.080 --> 0:12:07.760 a good tutor. They would give you what you need. 0:12:08.280 --> 0:12:11.720 And when we realize that you could steer or prompt 0:12:11.920 --> 0:12:15.800 these this new generation of models to really be not 0:12:15.880 --> 0:12:18.000 just pretend to be the tutor, but really have strong 0:12:18.120 --> 0:12:19.520 tutor moves, so to speak. 0:12:19.960 --> 0:12:21.440 We're like, Okay, this thing has. 0:12:21.280 --> 0:12:26.600 Issues, It hallucinates, it makes horrible math errors. But if 0:12:26.600 --> 0:12:28.599 we assume that those things can either be mitigated or 0:12:28.640 --> 0:12:31.000 they're going to get a lot better, which has happened. 0:12:31.360 --> 0:12:32.920 This is going to be a game changer for us 0:12:32.960 --> 0:12:34.600 being able to scale up what we think is world 0:12:34.600 --> 0:12:38.720 class education, which is more personalization. Giving what Navia had 0:12:38.760 --> 0:12:41.600 in sal what if every student we could eventually get there. 0:12:41.920 --> 0:12:44.240 So let's double click on that idea about an AI tutor, 0:12:44.280 --> 0:12:46.880 because I heard it talk from Steve Jobs way back 0:12:46.920 --> 0:12:48.640 in the day, maybe in the nineties or something. He said, 0:12:48.720 --> 0:12:51.560 just imagine if everyone had Aristotle as a tutor, if 0:12:51.600 --> 0:12:54.680 I'm remembering what he said, treg But I asked my 0:12:54.840 --> 0:12:56.839 son about that at someonhere. He's thirteen years old, and 0:12:56.920 --> 0:12:58.559 I said, you know, hey, what would it be like 0:12:58.600 --> 0:13:00.319 for you to have Aristotles a tutor? And he's said, 0:13:00.600 --> 0:13:02.680 I probably wouldn't do much with it. And the reason 0:13:02.720 --> 0:13:06.120 he said that is because, first of all, his internal 0:13:06.200 --> 0:13:09.160 model of what he knows and what Aristotle knows from 0:13:09.200 --> 0:13:11.520 all his writings they're different. So you wouldn't even know 0:13:11.559 --> 0:13:13.680 what to ask him. But besides that, what he wants 0:13:13.720 --> 0:13:15.840 to do is be with friends and play video games stuff. 0:13:16.200 --> 0:13:19.080 So that model of tutor, I think isn't going to work. 0:13:19.120 --> 0:13:22.360 But what you're doing is different. It's not just someone 0:13:22.400 --> 0:13:24.800 available to answer questions for you. So tell us about 0:13:24.800 --> 0:13:29.559 how you think about surfacing information in the learner. 0:13:30.000 --> 0:13:31.600 No, that's right, and I want to be clear where 0:13:31.600 --> 0:13:33.640 you know, it's a journey and it's only going to 0:13:33.679 --> 0:13:36.000 get hopefully better over time. But immediately when we started 0:13:36.000 --> 0:13:37.840 seeing all this, we said, all right, well, some of 0:13:37.840 --> 0:13:40.480 the limitations of classic con Academy is you could watch 0:13:40.520 --> 0:13:42.439 a five minute video, but what if you have a question. 0:13:42.840 --> 0:13:44.480 If you have a peer around, you could ask them. 0:13:44.520 --> 0:13:45.760 If you have a teacher around, you can ask them. 0:13:45.800 --> 0:13:48.120 But what if you're a young girl in Afghanistan and 0:13:48.480 --> 0:13:49.559 you don't have a school. 0:13:49.880 --> 0:13:50.560 What do you do? Then? 0:13:50.880 --> 0:13:53.880 If you're working on a on a exercise and you 0:13:54.000 --> 0:13:55.760 just got stuck. We try to make it as gradual 0:13:55.800 --> 0:13:57.760 as possible, but sometimes you just get stuck, and that's 0:13:57.800 --> 0:14:00.240 an that's the most likely time that you'll just engage. 0:14:00.800 --> 0:14:02.040 What if you could get unstuck? 0:14:02.280 --> 0:14:05.200 So the first iteration of what we called what we 0:14:05.240 --> 0:14:08.160 still call Conmigo, which is the AI tutor on con Academy, 0:14:08.880 --> 0:14:11.400 was supporting those use cases. Now, to your point and 0:14:11.440 --> 0:14:14.720 to your son's point, what we see is about ten 0:14:14.840 --> 0:14:17.839 fifteen percent. We haven't, you know, had a formal study 0:14:17.880 --> 0:14:19.920 of this, but what we're observing about ten fifteen percent 0:14:19.920 --> 0:14:22.600 of students make good use of that. They're the students 0:14:22.600 --> 0:14:24.720 that if they watch a video, they're still curious. They're like, 0:14:24.720 --> 0:14:26.440 what about this, or how about this? What if we 0:14:26.520 --> 0:14:29.920 change this variable? And so they know how to ask that. 0:14:30.240 --> 0:14:32.560 But there's another eighty percent of students who are like, oh, 0:14:32.640 --> 0:14:35.080 you know whatever, this isn't my thing. And so what 0:14:35.120 --> 0:14:38.920 we're realizing is to have the benefits of an Aristotle, 0:14:39.000 --> 0:14:41.920 and Aristotle was Alexander the great tutor. Aristotle didn't just 0:14:41.920 --> 0:14:43.800 sit there and wait, I'm guessing, didn't just sit there 0:14:43.840 --> 0:14:46.840 and wait for Alexander to ask him another question. He said, Okay, 0:14:46.840 --> 0:14:49.080 this is what we're going to do, young Alexander. And 0:14:49.360 --> 0:14:50.920 by the way, how come you didn't do your homework. 0:14:51.120 --> 0:14:53.200 I'm going to go talk to was it Prince phil 0:14:53.360 --> 0:14:55.720 King Philip or verbal Macedonia? 0:14:55.760 --> 0:14:57.040 Yeah, his dad, right. 0:14:57.200 --> 0:14:59.360 Philip of Macedonia I'm going to go talk to King 0:14:59.360 --> 0:15:01.840 Philip about you're not doing your work. 0:15:02.080 --> 0:15:03.640 And that's what I used to do with my cousins. 0:15:03.880 --> 0:15:06.160 Yes, I would answer their questions, but most of the 0:15:06.160 --> 0:15:07.920 time I was driving, I'd be like, this is what 0:15:07.960 --> 0:15:10.320 we're going to do today because I noticed this, And 0:15:10.400 --> 0:15:12.920 if they showed up unprepared, I would hold them accountable. 0:15:12.920 --> 0:15:14.480 And sometimes I would call their mom or their dad 0:15:14.560 --> 0:15:16.800 up and say, look, you got to make sure they 0:15:16.840 --> 0:15:19.040 have time for this. And so we are working on 0:15:19.120 --> 0:15:20.960 ways that the AI can be more proactive. 0:15:21.400 --> 0:15:21.600 Now. 0:15:21.600 --> 0:15:24.080 At the end of the day, we do think that 0:15:23.760 --> 0:15:27.040 that teacher and if a student has access to a teacher, 0:15:27.640 --> 0:15:29.320 which is true of most of the world we live in, 0:15:29.360 --> 0:15:30.960 there's parts of the world where they don't, But if 0:15:31.000 --> 0:15:33.320 they have access to a teacher, the teacher is still 0:15:33.360 --> 0:15:37.920 the biggest motivational mechanic for students. And so we are 0:15:37.960 --> 0:15:40.360 thinking about how can the AI both support the teacher 0:15:40.400 --> 0:15:42.840 as a teaching assistant helping them with some of their tasks, 0:15:43.280 --> 0:15:46.680 but then also be a connective tissue and the teacher 0:15:46.680 --> 0:15:49.480 can hold the student accountable on working with the A 0:15:49.520 --> 0:15:51.040 and then the a I can report back to the 0:15:51.120 --> 0:15:54.360 to the teacher. So it's a journey on how we 0:15:54.400 --> 0:15:56.920 can make it more and more part of that loop 0:15:57.120 --> 0:16:00.000 and it can drive more and more, but once again 0:16:00.360 --> 0:16:03.320 not replacing the humans, but but optimizing the connection with. 0:16:03.320 --> 0:16:04.960 The humans so that they don't have to worry about 0:16:05.000 --> 0:16:07.000 some of the busy work. Great. 0:16:07.040 --> 0:16:08.560 So I want to come back to that point about 0:16:08.560 --> 0:16:11.080 what this will mean for teachers, but I want to 0:16:11.160 --> 0:16:13.600 quickly interject something, which is so I'm sort of a 0:16:13.720 --> 0:16:16.320 history buff, but I blanked on this issue about King 0:16:16.320 --> 0:16:19.800 Philip of Macedonia and so, but my question is, I 0:16:19.800 --> 0:16:23.040 mean the question everyone's asking currently is, you know, with 0:16:23.200 --> 0:16:26.720 AI in the world and that's not going away, what 0:16:26.880 --> 0:16:29.920 is the importance of memorizing facts? 0:16:29.960 --> 0:16:31.600 How will education change? 0:16:32.040 --> 0:16:34.800 Will it be important for children to learn a lot, 0:16:34.840 --> 0:16:36.920 you know, the Battle of Hastings was ten sixty six, 0:16:37.280 --> 0:16:38.920 or will it not be so important because they can 0:16:38.960 --> 0:16:40.640 retrieve that information to a point? 0:16:40.720 --> 0:16:41.680 What's your take on that. 0:16:42.360 --> 0:16:45.160 I'm solidly in the camp that content knowledge is really 0:16:45.200 --> 0:16:49.640 valuable because and we've seen this pattern before Calculator comes 0:16:49.640 --> 0:16:51.040 out in the sixties and seventies, We're like. 0:16:51.000 --> 0:16:52.760 Oh, no, one has to learned arithmetic anymore. Well, it 0:16:52.760 --> 0:16:53.120 turns out. 0:16:53.120 --> 0:16:55.040 The people who benefit the most from calculators are the 0:16:55.040 --> 0:16:58.680 people who understood arithmetic and why why They know how 0:16:58.720 --> 0:16:59.960 to use the tool. They know when the tool can 0:17:00.200 --> 0:17:01.760 be used, they know when the tool can't be used. 0:17:01.800 --> 0:17:04.639 They know when there's a precision error in your calculator. 0:17:05.040 --> 0:17:06.840 You know, they know that one third is the same 0:17:06.840 --> 0:17:09.439 thing as point three repeating, and so they know that, 0:17:09.560 --> 0:17:11.800 Like when your calculator shows zero point three two two 0:17:11.800 --> 0:17:13.639 two two two two two three, they're like, oh, one third, 0:17:14.000 --> 0:17:16.399 And that's a simple example, but you could imagine other ones. 0:17:16.520 --> 0:17:18.800 You could not send people to the moon if you 0:17:18.920 --> 0:17:22.400 only use a calculator but you didn't have an understanding 0:17:22.400 --> 0:17:24.520 of things like floating point errors, or if you didn't 0:17:24.600 --> 0:17:27.720 understand things of whereas the calculator have a limitation. Similar thing. 0:17:28.440 --> 0:17:31.000 Internet comes out, Google comes out. People say, oh, you 0:17:31.040 --> 0:17:33.639 can web search everything. Why do you need to have 0:17:33.680 --> 0:17:37.720 any knowledge anymore? It turns out that people with knowledge 0:17:37.760 --> 0:17:40.359 are going to have much better information. For example, you 0:17:40.400 --> 0:17:42.120 might have forgotten that it was the Battle of Hastings, 0:17:42.440 --> 0:17:46.200 but you're you'll be like, hey, you know google, William 0:17:46.280 --> 0:17:47.600 the Conqueror, what was that battle again? 0:17:48.720 --> 0:17:51.199 Or you know, I remember that the card English. 0:17:51.280 --> 0:17:53.240 You know, English has a lot of French words in 0:17:53.280 --> 0:17:56.199 it was it was that related to the Battle of 0:17:56.200 --> 0:18:00.040 Hastings and and and the Norman conquest. And if you 0:18:00.160 --> 0:18:02.439 at least have some of that, you wouldn't even begin 0:18:02.600 --> 0:18:04.720 to ask the right questions. Or let's say that you 0:18:04.800 --> 0:18:06.680 just have a homework assignment and you copy and paste 0:18:06.680 --> 0:18:09.280 it into Google this is twenty years ago. You got 0:18:09.520 --> 0:18:12.960 you got five search results, or YouTube videos start showing up. 0:18:13.720 --> 0:18:16.760 Unless you have some context and some critical thinking of 0:18:16.840 --> 0:18:21.080 judging what's solid information or not, you could just be 0:18:21.080 --> 0:18:22.639 completely misguided. Like if you didn't know it was the 0:18:22.640 --> 0:18:24.960 Battle of Hastings and you click on a video it's 0:18:25.000 --> 0:18:26.679 like we're going to talk about William the Conqueror and 0:18:26.720 --> 0:18:29.720 the Battle of Pudding and you're like, oh, it's the 0:18:29.720 --> 0:18:32.359 Battle of Pudding. And it might have been a joke video, 0:18:32.440 --> 0:18:34.119 but you didn't even watch enough of it to like 0:18:34.480 --> 0:18:37.520 that's going to be a problem. And I see pre AI, 0:18:37.720 --> 0:18:40.000 those with the most content knowledge are the people in 0:18:40.000 --> 0:18:42.960 the position to use the Internet for their knowledge work, 0:18:43.160 --> 0:18:45.480 and they're the ones that I see the difference between 0:18:46.080 --> 0:18:48.520 when I web search versus when a younger child does, 0:18:48.600 --> 0:18:51.800 or even some people you know in my family who 0:18:51.880 --> 0:18:55.000 might not have the full context of what they're searching for. 0:18:55.320 --> 0:18:57.639 They're doing very superficial searches. They don't know the right 0:18:57.680 --> 0:18:59.960 questions to ask. Same thing with AI. AI is like 0:19:00.480 --> 0:19:04.159 can be an intelligent partner with you. Imagine trying to 0:19:04.200 --> 0:19:07.720 have a philosophical conversation with Aristotle without having read any 0:19:07.720 --> 0:19:10.560 of Aristotle, or without having read any philosophy, or at 0:19:10.640 --> 0:19:12.840 least without having talked about it. Or imagine you know, 0:19:12.880 --> 0:19:16.240 being able to meet Isaac Newton, but you don't. You 0:19:16.560 --> 0:19:20.080 haven't gotten the basics of physics or calculus down. You 0:19:20.119 --> 0:19:21.800 can't just tell them like, what's going to happen to 0:19:21.840 --> 0:19:24.800 this rock? If you should say, hey, does Newton's third 0:19:24.880 --> 0:19:26.760 law apply here? Because this seems like an edge case, 0:19:26.800 --> 0:19:27.879 and then you're going to be able to go much 0:19:27.920 --> 0:19:30.480 much further. So I think content knowledge is actually more 0:19:30.520 --> 0:19:33.520 important now because if you want to leverage AI, if 0:19:33.520 --> 0:19:35.199 you want it to amplify your intent, you have to 0:19:35.240 --> 0:19:36.000 hang with the AI. 0:19:36.280 --> 0:19:38.600 And there's still going to be work, human centered work. 0:19:38.640 --> 0:19:41.040 There's going to be taking care of seniors, things that 0:19:41.040 --> 0:19:44.840 are very more manual or more human driven hospitality. But 0:19:44.880 --> 0:19:47.159 I think the knowledge economy where AI is going to 0:19:47.160 --> 0:19:48.760 become a big part of it, you have to be 0:19:48.800 --> 0:19:50.720 able to hang with it, and content will matter more. 0:19:51.000 --> 0:19:54.280 I agree with you, although I do wonder about memorization 0:19:54.600 --> 0:19:57.840 versus critical thinking. So how do you think about teaching 0:19:57.920 --> 0:19:59.120 children critical thinking? 0:19:59.640 --> 0:20:02.560 At this point here, I'll also sound a little traditionalist. 0:20:02.600 --> 0:20:03.240 I think it's both. 0:20:03.680 --> 0:20:07.520 I think it's become very I definitely disagree, like I 0:20:07.520 --> 0:20:11.200 don't like pure like Victorian era memorize everything like that's 0:20:11.240 --> 0:20:15.240 not good. But I do think in like Western school systems, 0:20:15.240 --> 0:20:19.239 it's become fashionable to say any type of memorization is 0:20:19.280 --> 0:20:22.040 not good, and we're only going to focus on critical 0:20:22.040 --> 0:20:24.960 thinking and skill development. And the reality is is that 0:20:25.720 --> 0:20:27.520 I'll speak for myself, A lot of the connections that 0:20:27.520 --> 0:20:31.040 have formed in my mind are because I knew something 0:20:31.880 --> 0:20:35.080 and then my brain at some point, Like you know, 0:20:35.200 --> 0:20:37.920 I every American kid learns about the Louisiana purchase in 0:20:37.960 --> 0:20:40.640 eighteen oh three, and then later most American kids don't 0:20:40.680 --> 0:20:42.520 learn about the Napoleonic Wars, even though that was a 0:20:42.600 --> 0:20:43.480 much more important thing. 0:20:43.960 --> 0:20:45.760 But if you do learn about the Napoleonic Wars, and 0:20:45.760 --> 0:20:46.600 they're normally. 0:20:46.280 --> 0:20:50.240 Not connected for anyone right, But at some point you're like, oh, wait, 0:20:50.720 --> 0:20:53.200 in that other class, I learned like he's sitting here 0:20:53.240 --> 0:20:56.320 fighting all of these other European powers, and eighteen oh 0:20:56.320 --> 0:21:00.280 three he actually his navy was defeated. I wonder whether 0:21:00.280 --> 0:21:03.399 that has anything to do with the Louisiana purchase. And 0:21:03.480 --> 0:21:05.119 even though they never teach us you in history class, 0:21:05.119 --> 0:21:07.760 has everything to do with the Louisiana purchase. Napoleon sold 0:21:07.800 --> 0:21:09.560 it because there's no way he could defend it. It turns 0:21:09.600 --> 0:21:12.800 out Thomas Jefferson could have just taken Louisiana and Napoleon 0:21:12.840 --> 0:21:15.320 had no navy to send out because the British had 0:21:15.359 --> 0:21:19.080 just destroyed it. And that's critical thinking. But in order 0:21:19.119 --> 0:21:24.040 to create those connections, you had to put the Napoleonic 0:21:24.080 --> 0:21:27.679 Wars in time and space with the Louisiana purchase, and 0:21:27.720 --> 0:21:29.280 you have to know where these things are. You have 0:21:29.320 --> 0:21:32.199 to know where Napoleon is, where England is, where the 0:21:32.280 --> 0:21:35.359 navy was defeated, where Louisiana is. You're like, oh, critical thinking, 0:21:35.720 --> 0:21:38.359 I'd be awfully hard to defend Louisiana if I'm sitting 0:21:38.359 --> 0:21:40.440 here battling all these Europeans and I don't have no 0:21:40.520 --> 0:21:41.160 navy anymore. 0:21:55.320 --> 0:21:56.120 Well, let me see two. 0:21:56.040 --> 0:21:58.440 Things from the brain point of view, which is four 0:21:58.560 --> 0:22:03.199 plasticity to happen. For knowledge to stick, it has to 0:22:03.240 --> 0:22:06.199 be relevant in some way. And so one of the 0:22:06.200 --> 0:22:08.280 ways that knowledge becomes relevant for us is when we 0:22:08.320 --> 0:22:10.359 are curious about something. So if you think, hey, wait, 0:22:10.520 --> 0:22:12.320 did the employee ack wars have anything to do with it? 0:22:12.400 --> 0:22:14.040 Then you're curious and you look it up and then 0:22:14.080 --> 0:22:15.120 it sticks the rest of your life. 0:22:15.240 --> 0:22:15.520 Great. 0:22:15.960 --> 0:22:20.000 But how do we, as educators and using AI, how 0:22:20.000 --> 0:22:22.960 do we make things relevant to kids? And how do 0:22:23.119 --> 0:22:25.679 two AI tutors play part in that? Yeah, That's what 0:22:25.680 --> 0:22:26.440 I'm excited about. 0:22:26.600 --> 0:22:29.399 And I'll keep giving display because I don't want to 0:22:29.440 --> 0:22:31.240 make it sound like AI to mar Is is going to 0:22:31.280 --> 0:22:34.160 make everything perfect. But we're on a path and we've 0:22:34.200 --> 0:22:37.119 already seen once again that fifteen twenty percent of students 0:22:37.119 --> 0:22:40.240 who are let's call it proactive and showing more of 0:22:40.240 --> 0:22:42.000 that curiosity. I've seen it in front of my eyes 0:22:42.000 --> 0:22:44.679 where they're learning about the amendments of the Constitution. But 0:22:44.720 --> 0:22:46.439 now with generative AI, they could say, hey, why is 0:22:46.440 --> 0:22:48.240 this relevant now? And then the IA says, well, actually, 0:22:49.119 --> 0:22:51.920 back just recently in twenty twenty there was this Supreme 0:22:51.960 --> 0:22:54.679 Court case and actually that makes a big difference on 0:22:54.840 --> 0:22:57.359 like executive power, the Chevron doctrine, and. 0:22:57.359 --> 0:22:59.280 Are like, oh wow, this is a big deal. I 0:22:59.359 --> 0:22:59.919 never realized. 0:23:00.440 --> 0:23:02.600 So there's apps or like hey, I'm studying entroped, why 0:23:02.600 --> 0:23:04.320 does this matter? And it's like, no, it actually matters 0:23:04.320 --> 0:23:06.480 a ton because if you go into this or there's 0:23:06.560 --> 0:23:09.280 just a study recently, or if you're trying to and 0:23:09.560 --> 0:23:11.560 so I think that's one of the really powerful things 0:23:11.600 --> 0:23:14.320 that it. You know, the classic question for every kid, 0:23:14.440 --> 0:23:16.560 when am I going to need to use this? I 0:23:16.600 --> 0:23:18.480 consider myself better than most people and being able to 0:23:18.520 --> 0:23:21.679 answer that question, but even I'll fumble sometimes. But the 0:23:21.680 --> 0:23:23.920 AI is very good at that, and it's very good 0:23:23.920 --> 0:23:26.959 at like on conmego, if a student literally asked why 0:23:27.000 --> 0:23:28.199 do I need to learn this? The first thing that 0:23:28.200 --> 0:23:29.840 the AI typically does is is, well, what do you 0:23:29.840 --> 0:23:31.800 care about? And the students I want to be a 0:23:31.840 --> 0:23:33.639 professional athlete, I want to do this, and it'll actually 0:23:33.640 --> 0:23:35.520 the I will remember that there's ways to reset it 0:23:35.560 --> 0:23:37.679 and clear it and so it's not creepy. But like 0:23:38.200 --> 0:23:40.840 then in the future, it can proactively say, well, you know, 0:23:40.880 --> 0:23:42.240 since you said you want to be an athlete. 0:23:42.640 --> 0:23:45.040 This molecule, you know, ATP is crucial. 0:23:45.520 --> 0:23:47.280 You know, you want to maximize your ATP if you're 0:23:47.280 --> 0:23:49.480 about to you know, run a marathon or something. 0:23:49.800 --> 0:23:51.480 So that's the hope. 0:23:51.640 --> 0:23:54.560 And as AI gets better and better, as it gets multimodal, 0:23:54.560 --> 0:23:58.000 as it gets you know, we can completely imagine ten 0:23:58.080 --> 0:24:00.200 years in the future, for sure you're going to You'll 0:24:00.240 --> 0:24:03.439 put on augmented reality glasses or virtual reality glasses and 0:24:04.080 --> 0:24:06.679 travel to ancient Rome and try to stop, you know, 0:24:06.720 --> 0:24:09.919 the assassination of Caesar, or travel into the human brain 0:24:10.320 --> 0:24:12.600 and try to, you know, diagnose whether this is a 0:24:12.960 --> 0:24:15.760 you know, an Alzheimer's case, or you know, are there 0:24:15.800 --> 0:24:17.159 ways that you might be able to address it. So 0:24:17.520 --> 0:24:20.439 I think that relevance question AI will help. It's not 0:24:20.680 --> 0:24:23.800 solved it now. I'll say one other thing though, you know, 0:24:23.840 --> 0:24:26.040 I in my first book, I gave the example of 0:24:26.400 --> 0:24:28.879 you know, people always talk about relevance, but I was 0:24:28.920 --> 0:24:31.000 this I don't know. I used to think I was 0:24:31.000 --> 0:24:33.000 five nine, but I now realize probably closer to five 0:24:33.000 --> 0:24:33.639 to seven. 0:24:33.560 --> 0:24:34.000 Five eight. 0:24:34.200 --> 0:24:37.919 But I was this that tall kid in high school. 0:24:38.160 --> 0:24:40.359 But when I was in Pe, I never said, why 0:24:40.359 --> 0:24:42.160 am I going to have to take an orange ball 0:24:42.200 --> 0:24:44.159 and have to put it into a ten foot hoop, 0:24:44.240 --> 0:24:46.320 and this has no relev It had no relevance to 0:24:46.640 --> 0:24:49.440 my but I was engaged because my friends were there, 0:24:49.520 --> 0:24:53.080 I was having fun. I didn't feel disengaged. I do 0:24:53.119 --> 0:24:56.520 think that sometimes the relevance question is missing the mark 0:24:56.560 --> 0:24:59.040 a bit like math to some degree, the real beauty 0:24:59.080 --> 0:25:01.040 of math isn't in its ow implication. It's in its 0:25:01.040 --> 0:25:04.159 inherent beauty in it. It's the thought process of it. 0:25:04.400 --> 0:25:05.680 And I think the reason why a lot of kids 0:25:05.680 --> 0:25:07.800 start saying, oh, when am I going to learn this 0:25:08.400 --> 0:25:10.600 is they have so many gaps that they've disengaged, so 0:25:10.600 --> 0:25:12.880 they're just protecting their self esteem if you can fill 0:25:12.880 --> 0:25:14.480 in those gaps. I saw this with my cousins. Some 0:25:14.520 --> 0:25:16.760 of my cousins were saying, I'm not a math person. 0:25:16.840 --> 0:25:17.440 I hate math. 0:25:17.560 --> 0:25:20.280 Is that. But once they started getting that personal attention 0:25:20.320 --> 0:25:22.640 and I'm like, Hey, isn't this cool? Isn't this cool? 0:25:22.680 --> 0:25:24.600 You can do it this way or this way? Isn't 0:25:24.600 --> 0:25:27.280 this amazing that this can be described or predicted? They 0:25:27.280 --> 0:25:29.359 started becoming math people. And not that they thought that 0:25:29.400 --> 0:25:30.399 they were going to use it in their life, but 0:25:30.400 --> 0:25:31.760 they just started seeing the beauty in it and they 0:25:31.800 --> 0:25:33.040 start feeling confident in it. 0:25:33.240 --> 0:25:34.199 That's great. You know. 0:25:34.280 --> 0:25:37.679 Another thing is we as humans are so sensitive to 0:25:37.920 --> 0:25:41.520 story as opposed to let's say, a bulleted list of points. 0:25:42.000 --> 0:25:44.080 And so this is one of the things I'm enthusiastic about 0:25:44.080 --> 0:25:47.320 with AI is can it take complicated things, whether it's 0:25:47.400 --> 0:25:49.560 history or math or whatever and tell you a story 0:25:49.600 --> 0:25:50.080 about it. 0:25:50.440 --> 0:25:53.120 Absolutely, I can create a story. It could create a poem, 0:25:53.280 --> 0:25:57.160 a song. It can. I think it can create simulations 0:25:57.440 --> 0:26:00.720 for you where yes, it becomes like the ultimate choose 0:26:00.720 --> 0:26:01.960 your own eventure. I mean, right now, it could be 0:26:01.960 --> 0:26:04.760 text based, it's soon going to be a video and 0:26:05.359 --> 0:26:08.200 visual based. But as we said in the not too 0:26:08.240 --> 0:26:11.280 far off future, let's call it five ten years, it's 0:26:11.280 --> 0:26:12.679 going to be able to create a lucid dream for 0:26:12.720 --> 0:26:15.160 you that puts you in the middle of it all. 0:26:15.640 --> 0:26:15.960 Yeah. 0:26:16.119 --> 0:26:17.560 So I want to come back to this point about 0:26:17.600 --> 0:26:20.639 what this means for teachers. Tell us your thoughts on that. 0:26:21.119 --> 0:26:24.399 I've been saying this for years because even when I 0:26:24.440 --> 0:26:27.399 started making videos and software, some people said, oh, on 0:26:27.480 --> 0:26:31.879 demand video does this replace teachers? And my response is 0:26:31.960 --> 0:26:34.960 I don't think it does. If all a teacher is 0:26:35.040 --> 0:26:40.600 doing is going up, giving a lecture and leaving. Then yes, 0:26:40.680 --> 0:26:43.960 maybe on demand video could replace that. And you see 0:26:44.000 --> 0:26:45.760 that happening in a lot of colleges. They've you know, 0:26:45.800 --> 0:26:47.119 no one goes in med schools, no one goes to 0:26:47.119 --> 0:26:49.520 the lecture anymore, because that's what the lectures were. People 0:26:49.560 --> 0:26:52.439 are watching the recordings of the lectures on double speed, 0:26:52.720 --> 0:26:54.240 and now they can ask questions to an AI. 0:26:55.359 --> 0:26:55.920 So that's that. 0:26:56.040 --> 0:26:58.199 But and I've been saying this for years, the ideal 0:26:58.240 --> 0:27:01.119 classroom has always been, and this go back to Socrates 0:27:02.200 --> 0:27:05.760 and Plato and Aristotle, is if human beings are getting 0:27:05.760 --> 0:27:08.800 together in a room, make them interact with each other, 0:27:09.320 --> 0:27:12.520 have a Socratic dialogue, have a simulation. Have students go 0:27:12.560 --> 0:27:15.080 into groups and tackle the problem a little bit. Yes, 0:27:15.119 --> 0:27:16.680 you could give a little bit of a mini lecture, 0:27:17.080 --> 0:27:20.840 but then make it interactive or maybe out of the 0:27:20.840 --> 0:27:23.440 thirty students, ten of them really need help with this concept, 0:27:23.440 --> 0:27:25.040 while the other twenty can actually learn at their own 0:27:25.040 --> 0:27:27.560 pace or keep working on their project, or work on connuiccuting, whatever. 0:27:27.800 --> 0:27:30.199 So do the focus intervention with these ten, and then 0:27:30.240 --> 0:27:32.560 you don't have to bore or lose the other twenty. 0:27:32.600 --> 0:27:34.600 And then do another focus intervention with the other ten 0:27:35.040 --> 0:27:38.080 and that type of work. I think most people go 0:27:38.119 --> 0:27:40.080 into the teaching profession because they actually want to be 0:27:40.119 --> 0:27:44.200 that teacher that unlocks little sal or, little David, makes 0:27:44.200 --> 0:27:47.399 them believe in themselves. You know, we all can list 0:27:47.880 --> 0:27:50.600 five to ten teachers at every stage of our life. 0:27:50.640 --> 0:27:52.480 It's like that was the teacher that made me believe 0:27:52.480 --> 0:27:55.200 in myself or made me see wonder in the world, 0:27:55.480 --> 0:27:57.760 and they change my trajectory. And every teacher wants to 0:27:57.800 --> 0:28:00.920 be that teacher. And I think whether it's video, whether 0:28:00.960 --> 0:28:04.000 it's software, and it's AI, if it's used well and 0:28:04.040 --> 0:28:05.840 the AI is acting as assistant but not trying to 0:28:05.920 --> 0:28:07.919 replace the teacher, which I don't think it should. I 0:28:07.920 --> 0:28:09.439 think it could actually free up the teacher to do 0:28:09.480 --> 0:28:11.440 a lot of that more human element to work. 0:28:11.840 --> 0:28:13.119 And I know one of the things that you've been 0:28:13.160 --> 0:28:17.480 looking at is how AI could improve debating, as in, 0:28:17.560 --> 0:28:18.720 I'm going to take a point. 0:28:18.480 --> 0:28:19.840 And the AI is going to give you the opposite 0:28:19.880 --> 0:28:20.840 point and tell us about that. 0:28:21.080 --> 0:28:23.520 Yeah, and it's you know, there's actually two projects on 0:28:23.560 --> 0:28:26.840 the debating let's call it debate, and one is very 0:28:26.880 --> 0:28:28.840 AI and one is very non AI. But I think 0:28:28.880 --> 0:28:32.000 they might actually converge. So on Conmigo, we actually have 0:28:32.080 --> 0:28:34.199 an activity where you can get into a debate with 0:28:34.200 --> 0:28:37.080 an AI on anything, you know, should the government cancel 0:28:37.160 --> 0:28:40.560 student debt, should the US maintain military bases around the world, 0:28:40.640 --> 0:28:43.160 or should it you know, kind of become more isolationist 0:28:43.600 --> 0:28:46.360 and you can pick either side. The AI will have 0:28:46.400 --> 0:28:48.480 a polite debate and then actually give you feedback and 0:28:48.480 --> 0:28:50.280 then you can switch side. So that's where we've got 0:28:50.280 --> 0:28:53.040 a lot of positive feedback. Students enjoy that it's a 0:28:53.080 --> 0:28:55.840 safe environment to test ideas without being embarrassed. 0:28:56.240 --> 0:28:57.360 Now we have another project. 0:28:57.400 --> 0:28:59.960 There's a sister nonprofit to Kind Academy that we still 0:29:00.120 --> 0:29:03.200 during the pandemic called Schoolhouse dot World. Schoolhouse dot World 0:29:03.280 --> 0:29:06.719 is all about free live tutoring from other human beings. 0:29:07.520 --> 0:29:09.960 The way that we're able to give people free live 0:29:10.040 --> 0:29:13.120 over zoom tutoring from other human beings is through volunteers, 0:29:13.360 --> 0:29:15.960 and anyone listening can go be a volunteer, and there's 0:29:15.960 --> 0:29:19.200 a whole mechanism to certify yourself or you can get tutoring. 0:29:19.640 --> 0:29:21.520 But one of the things we realize a lot of 0:29:21.600 --> 0:29:27.160 universities and some of the top universities you, Chicago, MIT, Brown, Yale, Caltech. 0:29:27.560 --> 0:29:30.640 There's thirty others said, Hey, if someone is tutoring on 0:29:30.680 --> 0:29:33.120 Schoolhouse dot World, so they've certified that they say no 0:29:33.200 --> 0:29:35.600 physics or they know biology, and they're tutoring other people, 0:29:35.960 --> 0:29:38.560 we want to know about that for college admissions. And 0:29:38.600 --> 0:29:40.520 so they have been using it for college admissions. But 0:29:40.520 --> 0:29:42.920 on top of that, when we went last year, they're like, hey, 0:29:42.920 --> 0:29:44.520 is there any way that we could leverage this platform 0:29:44.560 --> 0:29:49.479 to foster like civil dialogue between people, civil debate to 0:29:49.520 --> 0:29:51.640 your point, And we're like, okay, let's try something a 0:29:51.680 --> 0:29:54.920 little bit wild. Let's have real people fill out a 0:29:54.960 --> 0:29:59.000 survey on all the hottest button issues Israel, Palestine, abortion, 0:29:59.400 --> 0:30:04.080 affirmative action, free speech, gun control, gun control, go down 0:30:04.080 --> 0:30:06.040 the list, the stuff that people are afraid to talk 0:30:06.040 --> 0:30:07.640 about because they don't want to lose friendships. 0:30:08.200 --> 0:30:09.040 Students fill it out. 0:30:09.200 --> 0:30:11.920 You get paired, either one on one, sometimes two on 0:30:11.960 --> 0:30:16.000 two with someone with a diametrically opposite point of view. 0:30:16.400 --> 0:30:19.760 Now what happens on this dialogue project is you're not 0:30:19.760 --> 0:30:22.360 supposed to it's not a debate, but you're supposed to 0:30:22.400 --> 0:30:24.520 listen to each other. And at the end of it, 0:30:25.040 --> 0:30:27.959 you have to with the form represent the other person's 0:30:28.000 --> 0:30:30.240 point of view. You also have to reflect on whether 0:30:30.280 --> 0:30:32.840 you think they heard you and represent and then any 0:30:32.880 --> 0:30:35.560 other reflections on the interaction that you had. And what 0:30:35.680 --> 0:30:38.280 happens then is you get a rating on your ability 0:30:38.280 --> 0:30:42.480 to engage in hard conversations. And these colleges are very 0:30:42.520 --> 0:30:44.480 interested in this as they don't carry your point of 0:30:44.520 --> 0:30:46.480 view on these issues. They care that they like that 0:30:46.520 --> 0:30:48.600 you have a point of view, but that you were 0:30:48.640 --> 0:30:50.640 able to engage in a respectful and thoughtful and you're 0:30:50.680 --> 0:30:52.760 able to steal man the other person's argument, you're able 0:30:52.840 --> 0:30:55.840 to represent it. Now, those are both ways to have 0:30:55.960 --> 0:31:00.000 dialogue in a world where it's getting increasingly polarized, increasingly 0:31:00.120 --> 0:31:03.240 hard to have these conversations and feel safe. Over time, 0:31:03.280 --> 0:31:05.440 these might converge a little bit. We are imagining ways 0:31:05.440 --> 0:31:08.400 that AI. We are already using AI on schoolhouse to 0:31:08.440 --> 0:31:11.080 give the tutor's feedback after the fact to saying, hey, 0:31:11.160 --> 0:31:13.920 you know, you probably didn't ask enough questions, or you 0:31:13.960 --> 0:31:16.840 ignored this one student, or they were actually asking this. 0:31:17.320 --> 0:31:19.360 Over time is going to be real time. We also 0:31:19.400 --> 0:31:22.040 imagine this dialogue project the AI might be able to facilitate. 0:31:22.080 --> 0:31:24.240 We actually haven't had any where people started yelling at 0:31:24.280 --> 0:31:26.760 each other or got on out. But maybe it can 0:31:26.800 --> 0:31:29.959 make sure that the conversation. Oh oh, David, you know 0:31:30.200 --> 0:31:31.760 I can tell your feeling a little sensitive about what 0:31:31.840 --> 0:31:33.280 Sala said. Sal, is there another way that you might 0:31:33.320 --> 0:31:35.360 have said that that you know might be a little 0:31:35.440 --> 0:31:35.959 less triggering? 0:31:36.080 --> 0:31:38.440 Or David, can you tell sal a way. 0:31:38.240 --> 0:31:40.360 That he might have said that that you would have 0:31:40.360 --> 0:31:43.120 appreciated things like that? I could imagine AI facilitating that 0:31:43.120 --> 0:31:43.640 type of thing. 0:31:43.800 --> 0:31:45.959 Oh great, And what do you think is the future 0:31:46.000 --> 0:31:49.800 of AI in terms of assessments instead of taking standardized tests? 0:31:50.240 --> 0:31:52.800 What's the future in five years? 0:31:53.240 --> 0:31:53.840 Like everything? 0:31:53.880 --> 0:31:55.640 It's I mean, if if we're living in a science 0:31:55.640 --> 0:32:00.800 fiction book and standardized tests I have historically defended because 0:32:00.800 --> 0:32:03.200 I've said, look, to get good at anything, you have 0:32:03.240 --> 0:32:06.280 to measure it, And would you prefer unstandardized measurement you 0:32:06.320 --> 0:32:08.560 want to be able to benchmark. Now, the problem that 0:32:08.560 --> 0:32:10.719 most people have with sanitized tests, which I also agree with, 0:32:10.840 --> 0:32:14.840 is if you want something that's consistent and standardized and scalable, 0:32:15.280 --> 0:32:18.200 it's got to be you know, fairly narrow in what 0:32:18.240 --> 0:32:21.040 it can measure. So most standardized tests are multiple choice 0:32:21.120 --> 0:32:23.720 or just numeric entry. They don't let you do a 0:32:23.760 --> 0:32:26.840 lot of open ended things. And unfortunately, yes, they do 0:32:26.880 --> 0:32:31.080 measure some things that are of value, but they can't 0:32:31.120 --> 0:32:33.120 measure how well you could write a longer paper, or 0:32:33.160 --> 0:32:35.240 how well you can design something, or how well you 0:32:35.280 --> 0:32:39.080 can problem solve things. What's exciting about generative AI is yes, 0:32:39.120 --> 0:32:41.040 I do think in the next in fact, we're working 0:32:41.040 --> 0:32:43.840 on this kind of the future of assessment. We can 0:32:43.880 --> 0:32:47.520 incrementally add layers on top of traditional standardized assessment. So 0:32:47.560 --> 0:32:50.000 you can ask a student, Okay, you picked C. Can 0:32:50.000 --> 0:32:52.280 you explain your reasoning? Why do you think you pick C? 0:32:53.080 --> 0:32:54.960 Or even after you you actually got this one wrong, 0:32:55.000 --> 0:32:56.440 do you think it was a careless mistake or do 0:32:56.440 --> 0:32:57.960 you think you know there was an error? 0:32:57.960 --> 0:32:58.680 What was your error? 0:32:58.920 --> 0:33:00.960 And if you can give some of that subjective insights 0:33:00.960 --> 0:33:03.520 to teachers, it becomes more actionable. Then they know what 0:33:03.560 --> 0:33:05.560 to do with it. Reading comprehension instead of what was 0:33:05.600 --> 0:33:08.840 the author's intent? You know ABCD, you could say, before 0:33:08.880 --> 0:33:11.080 you see the choices in your own words, what do 0:33:11.120 --> 0:33:12.280 you think the author's intent was? 0:33:12.640 --> 0:33:13.640 Now look at the choices. 0:33:13.960 --> 0:33:16.760 Now if you fast forward five years, ten years, I 0:33:16.760 --> 0:33:18.680 think it's going to feel very much like a job 0:33:18.720 --> 0:33:22.280 interview or like an oral thesis defense at a university. 0:33:22.800 --> 0:33:25.160 And I know people's alarm bells start going off of 0:33:25.200 --> 0:33:27.200 a bias and this and that and a it can 0:33:27.240 --> 0:33:28.840 be creepy and data and all that, But I always 0:33:28.840 --> 0:33:34.000 point out, like job interviews and these rich assessments like 0:33:34.120 --> 0:33:38.440 thesis defense oral exams are rich, but they're incredibly biased, 0:33:38.600 --> 0:33:41.479 and they're incredibly inconsistent. Right now, you know, even if 0:33:41.480 --> 0:33:44.400 you're getting interviewed by the same person, depending on their mood, 0:33:44.800 --> 0:33:47.120 depending on the day of the week, you could get 0:33:47.120 --> 0:33:49.760 a completely different experience with that person. If we can 0:33:49.800 --> 0:33:51.840 get AI, and if we can benchmark AI and test 0:33:51.840 --> 0:33:54.760 AID and make sure, okay, two people equivalent, one is 0:33:54.800 --> 0:33:58.000 African American, one is Asian, is giving the same signal 0:33:58.040 --> 0:34:02.560 because other than race, these two simulated candidates are completely identical. 0:34:02.560 --> 0:34:04.280 So you can benchmark an AI in ways that you 0:34:04.280 --> 0:34:06.640 can't benchmark a human being. But if you can do that, 0:34:06.720 --> 0:34:08.320 I think AI is going to You're going to be 0:34:08.320 --> 0:34:12.200 able to have standardized assessments that feel a lot like 0:34:12.560 --> 0:34:16.279 a job interview, that feel a lot like a simulation 0:34:16.520 --> 0:34:19.279 or a game or a problem solving that I think 0:34:19.360 --> 0:34:21.400 can really broaden the oppertunity. I mean even things like 0:34:21.440 --> 0:34:25.800 sense of humor, or communication skills or body language. 0:34:25.880 --> 0:34:27.280 I think the AI could give feedback. 0:34:27.320 --> 0:34:28.880 Now, I'm a big believer you shouldn't just say, oh, 0:34:28.920 --> 0:34:31.479 that person's body language was off, never give them a job. 0:34:31.840 --> 0:34:33.800 It should be like, hey, you know what your body 0:34:33.880 --> 0:34:35.560 language right now? You were a little bit you weren't 0:34:35.560 --> 0:34:37.839 making eye contact, et cetera. I'd give you a three 0:34:37.880 --> 0:34:40.240 out of five. Here's tips. Come back to me tomorrow 0:34:40.239 --> 0:34:42.200 and take the assessment again. Mastery learning. 0:34:42.520 --> 0:34:45.320 Oh great, so you've touched on a few things about 0:34:45.400 --> 0:34:47.359 what the world might look like five ten years ago. 0:34:47.520 --> 0:34:50.200 When I think of five years ago, I mean, the 0:34:50.239 --> 0:34:52.640 only people who even knew these llms were working was 0:34:52.680 --> 0:34:54.439 you know, some researchers that could will whatever. 0:34:54.480 --> 0:34:56.720 But it hadn't burst onto the world yet. 0:34:57.239 --> 0:35:00.000 And so five years hence, what else do you think 0:35:00.040 --> 0:35:02.040 it's going to be different about, for example, what you're 0:35:02.040 --> 0:35:04.000 doing with con migo or whatever is next. 0:35:04.040 --> 0:35:06.120 What does it look like for students? It's hard. 0:35:06.360 --> 0:35:08.359 Yeah, and you said five years ago. Even five years ago, 0:35:08.360 --> 0:35:11.799 the AI researchers themselves didn't even appreciate I remember when 0:35:12.480 --> 0:35:14.840 Greg and Sam gave us access to GPT four in 0:35:14.880 --> 0:35:17.279 the summer of twenty twenty two, and my first question 0:35:17.320 --> 0:35:18.520 before I even tried it out, I was like, does 0:35:18.560 --> 0:35:20.520 this work in other languages? And they said, oh, we 0:35:20.520 --> 0:35:24.239 don't think so. And then I barely speak Bengali, which 0:35:24.280 --> 0:35:26.920 is where my family. My family comes from Bengal, and 0:35:27.000 --> 0:35:30.000 I said some it wrote back to me to Bengali, 0:35:30.040 --> 0:35:32.280 and I took a screenshot I sent to say They're like, oh, 0:35:32.400 --> 0:35:34.320 oh yeah, we just figured that out when you asked, 0:35:34.600 --> 0:35:36.640 so they didn't even know. And then one of my 0:35:36.960 --> 0:35:40.759 friend's parents as a researcher at another AI firm, and 0:35:41.200 --> 0:35:43.759 he's like, you know what, we didn't realize that you 0:35:43.800 --> 0:35:45.239 guys did Kin Academy did. 0:35:45.280 --> 0:35:47.080 I'm like, oh wow, what's our role here? 0:35:47.400 --> 0:35:49.640 He's like, we didn't realize the power of prompting that 0:35:49.719 --> 0:35:51.960 A lot of the AI researchers, if you go five 0:35:52.000 --> 0:35:54.640 years ago, four years ago, three years ago, they were 0:35:54.680 --> 0:35:56.839 just like going to make a better model with more parameters, 0:35:56.920 --> 0:35:59.960 better training data. And I think one of the breakthrough 0:36:00.120 --> 0:36:02.520 that happened both with GPT four and chat GPT to 0:36:02.560 --> 0:36:05.400 some degree one was a chat interface that unlocked a 0:36:05.400 --> 0:36:08.080 lot and then the other was that you can actually 0:36:08.200 --> 0:36:10.520 and GPT four in particular was very steerable, which means 0:36:10.560 --> 0:36:12.480 you could prompt it and have it take on personas 0:36:12.480 --> 0:36:14.800 as a tutor or as a poet, or as a 0:36:14.880 --> 0:36:16.839 rapper or whatever you want to make it do. And 0:36:17.080 --> 0:36:19.680 they I don't think the researchers had the time to 0:36:19.760 --> 0:36:21.920 even sit there and prompt it. So it is amazing 0:36:21.960 --> 0:36:24.600 even they didn't realize how far you could get with 0:36:24.640 --> 0:36:27.960 something like that five years in the future. You know, 0:36:28.520 --> 0:36:30.680 there's some people are arguing that, you know, we're hitting 0:36:30.680 --> 0:36:33.279 some type of an asymptote. You know, the jump from 0:36:33.320 --> 0:36:36.759 GPT three to GPT four was massive. The jump from 0:36:36.800 --> 0:36:40.719 GPT four now two years later to GPT four Omni 0:36:41.320 --> 0:36:44.719 one reasoning model three or whatever, or Gemini three or 0:36:45.400 --> 0:36:46.480 Claude three point five. 0:36:46.920 --> 0:36:47.520 It's better. 0:36:47.640 --> 0:36:50.520 I mean, the hallucinations are way down, the math errors 0:36:50.520 --> 0:36:50.960 the way down. 0:36:51.080 --> 0:36:52.560 It is getting better at reasoning. 0:36:52.239 --> 0:36:54.239 But it's not as much of a you know, we 0:36:54.280 --> 0:36:56.480 thought GPT five would be around by now, and that 0:36:56.680 --> 0:37:00.000 you know, maybe it would be approaching you know, something 0:37:00.120 --> 0:37:03.839 even more mind blowing or even transcending human intelligence that 0:37:03.920 --> 0:37:06.640 hasn't happened yet. So I think there's two scenarios. I 0:37:06.640 --> 0:37:08.080 think there's a world where in the next five years 0:37:08.120 --> 0:37:10.880 we do and then that's a wild world. I think 0:37:10.920 --> 0:37:13.360 you're going to see massive acceleration. I mean in your world, 0:37:13.400 --> 0:37:17.919 understanding the brain, understanding complex systems, drug developments. 0:37:17.960 --> 0:37:18.400 Science. 0:37:18.520 --> 0:37:20.759 I've heard people say we're gonna have one hundred years 0:37:20.760 --> 0:37:23.040 of science breakthroughs in the next ten I don't know. 0:37:23.440 --> 0:37:25.799 I don't know. I think that's at least true. 0:37:25.880 --> 0:37:28.320 I've actually made the argument that A is going to 0:37:28.440 --> 0:37:32.120 help in science in several ways. The main way it's 0:37:32.160 --> 0:37:35.040 going to do it is simply by pulling together facts 0:37:35.360 --> 0:37:37.319 that I couldn't possibly know because I couldn't have read 0:37:37.320 --> 0:37:40.160 all those journal articles. But the thing I'm really looking 0:37:40.160 --> 0:37:45.839 forward to now, which current lms cannot do, is scientific 0:37:45.920 --> 0:37:49.920 discovery of the type that humans do, which is extrapolating 0:37:49.960 --> 0:37:52.759 thinking of a new framework, saying what if I were 0:37:52.800 --> 0:37:55.120 writing on that photon of light, what would the world 0:37:55.200 --> 0:37:58.360 look like? Oh? That leads me to special theory of relativity. 0:37:58.600 --> 0:38:02.839 That's the kind of thing the current llms are incredible 0:38:02.880 --> 0:38:06.879 at interpolating, but not at extrapolating. So that's what I'm 0:38:06.920 --> 0:38:09.319 looking forward to in science. But actually, let me ask 0:38:09.360 --> 0:38:12.720 you a question on this, And actually you mentioned hallucinations 0:38:12.719 --> 0:38:15.520 a moment ago, and I always think about what I 0:38:15.680 --> 0:38:18.920 like about hallucinations in MS, which is that you can 0:38:18.960 --> 0:38:22.919 also think of it as hypotheses, which is to say, 0:38:23.360 --> 0:38:25.240 and this is what scientists do all the time. 0:38:25.080 --> 0:38:26.359 Is hey, what if? What if? 0:38:26.400 --> 0:38:28.200 You know, we're trying things out all the time. Most 0:38:28.200 --> 0:38:30.440 of them don't lead anywhere. Occasionally you can build a 0:38:30.480 --> 0:38:32.279 bridge to that and say, oh my gosh, I just 0:38:32.320 --> 0:38:33.160 made real progress. 0:38:33.719 --> 0:38:36.120 So what do you think is the effect. 0:38:35.960 --> 0:38:41.280 Of AIS in education on creativity in students. 0:38:41.880 --> 0:38:43.520 I read a lot about it in my book Brave 0:38:43.600 --> 0:38:46.279 New Words, and I think it's going to be a 0:38:46.360 --> 0:38:48.759 creativity amplifier. You know, some people would argue like, oh, 0:38:48.800 --> 0:38:50.719 if this thing can create a lazy student's just not 0:38:50.719 --> 0:38:53.000 going to create themselves. And look, lazy students are going 0:38:53.000 --> 0:38:54.480 to figure out ways to be lazy. There's ways to 0:38:54.480 --> 0:38:57.040 police them. We're working on that too. But I always 0:38:57.040 --> 0:38:59.680 point out the most creative moments in my life have 0:38:59.760 --> 0:39:01.640 been when I'm in a room with other creative people. 0:39:01.880 --> 0:39:03.600 If you look at all the creative moments in history, 0:39:03.640 --> 0:39:06.120 you know, Florence during the Renaissance, or some of the 0:39:07.040 --> 0:39:09.640 you know beat nicks in you know, different cities in 0:39:09.640 --> 0:39:11.040 the world, or Silicon Valley. 0:39:11.120 --> 0:39:12.520 Right now, it's there. 0:39:12.680 --> 0:39:16.760 There is this effect of people meeting other creative people 0:39:16.800 --> 0:39:19.480 and then collaborating with them, and the power of AI 0:39:19.760 --> 0:39:22.040 is you know, like I remember being in middle school 0:39:22.040 --> 0:39:24.799 in high school and I was a curious kid, but 0:39:24.840 --> 0:39:27.800 I had to suppress that curiosity in the classroom otherwise 0:39:27.800 --> 0:39:29.719 I would get beat up. But I had no one 0:39:29.719 --> 0:39:31.520 to talk about this stuff with, and you just kind 0:39:31.520 --> 0:39:33.640 of sit been by yourself. It's like, I wonder, I 0:39:33.680 --> 0:39:35.239 wonder if this is different, and I wonder if I 0:39:35.280 --> 0:39:39.120 could do this. But now, if young sal had had 0:39:39.200 --> 0:39:41.520 chat GBT or Gemini or something, I. 0:39:41.440 --> 0:39:42.880 Could have played around. I was like, well, what do 0:39:42.920 --> 0:39:44.279 you mean by that? Or how does this? 0:39:44.680 --> 0:39:46.640 And I do think you know, to your point, some 0:39:46.680 --> 0:39:50.719 of the big breakthroughs are naive thinking, and some of 0:39:50.719 --> 0:39:53.200 the best naive thinking is coming from people who haven't 0:39:53.200 --> 0:39:56.480 been indoctrinated yet. And every couple of weeks I do 0:39:56.480 --> 0:39:59.000 get an email from a student saying, Oh, I watched 0:39:59.000 --> 0:40:01.240 your video on this, but I have a new theory 0:40:01.960 --> 0:40:05.560 that disproves you know Newton, And I'm like, okay, whatever, whatever, 0:40:05.719 --> 0:40:07.879 But you know, one out of every thousand of those 0:40:07.920 --> 0:40:10.279 kids is going to be right if they can start 0:40:10.320 --> 0:40:13.400 to validate or fine tune their ideas with an AI 0:40:13.520 --> 0:40:17.319 that could even bring what's already known out there. One 0:40:17.360 --> 0:40:19.200 of them is like, wait, I think I'm onto something, 0:40:19.640 --> 0:40:21.560 and maybe the AI says, you know, you might be 0:40:21.680 --> 0:40:41.440 onto something that might reconcile gravity with quantum mechanics. 0:40:43.680 --> 0:40:46.080 It feels to me like the two most important things 0:40:46.680 --> 0:40:49.440 for educators is, you know, roots and wings. This was 0:40:49.480 --> 0:40:52.919 something Gerta said two hundred and fifty years ago that 0:40:52.560 --> 0:40:54.960 that's our job is to give kids roots and wings, 0:40:55.040 --> 0:40:58.279 meaning all the content knowledge as you mentioned, and the 0:40:58.280 --> 0:41:02.080 wings being the creativity part. So if students have a 0:41:02.239 --> 0:41:05.319 partner with which they can do it, great, that makes 0:41:05.320 --> 0:41:08.279 things better. How do we make sure that kids are 0:41:08.680 --> 0:41:10.880 using it as a partner as opposed to being the 0:41:10.960 --> 0:41:13.480 lazy kid and just letting let me do it. 0:41:13.880 --> 0:41:15.879 Yeah. I wrote about a whole in my book. 0:41:15.880 --> 0:41:18.880 I wrote a whole chapter about cheating, and I always 0:41:18.920 --> 0:41:20.960 like to start like, you know, everyone's concerned about cheating 0:41:20.960 --> 0:41:23.120 because of chat GEPT and it can cheat. I was like, 0:41:23.120 --> 0:41:25.120 but what was the state of cheating before, and it 0:41:25.160 --> 0:41:26.839 turns out there's been a lot of cheating, in fact, 0:41:26.920 --> 0:41:29.960 even since you and I were in school. Unfortunately cheating, 0:41:30.040 --> 0:41:32.799 you know, I think the internet, there's services online that 0:41:33.040 --> 0:41:35.640 a human being will write your paper in Kenya, a 0:41:35.680 --> 0:41:37.760 PhD in English will write your paper for five dollars 0:41:37.800 --> 0:41:40.440 a page, write your term paper for you. And that's 0:41:40.680 --> 0:41:43.480 arguably and that's been around for twenty years. So and 0:41:43.760 --> 0:41:47.880 sorority A sororities sisters and fraternity brothers are you know, sharing, 0:41:48.280 --> 0:41:51.440 So that's always been there and so our solution to 0:41:51.480 --> 0:41:53.640 it is, yeah, you're right, chat GPT could be used. 0:41:53.680 --> 0:41:55.200 In fact, I'm sure people are using it. 0:41:55.560 --> 0:41:57.440 And anyone who says that there's tools that can detect 0:41:57.440 --> 0:41:59.640 AI cheating is it snake oil. In fact, there's a 0:41:59.640 --> 0:42:02.640 lot of FA accusations happening right now, especially in higher education. 0:42:03.160 --> 0:42:05.640 But I think the real solution is and it doesn't 0:42:05.640 --> 0:42:07.920 just solve the cheating problem from AI. I think it 0:42:08.120 --> 0:42:10.720 solved the cheating problem generally, but it can also support 0:42:10.760 --> 0:42:14.880 students and teachers better is have the students have the 0:42:14.920 --> 0:42:17.440 teacher signed through the AI, have the students work with 0:42:17.480 --> 0:42:19.520 the AI on the paper, and then have the AI 0:42:19.600 --> 0:42:22.839 make the entire process transparent to the professor or to 0:42:22.880 --> 0:42:26.520 the teacher, because then and we're building this. But then 0:42:26.680 --> 0:42:29.400 if the AI could say, yes, professor, we worked on 0:42:29.440 --> 0:42:31.440 this for four hours. David had some trouble coming up 0:42:31.440 --> 0:42:33.360 with a thesis statement. This is where we worked. You 0:42:33.440 --> 0:42:35.120 can see two hours into it. We had a lot 0:42:35.160 --> 0:42:36.920 of back and forth on this part. You can see 0:42:36.920 --> 0:42:39.359 his edit history, but you know, three hours into it, 0:42:39.400 --> 0:42:41.600 he just copy and pasted this paragraph into this. I 0:42:41.640 --> 0:42:43.359 do not know where it comes from. It actually doesn't 0:42:43.360 --> 0:42:46.279 look like his writing. We should look into this. Don't 0:42:46.280 --> 0:42:48.880 accuse him, ask him about that paragraph. I think that 0:42:48.880 --> 0:42:51.280 could be healthy. Or if the whole thing just appears 0:42:51.680 --> 0:42:54.080 or it just comes linearly on, and you know, it 0:42:54.080 --> 0:42:56.160 doesn't look like something that someone worked on, then yeah, 0:42:56.160 --> 0:42:58.239 that's the red flag. But that would have flagged even 0:42:58.320 --> 0:43:01.600 the situation where you hired someone in Africa to write 0:43:01.640 --> 0:43:03.000 your paper for you and. 0:43:02.960 --> 0:43:05.640 It can quiz you on it. Yes, that's excellent. 0:43:06.280 --> 0:43:08.680 One question just to push back on that is will 0:43:08.719 --> 0:43:12.720 teachers want to look through the scrolls of data about 0:43:12.920 --> 0:43:14.440 this paragraph appeared and so on? 0:43:14.560 --> 0:43:16.440 Or is that too much work for them? Well, it 0:43:16.520 --> 0:43:16.759 is true. 0:43:16.840 --> 0:43:18.520 Well, That's why it's useful for the AI, because the 0:43:18.520 --> 0:43:21.920 AI could say so right now, our product manager, who 0:43:21.960 --> 0:43:23.360 leads what we call writing coach, she used to be 0:43:23.400 --> 0:43:25.680 an English teacher, and she used to have one hundred 0:43:25.800 --> 0:43:30.440 seventh grade students on her docket. She assigns one paper 0:43:31.320 --> 0:43:34.200 to grade. Those hundred papers would take her seventeen hours. 0:43:34.960 --> 0:43:38.000 That is not a great like that seventeen hours above 0:43:38.000 --> 0:43:40.799 and beyond being at the school for forty like it's incredible, 0:43:41.760 --> 0:43:44.040 and teachers are doing this right now. So imagine this 0:43:44.120 --> 0:43:47.600 new world where those hundred papers are done. And even 0:43:47.600 --> 0:43:49.440 those papers you don't know if Chad Gpt did them, 0:43:49.440 --> 0:43:52.320 you didn't know what happened. Now imagine this new world 0:43:52.560 --> 0:43:55.399 the papers are all done. The AI, first of all, 0:43:55.520 --> 0:43:58.560 can say, you know her name is Sarah Sarah, based 0:43:58.600 --> 0:44:01.800 on our rubric we constructed together, here's my preliminary grade 0:44:02.040 --> 0:44:05.239 and my justification of why. But you should review it. 0:44:05.800 --> 0:44:08.319 So what used to take seventeen hours can now maybe 0:44:08.320 --> 0:44:10.759 take you two hours. You still should review it, but 0:44:10.840 --> 0:44:12.360 like oh this, and in a lot of ways, it 0:44:12.400 --> 0:44:14.840 might be far more consistent because if you're grading one 0:44:14.920 --> 0:44:19.200 hundred seventh grade papers on the great Gatsby that ninetieth paper. 0:44:19.480 --> 0:44:21.320 You know you're not going to be consistent. I wouldn't 0:44:21.360 --> 0:44:24.920 be consistent one hundred percent. But also yes, and I'm 0:44:24.960 --> 0:44:26.680 confident that this is David's work, et cetera. 0:44:26.800 --> 0:44:27.279 It's just giving. 0:44:27.360 --> 0:44:29.720 It can give the synopsis, but then Sarah can ask questions. 0:44:29.719 --> 0:44:31.520 She's like, tell me more. Are there some themes you see? 0:44:31.640 --> 0:44:33.160 But yeah, it turns out a lot of your students 0:44:33.200 --> 0:44:35.960 are having trouble with thesis statements. So Sarah doesn't have 0:44:35.960 --> 0:44:39.520 to read every transcript that's not But once again, this 0:44:39.600 --> 0:44:41.640 acts as a teaching assistant and it can do all 0:44:41.640 --> 0:44:44.040 of that grunt work, but give Sarah way way more 0:44:44.080 --> 0:44:48.160 insights on where the students are, streamline the grading, and 0:44:48.400 --> 0:44:50.160 undermine any kind of cheating that might happen. 0:44:50.560 --> 0:44:50.959 I see. 0:44:51.000 --> 0:44:53.680 And it's not only grading the individual papers, but it's 0:44:53.680 --> 0:44:56.239 also giving her feedback about how the group did. 0:44:56.680 --> 0:44:57.759 Yeah, yeah, we should do. 0:44:57.920 --> 0:45:00.520 Let's let's modify tomorrow's lesson plan to be focused on outlining, 0:45:00.560 --> 0:45:02.080 because that's where a lot of students struggled. 0:45:02.280 --> 0:45:05.239 In your experience, how are teachers currently feeling about AAR 0:45:05.400 --> 0:45:08.279 feeling excited? Are they feeling threatened? And what advice do 0:45:08.280 --> 0:45:12.160 you give to teachers. I've been pleasantly surprised. You know, 0:45:12.200 --> 0:45:14.520 when we started working on kon Mega, we had we 0:45:14.560 --> 0:45:16.440 had all these debates inside of our organization, people like 0:45:16.520 --> 0:45:19.560 it's communities for cheating, and had hallucinates and and what 0:45:19.680 --> 0:45:21.319 I what I told the team at the time is like, look, 0:45:21.360 --> 0:45:23.319 we we got to just this is going to be 0:45:23.320 --> 0:45:23.560 a thing. 0:45:23.560 --> 0:45:25.239 It's going to be it's going to improve. We have 0:45:25.360 --> 0:45:27.120 these are real concerns, but we have to turn them 0:45:27.160 --> 0:45:29.080 into features. So we said, Okay, we're gonna create oversight. 0:45:29.080 --> 0:45:32.040 We're going to moderate what students are doing. We're gonna 0:45:32.840 --> 0:45:34.319 cot Migo's not going to give you the answer. It's 0:45:34.320 --> 0:45:36.480 going to be socratic. It's going to ask questions. But 0:45:36.560 --> 0:45:38.919 I was still afraid that when we launched, like there's 0:45:38.960 --> 0:45:40.879 there's gonna be a lot of blowback. Con Academy's going 0:45:40.880 --> 0:45:43.319 all in on AI. It's not perfect, et cetera. We 0:45:43.320 --> 0:45:45.240 were working on that, and then all of a sudden 0:45:45.280 --> 0:45:47.520 and we were all under a non disclosure agreement with 0:45:47.600 --> 0:45:50.839 chat GPT, with open Ai all lawyered up, and and 0:45:50.880 --> 0:45:52.879 then and we were supposed to launch March of twenty 0:45:52.920 --> 0:45:56.560 twenty three, November twenty twenty two, the last day of November. 0:45:56.239 --> 0:45:58.719 Chat GPT comes out, and I remember, like, what's going on. 0:45:58.719 --> 0:46:00.720 You guys said you were going to launch anything until March, 0:46:01.160 --> 0:46:03.120 and then I slacked Greg Brock and he's like, oh no, 0:46:03.160 --> 0:46:04.040 we all we did is put it. 0:46:04.040 --> 0:46:06.040 We launched all these apps on our old model. 0:46:06.719 --> 0:46:09.200 Chat GPT was one of them, just a chat interface 0:46:09.200 --> 0:46:12.719 on GPT three point five. We too are surprised. We 0:46:13.280 --> 0:46:15.120 didn't think anyone would care until GPT. 0:46:14.880 --> 0:46:15.760 Four came out. 0:46:15.920 --> 0:46:17.960 We were really wired because that was the moment where 0:46:17.960 --> 0:46:19.520 everyone says, oh my god, this thing has a lot 0:46:19.560 --> 0:46:22.320 of errors, it's bad at math, and it's a cheating tool. 0:46:22.520 --> 0:46:25.560 And then you saw these school districts start to ban it, etc. 0:46:25.880 --> 0:46:27.359 And we're like, oh no, we're working on this thing. 0:46:27.360 --> 0:46:29.600 We're gonna release in March, and everyone's already assuming that 0:46:29.680 --> 0:46:32.040 this is evil. It ended up that that was actually 0:46:32.080 --> 0:46:35.040 a blessing because by the time March came around four 0:46:35.040 --> 0:46:38.840 months later, the school system actually had said or started 0:46:38.840 --> 0:46:41.280 to think, well, you know what, this does have issues, 0:46:41.960 --> 0:46:44.239 but if only someone were to mitigate the risks, but 0:46:44.320 --> 0:46:46.440 this is going to be part of kids' lives and 0:46:46.560 --> 0:46:48.960 it couldn't theory be used to teach them? What if 0:46:49.000 --> 0:46:50.319 we could use it that way? And so when we 0:46:50.400 --> 0:46:53.799 launched Conmego, it had positive reaction. And I think what 0:46:53.840 --> 0:46:56.279 we're seeing from teachers is for some teachers there is 0:46:56.280 --> 0:46:58.839 a little trepidation, but as soon as we make clear, like, look, 0:46:59.000 --> 0:47:01.160 if I harted teaching sistant for every one of you, you 0:47:01.080 --> 0:47:03.359 would love that and you wouldn't feel threatened no matter 0:47:03.400 --> 0:47:05.879 how amazing that teaching assistant is. As long as you're 0:47:05.880 --> 0:47:08.200 in charge, that's what this is. And then Amani like, 0:47:08.239 --> 0:47:11.080 oh wow, yeah, this is really a gift. But and 0:47:11.239 --> 0:47:13.279 the other benefit is teachers are immediately seeing how it 0:47:13.320 --> 0:47:15.960 benefits their lives. I give that example of an English 0:47:15.960 --> 0:47:19.440 teacher having to spend seventeen hours grading one assignment, or 0:47:19.480 --> 0:47:21.960 if you're spending ten hours a week lesson planning or 0:47:21.960 --> 0:47:24.600 writing progress reports, and if you can shorten that by 0:47:24.600 --> 0:47:27.120 a factor of five or a factor of ten, that's 0:47:27.160 --> 0:47:28.720 an immediate benefit for the teacher. 0:47:28.920 --> 0:47:30.360 And it's a benefit for the teacher. 0:47:30.560 --> 0:47:32.440 If you have a classroom of thirty kids and they're 0:47:32.440 --> 0:47:35.800 all at different levels, you can't replicate yourself to address 0:47:35.800 --> 0:47:37.719 every one of their needs. So if more of those 0:47:37.760 --> 0:47:40.320 kids are having their gaps filled in and you're getting 0:47:40.320 --> 0:47:42.600 that information and then you can up level the lessons 0:47:42.600 --> 0:47:45.000 and it takes you less time to prepare them. That's 0:47:45.000 --> 0:47:48.359 starting to get to, you know, better and better from 0:47:48.360 --> 0:47:49.239 the teacher's point of view. 0:47:49.680 --> 0:47:52.520 What is the most surprising way that teachers or students 0:47:52.560 --> 0:47:53.680 have used your platform? 0:47:54.280 --> 0:47:57.759 I think early on I write about this in Brave 0:47:57.840 --> 0:48:00.560 New Words. This was I saw my opened the book 0:48:00.560 --> 0:48:02.600 this way when my daughter. This was in the early 0:48:02.680 --> 0:48:05.000 days before we had launched con Migo. We had access 0:48:05.040 --> 0:48:08.080 to GPT four no one else did. I had prompted 0:48:08.120 --> 0:48:10.160 it to just kind of co write a story with her, 0:48:10.640 --> 0:48:12.720 and it was a story about the social media influencer 0:48:12.760 --> 0:48:15.239 Samantha who's stuck on a desert island and she was 0:48:15.280 --> 0:48:17.600 having like a panic attack because she couldn't share how 0:48:17.640 --> 0:48:21.200 beautiful it was with her friends. And then my daughter said, 0:48:21.200 --> 0:48:24.799 can I talk to Samantha. I'm like, I guess you could, 0:48:24.880 --> 0:48:26.320 and so she said, I'd like to talk to Samantha, 0:48:26.320 --> 0:48:30.560 and then the aimedia says, hey, Samantha, here, I don't 0:48:30.560 --> 0:48:32.440 know what to do. I can't share this, and my 0:48:32.520 --> 0:48:35.600 daughter's like, it's okay, you know, just enjoy the beauty 0:48:35.760 --> 0:48:36.160 and like. 0:48:36.480 --> 0:48:37.839 And that was the first one. I was like, this 0:48:37.880 --> 0:48:38.560 is science fiction. 0:48:38.680 --> 0:48:40.840 Like, my daughter is not only writing a story with 0:48:40.880 --> 0:48:43.520 an AI, but now she can actually talk to a 0:48:43.600 --> 0:48:47.799 character that they constructed together in her story. And that 0:48:47.960 --> 0:48:50.360 was some of the idea behind Now we have activities 0:48:50.400 --> 0:48:53.319 on con Migo where you can talk to AI simulations 0:48:53.360 --> 0:48:57.160 of literary figures or historical characters. And you know, we 0:48:57.560 --> 0:48:59.200 have something called con World School, which is a virtual 0:48:59.280 --> 0:49:02.000 high school, and one of there's a student in India 0:49:02.440 --> 0:49:04.600 and she was starting The Great Gatsby. She had a huge, 0:49:04.680 --> 0:49:07.520 long conversation with Jay Gatsby AI simulation of Jay Gatsby, 0:49:07.560 --> 0:49:09.640 And you know, she told me the whole conversation. She's like, 0:49:09.680 --> 0:49:11.439 and I apologize for taking up all of his time. 0:49:11.719 --> 0:49:13.799 And she knew that he's an AI. It's not like 0:49:13.880 --> 0:49:16.680 she's you know, but and it was a pretty meaningful 0:49:16.719 --> 0:49:20.279 conversation about you know, goals and aspirations and never being 0:49:20.280 --> 0:49:22.880 able to reach them, et cetera. You know, I'm a 0:49:22.880 --> 0:49:25.520 Star trek nerd, especially you know, the next generation of 0:49:25.520 --> 0:49:27.200 the Holidack. And I used to think that was the 0:49:27.239 --> 0:49:29.520 most fake part of Star Trek, Like even in the 0:49:29.560 --> 0:49:31.480 twenty fourth century. Surely we're not going to be able 0:49:31.480 --> 0:49:35.319 to have like conversations with Albert Einstein. But literally, that's 0:49:35.400 --> 0:49:38.200 already hap there, that's already happening, and it's only going 0:49:38.239 --> 0:49:39.000 to get better and better. 0:49:43.640 --> 0:49:45.840 That was my interview with Saul Khan, founder of the 0:49:45.880 --> 0:49:48.640 Kahn Academy. One of the things I thought was interesting 0:49:48.680 --> 0:49:51.719 is that he's in favor of teaching wrote knowledge. Sometimes 0:49:51.760 --> 0:49:54.600 he argues that it is to some degree a really 0:49:54.640 --> 0:49:57.840 important factor. But of all the things we covered today, 0:49:57.880 --> 0:50:00.560 I have to say that my absolute favorite is the 0:50:00.680 --> 0:50:05.920 project of learning critical thinking by debating with an AI, 0:50:06.239 --> 0:50:09.080 one that is friendly but firm, and the idea that 0:50:09.080 --> 0:50:11.360 we can start doing this right away. We can start 0:50:11.440 --> 0:50:14.720 getting our students to take a side on some issue 0:50:14.960 --> 0:50:19.120 and put forward logical arguments about it, and get graded 0:50:19.200 --> 0:50:22.480 on the quality of their arguments, and then on their 0:50:22.600 --> 0:50:26.680 ability to represent the argument of the other side, and 0:50:26.760 --> 0:50:30.279 then to switch sides and do it all again. That 0:50:30.360 --> 0:50:33.240 feels to me like a place where AI can fill 0:50:33.280 --> 0:50:36.400 a niche that is not currently filled, and it can 0:50:36.480 --> 0:50:40.680 do this with the loving patience that we would devoutly 0:50:40.719 --> 0:50:44.320 wish for when arguing with our classmates. Or our children 0:50:44.400 --> 0:50:48.200 or our students. The AI never gets tired or bored 0:50:48.440 --> 0:50:52.279 or offended. So in the end, as we start each 0:50:52.320 --> 0:50:55.920 school year in a world that changes faster and faster, 0:50:56.480 --> 0:51:00.560 I have high hopes that will continue to discover new 0:51:00.600 --> 0:51:04.680 ways to teach our students things that last, that we 0:51:04.800 --> 0:51:07.400 will give them roots and wings. 0:51:07.719 --> 0:51:09.279 And if we can use AI. 0:51:09.360 --> 0:51:13.480 To help students sharpen their reasoning and engage in deeper 0:51:13.520 --> 0:51:17.960 conversations and see the world from multiple perspectives, then we're 0:51:17.960 --> 0:51:21.719 not just preparing them to take the standardized tests. We 0:51:21.800 --> 0:51:26.560 are preparing them for life. The future of education isn't 0:51:26.600 --> 0:51:29.759 just about what we learn, but how we learn, and 0:51:29.800 --> 0:51:33.000 with the right technology, I have no doubt that we 0:51:33.080 --> 0:51:42.520 can make that future and our students quite bright. Go 0:51:42.560 --> 0:51:45.600 to Eagleman dot com slash podcasts for more information and 0:51:45.640 --> 0:51:47.000 to find further reading. 0:51:47.480 --> 0:51:48.800 Send me an email. 0:51:48.480 --> 0:51:51.759 At podcasts at eagleman dot com with questions or discussion, 0:51:52.120 --> 0:51:55.319 and check out and subscribe to Intercosmos on YouTube for 0:51:55.480 --> 0:51:59.439 videos of each episode and to leave comments until next time. 0:51:59.560 --> 0:52:02.560 I'm Dave Eagelman and this is inner Cosmos.