1
00:00:00,070 --> 00:00:02,830
Speaker 1: Hey, Odd Lots listeners, the Odd Lots tour continues and

2
00:00:02,890 --> 00:00:04,250
Speaker 1: our next stop is in Chicago.

3
00:00:04,510 --> 00:00:06,319
Speaker 2: That's right. Joe and I will be at the City

4
00:00:06,460 --> 00:00:10,539
Speaker 2: Winery Chicago on October 15th for a live Odd Lots recording.

5
00:00:10,700 --> 00:00:14,640
Speaker 2: Tickets are on sale now at Bloomberg.com forward slash Odd Lots.

6
00:00:14,940 --> 00:00:17,390
Speaker 1: And of course, a special thank you to Barclays for

7
00:00:17,450 --> 00:00:18,970
Speaker 1: supporting Odd Lots Live.

8
00:00:19,210 --> 00:00:22,930
Speaker 2: So that's October 15th at City Winery in Chicago. Get

9
00:00:23,010 --> 00:00:23,910
Speaker 2: your tickets now.

10
00:00:26,950 --> 00:00:43,279
Speaker 1: Bloomberg Audio Studios. Podcasts, radio, news. Hello and welcome to

11
00:00:43,320 --> 00:00:46,979
Speaker 1: another episode of the Odd Laws Podcast. I'm Joe Weisenthal.

12
00:00:47,120 --> 00:00:48,080
Speaker 2: And I'm Tracy Allaway.

13
00:00:48,479 --> 00:00:51,420
Speaker 1: So Tracy, with AI, you know, obviously one of the

14
00:00:51,460 --> 00:00:55,360
Speaker 1: big concerns is like job loss. What are we going to,

15
00:00:55,580 --> 00:00:57,820
Speaker 1: in the future, what are humans going to be better

16
00:00:58,060 --> 00:00:59,730
Speaker 1: at and What.

17
00:00:59,570 --> 00:01:00,510
Speaker 2: Are we going to do all day?

18
00:01:00,530 --> 00:01:02,330
Speaker 1: What are we going to do all day, et cetera?

19
00:01:02,410 --> 00:01:04,390
Speaker 1: And we really don't know the answer, I think, to

20
00:01:04,530 --> 00:01:09,010
Speaker 1: any of these questions yet. But I am aware, obviously,

21
00:01:09,209 --> 00:01:11,130
Speaker 1: the models are getting very good at math, and at

22
00:01:11,190 --> 00:01:14,990
Speaker 1: least some mathematicians perceive, you know, they're in a state

23
00:01:15,010 --> 00:01:20,600
Speaker 1: of existential angst about this, and certainly crisis, perhaps, some

24
00:01:20,680 --> 00:01:23,260
Speaker 1: feel about, like, well, what are we doing here? These

25
00:01:23,360 --> 00:01:27,440
Speaker 1: problems that I devoted my life to solving and the

26
00:01:27,480 --> 00:01:31,750
Speaker 1: model solves it in a weekend. Is this what's potentially

27
00:01:31,770 --> 00:01:32,569
Speaker 1: coming for many of us?

28
00:01:32,709 --> 00:01:35,449
Speaker 2: Yeah. Math is an interesting one because you can tell

29
00:01:35,510 --> 00:01:38,070
Speaker 2: that a lot of the AI companies themselves have an

30
00:01:38,110 --> 00:01:40,550
Speaker 2: interest in math. They see math as a sort of

31
00:01:40,610 --> 00:01:43,089
Speaker 2: holy grail for, I guess, proving that the models are

32
00:01:43,190 --> 00:01:46,360
Speaker 2: able to reason. It's the thing they need to do

33
00:01:46,440 --> 00:01:49,280
Speaker 2: and able to say that they've achieved AGI, I think,

34
00:01:49,360 --> 00:01:52,760
Speaker 2: is the way it's couched. And it also makes good headlines.

35
00:01:53,100 --> 00:01:56,900
Speaker 2: So even though I have no idea what the Navier-Stokes

36
00:01:57,000 --> 00:01:59,660
Speaker 2: problem actually is. I don't know even if I'm pronouncing

37
00:01:59,700 --> 00:02:03,660
Speaker 2: that correctly. Is it Navier or Navier? Okay, Navier-Stokes. I

38
00:02:03,740 --> 00:02:06,560
Speaker 2: see the headline and I'm like, ooh, a model has

39
00:02:06,740 --> 00:02:09,919
Speaker 2: solved the Navier-Stokes problem, as if I know what that

40
00:02:10,000 --> 00:02:12,669
Speaker 2: actually means and the significance for the mathematics profession.

41
00:02:13,230 --> 00:02:17,030
Speaker 1: It's also funny because I downloaded the paper and it's

42
00:02:17,050 --> 00:02:19,990
Speaker 1: like one paragraph and I have no idea. I can't

43
00:02:20,010 --> 00:02:25,980
Speaker 1: even begin to parse the output. Another fascinating aspect of

44
00:02:26,040 --> 00:02:29,540
Speaker 1: this is just, like, it would not have been intuitive

45
00:02:29,780 --> 00:02:33,040
Speaker 1: to me, say, two years ago or three years ago,

46
00:02:33,490 --> 00:02:37,070
Speaker 1: that a language model would be so good at mathematics.

47
00:02:37,290 --> 00:02:39,150
Speaker 1: In fact, at one point, they weren't very good at

48
00:02:39,230 --> 00:02:41,430
Speaker 1: math at all. Like, go back to, like, I don't know,

49
00:02:41,470 --> 00:02:44,089
Speaker 1: GPT-3 or GPT-3.5.

50
00:02:44,690 --> 00:02:46,890
Speaker 2: If you asked them to count, like, the number of

51
00:02:46,990 --> 00:02:49,470
Speaker 2: letters in a word or something, they would get it wrong.

52
00:02:49,650 --> 00:02:49,910
Speaker 3: Yeah.

53
00:02:50,169 --> 00:02:54,010
Speaker 1: And weirdly, they still, like, struggle with certain things like that.

54
00:02:54,470 --> 00:02:58,770
Speaker 1: But it's not obvious to me, like, okay, you scan

55
00:02:58,850 --> 00:03:02,420
Speaker 1: all of the text on text, you know, language, et cetera.

56
00:03:02,840 --> 00:03:07,980
Speaker 1: And yet somehow that generalizes into this extraordinary math capability.

57
00:03:09,220 --> 00:03:10,369
Speaker 1: That too is very surprising.

58
00:03:10,380 --> 00:03:10,470
Speaker 3: Yeah.

59
00:03:10,950 --> 00:03:13,450
Speaker 2: In this conversation, can I just say before we start,

60
00:03:13,590 --> 00:03:16,560
Speaker 2: I would also just like to learn what advanced mathematics

61
00:03:16,740 --> 00:03:20,160
Speaker 2: actually is. Because everything I know about it, which is nothing,

62
00:03:20,900 --> 00:03:23,320
Speaker 2: comes from like watching Good Will Hunting and just seeing,

63
00:03:23,360 --> 00:03:25,140
Speaker 2: you know, a bunch of formulas on a chalkboard and

64
00:03:25,180 --> 00:03:28,300
Speaker 2: people get really excited when they start crossing off certain

65
00:03:28,360 --> 00:03:31,210
Speaker 2: parentheses and stuff like that. I have no idea what

66
00:03:31,250 --> 00:03:33,450
Speaker 2: it is. I have no idea how it differs from

67
00:03:33,550 --> 00:03:37,010
Speaker 2: pure maths of like two plus two equals four. And

68
00:03:37,090 --> 00:03:39,450
Speaker 2: so I would just like to come away in this

69
00:03:39,490 --> 00:03:42,460
Speaker 2: conversation with a better understanding of what it is overall

70
00:03:42,580 --> 00:03:44,440
Speaker 2: and how AI actually fits into it.

71
00:03:44,920 --> 00:03:48,560
Speaker 1: Absolutely. So like I was thinking along the same lines, like, okay,

72
00:03:48,600 --> 00:03:51,520
Speaker 1: for like someone who like devotes their life to like

73
00:03:51,900 --> 00:03:56,650
Speaker 1: advanced theoretical math and, you know, other also like things

74
00:03:56,690 --> 00:04:00,620
Speaker 1: like advanced theoretical physics and, that don't necessarily have like

75
00:04:00,780 --> 00:04:05,120
Speaker 1: obvious like industrial applications, etc. Who is the type of person?

76
00:04:05,240 --> 00:04:08,050
Speaker 1: What are they hoping to achieve? Why is there still

77
00:04:08,290 --> 00:04:12,210
Speaker 1: interest in some of these things that perhaps the sort

78
00:04:12,250 --> 00:04:18,940
Speaker 1: of applicability is not necessarily so obvious? But then like obviously,

79
00:04:19,000 --> 00:04:21,719
Speaker 1: you know, like what is the role of the mathematician here?

80
00:04:22,420 --> 00:04:22,800
Speaker 3: At this point.

81
00:04:23,040 --> 00:04:27,220
Speaker 1: And then actually specifically it's like, okay, across universe. This

82
00:04:27,279 --> 00:04:29,440
Speaker 1: is actually the thing I'm very curious about, which is

83
00:04:30,180 --> 00:04:33,200
Speaker 1: my son studies math, right? Both my kids study math

84
00:04:33,279 --> 00:04:33,779
Speaker 1: and like.

85
00:04:33,960 --> 00:04:36,380
Speaker 2: Hopefully not theoretical math just yet.

86
00:04:36,460 --> 00:04:37,020
Speaker 1: Not yet.

87
00:04:37,060 --> 00:04:37,680
Speaker 3: How old is he?

88
00:04:38,000 --> 00:04:41,880
Speaker 1: My daughter's 10. My son is seven. They take math,

89
00:04:42,279 --> 00:04:45,870
Speaker 1: et cetera. But like setting aside like what the future holds,

90
00:04:46,330 --> 00:04:48,900
Speaker 1: I'm very interested in just the present. Because it's like

91
00:04:48,920 --> 00:04:51,719
Speaker 1: you have all these math departments at universities all around

92
00:04:52,260 --> 00:04:54,520
Speaker 1: the country and like maybe some are excited about AI,

93
00:04:54,860 --> 00:04:58,200
Speaker 1: et cetera. But like, what do you do today in

94
00:04:58,260 --> 00:05:02,290
Speaker 1: terms of like rethinking pedagogies? Because it must be, or

95
00:05:02,350 --> 00:05:06,410
Speaker 1: I would assume it's pretty different in 2026 than like

96
00:05:06,430 --> 00:05:10,390
Speaker 1: those same departments in 2021 or something like that.

97
00:05:10,490 --> 00:05:10,950
Speaker 2: Absolutely.

98
00:05:10,970 --> 00:05:16,610
Speaker 1: And like, how are professors, teachers, et cetera, thinking about

99
00:05:16,670 --> 00:05:19,800
Speaker 1: just the sort of the process of teaching math right now.

100
00:05:20,080 --> 00:05:22,580
Speaker 1: It's just like, just sort of like very straightforwardly, how

101
00:05:22,640 --> 00:05:25,380
Speaker 1: is it affecting their jobs? I used to be really,

102
00:05:25,440 --> 00:05:27,960
Speaker 1: really good at sort of mental math, like times tables

103
00:05:27,980 --> 00:05:28,500
Speaker 1: and stuff like that.

104
00:05:28,520 --> 00:05:30,140
Speaker 2: This is a question I have too. If you are

105
00:05:30,300 --> 00:05:34,410
Speaker 2: an advanced mathematician doing theoretical maths, how often do you

106
00:05:34,450 --> 00:05:35,570
Speaker 2: actually use a calculator?

107
00:05:36,310 --> 00:05:37,110
Speaker 1: I'm curious about that.

108
00:05:37,130 --> 00:05:38,010
Speaker 2: That's going to be my first question.

109
00:05:38,210 --> 00:05:41,390
Speaker 1: Actually, Jensen Wong, I don't know if you read the

110
00:05:41,430 --> 00:05:45,430
Speaker 1: transcript of his interview with Ezra Klein a couple of

111
00:05:45,470 --> 00:05:45,920
Speaker 1: weeks ago.

112
00:05:45,940 --> 00:05:46,210
Speaker 3: Mm-hmm.

113
00:05:46,480 --> 00:05:48,660
Speaker 1: But he said something. It's like, oh, maybe it's just not.

114
00:05:49,460 --> 00:05:51,560
Speaker 1: It's a little kind of weird interview. He said something like, well,

115
00:05:51,580 --> 00:05:54,300
Speaker 1: maybe like knowing your times tables isn't going to be

116
00:05:54,320 --> 00:05:55,500
Speaker 1: an important skill in the future.

117
00:05:55,580 --> 00:05:56,300
Speaker 2: Or your address.

118
00:05:56,440 --> 00:05:59,080
Speaker 1: Yeah, which is like kind of weird. Like, I still

119
00:05:59,140 --> 00:06:00,550
Speaker 1: want my kids to.

120
00:06:01,370 --> 00:06:05,590
Speaker 2: No, but I think it raises a very important question,

121
00:06:05,610 --> 00:06:08,500
Speaker 2: which is you can go to a model and have

122
00:06:08,560 --> 00:06:12,340
Speaker 2: them verify a proof or a theorem or something like that.

123
00:06:12,779 --> 00:06:16,300
Speaker 2: But if you don't understand how you actually got there

124
00:06:16,360 --> 00:06:19,480
Speaker 2: and how the model is getting to its verification, then

125
00:06:19,500 --> 00:06:20,740
Speaker 2: what are we actually learning here?

126
00:06:21,180 --> 00:06:24,730
Speaker 1: Right. And then there's the question of like. people learn

127
00:06:24,770 --> 00:06:27,210
Speaker 1: a lot in the process, right? So it's like even

128
00:06:27,680 --> 00:06:30,860
Speaker 1: presumably in just the people work on some of these

129
00:06:30,880 --> 00:06:33,800
Speaker 1: problems for years and years, and maybe they never arrive

130
00:06:33,839 --> 00:06:37,279
Speaker 1: at any proof, but intuitively you would think there is something,

131
00:06:37,680 --> 00:06:40,420
Speaker 1: maybe it's exercising a muscle or something, or sort of

132
00:06:41,279 --> 00:06:44,870
Speaker 1: other benefits to just the process of learning. How's that change? Anyway,

133
00:06:45,990 --> 00:06:47,690
Speaker 1: I have a lot of questions about this and we

134
00:06:47,730 --> 00:06:49,820
Speaker 1: really do have the perfect guest. We're going to be

135
00:06:49,860 --> 00:06:54,180
Speaker 1: speaking with Justin Solomon. He's Associate Dean of Engineering Education

136
00:06:54,380 --> 00:06:58,300
Speaker 1: at MIT and very interested and involved in math education

137
00:06:58,400 --> 00:07:02,000
Speaker 1: and thinking about the intersection of math and AI. Justin,

138
00:07:02,120 --> 00:07:03,750
Speaker 1: thank you so much for coming on OddLots.

139
00:07:03,930 --> 00:07:04,560
Speaker 3: Thanks for having me.

140
00:07:05,250 --> 00:07:08,330
Speaker 1: I mean, Tracy sort of already suggested, but let's set

141
00:07:08,370 --> 00:07:12,309
Speaker 1: aside AI for a second. What is it that people

142
00:07:12,350 --> 00:07:15,390
Speaker 1: who end up going into sort of like advanced math

143
00:07:15,430 --> 00:07:18,730
Speaker 1: or advanced theoretical math, like who are these people?

144
00:07:19,970 --> 00:07:20,290
Speaker 2: Why?

145
00:07:20,350 --> 00:07:21,790
Speaker 1: Like, what are they hoping to do?

146
00:07:22,670 --> 00:07:25,360
Speaker 3: Yeah, it's true. I feel like the typical caricature that

147
00:07:25,380 --> 00:07:28,380
Speaker 3: you see is maybe this character on, what, Good Will Hunting,

148
00:07:28,580 --> 00:07:31,280
Speaker 3: which incidentally, they're always writing math on Windows, which is

149
00:07:31,520 --> 00:07:35,260
Speaker 3: very impractical. Good visuals, though. Yeah, that's true. It's great

150
00:07:35,280 --> 00:07:37,560
Speaker 3: for a camera. But that aside, I think it's an

151
00:07:37,600 --> 00:07:41,400
Speaker 3: intensely personal thing. So to be clear, my personal career,

152
00:07:41,440 --> 00:07:43,750
Speaker 3: like I'm a professor of engineering, is in an area

153
00:07:43,760 --> 00:07:46,050
Speaker 3: of math called applied math, which sometimes is applied and

154
00:07:46,090 --> 00:07:48,830
Speaker 3: sometimes not. That's a discussion for another day. But essentially,

155
00:07:48,870 --> 00:07:50,590
Speaker 3: some people go into math because they want to see

156
00:07:50,630 --> 00:07:54,250
Speaker 3: it translated into applications or to bring fundamental insight into

157
00:07:54,290 --> 00:07:56,030
Speaker 3: the kind of things we do every day, whether it's

158
00:07:56,050 --> 00:07:59,470
Speaker 3: predicting markets or physics, what have you. Other people in

159
00:07:59,710 --> 00:08:02,540
Speaker 3: areas of math, sometimes you'll call them pure math. I

160
00:08:02,580 --> 00:08:05,800
Speaker 3: don't love that terminology, like pure versus applied. But in

161
00:08:05,840 --> 00:08:07,860
Speaker 3: other areas of math, it's almost just for the joy

162
00:08:07,880 --> 00:08:10,440
Speaker 3: of the game or understanding the universe around you. All

163
00:08:10,460 --> 00:08:13,300
Speaker 3: of these are totally legitimate reasons and sort of.

164
00:08:13,940 --> 00:08:17,370
Speaker 1: intellectual pursuit for the sake of intellectual pursuit.

165
00:08:17,530 --> 00:08:20,050
Speaker 3: That's right. But along the way, I think you develop

166
00:08:20,090 --> 00:08:22,950
Speaker 3: a lot of really important reasoning skills and skills that

167
00:08:22,990 --> 00:08:24,560
Speaker 3: support work in all kinds of different areas.

168
00:08:25,060 --> 00:08:29,100
Speaker 2: And how often do advanced mathematicians, whether they're in, I guess,

169
00:08:29,160 --> 00:08:33,040
Speaker 2: applied maths or purely theoretical maths, how often do they

170
00:08:33,100 --> 00:08:33,760
Speaker 2: use a calculator?

171
00:08:34,530 --> 00:08:37,650
Speaker 3: Very often. Okay. Yeah, no. But my impression, again.

172
00:08:37,690 --> 00:08:40,750
Speaker 2: Is everyone is doing this by hand on chalkboards and windows.

173
00:08:41,110 --> 00:08:43,290
Speaker 3: I mean, there's like lots of Greek symbols and letters

174
00:08:43,330 --> 00:08:46,210
Speaker 3: and variables and things. But no, I'm terrible at arithmetic.

175
00:08:46,390 --> 00:08:48,820
Speaker 3: I've got a calculator out all the time. Oh, yeah.

176
00:08:49,390 --> 00:08:53,140
Speaker 2: This gives me hope, actually, that maybe I could understand

177
00:08:53,420 --> 00:08:56,580
Speaker 2: some of this, even though I'm pretty bad at arithmetic itself.

178
00:08:56,980 --> 00:08:57,520
Speaker 3: Oh, absolutely.

179
00:08:58,160 --> 00:09:00,380
Speaker 2: What's the difference between a math problem then and a

180
00:09:00,760 --> 00:09:03,820
Speaker 2: math proof or a theorem? These sort of big picture

181
00:09:03,900 --> 00:09:06,370
Speaker 2: things that we're talking about AI models actually doing now.

182
00:09:06,550 --> 00:09:08,810
Speaker 3: That's a great question. And it's really important to dig

183
00:09:08,870 --> 00:09:11,410
Speaker 3: into exactly what the models are doing and not doing

184
00:09:11,570 --> 00:09:14,870
Speaker 3: and who deserves credit, by the way. But essentially, in mathematics,

185
00:09:14,929 --> 00:09:18,710
Speaker 3: it's all about verifying, in some sense, verifying true facts

186
00:09:18,770 --> 00:09:21,679
Speaker 3: is one part of the puzzle. But it's also all

187
00:09:21,720 --> 00:09:24,449
Speaker 3: about figuring out what are the interesting problems to study

188
00:09:24,570 --> 00:09:27,330
Speaker 3: and what insight they bring to the broader world, right?

189
00:09:27,390 --> 00:09:30,319
Speaker 3: So in some areas of mathematics, there are all kinds

190
00:09:30,340 --> 00:09:31,740
Speaker 3: of different questions you could ask. And some of them

191
00:09:31,780 --> 00:09:33,819
Speaker 3: are super uninteresting, but you could ask a computer to

192
00:09:33,860 --> 00:09:36,500
Speaker 3: prove them. And others maybe tell you something about a

193
00:09:36,520 --> 00:09:39,180
Speaker 3: physical system or about a system of equations or something

194
00:09:39,220 --> 00:09:42,800
Speaker 3: like that, that bring broader understanding. And so this exercise

195
00:09:42,840 --> 00:09:47,410
Speaker 3: of proofwriting is all about sort of writing a mathematically

196
00:09:47,450 --> 00:09:50,990
Speaker 3: convincing argument that tells mathematicians. So there's a human aspect,

197
00:09:51,050 --> 00:09:53,880
Speaker 3: which is an important one. that a certain fact is

198
00:09:53,960 --> 00:09:56,960
Speaker 3: actually maybe reality, right? It aligns with logic. There's not

199
00:09:56,980 --> 00:09:58,350
Speaker 3: some flaw hiding in there somewhere.

200
00:09:59,350 --> 00:10:03,020
Speaker 2: And where do you ideas for a particular mathematical proof

201
00:10:03,200 --> 00:10:05,579
Speaker 2: or theory come from? Because again, I'm going to come

202
00:10:05,620 --> 00:10:07,719
Speaker 2: at this from, I guess, the movie angle, but I

203
00:10:07,740 --> 00:10:10,719
Speaker 2: think about A Beautiful Mind, right? And the famous bar

204
00:10:10,780 --> 00:10:13,900
Speaker 2: scene and it was John Nash, right? That was his name.

205
00:10:14,350 --> 00:10:17,000
Speaker 2: And he has this like big realization while he's at

206
00:10:17,040 --> 00:10:20,800
Speaker 2: the bar with these women and it ends up turning

207
00:10:20,820 --> 00:10:24,120
Speaker 2: into a famous mathematical theory. Is it up, like where

208
00:10:24,140 --> 00:10:26,840
Speaker 2: did the ideas actually come from for the rules that

209
00:10:26,880 --> 00:10:29,240
Speaker 2: you are trying to prove in maths or these big

210
00:10:29,280 --> 00:10:30,059
Speaker 2: picture ideas?

211
00:10:30,520 --> 00:10:33,990
Speaker 3: That's a great question. So sometimes in mathematics, there are

212
00:10:34,000 --> 00:10:36,370
Speaker 3: some really natural questions out there in the universe that

213
00:10:36,670 --> 00:10:38,530
Speaker 3: people have been asking for a long time. So for example,

214
00:10:38,570 --> 00:10:40,030
Speaker 3: these days in the news, we hear about some of

215
00:10:40,070 --> 00:10:42,730
Speaker 3: these really high profile math problems that have been around forever.

216
00:10:43,880 --> 00:10:46,600
Speaker 3: There are other types of mathematicians that are more theory builders,

217
00:10:46,660 --> 00:10:48,920
Speaker 3: where they're the ones asking the questions rather than answering

218
00:10:48,970 --> 00:10:51,910
Speaker 3: someone else's questions. There are definitely different approaches that are

219
00:10:51,970 --> 00:10:54,890
Speaker 3: out there. Now, when it comes to where these questions

220
00:10:54,950 --> 00:10:58,270
Speaker 3: are coming from, some of them are inspired purely mathematically,

221
00:10:58,550 --> 00:11:00,949
Speaker 3: and other ones maybe come from a very practical setting.

222
00:11:00,990 --> 00:11:03,570
Speaker 3: Maybe somebody observed some phenomenon out there in the real world,

223
00:11:03,630 --> 00:11:07,340
Speaker 3: and they're trying to explain it with the smallest model possible. Now,

224
00:11:07,510 --> 00:11:09,280
Speaker 3: one thing I should clarify is that math is a

225
00:11:09,340 --> 00:11:12,420
Speaker 3: really social exercise. I think that's something that people don't

226
00:11:12,460 --> 00:11:15,699
Speaker 3: necessarily understand, especially because these movies really play into this

227
00:11:16,140 --> 00:11:18,560
Speaker 3: almost great man theory of mathematics. But that's just not

228
00:11:18,580 --> 00:11:21,000
Speaker 3: how it works. There's this huge community of people that

229
00:11:21,040 --> 00:11:25,060
Speaker 3: are working on important problems. They're communicating with one another, collaborating.

230
00:11:25,500 --> 00:11:27,559
Speaker 3: And I would say, for example, from my own research,

231
00:11:27,660 --> 00:11:30,880
Speaker 3: almost everything that our team has worked on comes as

232
00:11:30,920 --> 00:11:32,820
Speaker 3: a team effort. We're all sitting in the same room.

233
00:11:32,840 --> 00:11:35,250
Speaker 3: We're at the blackboard writing computer models, what have you.

234
00:11:35,600 --> 00:11:37,220
Speaker 3: And that's where the insight is coming from. So it's

235
00:11:37,240 --> 00:11:39,540
Speaker 3: not just sitting in your ivory tower. It's being out

236
00:11:39,570 --> 00:11:42,470
Speaker 3: there in the world, observing things happening and talking to

237
00:11:42,490 --> 00:11:43,110
Speaker 3: your colleagues.

238
00:11:43,830 --> 00:11:46,590
Speaker 1: Before we get more into the sort of AI question,

239
00:11:46,890 --> 00:11:50,450
Speaker 1: I've actually been curious, what's applied math? What is the

240
00:11:50,510 --> 00:11:52,790
Speaker 1: difference between applied math and pure math?

241
00:11:52,970 --> 00:11:55,490
Speaker 3: Yeah, it's a great question. So there's sort of a

242
00:11:55,530 --> 00:11:58,570
Speaker 3: divide in the math world, but I think it's fuzzier

243
00:11:58,610 --> 00:12:01,069
Speaker 3: than maybe it sounds when I describe it here. So roughly,

244
00:12:01,130 --> 00:12:04,130
Speaker 3: there's an area of mathematics that you might call applied math.

245
00:12:04,540 --> 00:12:07,030
Speaker 3: where the goal is to either derive theoretical results or

246
00:12:07,070 --> 00:12:10,190
Speaker 3: even come up with computer algorithms that explain phenomena out

247
00:12:10,210 --> 00:12:12,670
Speaker 3: there in the real world. So, for example, in applied math,

248
00:12:13,110 --> 00:12:16,190
Speaker 3: maybe you work on algorithms that simulate the weather, and

249
00:12:16,230 --> 00:12:19,130
Speaker 3: behind the scenes is a really complex mathematical model, and

250
00:12:19,170 --> 00:12:21,410
Speaker 3: it's all about making sure that your model is correct

251
00:12:21,490 --> 00:12:25,280
Speaker 3: and efficient and well-behaved, all that good stuff. In pure mathematics,

252
00:12:25,500 --> 00:12:28,420
Speaker 3: sometimes the motivation is a little bit less attached to

253
00:12:28,440 --> 00:12:31,360
Speaker 3: a particular application, and more just attached to the math

254
00:12:31,380 --> 00:12:32,200
Speaker 3: problem itself.

255
00:12:32,240 --> 00:12:35,480
Speaker 1: So something like Fermat's Last Theorem or something like that,

256
00:12:35,520 --> 00:12:39,020
Speaker 1: where he's like, okay, I forget what it is exactly,

257
00:12:39,070 --> 00:12:41,050
Speaker 1: but that would not be applied math.

258
00:12:41,170 --> 00:12:43,390
Speaker 3: Right, so Fermat's Last Theorem, and many things I think

259
00:12:43,429 --> 00:12:46,270
Speaker 3: in number theory fall into that category, but also many

260
00:12:46,309 --> 00:12:48,630
Speaker 3: times these things bleed into one another. So for example,

261
00:12:48,750 --> 00:12:52,300
Speaker 3: a lot of modern developments in cryptography came from areas

262
00:12:52,340 --> 00:12:54,760
Speaker 3: of mathematics that we used to think of as extremely

263
00:12:54,880 --> 00:12:56,100
Speaker 3: pure and abstract.

264
00:12:56,460 --> 00:12:57,020
Speaker 1: Oh, interesting.

265
00:12:57,400 --> 00:12:59,600
Speaker 2: This is also Good Will Hunting, by the way.

266
00:12:59,800 --> 00:13:00,699
Speaker 3: It all goes back to that.

267
00:13:00,820 --> 00:13:01,260
Speaker 2: That's right.

268
00:13:01,380 --> 00:13:01,660
Speaker 3: That's right.

269
00:13:02,300 --> 00:13:03,560
Speaker 2: You worked at Pixar before.

270
00:13:03,920 --> 00:13:04,520
Speaker 3: Many years ago.

271
00:13:04,679 --> 00:13:06,740
Speaker 2: Was that applied maths working at Pixar?

272
00:13:06,800 --> 00:13:09,650
Speaker 3: Oh, I love that question. The answer is absolutely yes. Okay. Yeah.

273
00:13:09,770 --> 00:13:12,569
Speaker 3: So the movie studios are a great example of a

274
00:13:12,610 --> 00:13:14,469
Speaker 3: place where I would say state of the art and

275
00:13:14,510 --> 00:13:17,410
Speaker 3: certain types of mathematics are taking place. So when you

276
00:13:17,429 --> 00:13:21,350
Speaker 3: watch a typical movie with different movie effects, there's cloth,

277
00:13:21,470 --> 00:13:25,579
Speaker 3: there's smoke, there's all kinds of fluids, all rigid bodies

278
00:13:25,620 --> 00:13:28,570
Speaker 3: flying around. All the dynamics of those are governed by

279
00:13:28,610 --> 00:13:31,800
Speaker 3: things called partial differential equations. And coming up with the

280
00:13:31,860 --> 00:13:34,260
Speaker 3: right algorithms for simulating those, and by the way, making

281
00:13:34,280 --> 00:13:36,940
Speaker 3: them so well behaved that an artist doesn't have to

282
00:13:36,980 --> 00:13:40,630
Speaker 3: think about math, is a really challenging mathematical problem. So

283
00:13:40,970 --> 00:13:43,330
Speaker 3: I typically tell my students that if you're looking for

284
00:13:43,370 --> 00:13:45,210
Speaker 3: really challenging math problems, you don't have to make them up.

285
00:13:45,230 --> 00:13:46,870
Speaker 3: They're out there in the real world. And that's a

286
00:13:46,910 --> 00:13:47,330
Speaker 3: nice example.

287
00:14:03,390 --> 00:14:04,380
Speaker 1: What's Navier-Stokes?

288
00:14:04,860 --> 00:14:07,979
Speaker 3: Okay. So Navier-Stokes has been in the news a lot lately.

289
00:14:08,780 --> 00:14:10,480
Speaker 3: And it's been great for those of us with a

290
00:14:10,480 --> 00:14:13,580
Speaker 3: little bit of background in differential equations.

291
00:14:14,860 --> 00:14:16,380
Speaker 1: Getting lots of media inquiries.

292
00:14:16,500 --> 00:14:18,500
Speaker 3: That's exactly right. Yeah, we have our 50 minutes of

293
00:14:18,540 --> 00:14:22,000
Speaker 3: fame here. It'll die fast, but we'll enjoy it for now. Right.

294
00:14:22,020 --> 00:14:24,530
Speaker 3: So the Navier-Stokes problem is one that comes from the

295
00:14:24,550 --> 00:14:27,890
Speaker 3: world of fluid dynamics. So the Navier-Stokes equations were this model.

296
00:14:27,930 --> 00:14:30,260
Speaker 3: They go back quite a long time. that explain the

297
00:14:30,300 --> 00:14:32,660
Speaker 3: motion of, for example, liquid in a cup. So like

298
00:14:32,680 --> 00:14:34,460
Speaker 3: both of you guys have coffee cups here on the table.

299
00:14:34,900 --> 00:14:36,500
Speaker 3: And if you like shake that coffee cup up, you

300
00:14:36,520 --> 00:14:39,260
Speaker 3: know the initial velocity, you know what's inside of the cup.

301
00:14:39,920 --> 00:14:43,560
Speaker 3: Then the Navier-Stokes equations are trying to predict like where

302
00:14:43,600 --> 00:14:47,080
Speaker 3: the fluid moves over time, right? Like where is the velocity, vorticity,

303
00:14:47,120 --> 00:14:51,470
Speaker 3: all these different things. Now, the Navier-Stokes equations are really complicated.

304
00:14:51,500 --> 00:14:53,380
Speaker 3: I mean, when you look at your latte, when you

305
00:14:53,400 --> 00:14:55,270
Speaker 3: stir it in the morning and you see the milk

306
00:14:55,330 --> 00:14:57,910
Speaker 3: making these cool shapes and so on, there's all kinds

307
00:14:57,950 --> 00:15:01,800
Speaker 3: of complex behavior that can happen. And so the Navier-Stokes equations,

308
00:15:02,100 --> 00:15:04,100
Speaker 3: although they're easy to write down, they're very hard to

309
00:15:04,120 --> 00:15:07,020
Speaker 3: say a whole lot about. And so there was this

310
00:15:07,060 --> 00:15:09,300
Speaker 3: big open math problem, which I think is what you

311
00:15:09,320 --> 00:15:10,780
Speaker 3: guys are referring to because it's been in the news,

312
00:15:11,380 --> 00:15:14,570
Speaker 3: which involves existence and uniqueness of solutions. So the basic

313
00:15:14,850 --> 00:15:18,770
Speaker 3: idea here, I think actually my undergrad differential equation professor

314
00:15:18,810 --> 00:15:20,370
Speaker 3: years ago explained it to me in kind of a

315
00:15:20,390 --> 00:15:22,420
Speaker 3: funny way. He sort of said, you know, all you

316
00:15:22,440 --> 00:15:24,000
Speaker 3: got to do to make a million dollars is prove

317
00:15:24,020 --> 00:15:27,580
Speaker 3: that water doesn't explode. And that's roughly right. I mean,

318
00:15:27,600 --> 00:15:30,020
Speaker 3: with a lot of asterisks. Essentially, what we didn't know

319
00:15:30,040 --> 00:15:33,750
Speaker 3: is if you know this initial configuration of your fluid

320
00:15:33,870 --> 00:15:35,930
Speaker 3: and you let it kind of go under its dynamics

321
00:15:35,990 --> 00:15:39,730
Speaker 3: over time, then whether it exists for all time and

322
00:15:39,770 --> 00:15:43,710
Speaker 3: doesn't have some really crazy turbulent behavior, or if, for example,

323
00:15:43,750 --> 00:15:45,810
Speaker 3: maybe it blows up or it develops a singularity or

324
00:15:45,850 --> 00:15:47,550
Speaker 3: something like that. So this was kind of a yes

325
00:15:47,580 --> 00:15:50,700
Speaker 3: or no question, like prove or disprove. And I guess

326
00:15:50,740 --> 00:15:53,780
Speaker 3: the recent news was actually a counterexample. So in other words,

327
00:15:53,840 --> 00:15:57,980
Speaker 3: it showed in some finite amount of time that with

328
00:15:58,040 --> 00:16:00,800
Speaker 3: an asterisk, which is with a very crazy spoon, if

329
00:16:00,820 --> 00:16:02,980
Speaker 3: you stir your fluid just the right way, you really

330
00:16:03,020 --> 00:16:07,400
Speaker 3: can create this totally singular and frankly, non-physical behavior. Interesting.

331
00:16:07,850 --> 00:16:12,530
Speaker 1: Proving in math is like one of these things. Like

332
00:16:12,570 --> 00:16:15,730
Speaker 1: I think I, the formal proof that two plus two

333
00:16:15,810 --> 00:16:19,260
Speaker 1: equals four, it's like a nine step thing, isn't it?

334
00:16:19,330 --> 00:16:19,490
Speaker 3: Right?

335
00:16:19,530 --> 00:16:22,270
Speaker 1: Like if you are like, sometimes eight, sometimes eight, but like,

336
00:16:23,460 --> 00:16:26,380
Speaker 1: Can you explain what a proof is?

337
00:16:26,920 --> 00:16:27,180
Speaker 3: Sure.

338
00:16:27,240 --> 00:16:29,440
Speaker 1: I know 2 plus 2 equals 4. It seems obvious.

339
00:16:30,040 --> 00:16:35,780
Speaker 1: But explain what is a proof such that to say

340
00:16:35,820 --> 00:16:38,400
Speaker 1: that to prove that 2 plus 2 equals 4 is

341
00:16:38,440 --> 00:16:40,040
Speaker 1: something that requires eight steps.

342
00:16:40,340 --> 00:16:43,730
Speaker 3: Right. This idea of proof writing, I think, is actually

343
00:16:43,750 --> 00:16:45,280
Speaker 3: you can think of it as almost the result of

344
00:16:45,300 --> 00:16:47,460
Speaker 3: an identity crisis among mathematicians maybe 100, 200 years ago.

345
00:16:49,080 --> 00:16:50,500
Speaker 3: where it used to be a lot of math was

346
00:16:50,620 --> 00:16:53,320
Speaker 3: drawing pictures and kind of working in the same fashion

347
00:16:53,340 --> 00:16:56,030
Speaker 3: that you're talking about, kind of intuitively. But at some point,

348
00:16:56,070 --> 00:16:57,770
Speaker 3: people realized that they wanted to make sure that the

349
00:16:57,790 --> 00:17:01,580
Speaker 3: field had really strong foundations. And so mathematicians will agree

350
00:17:01,820 --> 00:17:04,260
Speaker 3: on a certain set of maybe logical principles or rules,

351
00:17:05,140 --> 00:17:07,100
Speaker 3: I would say that's more the domain of philosophy, precisely

352
00:17:07,119 --> 00:17:10,260
Speaker 3: what those should be. And then the idea of proofreading

353
00:17:10,359 --> 00:17:13,070
Speaker 3: is to build up from these kind of basically, we

354
00:17:13,109 --> 00:17:16,189
Speaker 3: agree that these five facts are true, to a much

355
00:17:16,230 --> 00:17:19,170
Speaker 3: more complicated fact by combining them in different and creative ways.

356
00:17:19,690 --> 00:17:23,359
Speaker 3: And so theoretical mathematics oftentimes is all about, I don't

357
00:17:23,380 --> 00:17:25,260
Speaker 3: want to just say mixing and matching different facts, but

358
00:17:25,280 --> 00:17:29,180
Speaker 3: coming up with smart ways to justify mathematical ideas built

359
00:17:29,220 --> 00:17:31,600
Speaker 3: on very simple principles and steps from logic.

360
00:17:32,230 --> 00:17:35,550
Speaker 2: So when it comes to AI currently, is the role

361
00:17:35,670 --> 00:17:40,490
Speaker 2: more in, I guess, proof checking versus proof generation or

362
00:17:40,570 --> 00:17:43,619
Speaker 2: idea generation? Like what are the models actually doing.

363
00:17:43,380 --> 00:17:45,580
Speaker 3: At the moment? Well, they're sort of doing it all

364
00:17:45,600 --> 00:17:48,760
Speaker 3: and they're sort of doing nothing. So, right. The current

365
00:17:48,840 --> 00:17:51,520
Speaker 3: AI models in terms of the types of mathematics they're doing,

366
00:17:52,260 --> 00:17:53,740
Speaker 3: I don't think it's quite right to say that it's

367
00:17:53,760 --> 00:17:56,360
Speaker 3: just proof checking. And in fact, actually, they're quite bad

368
00:17:56,400 --> 00:17:59,260
Speaker 3: at proof checking. And one of the revolutions in, I

369
00:17:59,260 --> 00:18:01,919
Speaker 3: would say, the last year or so is the realization

370
00:18:02,570 --> 00:18:04,880
Speaker 3: that actually what you have to do is to provide

371
00:18:04,950 --> 00:18:07,439
Speaker 3: your model, like your large language model, with a second

372
00:18:07,500 --> 00:18:12,000
Speaker 3: module that can check the AI's proof. And so, for example,

373
00:18:12,020 --> 00:18:14,800
Speaker 3: there's a programming language called Lean, which is basically just

374
00:18:14,859 --> 00:18:18,619
Speaker 3: a sort of codified version of these mathematical rules. And

375
00:18:18,640 --> 00:18:20,260
Speaker 3: these days what people will do is they'll put in

376
00:18:20,280 --> 00:18:22,070
Speaker 3: their conjecture, so like the thing that they want to

377
00:18:22,160 --> 00:18:24,629
Speaker 3: prove or disprove, and then ask the AI system to

378
00:18:24,670 --> 00:18:26,790
Speaker 3: generate a proof. But then they'll say rather than just

379
00:18:26,830 --> 00:18:29,670
Speaker 3: writing it in human-readable language, which is the substrate that

380
00:18:29,910 --> 00:18:32,760
Speaker 3: mathematicians use to communicate with each other, maybe in addition

381
00:18:32,780 --> 00:18:35,460
Speaker 3: to that, they'll actually produce this lean code so that

382
00:18:36,220 --> 00:18:38,100
Speaker 3: there's a separate piece of software out there that can

383
00:18:38,140 --> 00:18:41,359
Speaker 3: check that this proof truly is correct. So actually the

384
00:18:41,400 --> 00:18:44,710
Speaker 3: correctness part of it is usually deferred to another piece

385
00:18:44,730 --> 00:18:47,750
Speaker 3: of software, which isn't AI, but somehow because of the

386
00:18:47,790 --> 00:18:51,070
Speaker 3: interaction between the AI tool and this proof checker, that's

387
00:18:51,090 --> 00:18:52,970
Speaker 3: what's created a lot of progress that we can believe

388
00:18:52,990 --> 00:18:54,619
Speaker 3: in a little bit more than we did in the past,

389
00:18:54,690 --> 00:18:55,540
Speaker 3: if that makes sense.

390
00:18:56,460 --> 00:18:59,210
Speaker 1: I have heard of, only because, you know, I've been

391
00:18:59,680 --> 00:19:02,139
Speaker 1: reading the news and I've heard of this thing called Lean.

392
00:19:02,320 --> 00:19:02,500
Speaker 3: Yes.

393
00:19:02,840 --> 00:19:06,580
Speaker 1: And it sort of seems extraordinary to me. Can you

394
00:19:06,600 --> 00:19:08,979
Speaker 1: explain it a little bit? It seems extraordinary to me

395
00:19:09,020 --> 00:19:11,810
Speaker 1: that there is this, I don't know, is it software

396
00:19:11,850 --> 00:19:17,929
Speaker 1: or this framework such that these extraordinarily complex problems or solutions,

397
00:19:17,990 --> 00:19:20,790
Speaker 1: I guess is the answer, can be dropped into this

398
00:19:21,609 --> 00:19:24,940
Speaker 1: and it's capable of like, Yes, this is available.

399
00:19:24,960 --> 00:19:26,980
Speaker 2: But it's also kind of crazy that you need basically

400
00:19:27,020 --> 00:19:28,980
Speaker 2: a translator for it, right?

401
00:19:29,240 --> 00:19:29,540
Speaker 3: Yeah.

402
00:19:29,640 --> 00:19:32,500
Speaker 1: Can you explain a little bit like how Lean works, perhaps?

403
00:19:32,780 --> 00:19:34,359
Speaker 3: I can explain a little bit. To be clear, I'm

404
00:19:34,380 --> 00:19:38,510
Speaker 3: not an expert in this particular corner. But essentially, as

405
00:19:38,550 --> 00:19:40,670
Speaker 3: we all know, things that come out of AI are unreliable.

406
00:19:40,970 --> 00:19:41,230
Speaker 2: Yeah.

407
00:19:41,410 --> 00:19:44,130
Speaker 3: All of your listeners should know this. We all know this.

408
00:19:44,510 --> 00:19:46,689
Speaker 3: And that goes for mathematics as well. And math we

409
00:19:46,730 --> 00:19:49,070
Speaker 3: think of as this area that is extremely reliable, right?

410
00:19:49,090 --> 00:19:51,580
Speaker 3: Everything should be true or false. Now, there are gray areas,

411
00:19:51,660 --> 00:19:54,460
Speaker 3: but certainly we try to avoid that. So this programming

412
00:19:54,500 --> 00:19:56,680
Speaker 3: language lean, what it does is it has a few

413
00:19:56,760 --> 00:19:59,679
Speaker 3: simple maybe axioms that are built in. So math things

414
00:19:59,740 --> 00:20:02,369
Speaker 3: that we sort of agree are the starting points for

415
00:20:02,420 --> 00:20:05,530
Speaker 3: writing proofs, like the steps of basic logic. And then

416
00:20:05,570 --> 00:20:08,550
Speaker 3: what happens is that you write a theorem in this

417
00:20:08,670 --> 00:20:10,770
Speaker 3: lean language. So rather than in English, you write it

418
00:20:10,930 --> 00:20:13,770
Speaker 3: in this programming language. And then it's kind of cleverly

419
00:20:13,810 --> 00:20:17,550
Speaker 3: designed so that once the software checks that that theorem

420
00:20:17,609 --> 00:20:19,649
Speaker 3: is true, it can get added to the library of

421
00:20:19,690 --> 00:20:22,140
Speaker 3: sort of true facts, and then you can build up.

422
00:20:22,160 --> 00:20:24,080
Speaker 3: So you make this more and more complex edifice out

423
00:20:24,100 --> 00:20:26,280
Speaker 3: of these simple steps. And it's been really impressive what

424
00:20:26,320 --> 00:20:28,620
Speaker 3: that can do. I mean, some of the modern proofs

425
00:20:28,640 --> 00:20:31,220
Speaker 3: that you're seeing, especially the AI-generated ones, are like hundreds

426
00:20:31,240 --> 00:20:33,260
Speaker 3: of thousands of lines of lean code. I don't think

427
00:20:33,300 --> 00:20:35,340
Speaker 3: a human could check it. It also, by the way,

428
00:20:35,380 --> 00:20:38,859
Speaker 3: means that we're trusting that someone wrote That software, frankly,

429
00:20:39,619 --> 00:20:42,260
Speaker 3: which is a philosophical question. But, you know, if you

430
00:20:42,280 --> 00:20:44,419
Speaker 3: believe that Lean is good at checking proofs, then it

431
00:20:44,480 --> 00:20:46,720
Speaker 3: provides a lot of certainty that the things that the

432
00:20:46,770 --> 00:20:48,310
Speaker 3: AIs are writing down are actually correct.

433
00:20:49,050 --> 00:20:53,070
Speaker 2: Devil's advocate question. But why is it important that we

434
00:20:53,109 --> 00:20:57,770
Speaker 2: need to understand the actual problem and the proof? Because

435
00:20:57,910 --> 00:21:01,330
Speaker 2: if I'm an engineer and I don't know, some model

436
00:21:01,450 --> 00:21:04,970
Speaker 2: somewhere comes up with this really new, interesting mathematical theory

437
00:21:05,070 --> 00:21:07,050
Speaker 2: and I can apply it to my day to day work.

438
00:21:07,230 --> 00:21:07,410
Speaker 3: Yes.

439
00:21:08,109 --> 00:21:10,760
Speaker 2: Is that not enough that like I can use it?

440
00:21:10,950 --> 00:21:13,680
Speaker 2: It seems to work for the most part. Maybe it's

441
00:21:13,700 --> 00:21:17,700
Speaker 2: been rigorously tested in a physical environment or whatever. Why

442
00:21:17,720 --> 00:21:19,760
Speaker 2: do I need to understand everything that's gone into it?

443
00:21:19,980 --> 00:21:23,600
Speaker 3: Oh, that's a fantastic question. So let's see. My research

444
00:21:23,640 --> 00:21:26,020
Speaker 3: group works a lot on simulation algorithms. So like I

445
00:21:26,060 --> 00:21:28,700
Speaker 3: have some physical system and I develop a technique for

446
00:21:28,840 --> 00:21:31,240
Speaker 3: predicting what it will look like in the future. And

447
00:21:31,359 --> 00:21:33,740
Speaker 3: we spend a lot of time writing theoretical results to

448
00:21:33,780 --> 00:21:36,179
Speaker 3: verify that our algorithms are correct. They behave the way

449
00:21:36,200 --> 00:21:39,330
Speaker 3: we expect and so on. And that's absolutely critical because

450
00:21:39,350 --> 00:21:42,490
Speaker 3: without that, you really can't trust what your software is producing.

451
00:21:42,930 --> 00:21:45,149
Speaker 3: So for example, if I'm trying to say yes or no,

452
00:21:45,190 --> 00:21:47,750
Speaker 3: is this building going to collapse during an earthquake?

453
00:21:47,930 --> 00:21:48,550
Speaker 2: Seems important.

454
00:21:48,630 --> 00:21:51,490
Speaker 3: That's right. And all of that is built on mathematical models,

455
00:21:51,530 --> 00:21:54,550
Speaker 3: techniques like finite element modeling and so on. They're not

456
00:21:54,570 --> 00:21:57,690
Speaker 3: too exciting outside of my universe, but you're all trusting

457
00:21:57,730 --> 00:22:01,100
Speaker 3: these things to do really important engineering calculations and things

458
00:22:01,140 --> 00:22:04,280
Speaker 3: that sure, like one way to justify your software is

459
00:22:04,300 --> 00:22:06,760
Speaker 3: that your building hasn't collapsed yet. but that's not a

460
00:22:06,780 --> 00:22:09,180
Speaker 3: whole lot of, you know, I hope I can trust

461
00:22:09,220 --> 00:22:10,140
Speaker 3: a little bit better than that.

462
00:22:10,260 --> 00:22:13,159
Speaker 1: You know, my dad was a physics professor and I

463
00:22:13,200 --> 00:22:20,369
Speaker 1: have memories of watching a video of the famous Tacoma

464
00:22:20,410 --> 00:22:25,730
Speaker 1: Narrows bridge collapse. You know, that video, you know, like

465
00:22:25,750 --> 00:22:28,389
Speaker 1: that was like for him, like this, like, you know, I,

466
00:22:28,510 --> 00:22:31,770
Speaker 1: that was, I've watched it several times in my household.

467
00:22:31,869 --> 00:22:33,890
Speaker 3: Our engineering students have to take a lot of ethics

468
00:22:33,930 --> 00:22:36,290
Speaker 3: courses that are pretty fraught that way.

469
00:22:36,830 --> 00:22:39,990
Speaker 1: Oh, interesting. So why don't you give us, I mean,

470
00:22:40,010 --> 00:22:42,390
Speaker 1: there's a couple avenues I want to explore, including just

471
00:22:42,410 --> 00:22:47,080
Speaker 1: the process of teaching and how it's different in 2026

472
00:22:47,080 --> 00:22:52,240
Speaker 1: versus 2016, given the new technology. But maybe give us

473
00:22:52,380 --> 00:22:55,940
Speaker 1: a lay of the land for the math community. Because

474
00:22:55,980 --> 00:22:57,580
Speaker 1: I know there are some people in despair.

475
00:22:59,170 --> 00:22:59,710
Speaker 3: I believe.

476
00:23:00,090 --> 00:23:03,360
Speaker 1: We're very anxious about what this means for the future

477
00:23:03,380 --> 00:23:08,020
Speaker 1: of math as an intellectual discipline, etc. And then others

478
00:23:08,080 --> 00:23:11,760
Speaker 1: seem to be very enthusiastic about the opportunity of human

479
00:23:12,700 --> 00:23:18,190
Speaker 1: and AI exploring new ground. But what are the conversations

480
00:23:18,250 --> 00:23:21,330
Speaker 1: that you're in these days with fellow mathematicians?

481
00:23:21,590 --> 00:23:24,490
Speaker 3: Oh my gosh, things are changing by the day. And really,

482
00:23:24,510 --> 00:23:27,480
Speaker 3: it's a Huge collection of individuals. Every single person has

483
00:23:27,540 --> 00:23:29,780
Speaker 3: their own opinion here. So I can tell you my

484
00:23:29,820 --> 00:23:31,880
Speaker 3: view of the world. And if you talk to somebody else,

485
00:23:31,940 --> 00:23:34,480
Speaker 3: I'm sure they'll have a very different one. Math is

486
00:23:34,530 --> 00:23:36,550
Speaker 3: an academic discipline. It also shows up in a lot

487
00:23:36,570 --> 00:23:39,730
Speaker 3: of applications, as we mentioned. But every aspect of the

488
00:23:39,770 --> 00:23:43,670
Speaker 3: work is changing from how people produce mathematics and choose

489
00:23:43,710 --> 00:23:47,490
Speaker 3: problems to work on to how mathematics is sort of

490
00:23:48,190 --> 00:23:51,040
Speaker 3: measured in terms of productivity, right? When we talk about

491
00:23:51,140 --> 00:23:53,760
Speaker 3: hiring faculty, for example, like what are the standards to

492
00:23:54,160 --> 00:23:56,480
Speaker 3: evaluate a person's body of work and what did the

493
00:23:56,540 --> 00:23:59,800
Speaker 3: human contribute, I think are really critical questions to like,

494
00:23:59,840 --> 00:24:01,879
Speaker 3: why do this exercise at all? Maybe leave this to

495
00:24:01,920 --> 00:24:06,170
Speaker 3: a computer. There's all kinds of identity crises that are happening. Now,

496
00:24:06,310 --> 00:24:09,630
Speaker 3: among mathematicians, there are people that are maybe theory builders

497
00:24:09,710 --> 00:24:11,629
Speaker 3: and there are people that are like theorem provers. The

498
00:24:11,670 --> 00:24:13,730
Speaker 3: theorem provers, I think, are especially under threat at the

499
00:24:13,770 --> 00:24:16,469
Speaker 3: current moment where like maybe you've been working on one

500
00:24:16,510 --> 00:24:19,129
Speaker 3: problem and it's well-defined and you've known about it for,

501
00:24:19,210 --> 00:24:22,350
Speaker 3: you know, 50 years and it's this crowning question that

502
00:24:22,369 --> 00:24:24,810
Speaker 3: you want to answer about the universe. And these AI

503
00:24:24,850 --> 00:24:26,409
Speaker 3: tools are sort of at least good at sort of

504
00:24:26,690 --> 00:24:28,770
Speaker 3: cherry picking the ones that are somehow close to being

505
00:24:28,850 --> 00:24:32,350
Speaker 3: proved and taking that last mile in some cases. And

506
00:24:32,650 --> 00:24:34,810
Speaker 3: other people disagree with that characterization.

507
00:24:34,950 --> 00:24:37,070
Speaker 1: What do you mean by when you say they disagree

508
00:24:37,109 --> 00:24:37,409
Speaker 1: with it?

509
00:24:37,990 --> 00:24:41,310
Speaker 3: Like exactly how interesting, for example, a mathematical proof is,

510
00:24:41,330 --> 00:24:43,600
Speaker 3: is a very human judgment, as is what problems you

511
00:24:43,630 --> 00:24:46,359
Speaker 3: should or should not actually go about working on in

512
00:24:46,400 --> 00:24:48,590
Speaker 3: the real world. So There are a lot of true facts.

513
00:24:48,650 --> 00:24:50,430
Speaker 3: It doesn't mean we have to write them all down, essentially.

514
00:24:50,450 --> 00:24:53,920
Speaker 3: And because of that, this is definitely a human exercise, right?

515
00:24:54,080 --> 00:24:57,460
Speaker 3: To a large extent, mathematical research is about humans communicating

516
00:24:57,500 --> 00:24:59,900
Speaker 3: to other humans, right? It's not just about verifying true

517
00:24:59,920 --> 00:25:03,100
Speaker 3: things about the universe, but bringing insight and helping people

518
00:25:03,160 --> 00:25:06,110
Speaker 3: understand systems and so on. So I think you mentioned

519
00:25:06,150 --> 00:25:08,770
Speaker 3: an identity crisis. I think what's happened is, and this

520
00:25:08,830 --> 00:25:11,630
Speaker 3: is reflected, there's a really nice write-up by Terence Tao,

521
00:25:11,650 --> 00:25:14,130
Speaker 3: who's a famous mathematician, that kind of talks about how

522
00:25:14,170 --> 00:25:17,060
Speaker 3: there were a lot of different sort of signs of

523
00:25:17,080 --> 00:25:20,760
Speaker 3: mathematical progress that used to be correlated with one another, right?

524
00:25:20,780 --> 00:25:24,129
Speaker 3: So for example, if I wanted to judge whether a

525
00:25:24,150 --> 00:25:27,050
Speaker 3: mathematician has quality research or output, maybe I look at

526
00:25:27,550 --> 00:25:30,150
Speaker 3: where they're publishing, the number of things that they're writing,

527
00:25:30,190 --> 00:25:32,810
Speaker 3: the interest of their result, whether they're earning these prizes

528
00:25:32,869 --> 00:25:35,050
Speaker 3: and so on. And now these things are a bit

529
00:25:35,090 --> 00:25:37,620
Speaker 3: at odds. So for example, the AI companies have figured out,

530
00:25:37,670 --> 00:25:41,020
Speaker 3: as you guys pointed out, that proving some big theoretical

531
00:25:41,060 --> 00:25:45,170
Speaker 3: result maybe is an interesting press release, right? And now

532
00:25:45,230 --> 00:25:46,450
Speaker 3: there's an economic component.

533
00:25:46,470 --> 00:25:47,990
Speaker 1: Knowing how to do the right prompt, right?

534
00:25:48,270 --> 00:25:51,390
Speaker 3: That's right. And then there's a big question, which is, okay, so,

535
00:25:51,609 --> 00:25:53,250
Speaker 3: you know, at least in the sort of cartoon version

536
00:25:53,270 --> 00:25:55,110
Speaker 3: of a mathematician, you sit in a room, you think

537
00:25:55,170 --> 00:25:57,310
Speaker 3: about a really hard problem, you write down a proof

538
00:25:57,350 --> 00:26:00,890
Speaker 3: that's subtle and challenging. Now, as you say, maybe you're

539
00:26:00,910 --> 00:26:02,850
Speaker 3: the guy that wrote down the prompt. If the prompt,

540
00:26:02,869 --> 00:26:06,340
Speaker 3: for example, were prove or disprove Navier-Stokes equation, did that

541
00:26:06,380 --> 00:26:08,060
Speaker 3: guy really bring a whole lot of insight to that

542
00:26:08,080 --> 00:26:10,420
Speaker 3: problem is an interesting question and one that I think

543
00:26:10,820 --> 00:26:12,060
Speaker 3: we're really grappling with today.

544
00:26:28,190 --> 00:26:30,770
Speaker 2: I want to ask more questions about credit, but just

545
00:26:31,210 --> 00:26:35,780
Speaker 2: on the models themselves, does sycophancy exist when it comes

546
00:26:35,859 --> 00:26:39,580
Speaker 2: to math prompts the same way it exists for other,

547
00:26:39,660 --> 00:26:42,960
Speaker 2: I guess, language-based queries? Does the model go like, oh,

548
00:26:43,000 --> 00:26:46,740
Speaker 2: what an interesting problem. You're on the right track.

549
00:26:47,140 --> 00:26:50,060
Speaker 3: You're on the right track. Judging from my email box,

550
00:26:50,140 --> 00:26:52,159
Speaker 3: I think the answer is yes. I'll tell you a

551
00:26:52,160 --> 00:26:55,440
Speaker 3: funny anecdote. Right, so all the time I get emails

552
00:26:55,480 --> 00:26:57,920
Speaker 3: from people that are looking for an expert to maybe verify,

553
00:26:57,940 --> 00:27:00,000
Speaker 3: you know. Oh, yeah. And what I've noticed lately is

554
00:27:00,040 --> 00:27:01,980
Speaker 3: I get a lot of emails from outside folks that

555
00:27:02,020 --> 00:27:04,520
Speaker 3: sort of, they've been playing with their AI system, they've

556
00:27:04,540 --> 00:27:06,320
Speaker 3: proven a new result, and the AI told them it

557
00:27:06,359 --> 00:27:08,960
Speaker 3: was really important and they need an MIT professor to validate,

558
00:27:09,180 --> 00:27:11,290
Speaker 3: which is absolutely fascinating. In fact, for fun, I played

559
00:27:11,330 --> 00:27:13,169
Speaker 3: with some of these tools and put in a few

560
00:27:13,210 --> 00:27:16,570
Speaker 3: open problems in my own domain. Interestingly, they did not

561
00:27:16,590 --> 00:27:19,370
Speaker 3: get solved, so apparently my corner of math is safe

562
00:27:19,410 --> 00:27:22,740
Speaker 3: at the moment. But... the first thing that it did,

563
00:27:22,840 --> 00:27:26,480
Speaker 3: it proved some, from my perspective, pretty superficial thing. And

564
00:27:26,500 --> 00:27:28,100
Speaker 3: then it had a little footnote on the bottom being like,

565
00:27:28,119 --> 00:27:29,359
Speaker 3: as far as I can tell, this doesn't appear in

566
00:27:29,380 --> 00:27:31,560
Speaker 3: the literature. Like, would you like me to convert this

567
00:27:31,600 --> 00:27:34,740
Speaker 3: into the template for a journal? It was an interesting,

568
00:27:34,780 --> 00:27:36,240
Speaker 3: I think it explains a lot what's going on in

569
00:27:36,260 --> 00:27:36,640
Speaker 3: my email.

570
00:27:36,700 --> 00:27:39,660
Speaker 2: Wait, so you can imagine a future where I guess

571
00:27:39,869 --> 00:27:44,020
Speaker 2: they're like, you're sort of overwhelmed with new proofs and

572
00:27:44,060 --> 00:27:48,280
Speaker 2: the bottleneck becomes the ability to actually verify them with humans.

573
00:27:48,420 --> 00:27:50,560
Speaker 3: That is a problem right now. And in fact, it's

574
00:27:50,600 --> 00:27:53,740
Speaker 3: not just in mathematics, it is across academia. So to

575
00:27:53,760 --> 00:27:56,520
Speaker 3: give you an example, actually, interestingly, I would say the

576
00:27:56,580 --> 00:27:59,359
Speaker 3: academic field that's been affected most by this phenomenon you're

577
00:27:59,380 --> 00:28:03,139
Speaker 3: talking about is actually AI and machine learning itself. So

578
00:28:03,500 --> 00:28:06,900
Speaker 3: in that particular corner of academia, research is driven by conferences.

579
00:28:06,960 --> 00:28:09,450
Speaker 3: So you write your research paper, and there's a sort

580
00:28:09,470 --> 00:28:12,210
Speaker 3: of annual cycle where you submit and it undergoes peer review,

581
00:28:12,510 --> 00:28:14,389
Speaker 3: and the whole community spends like a month or two

582
00:28:14,890 --> 00:28:16,590
Speaker 3: reviewing each other's papers, sort of like, you know, you

583
00:28:17,090 --> 00:28:19,190
Speaker 3: pass your paper to the left in class and grade.

584
00:28:19,210 --> 00:28:22,010
Speaker 3: It's sort of a very complicated version of that. If

585
00:28:22,050 --> 00:28:24,250
Speaker 3: you look at the number of submissions to the machine

586
00:28:24,270 --> 00:28:27,890
Speaker 3: learning conferences, it is wild. So in the last 10 years,

587
00:28:27,930 --> 00:28:29,590
Speaker 3: I think about 10 years ago, it was maybe one

588
00:28:29,609 --> 00:28:33,990
Speaker 3: or 2,000, the deadline for ICLR, the International Conference on

589
00:28:34,030 --> 00:28:37,090
Speaker 3: Learning Representations, one of the big ones, I think it

590
00:28:37,150 --> 00:28:39,930
Speaker 3: had 60,000 submissions. And if you think about it, it

591
00:28:39,990 --> 00:28:42,620
Speaker 3: completely breaks the academic system the way that we're used

592
00:28:42,640 --> 00:28:45,180
Speaker 3: to thinking about it. Yeah. We don't have bandwidth to,

593
00:28:46,440 --> 00:28:48,320
Speaker 3: never mind just check all of these proofs, even open

594
00:28:48,340 --> 00:28:51,240
Speaker 3: up the PDF. Yeah. And so you're then relying on

595
00:28:51,840 --> 00:28:56,050
Speaker 3: extremely superficial review process. There's many anecdotes of like high

596
00:28:56,090 --> 00:28:58,770
Speaker 3: school students that have somehow gotten involved in peer review

597
00:28:58,850 --> 00:29:01,209
Speaker 3: because they had a project once and the system decided

598
00:29:01,250 --> 00:29:04,390
Speaker 3: they were in okay. You know, which is frustrating to

599
00:29:04,410 --> 00:29:08,330
Speaker 3: build an academic career. Yeah. Essentially, some of the stamps

600
00:29:08,370 --> 00:29:10,330
Speaker 3: of quality that we're used to in the field, you

601
00:29:10,350 --> 00:29:12,630
Speaker 3: really can't rely on for exactly the reason you identified.

602
00:29:13,070 --> 00:29:14,490
Speaker 3: The way that Terence Tao put it is that it

603
00:29:14,570 --> 00:29:16,770
Speaker 3: used to be we were in this era of proof scarcity,

604
00:29:16,790 --> 00:29:20,130
Speaker 3: that mathematicians, it was really hard. And it was this

605
00:29:20,190 --> 00:29:23,030
Speaker 3: very bespoke object that took a lot of craftwork and

606
00:29:23,070 --> 00:29:26,890
Speaker 3: training to produce. Now we're in this era of proof abundance.

607
00:29:26,930 --> 00:29:28,560
Speaker 3: So now we have to kind of think about different

608
00:29:28,600 --> 00:29:31,800
Speaker 3: ways to do our job and to do mathematics in

609
00:29:31,840 --> 00:29:33,020
Speaker 3: a productive and interesting way.

610
00:29:33,460 --> 00:29:36,750
Speaker 1: Yeah, I think a lot of fields are dealing with

611
00:29:38,090 --> 00:29:42,900
Speaker 1: very similar issues. in which we used to have like certain,

612
00:29:42,920 --> 00:29:45,780
Speaker 1: I mean, even take writing, for example. We used to

613
00:29:45,800 --> 00:29:50,270
Speaker 1: have these certain heuristics that we, maybe like shortcuts that

614
00:29:50,310 --> 00:29:53,750
Speaker 1: we use. It's like, okay, if this person can like

615
00:29:54,590 --> 00:29:58,340
Speaker 1: construct a series of, like, well-written sentences.

616
00:29:58,800 --> 00:29:59,240
Speaker 3: We might.

617
00:29:59,260 --> 00:30:00,400
Speaker 2: And no spelling errors.

618
00:30:00,480 --> 00:30:03,959
Speaker 1: Yeah. Like, how I've... I never make spelling errors. We

619
00:30:04,000 --> 00:30:06,480
Speaker 1: might start... You know, we start from this... Okay, this

620
00:30:06,640 --> 00:30:09,320
Speaker 1: person might be worth, like, reading. They might have something...

621
00:30:09,120 --> 00:30:11,310
Speaker 1: You know, we all have to, like, operate with these

622
00:30:11,770 --> 00:30:12,770
Speaker 1: cheats and shortcuts.

623
00:30:12,930 --> 00:30:13,190
Speaker 3: That's right.

624
00:30:13,210 --> 00:30:16,430
Speaker 1: And now, like, obviously, producing text with no spelling errors

625
00:30:16,490 --> 00:30:19,710
Speaker 1: or anything is available to literally everyone. So it's like

626
00:30:19,730 --> 00:30:21,490
Speaker 1: we have to go a step further, et cetera.

627
00:30:21,510 --> 00:30:22,150
Speaker 3: I think that's right.

628
00:30:22,190 --> 00:30:24,760
Speaker 1: And so, like, You know, all these like heuristics, you know,

629
00:30:24,800 --> 00:30:28,520
Speaker 1: speaking of like everyone now, like thinking they can come

630
00:30:28,560 --> 00:30:32,800
Speaker 1: up with a proof. Do you remember last year, Travis Kalanick,

631
00:30:32,960 --> 00:30:35,960
Speaker 1: he was the Uber founder. He was on the All

632
00:30:36,020 --> 00:30:38,840
Speaker 1: In podcast and he was talking about how he was

633
00:30:39,140 --> 00:30:41,440
Speaker 1: he was doing vibe physics with Brock.

634
00:30:41,620 --> 00:30:41,900
Speaker 3: Oh, boy.

635
00:30:41,980 --> 00:30:45,010
Speaker 1: And he felt that he was, quote. Getting pretty damn

636
00:30:45,070 --> 00:30:50,060
Speaker 1: close to some interesting breakthroughs. Vibe physics was.

637
00:30:49,920 --> 00:30:51,680
Speaker 2: The crazy thing is he could just hire a bunch

638
00:30:51,700 --> 00:30:54,200
Speaker 2: of physicists to like verify the research and then you'd

639
00:30:54,220 --> 00:30:55,680
Speaker 2: have a real sycophancy problem.

640
00:30:56,260 --> 00:30:58,580
Speaker 3: And I think beyond that, like we're currently, so I

641
00:30:58,620 --> 00:31:01,540
Speaker 3: work in education. Yeah. One of the things that we

642
00:31:01,560 --> 00:31:05,260
Speaker 3: really rely on is somehow the friction in the creative

643
00:31:05,280 --> 00:31:07,760
Speaker 3: process and the technical process. I mean, you mentioned writing

644
00:31:07,800 --> 00:31:10,350
Speaker 3: is a fantastic example. How do you learn how to write? Well,

645
00:31:10,650 --> 00:31:13,050
Speaker 3: you do a lot of writing. And now it's very

646
00:31:13,290 --> 00:31:16,510
Speaker 3: difficult to convince our students to sit down and write

647
00:31:16,550 --> 00:31:18,510
Speaker 3: an essay because they have these AI tools that can

648
00:31:18,530 --> 00:31:22,010
Speaker 3: do it. But then writing a five-paragraph essay was never

649
00:31:22,310 --> 00:31:24,530
Speaker 3: the goal. It was to teach you to do other stuff.

650
00:31:24,840 --> 00:31:28,020
Speaker 3: Mathematics is similarly experiencing a very similar challenge.

651
00:31:28,170 --> 00:31:32,560
Speaker 1: So let's talk about the field of education itself.

652
00:31:32,820 --> 00:31:33,060
Speaker 3: I mean.

653
00:31:34,210 --> 00:31:35,990
Speaker 1: Are the students very anxious right now?

654
00:31:36,030 --> 00:31:36,950
Speaker 3: Absolutely.

655
00:31:36,970 --> 00:31:41,290
Speaker 1: Let's start there. What's the vibe among students, especially maybe

656
00:31:41,330 --> 00:31:45,540
Speaker 1: grad students who started their career before the idea of

657
00:31:45,670 --> 00:31:48,560
Speaker 1: AI-generated resources? The thing is suddenly they're in grad school.

658
00:31:49,120 --> 00:31:51,580
Speaker 1: What's going through their minds these days?

659
00:31:51,660 --> 00:31:55,160
Speaker 3: Yeah, the landscape of both graduate and undergraduate education is

660
00:31:55,200 --> 00:31:58,600
Speaker 3: changing a lot. Now, in the graduate school world, for me,

661
00:31:58,640 --> 00:32:01,850
Speaker 3: that would mean PhD. In other worlds, that's maybe master's

662
00:32:01,870 --> 00:32:05,720
Speaker 3: degrees and so on. They're It sort of divides into

663
00:32:05,840 --> 00:32:08,980
Speaker 3: maybe professional education, preparing you for a job, and then

664
00:32:09,030 --> 00:32:12,090
Speaker 3: maybe academic style, like PhD work. In the PhD world,

665
00:32:12,130 --> 00:32:14,590
Speaker 3: of course, they're being strongly affected by just the larger

666
00:32:14,650 --> 00:32:18,050
Speaker 3: trends and changes in the research universe. And then when

667
00:32:18,090 --> 00:32:20,390
Speaker 3: it comes to professional degrees, the main question we have

668
00:32:20,410 --> 00:32:22,090
Speaker 3: to ask is, what is the right thing to teach

669
00:32:22,110 --> 00:32:24,310
Speaker 3: our students in the current moment? So, for example, I'm

670
00:32:24,350 --> 00:32:27,170
Speaker 3: in a department that teaches computer science. The field of

671
00:32:27,190 --> 00:32:29,640
Speaker 3: computer science is changing by the day, right? It used

672
00:32:29,660 --> 00:32:31,700
Speaker 3: to be, for example, we talked about programming as this

673
00:32:31,820 --> 00:32:35,010
Speaker 3: core skill and thing that you should... Everybody should learn

674
00:32:35,050 --> 00:32:38,430
Speaker 3: from day one. I would argue that's probably still the case,

675
00:32:38,610 --> 00:32:41,010
Speaker 3: the same way that maybe learning to write and be

676
00:32:41,050 --> 00:32:43,780
Speaker 3: literate is a key part of being a person in

677
00:32:43,800 --> 00:32:45,660
Speaker 3: the society, even if you're not doing that every day.

678
00:32:46,360 --> 00:32:49,880
Speaker 3: But jobs as an actual programmer are becoming more rare, right?

679
00:32:49,900 --> 00:32:53,560
Speaker 3: We see a decline, for example, in enrollment in computer

680
00:32:53,590 --> 00:32:55,330
Speaker 3: science for the first time in like 20 years.

681
00:32:55,810 --> 00:33:00,570
Speaker 1: Since you mentioned computer science and unsolved questions, what is

682
00:33:00,770 --> 00:33:03,110
Speaker 1: P equals NP all about?

683
00:33:03,430 --> 00:33:05,850
Speaker 3: I see, I can answer all your math questions today, absolutely.

684
00:33:05,950 --> 00:33:08,120
Speaker 1: These are just all these questions I've stored up in

685
00:33:08,170 --> 00:33:08,620
Speaker 1: my head.

686
00:33:08,690 --> 00:33:10,180
Speaker 3: Right, and we should come back to, I don't think

687
00:33:10,220 --> 00:33:12,180
Speaker 3: I actually told you the end of the Navier-Stokes story,

688
00:33:12,200 --> 00:33:14,240
Speaker 3: so I can come back to that. But yeah, so

689
00:33:14,260 --> 00:33:17,200
Speaker 3: P versus NP and Navier-Stokes are examples of very famous

690
00:33:17,240 --> 00:33:20,140
Speaker 3: mathematical problems that have been identified by the Clay Math

691
00:33:20,200 --> 00:33:24,650
Speaker 3: Institute as sort of this benchmark for progress in modern mathematics, right?

692
00:33:24,670 --> 00:33:26,310
Speaker 3: So they're open questions we don't know how to answer.

693
00:33:26,940 --> 00:33:29,980
Speaker 3: P versus NP is a computer science problem in the

694
00:33:30,020 --> 00:33:34,030
Speaker 3: sense that it's basically asking are all problems solvable in

695
00:33:34,070 --> 00:33:37,690
Speaker 3: polynomial time? Meaning can your computer solve, or all is

696
00:33:37,710 --> 00:33:40,290
Speaker 3: a little strong, but there's certain classes of problems that

697
00:33:40,330 --> 00:33:43,770
Speaker 3: we know are solvable pretty efficiently. So for example, your

698
00:33:43,810 --> 00:33:45,710
Speaker 3: computer can sort a list of numbers really fast. There

699
00:33:46,090 --> 00:33:48,710
Speaker 3: are other problems that we don't know how to solve efficiently,

700
00:33:48,750 --> 00:33:51,570
Speaker 3: but that's different from saying we can't solve them efficiently.

701
00:33:52,230 --> 00:33:54,750
Speaker 3: And so that's the sort of fundamental question in that universe,

702
00:33:54,810 --> 00:33:57,550
Speaker 3: like do there exist problems Is this.

703
00:33:57,330 --> 00:34:01,530
Speaker 2: Like the known unknowns and unknown unknowns equivalent of the

704
00:34:01,550 --> 00:34:02,080
Speaker 2: computer world?

705
00:34:02,120 --> 00:34:04,860
Speaker 3: Yeah, yeah. It's hard to communicate about that way. That's

706
00:34:04,900 --> 00:34:07,830
Speaker 3: absolutely right. In fact, I loved... There was... Years ago,

707
00:34:07,850 --> 00:34:10,250
Speaker 3: there was a really terrible movie that came out that

708
00:34:10,310 --> 00:34:13,840
Speaker 3: involved like P versus NP and that secretly some mathematician

709
00:34:13,880 --> 00:34:15,680
Speaker 3: had proved that they were equal, but there's a government

710
00:34:15,719 --> 00:34:19,600
Speaker 3: conspiracy because the markets were all gonna collapse. I think

711
00:34:19,620 --> 00:34:24,200
Speaker 3: it was called Traveling Salesman. Anyway, the reviews were really funny.

712
00:34:24,239 --> 00:34:25,520
Speaker 3: They were like, it turns out it's really hard to

713
00:34:25,560 --> 00:34:27,440
Speaker 3: make drama out of people in a room at a blackboard.

714
00:34:28,550 --> 00:34:29,110
Speaker 3: That aside.

715
00:34:29,130 --> 00:34:30,650
Speaker 2: They should have used Windows instead.

716
00:34:30,730 --> 00:34:32,029
Speaker 3: That's right. Yeah, yeah. It would have made it a

717
00:34:32,030 --> 00:34:34,950
Speaker 3: little visually compelling. Right. So a lot of these questions

718
00:34:34,989 --> 00:34:36,230
Speaker 3: are sort of yes or no. We don't know if

719
00:34:36,310 --> 00:34:39,310
Speaker 3: a fact is true. Navier-Stokes is another one. Either you

720
00:34:39,350 --> 00:34:41,670
Speaker 3: need to show that these equations have a solution for

721
00:34:41,710 --> 00:34:45,000
Speaker 3: all time, like fluid moving around, or give a counterexample

722
00:34:45,020 --> 00:34:47,520
Speaker 3: where they don't. So in this AI case, for example,

723
00:34:47,940 --> 00:34:49,879
Speaker 3: what the AI gave was a counterexample. It said this

724
00:34:49,920 --> 00:34:54,020
Speaker 3: conjecture is not true, that there exists a particular configuration

725
00:34:54,040 --> 00:34:57,200
Speaker 3: of your fluid that becomes infinitely turbulent in a finite

726
00:34:57,219 --> 00:34:57,640
Speaker 3: amount of time.

727
00:34:57,980 --> 00:34:58,940
Speaker 2: With a special spoon.

728
00:34:59,140 --> 00:35:01,719
Speaker 3: That's right, with a very special spoon. And the reason

729
00:35:01,760 --> 00:35:03,779
Speaker 3: I'm highlighting that is because I think it's a really

730
00:35:03,840 --> 00:35:08,380
Speaker 3: great example of the really complicated relationship between humans and AI.

731
00:35:08,980 --> 00:35:13,739
Speaker 3: So the construction of that spoon arguably was done in

732
00:35:13,780 --> 00:35:16,110
Speaker 3: large part by a team in Spain of human mathematicians

733
00:35:16,160 --> 00:35:19,230
Speaker 3: on pen and paper. And then essentially what happened was

734
00:35:19,250 --> 00:35:21,549
Speaker 3: the AI, there was also this sort of interpersonal fight

735
00:35:21,590 --> 00:35:24,350
Speaker 3: between two different AI teams. That's a whole hot mess.

736
00:35:24,370 --> 00:35:27,670
Speaker 3: But that aside... these two teams were sort of, in

737
00:35:27,750 --> 00:35:30,430
Speaker 3: some sense, leveraging these guys' progress over a very long

738
00:35:30,469 --> 00:35:32,950
Speaker 3: period of time and then doing that last mile, right?

739
00:35:33,170 --> 00:35:35,560
Speaker 3: So it's true that the AI made some really critical

740
00:35:36,140 --> 00:35:39,219
Speaker 3: insights in that particular counter example, but it isn't true

741
00:35:39,260 --> 00:35:41,480
Speaker 3: that it's like a totally surprising result that came out

742
00:35:41,520 --> 00:35:42,000
Speaker 3: of left field.

743
00:35:42,340 --> 00:35:43,880
Speaker 2: Well, so this seems like it's going to be a

744
00:35:43,920 --> 00:35:46,620
Speaker 2: scenario that academia is going to have to deal with

745
00:35:46,920 --> 00:35:49,910
Speaker 2: more and more. So you could have a theoretical situation

746
00:35:49,950 --> 00:35:52,489
Speaker 2: where like one person comes up with the problem, another

747
00:35:53,340 --> 00:35:55,680
Speaker 2: or a team, one person or a team comes up

748
00:35:55,719 --> 00:35:57,890
Speaker 2: with the solution. And then you have a model that

749
00:35:57,950 --> 00:36:01,390
Speaker 2: like generates the verification or the final leg of the

750
00:36:01,430 --> 00:36:06,390
Speaker 2: problem or whatever. Who deserves credit in that scenario? Like,

751
00:36:06,510 --> 00:36:07,609
Speaker 2: how do you actually handle that?

752
00:36:08,050 --> 00:36:10,590
Speaker 3: How do we handle that currently? Poorly. And it's a

753
00:36:10,610 --> 00:36:12,969
Speaker 3: really interesting and challenging discussion. I don't think we have

754
00:36:12,989 --> 00:36:15,799
Speaker 3: a good answer right now. It used to be, for example,

755
00:36:16,180 --> 00:36:18,380
Speaker 3: these clay math prizes, they're at the very top of

756
00:36:18,940 --> 00:36:20,420
Speaker 3: the universe there. I'm not going to get one. I'm

757
00:36:20,440 --> 00:36:22,880
Speaker 3: not going to get anywhere close. It used to be

758
00:36:22,910 --> 00:36:24,900
Speaker 3: that they were sort of signifiers of two things. One

759
00:36:24,960 --> 00:36:28,000
Speaker 3: is that the person that proved the theoretical result is

760
00:36:28,040 --> 00:36:31,419
Speaker 3: a really smart person. The other one is that there's

761
00:36:31,540 --> 00:36:34,299
Speaker 3: a big achievement in mathematics itself that we now know

762
00:36:34,340 --> 00:36:36,270
Speaker 3: something to be true that we didn't know before or false.

763
00:36:37,410 --> 00:36:39,990
Speaker 3: Now it's not clear that those two things are correlated,

764
00:36:40,090 --> 00:36:42,430
Speaker 3: as you say, right? Like if a computer generates the

765
00:36:42,469 --> 00:36:46,029
Speaker 3: interesting part of a proof, who deserves credit? Should that

766
00:36:46,070 --> 00:36:47,940
Speaker 3: person be hired if they're the one that made the prompt?

767
00:36:48,340 --> 00:36:50,600
Speaker 3: It really is an interesting question for a math or

768
00:36:50,620 --> 00:36:53,430
Speaker 3: an engineering department. And it's not so clear. I think

769
00:36:53,469 --> 00:36:55,430
Speaker 3: it really depends on your value systems. Like, what do

770
00:36:55,510 --> 00:36:58,820
Speaker 3: we get out as humans from this exercise of proofwriting?

771
00:36:59,340 --> 00:37:02,080
Speaker 3: I mean, for example, does a counterexample to Navier-Stokes, is

772
00:37:02,100 --> 00:37:05,160
Speaker 3: that going to revolutionize the markets or change the way

773
00:37:05,200 --> 00:37:08,020
Speaker 3: we write software? Almost certainly not, right? So this is

774
00:37:08,060 --> 00:37:11,380
Speaker 3: more of a demonstration of a technology than any particular

775
00:37:11,420 --> 00:37:13,570
Speaker 3: practical thing. And so I think we have to be

776
00:37:13,590 --> 00:37:17,650
Speaker 3: really careful with understanding the incentive system behind who's claiming

777
00:37:17,670 --> 00:37:20,290
Speaker 3: for credit and also making sure that the humans that

778
00:37:20,330 --> 00:37:23,850
Speaker 3: are maybe doing all of the labor of composing the

779
00:37:23,880 --> 00:37:26,400
Speaker 3: data that's going into these systems that's getting remixed into

780
00:37:26,440 --> 00:37:29,379
Speaker 3: a proof, somehow that we're able to make some attribution there,

781
00:37:29,560 --> 00:37:31,680
Speaker 3: which currently the technology is really bad at, right? Like

782
00:37:31,700 --> 00:37:34,299
Speaker 3: you talk to AI, it's not so clear where it's

783
00:37:34,320 --> 00:37:36,339
Speaker 3: getting its answers from, even though we know that it's

784
00:37:36,420 --> 00:37:40,589
Speaker 3: kind of echoing data that's maybe scraped off the internet

785
00:37:40,630 --> 00:37:42,690
Speaker 3: or someone else's research paper or what have you.

786
00:37:42,930 --> 00:37:46,480
Speaker 2: Right, and this might be like the... the time that

787
00:37:46,540 --> 00:37:49,640
Speaker 2: we will know that for sure. And five years from now,

788
00:37:49,660 --> 00:37:51,440
Speaker 2: like who knows where it's getting its data from? It

789
00:37:51,480 --> 00:37:54,700
Speaker 2: could be scraping it from itself or like other models

790
00:37:54,760 --> 00:37:56,910
Speaker 2: are coming up with its own stuff. I have a

791
00:37:56,950 --> 00:38:01,790
Speaker 2: slight aside, which is the prizes that you mentioned, which

792
00:38:01,969 --> 00:38:04,830
Speaker 2: have also come up in movies like Good Will Hunting.

793
00:38:05,550 --> 00:38:08,700
Speaker 2: The prize money that people get for that, is that-

794
00:38:09,090 --> 00:38:12,400
Speaker 2: For them personally, or is it supposed to go into

795
00:38:12,739 --> 00:38:13,520
Speaker 2: their research?

796
00:38:13,620 --> 00:38:17,520
Speaker 3: That's a good question. I have no idea. The Claiming

797
00:38:17,540 --> 00:38:19,560
Speaker 3: Award specifically, I'm not sure.

798
00:38:20,180 --> 00:38:21,840
Speaker 1: Because some of them are big.

799
00:38:21,940 --> 00:38:24,630
Speaker 2: And so now I'm envisioning the scenario where like maybe

800
00:38:24,770 --> 00:38:27,010
Speaker 2: one day we have a model that ends up winning

801
00:38:27,190 --> 00:38:31,810
Speaker 2: some big mathematics prize and the money... goes to the lab.

802
00:38:32,090 --> 00:38:33,339
Speaker 3: I guess? I don't know.

803
00:38:33,410 --> 00:38:34,830
Speaker 1: I need the $ 1 million.

804
00:38:34,890 --> 00:38:36,970
Speaker 2: Well, no, I'm just wondering.

805
00:38:37,210 --> 00:38:39,330
Speaker 3: It's actually a good question, and we now have an

806
00:38:39,370 --> 00:38:43,030
Speaker 3: example of that. So this Navier-Stokes problem, I should be clear,

807
00:38:43,090 --> 00:38:48,490
Speaker 3: so their proof has not been officially verified and checked,

808
00:38:48,570 --> 00:38:51,650
Speaker 3: and in particular, there's a rule that it has to

809
00:38:51,690 --> 00:38:53,509
Speaker 3: kind of exist in the community for some number of

810
00:38:53,550 --> 00:38:56,540
Speaker 3: years and not be refuted. But we have to ask

811
00:38:56,560 --> 00:39:01,360
Speaker 3: that question right now. So OpenAI... arguably because they rushed

812
00:39:01,380 --> 00:39:04,690
Speaker 3: at the last second, were maybe the first to release

813
00:39:04,870 --> 00:39:08,130
Speaker 3: something that satisfies the Clay criteria. There's a kind of

814
00:39:08,150 --> 00:39:11,640
Speaker 3: a choose-your-own-adventure situation. There's a few different conjectures, and you

815
00:39:11,660 --> 00:39:13,359
Speaker 3: could prove any one of them and then get this

816
00:39:13,380 --> 00:39:16,080
Speaker 3: particular prize. OpenAI was quick to say that they don't

817
00:39:16,100 --> 00:39:18,060
Speaker 3: need the million dollars, because of course they don't. In

818
00:39:18,239 --> 00:39:20,160
Speaker 3: two minutes, they're burning through a million dollars worth of

819
00:39:20,200 --> 00:39:23,580
Speaker 3: compute anyway. Which I think begs the question, like, well,

820
00:39:23,719 --> 00:39:26,140
Speaker 3: why are they doing this? It's not for the money. Yeah.

821
00:39:26,180 --> 00:39:27,340
Speaker 3: And now I don't think it was true that it

822
00:39:27,380 --> 00:39:29,930
Speaker 3: was for the money for the mathematicians either. But the

823
00:39:29,989 --> 00:39:31,890
Speaker 3: million dollars made a big difference to an individual.

824
00:39:32,400 --> 00:39:35,160
Speaker 1: There was some story about some Russian mathematician who sold

825
00:39:35,180 --> 00:39:36,510
Speaker 1: one of those big prizes, right?

826
00:39:36,530 --> 00:39:39,270
Speaker 3: And then declined the award altogether. He also declined the money.

827
00:39:39,310 --> 00:39:39,730
Speaker 3: That's right.

828
00:39:39,890 --> 00:39:42,110
Speaker 2: I'm not going to decline the money, for the record.

829
00:39:42,210 --> 00:39:44,529
Speaker 3: No, I would happily take it.

830
00:39:44,910 --> 00:39:48,930
Speaker 2: When I finally solve some unprovable mathematical theorem. But actually,

831
00:39:48,950 --> 00:39:52,419
Speaker 2: this leads to another more serious question, which is, okay,

832
00:39:52,680 --> 00:39:55,140
Speaker 2: it costs a lot of money for these models to

833
00:39:55,160 --> 00:39:57,719
Speaker 2: be run, and they consume vast amounts of tokens from

834
00:39:57,820 --> 00:40:02,840
Speaker 2: very advanced mathematics. Do you end up in a situation

835
00:40:02,880 --> 00:40:06,180
Speaker 2: where whoever has the most tokens wins the mathematics prize?

836
00:40:06,340 --> 00:40:08,860
Speaker 3: Arguably. That, I think, is one of the major concerns

837
00:40:08,900 --> 00:40:10,840
Speaker 3: in our community right now. And by the way, you

838
00:40:11,120 --> 00:40:13,480
Speaker 3: mentioned education. It's also a concern there. Does a student

839
00:40:13,500 --> 00:40:15,700
Speaker 3: that has a $ 200 account get to have a better

840
00:40:15,739 --> 00:40:17,779
Speaker 3: homework solution? Yeah. It's a real question we have to

841
00:40:17,820 --> 00:40:20,390
Speaker 3: grapple with right now. On the advanced end, I think

842
00:40:20,430 --> 00:40:23,210
Speaker 3: we saw that. So, for example, I don't think it's

843
00:40:23,230 --> 00:40:26,370
Speaker 3: controversial to say that essentially when the news that people

844
00:40:26,390 --> 00:40:30,590
Speaker 3: were getting close to this Navier-Stokes counterexample came out, it

845
00:40:30,630 --> 00:40:34,359
Speaker 3: appears that OpenAI suddenly poured some ridiculous amount of tokens

846
00:40:34,430 --> 00:40:38,080
Speaker 3: or money or whatever into writing their proof first. The

847
00:40:38,120 --> 00:40:39,459
Speaker 3: typical academic can't do that.

848
00:40:39,660 --> 00:40:43,000
Speaker 1: And I think there's an important element, I believe, with

849
00:40:43,040 --> 00:40:48,120
Speaker 1: an unreleased model. And so I'm curious, let's say you're

850
00:40:48,140 --> 00:40:51,340
Speaker 1: a mathematician and you're an enthusiast. Let's say you're, you

851
00:40:51,380 --> 00:40:53,920
Speaker 1: know what, I'm going to embrace this new world of

852
00:40:54,700 --> 00:41:00,330
Speaker 1: AI augmenting our understanding of mathematics. If you're a mathematician

853
00:41:00,590 --> 00:41:04,299
Speaker 1: at a university, it's certain that there is a model

854
00:41:04,340 --> 00:41:06,860
Speaker 1: that exists inside the labs that's better than the model

855
00:41:06,900 --> 00:41:08,020
Speaker 1: that you have access to.

856
00:41:08,060 --> 00:41:08,420
Speaker 3: That's right.

857
00:41:08,860 --> 00:41:10,960
Speaker 1: And so it's like you start to work on some problems.

858
00:41:11,000 --> 00:41:15,560
Speaker 1: You start to demonstrate that actually using AI, you can

859
00:41:15,580 --> 00:41:19,020
Speaker 1: make progress towards this. And then suddenly it's like, well,

860
00:41:19,040 --> 00:41:21,569
Speaker 1: they have a model two generations ahead of you. And

861
00:41:21,590 --> 00:41:22,890
Speaker 1: then they knock it out in a week.

862
00:41:23,030 --> 00:41:24,830
Speaker 3: And by the way, for all you know, their model

863
00:41:24,850 --> 00:41:28,780
Speaker 3: was trained partially on your chats, right? It's really pernicious

864
00:41:28,860 --> 00:41:31,000
Speaker 3: if you think about it. That's right. I mean, at

865
00:41:31,040 --> 00:41:34,859
Speaker 3: least in the current moment, the AI system is very centralized, right?

866
00:41:34,880 --> 00:41:37,569
Speaker 3: I mean, there's two or three tools that almost everybody

867
00:41:37,630 --> 00:41:42,069
Speaker 3: is using. And they're owned by these companies. When you

868
00:41:42,090 --> 00:41:44,669
Speaker 3: write your query, it goes to their servers. And they're

869
00:41:44,710 --> 00:41:47,549
Speaker 3: basically the guys controlling the faucet. is a big concern.

870
00:41:47,860 --> 00:41:51,350
Speaker 3: Another issue related to that is research funding. As your

871
00:41:51,370 --> 00:41:54,270
Speaker 3: listeners may or may not know, even independently of AI,

872
00:41:54,330 --> 00:41:57,390
Speaker 3: research funding currently is in a bit of a crisis.

873
00:41:57,430 --> 00:41:59,040
Speaker 3: I mean, it's in a much more challenging moment than

874
00:41:59,060 --> 00:42:01,120
Speaker 3: it was in the past. I can tell you, I'm

875
00:42:01,219 --> 00:42:04,360
Speaker 3: so fortunate to work at MIT. And even for me,

876
00:42:04,440 --> 00:42:06,620
Speaker 3: it's a challenge to pay for my graduate students and

877
00:42:06,660 --> 00:42:08,910
Speaker 3: so on. And the costs change every year. Now we

878
00:42:08,930 --> 00:42:12,600
Speaker 3: have to add onto that, for example, AI accounts. And

879
00:42:12,620 --> 00:42:15,600
Speaker 3: that sounds theoretical, but it's not. My PhD students, for example,

880
00:42:15,620 --> 00:42:18,899
Speaker 3: want the most expensive possible cloud account, which is great,

881
00:42:18,940 --> 00:42:23,200
Speaker 3: but it's, what, $ 200 per person per month almost becomes

882
00:42:24,160 --> 00:42:27,040
Speaker 3: equitable with the cost of another human. Now, something with.

883
00:42:27,880 --> 00:42:31,439
Speaker 1: I read, apparently, some people had said that of the

884
00:42:31,660 --> 00:42:37,850
Speaker 1: outstanding clay prizes, many had sort of surmised that Navier-Stokes

885
00:42:37,989 --> 00:42:41,450
Speaker 1: might be the first one solved. And I think it's interesting...

886
00:42:41,730 --> 00:42:46,549
Speaker 1: that it was solved via counter example. And it sort

887
00:42:46,590 --> 00:42:50,080
Speaker 1: of raises a question of like, did the model find

888
00:42:50,140 --> 00:42:53,919
Speaker 1: an elegant solution or did it just grind? Was this

889
00:42:53,960 --> 00:42:55,380
Speaker 1: a grindable problem?

890
00:42:55,640 --> 00:42:56,460
Speaker 3: That's exactly the right question.

891
00:42:56,480 --> 00:42:58,900
Speaker 1: Or just a grind through a lot of states and

892
00:42:58,960 --> 00:43:02,419
Speaker 1: then it hits on the magic spoon that disproves it

893
00:43:02,520 --> 00:43:04,900
Speaker 1: or et cetera. Which seems like a very, like if

894
00:43:04,960 --> 00:43:09,660
Speaker 1: finding that counter example is a process of grinding through

895
00:43:10,219 --> 00:43:14,270
Speaker 1: hundreds of thousands of different states. Like, is that elegant

896
00:43:14,350 --> 00:43:16,549
Speaker 1: math or is that just like some sort of like

897
00:43:16,610 --> 00:43:18,090
Speaker 1: sheer mechanical process?

898
00:43:18,170 --> 00:43:20,290
Speaker 3: Oh, that's a great question. And indeed, I think you

899
00:43:20,310 --> 00:43:22,860
Speaker 3: mentioned the beginning of this, this talk that you opened

900
00:43:22,880 --> 00:43:25,819
Speaker 3: that paper and it's like totally unreadable. And that's not

901
00:43:25,860 --> 00:43:27,680
Speaker 3: just whether you have a math degree or not. Like

902
00:43:27,700 --> 00:43:30,480
Speaker 3: I can't read past the first couple of pages, you know? Right.

903
00:43:30,540 --> 00:43:33,660
Speaker 3: So that particular result, I think there was some beautiful

904
00:43:33,700 --> 00:43:37,270
Speaker 3: insight again, arguably generated by humans, that went into the

905
00:43:37,350 --> 00:43:40,370
Speaker 3: overall construction. And then indeed, the proof is really horrifying.

906
00:43:40,410 --> 00:43:42,810
Speaker 3: Like there's a lot going on. It's technical. It's got

907
00:43:42,890 --> 00:43:45,530
Speaker 3: inequalities and bounds and indices and all kinds of crazy stuff.

908
00:43:46,070 --> 00:43:48,799
Speaker 3: And so I think indeed, as you say, in this

909
00:43:48,840 --> 00:43:54,080
Speaker 3: particular case, no, is it elegant? Probably not. Is it correct? Yes, probably.

910
00:43:54,800 --> 00:43:57,719
Speaker 3: So again, basic question, like why do we do this exercise?

911
00:43:57,780 --> 00:44:00,660
Speaker 3: Is it just to now we can say with confidence

912
00:44:00,700 --> 00:44:03,080
Speaker 3: that this conjecture is false? Or is it that we

913
00:44:03,120 --> 00:44:05,219
Speaker 3: learn something along the way that'll bring insight to some

914
00:44:05,260 --> 00:44:08,500
Speaker 3: other problem? Those two things are now divorced, right? If

915
00:44:08,520 --> 00:44:11,319
Speaker 3: you just have this frictionless way to prove a theorem,

916
00:44:11,830 --> 00:44:14,270
Speaker 3: then all the beautiful insight and maybe kind of fun

917
00:44:14,370 --> 00:44:17,190
Speaker 3: garden paths along the way that might have inspired some

918
00:44:17,230 --> 00:44:20,130
Speaker 3: new research field have been pruned out, which is a

919
00:44:20,150 --> 00:44:23,390
Speaker 3: big challenge, right? In some sense, that's really the key

920
00:44:23,430 --> 00:44:24,739
Speaker 3: human aspect of all of this.

921
00:44:40,870 --> 00:44:43,549
Speaker 2: So this was going to be my next question. And

922
00:44:44,180 --> 00:44:46,259
Speaker 2: I guess I'm really asking you to opine on whether

923
00:44:46,280 --> 00:44:48,560
Speaker 2: or not you think AGI is going to be a

924
00:44:48,620 --> 00:44:51,279
Speaker 2: thing or super intelligence or whatever we're calling it now.

925
00:44:51,760 --> 00:44:55,800
Speaker 2: But OK, so AI right now can sort of brute

926
00:44:55,900 --> 00:45:01,100
Speaker 2: force a solution. And it's not particularly elegant. Can it

927
00:45:01,420 --> 00:45:05,489
Speaker 2: come up with ideas that are separate to what we

928
00:45:05,650 --> 00:45:09,520
Speaker 2: kind of already know or questions that we are already positing? Like,

929
00:45:09,620 --> 00:45:12,060
Speaker 2: can it actually inform us that, no, the way you

930
00:45:12,100 --> 00:45:16,510
Speaker 2: think the world works currently is actually different? Or can

931
00:45:16,550 --> 00:45:19,609
Speaker 2: it sort of like unveil mathematical rules or rules about

932
00:45:19,650 --> 00:45:22,890
Speaker 2: the world that we are not even thinking about or

933
00:45:23,270 --> 00:45:23,770
Speaker 2: asking about?

934
00:45:23,790 --> 00:45:24,510
Speaker 3: New conjecture.

935
00:45:24,590 --> 00:45:25,509
Speaker 2: Yeah, exactly.

936
00:45:25,570 --> 00:45:28,690
Speaker 3: Exactly. It's a great question. So actually, my colleague Nestor

937
00:45:28,730 --> 00:45:31,440
Speaker 3: Guillen at NYU has a nice blog post that kind

938
00:45:31,460 --> 00:45:33,680
Speaker 3: of talks about this a bit. When you look at

939
00:45:33,719 --> 00:45:37,100
Speaker 3: a lot of the proofs and the new results that

940
00:45:37,120 --> 00:45:39,219
Speaker 3: are coming out of these AI systems, they're arguably in

941
00:45:39,500 --> 00:45:42,560
Speaker 3: what mathematicians might call the convex hole of existing knowledge,

942
00:45:42,640 --> 00:45:44,840
Speaker 3: meaning there's sort of a bunch of stuff we already know.

943
00:45:45,530 --> 00:45:47,750
Speaker 3: And then a lot of these proofs are really intricate

944
00:45:47,770 --> 00:45:49,930
Speaker 3: and interesting ways to mix and match those things. They're

945
00:45:49,950 --> 00:45:52,330
Speaker 3: like kind of inside of the span of knowledge that

946
00:45:52,370 --> 00:45:54,650
Speaker 3: we already have. And you've asked the key question, which

947
00:45:54,710 --> 00:45:59,130
Speaker 3: is can a mathematical AI system actually come up with

948
00:45:59,190 --> 00:46:03,550
Speaker 3: a truly surprising new result that's like not just mixing

949
00:46:03,570 --> 00:46:06,030
Speaker 3: and matching facts we already know, or maybe like finishing

950
00:46:06,070 --> 00:46:09,730
Speaker 3: off a calculation that somebody got stuck on, but making

951
00:46:09,930 --> 00:46:13,570
Speaker 3: an entirely new theory, for example, that just wasn't present before.

952
00:46:13,590 --> 00:46:15,570
Speaker 3: I don't think we've seen a whole lot of examples

953
00:46:15,610 --> 00:46:17,690
Speaker 3: of that so far. And now, I mean, this technology

954
00:46:17,730 --> 00:46:19,910
Speaker 3: changes by the day, like who knows, but at least

955
00:46:19,950 --> 00:46:21,799
Speaker 3: so far, I think for the most part, it's in

956
00:46:21,820 --> 00:46:25,400
Speaker 3: that first category of like, maybe making really interesting and

957
00:46:25,420 --> 00:46:29,750
Speaker 3: surprising connections between surprising, between, things that we already knew,

958
00:46:29,910 --> 00:46:33,230
Speaker 3: but not necessarily generating something entirely new out of left field, right?

959
00:46:33,350 --> 00:46:34,790
Speaker 3: That would be really the next step for a lot

960
00:46:34,830 --> 00:46:35,390
Speaker 3: of these tools.

961
00:46:35,870 --> 00:46:37,569
Speaker 1: I know that like one of the questions too, with

962
00:46:37,590 --> 00:46:39,230
Speaker 1: some of the, and I think maybe some of the

963
00:46:39,290 --> 00:46:43,700
Speaker 1: lesser prestigious proofs that have been established, there's the question, okay,

964
00:46:43,719 --> 00:46:46,160
Speaker 1: maybe they just grind their way to an answer. And

965
00:46:46,200 --> 00:46:49,080
Speaker 1: the other, I know another question is like, well, was

966
00:46:49,120 --> 00:46:51,460
Speaker 1: this already sort of proven somewhere? Maybe it was like,

967
00:46:51,780 --> 00:46:54,500
Speaker 1: there is some obscure Russian math journal where most of

968
00:46:54,520 --> 00:46:57,489
Speaker 1: the work had been done and most of them. mathematicians

969
00:46:57,510 --> 00:47:00,620
Speaker 1: were aware of that. But it was in the training data.

970
00:47:00,900 --> 00:47:01,440
Speaker 1: And so.

971
00:47:01,460 --> 00:47:07,020
Speaker 3: Hard to know. Data attribution in AI generally, mathematics or otherwise,

972
00:47:07,040 --> 00:47:09,279
Speaker 3: is a really critical problem. I mean, it's exactly the

973
00:47:09,300 --> 00:47:13,240
Speaker 3: same problem as like, my AI generated a cool new cartoon.

974
00:47:13,840 --> 00:47:16,520
Speaker 3: Did it borrow from Disney? And is that okay? In

975
00:47:16,540 --> 00:47:18,480
Speaker 3: some sense, it's a very similar problem. And the current

976
00:47:18,500 --> 00:47:19,719
Speaker 3: tools are very bad at answering it.

977
00:47:19,940 --> 00:47:21,940
Speaker 1: Why don't you just tell us a little bit about

978
00:47:22,940 --> 00:47:27,980
Speaker 1: being a professor in 2026? And like, what you're doing

979
00:47:28,020 --> 00:47:31,820
Speaker 1: differently and just that education.

980
00:47:32,020 --> 00:47:35,080
Speaker 3: Yes. So the education field is having its own, I

981
00:47:35,080 --> 00:47:37,719
Speaker 3: don't know about identity crisis, but certainly a lot of

982
00:47:37,780 --> 00:47:41,150
Speaker 3: navel gazing and thinking very hard about the fundamentals. MIT

983
00:47:41,230 --> 00:47:45,290
Speaker 3: actually released a really nice little report on AI and education.

984
00:47:45,330 --> 00:47:47,730
Speaker 3: I encourage everybody to take a look. My colleagues, Sam

985
00:47:47,850 --> 00:47:50,350
Speaker 3: Klopfer and Eric Madden, were in charge of writing that.

986
00:47:50,989 --> 00:47:54,540
Speaker 3: But every aspect of education has changed. I think One

987
00:47:54,580 --> 00:47:57,780
Speaker 3: thing that maybe is undervalued is the amount of education,

988
00:47:57,800 --> 00:48:01,700
Speaker 3: which is social, even if primarily the maybe artifacts that

989
00:48:01,719 --> 00:48:04,290
Speaker 3: you're producing are things like problem sets. So one thing

990
00:48:04,310 --> 00:48:07,830
Speaker 3: that we've seen among our sort of youngest students, for example,

991
00:48:07,870 --> 00:48:10,569
Speaker 3: the early undergraduates, is that it's really hard to draw

992
00:48:10,590 --> 00:48:12,650
Speaker 3: them out of their dorm room. There's an interesting perception

993
00:48:12,670 --> 00:48:17,090
Speaker 3: challenge here. You now have, again, this frictionless robot that

994
00:48:17,230 --> 00:48:19,310
Speaker 3: can help you with your problem sets, or frankly, do

995
00:48:19,350 --> 00:48:23,509
Speaker 3: your problem set for you. Is that learning? Probably not. Now,

996
00:48:23,650 --> 00:48:26,890
Speaker 3: the interesting thing is that there's a difference between perception

997
00:48:26,950 --> 00:48:29,200
Speaker 3: and reality. A lot of our students maybe perceive that

998
00:48:29,219 --> 00:48:32,400
Speaker 3: they're learning when they use these different AI tools. Yeah.

999
00:48:32,500 --> 00:48:34,120
Speaker 3: But then they go and take the exam and the

1000
00:48:34,160 --> 00:48:35,440
Speaker 3: score would say otherwise.

1001
00:48:35,620 --> 00:48:39,100
Speaker 1: So what do you do to sort of maximize the

1002
00:48:39,140 --> 00:48:40,620
Speaker 1: chances that your students are learning?

1003
00:48:41,140 --> 00:48:43,410
Speaker 3: You know, at MIT, they have a slogan. I think

1004
00:48:43,469 --> 00:48:45,640
Speaker 3: it roughly is like mind and hand learning. And then

1005
00:48:45,660 --> 00:48:47,299
Speaker 3: they added and heart on the end, which is good.

1006
00:48:48,110 --> 00:48:50,049
Speaker 3: And I think we're really leaning into that second one.

1007
00:48:50,170 --> 00:48:52,430
Speaker 3: We've known long before AI that putting a bunch of

1008
00:48:52,450 --> 00:48:54,549
Speaker 3: students in a classroom and talking at them and then

1009
00:48:54,570 --> 00:48:56,989
Speaker 3: having them work on a problem set alone is not

1010
00:48:57,010 --> 00:48:57,969
Speaker 3: a very effective way to teach.

1011
00:48:58,170 --> 00:48:58,450
Speaker 1: Okay.

1012
00:48:58,670 --> 00:49:01,820
Speaker 3: So now I think we're really focusing on experiential learning,

1013
00:49:02,010 --> 00:49:04,500
Speaker 3: on getting the students to show up physically and to

1014
00:49:04,620 --> 00:49:07,399
Speaker 3: ask that question or answer that question rather of why

1015
00:49:07,460 --> 00:49:09,279
Speaker 3: do we take all these students and put them on

1016
00:49:09,320 --> 00:49:11,239
Speaker 3: a college campus instead of having them work from a

1017
00:49:11,280 --> 00:49:14,160
Speaker 3: laptop at home? Now, I would argue that that's still

1018
00:49:14,180 --> 00:49:17,650
Speaker 3: a really valuable exercise. MIT, for example, has the luxury

1019
00:49:17,710 --> 00:49:20,850
Speaker 3: of having the world expert on just about anything. physically

1020
00:49:20,890 --> 00:49:24,310
Speaker 3: within a mile of your dorms. And so we're designing

1021
00:49:24,410 --> 00:49:26,629
Speaker 3: all kinds of programs that, for example, in the School

1022
00:49:26,650 --> 00:49:29,890
Speaker 3: of Engineering, try to bring the students in for research experience,

1023
00:49:30,219 --> 00:49:33,319
Speaker 3: for hands-on learning, for going into industry and seeing what

1024
00:49:33,340 --> 00:49:36,890
Speaker 3: things look like there. And trying to build the undergraduate

1025
00:49:36,930 --> 00:49:38,950
Speaker 3: experience more out of that and a little bit less

1026
00:49:39,010 --> 00:49:41,870
Speaker 3: out of exams and problem sets. Now, there remain a

1027
00:49:41,890 --> 00:49:44,299
Speaker 3: lot of questions. For example, assessment is a big challenge

1028
00:49:44,340 --> 00:49:46,700
Speaker 3: for us. What we observed in the last year or

1029
00:49:46,719 --> 00:49:48,180
Speaker 3: two is that our students are getting nearly 100% on

1030
00:49:48,200 --> 00:49:52,970
Speaker 3: all their homeworks. Maybe it's because MIT students are super bright.

1031
00:49:52,989 --> 00:49:56,370
Speaker 2: Can I just say, so I take the train from

1032
00:49:56,590 --> 00:49:59,469
Speaker 2: New York to between New Haven quite a lot. There

1033
00:49:59,489 --> 00:50:02,430
Speaker 2: are a lot of Yale students. Watching them on their

1034
00:50:02,489 --> 00:50:07,640
Speaker 2: laptops right now, they're all on, mostly Claude actually, writing

1035
00:50:07,719 --> 00:50:10,859
Speaker 2: essays or at least doing the basic research via chatbot.

1036
00:50:11,230 --> 00:50:13,850
Speaker 3: So like this semester, I'm teaching a course called shape

1037
00:50:13,870 --> 00:50:16,549
Speaker 3: analysis that covers discrete differential geometry, if you're into that

1038
00:50:16,570 --> 00:50:16,890
Speaker 3: kind of thing.

1039
00:50:17,830 --> 00:50:18,149
Speaker 1: Of course.

1040
00:50:18,690 --> 00:50:21,090
Speaker 3: Who isn't, right? This is a pretty mathy class. In

1041
00:50:21,110 --> 00:50:24,060
Speaker 3: the past, most of what the students did were two things.

1042
00:50:24,120 --> 00:50:27,180
Speaker 3: It was kind of written assignments and then an open-ended

1043
00:50:27,219 --> 00:50:28,880
Speaker 3: project where they kind of come up with a new

1044
00:50:28,920 --> 00:50:31,640
Speaker 3: result and write it down. And both of those had

1045
00:50:31,660 --> 00:50:34,680
Speaker 3: to change. And so in our course now, the way

1046
00:50:34,719 --> 00:50:38,060
Speaker 3: we structure it is there's still homeworks, but The analogy

1047
00:50:38,140 --> 00:50:40,670
Speaker 3: that I'm using, as with many of my colleagues at MIT,

1048
00:50:40,730 --> 00:50:45,469
Speaker 3: is closer to weightlifting. You don't do it because it's fun, necessarily.

1049
00:50:45,950 --> 00:50:49,549
Speaker 3: I do sometimes, but not everybody's like that. But you

1050
00:50:49,570 --> 00:50:52,240
Speaker 3: do it more because you're trying to achieve some other goal.

1051
00:50:52,260 --> 00:50:55,140
Speaker 3: You're preparing or you're being a stronger person. And then

1052
00:50:55,200 --> 00:50:57,420
Speaker 3: maybe what you do is you have to test understanding

1053
00:50:57,440 --> 00:50:58,980
Speaker 3: in some other fashion. So we have a little short

1054
00:50:59,040 --> 00:51:01,980
Speaker 3: quiz after the homework that basically is just a copy-pasted

1055
00:51:02,000 --> 00:51:04,710
Speaker 3: homework problem, but makes sure that the students at least

1056
00:51:04,750 --> 00:51:07,380
Speaker 3: comprehend what they did. We also had to restructure the

1057
00:51:07,420 --> 00:51:10,880
Speaker 3: project a bit, add like oral presentation component to it

1058
00:51:10,960 --> 00:51:13,780
Speaker 3: and bring some more interaction to make sure that essentially

1059
00:51:13,840 --> 00:51:18,049
Speaker 3: we're educating the human, not just testing the capabilities of Claude, right?

1060
00:51:18,070 --> 00:51:20,170
Speaker 3: Those are very different things. And I would argue the

1061
00:51:20,210 --> 00:51:22,970
Speaker 3: real goal of an institution like MIT is to prepare

1062
00:51:23,090 --> 00:51:25,430
Speaker 3: brilliant students for the next set of problems.

1063
00:51:25,790 --> 00:51:28,460
Speaker 1: We were talking about how like it's difficult to just

1064
00:51:28,480 --> 00:51:31,920
Speaker 1: look at the output of the Navier-Stokes solution.

1065
00:51:31,980 --> 00:51:32,240
Speaker 3: Yes.

1066
00:51:32,540 --> 00:51:36,920
Speaker 1: Evidently even for mathematicians. And I was thinking, okay, so

1067
00:51:36,960 --> 00:51:39,470
Speaker 1: what's the future then of math? And I had this

1068
00:51:39,810 --> 00:51:43,490
Speaker 1: analogy in my head, like we knew how to brew

1069
00:51:43,530 --> 00:51:46,870
Speaker 1: beer before we understood the science of yeast, right? So

1070
00:51:46,969 --> 00:51:49,640
Speaker 1: alcohol has been around a long time. And then at

1071
00:51:49,700 --> 00:51:52,120
Speaker 1: some point people understood and they knew the process, but

1072
00:51:52,140 --> 00:51:53,859
Speaker 1: they didn't understand why the process worked.

1073
00:51:53,920 --> 00:51:54,200
Speaker 3: That's right.

1074
00:51:54,219 --> 00:51:58,419
Speaker 1: And then eventually they discovered yeast and why this sequence works.

1075
00:51:58,940 --> 00:52:01,520
Speaker 1: And so I thought like, well, maybe like, could it

1076
00:52:01,580 --> 00:52:06,140
Speaker 1: be that the future of science is, yes, the AI

1077
00:52:06,200 --> 00:52:10,140
Speaker 1: models are going to find all these answers or be

1078
00:52:10,160 --> 00:52:12,319
Speaker 1: able to do all these things, but that there will

1079
00:52:12,360 --> 00:52:15,219
Speaker 1: still be a role then for humans to like, I

1080
00:52:15,239 --> 00:52:18,740
Speaker 1: don't know, translate it or explain it.

1081
00:52:18,750 --> 00:52:19,790
Speaker 3: I think that's right.

1082
00:52:20,230 --> 00:52:24,050
Speaker 1: But then I also wonder if it's like, well, probably

1083
00:52:24,250 --> 00:52:26,430
Speaker 1: the AI is also going to be better at translating

1084
00:52:26,469 --> 00:52:29,650
Speaker 1: and explaining things too. But could there be a new

1085
00:52:29,790 --> 00:52:35,260
Speaker 1: sort of general reorientation of math such that rather than

1086
00:52:35,300 --> 00:52:40,640
Speaker 1: discovering new things per se, it's like understanding these things

1087
00:52:40,680 --> 00:52:41,540
Speaker 1: that have been proven?

1088
00:52:41,600 --> 00:52:44,100
Speaker 3: That's right. I think the right way to think about

1089
00:52:44,120 --> 00:52:47,500
Speaker 3: these AI systems is that they're tools. And people's jobs

1090
00:52:47,700 --> 00:52:50,320
Speaker 3: evolve all the time, and apparently mine is no different.

1091
00:52:51,000 --> 00:52:52,080
Speaker 3: I want to be clear, I don't think that we

1092
00:52:52,100 --> 00:52:53,939
Speaker 3: need to be subservient to the AI and just checking

1093
00:52:53,980 --> 00:52:56,840
Speaker 3: what it makes. I guess there's a few different aspects

1094
00:52:56,900 --> 00:52:59,109
Speaker 3: of that. One is, of course, my job is both

1095
00:52:59,190 --> 00:53:02,009
Speaker 3: education and research. And I think the education part, even

1096
00:53:02,070 --> 00:53:03,770
Speaker 3: if what it looks like day to day has changed

1097
00:53:03,850 --> 00:53:05,950
Speaker 3: a bit, the mission is still there. And we still

1098
00:53:05,989 --> 00:53:08,529
Speaker 3: want to educate the world's best humans at MIT. And

1099
00:53:08,590 --> 00:53:10,850
Speaker 3: we do an okay job of that. In terms of

1100
00:53:10,910 --> 00:53:14,450
Speaker 3: the mathematical labor, I think one thing that's really undervalued

1101
00:53:14,469 --> 00:53:16,879
Speaker 3: in this field is that the typical mathematician is not

1102
00:53:17,600 --> 00:53:19,420
Speaker 3: working on the Clay Math Institute problems.

1103
00:53:19,480 --> 00:53:20,140
Speaker 1: It was really hard.

1104
00:53:20,340 --> 00:53:22,040
Speaker 3: There's some really brilliant guys. I wish I could do it.

1105
00:53:22,060 --> 00:53:24,760
Speaker 3: I'm not, I'm not, I can't compete there. It's similarly

1106
00:53:24,800 --> 00:53:28,050
Speaker 3: like math contest problems was another one that the AI

1107
00:53:28,150 --> 00:53:30,030
Speaker 3: companies were quick to kind of pounce on.

1108
00:53:29,930 --> 00:53:31,610
Speaker 1: As the math Olympia.

1109
00:53:31,650 --> 00:53:33,730
Speaker 3: Exactly. That was the sort of previous generation.

1110
00:53:33,750 --> 00:53:34,190
Speaker 1: Yeah.

1111
00:53:34,330 --> 00:53:36,569
Speaker 3: All of these are great in the sense that they're

1112
00:53:36,610 --> 00:53:38,790
Speaker 3: like problems that you can write down. They're well contained.

1113
00:53:38,810 --> 00:53:41,050
Speaker 3: They're in a little box, but a lot of mathematics

1114
00:53:41,210 --> 00:53:45,450
Speaker 3: is more about taste than it is about just proofwriting

1115
00:53:45,530 --> 00:53:48,710
Speaker 3: and verifying some other conjecture that somebody wrote down. So

1116
00:53:48,730 --> 00:53:51,910
Speaker 3: for example, in my particular area, and again, I want

1117
00:53:51,930 --> 00:53:53,330
Speaker 3: to be clear, you know, I'm an applied guy. A

1118
00:53:53,350 --> 00:53:55,150
Speaker 3: lot of pure mathematicians would absolutely look down on what

1119
00:53:55,190 --> 00:53:57,330
Speaker 3: I'm doing and say, ah, that's not real math. I

1120
00:53:57,350 --> 00:54:00,230
Speaker 3: would say otherwise, but that's, we'll have that debate another day.

1121
00:54:00,890 --> 00:54:02,589
Speaker 3: By the time we get to writing a proof, we

1122
00:54:02,630 --> 00:54:06,030
Speaker 3: probably already know it's true. Like it's more double check situation.

1123
00:54:06,800 --> 00:54:09,319
Speaker 3: But the really interesting part is on the modeling and

1124
00:54:09,360 --> 00:54:11,859
Speaker 3: figuring out which questions to ask and what insight they

1125
00:54:11,880 --> 00:54:15,130
Speaker 3: bring us to the universe. And the AI tools are

1126
00:54:15,170 --> 00:54:19,380
Speaker 3: not doing that, right? And so you're right. Maybe proofreading

1127
00:54:19,440 --> 00:54:22,279
Speaker 3: becomes more of an exercise in writing prompts, but writing

1128
00:54:22,300 --> 00:54:24,219
Speaker 3: the prompts is very much what we've been teaching our

1129
00:54:24,239 --> 00:54:28,770
Speaker 3: students to do for centuries now. And that skill, I think, persists. Yeah.

1130
00:54:28,950 --> 00:54:31,530
Speaker 2: I mean, broadly, it feels like the emphasis is shifting

1131
00:54:31,570 --> 00:54:34,930
Speaker 2: to being able to ask the right questions, come up

1132
00:54:34,969 --> 00:54:38,670
Speaker 2: with new ideas versus just come up with a basic answer.

1133
00:54:38,710 --> 00:54:41,150
Speaker 3: That's right. And the typical math student probably doesn't end

1134
00:54:41,210 --> 00:54:43,360
Speaker 3: up in theoretical math. They maybe end up in finance.

1135
00:54:43,420 --> 00:54:46,439
Speaker 3: They end up in engineering. And for those folks, that

1136
00:54:46,480 --> 00:54:49,080
Speaker 3: skill is the really critical one, right? It's exactly what

1137
00:54:49,100 --> 00:54:50,620
Speaker 3: you need to be a good manager, to be a

1138
00:54:50,640 --> 00:54:53,460
Speaker 3: good project designer, to lead and innovate in all kinds

1139
00:54:53,500 --> 00:54:56,739
Speaker 3: of different fields. And that skill is somehow becoming more

1140
00:54:56,780 --> 00:54:59,970
Speaker 3: important than ever. All that boring proofreading stuff that everybody's

1141
00:54:59,989 --> 00:55:02,549
Speaker 3: allergic to is actually kind of being pushed to the

1142
00:55:02,590 --> 00:55:03,069
Speaker 3: side a bit.

1143
00:55:03,610 --> 00:55:05,109
Speaker 1: So we should still teach kids math.

1144
00:55:05,330 --> 00:55:05,690
Speaker 3: I think so.

1145
00:55:07,790 --> 00:55:09,950
Speaker 1: Justin Solomon, thank you so much for coming.

1146
00:55:09,790 --> 00:55:10,230
Speaker 2: On the blog.

1147
00:55:10,489 --> 00:55:11,490
Speaker 3: Of course, it was a pleasure.

1148
00:55:18,660 --> 00:55:29,680
Speaker 1: Well, I think, look, to say the least, the future

1149
00:55:29,700 --> 00:55:32,410
Speaker 1: of sort of math and math education, it's like, we

1150
00:55:32,450 --> 00:55:34,670
Speaker 1: have no idea where this is going, right? Because the

1151
00:55:34,710 --> 00:55:37,680
Speaker 1: models are going to keep getting better. And even just

1152
00:55:37,719 --> 00:55:41,520
Speaker 1: the sort of like prosaic point about like, okay, here

1153
00:55:41,560 --> 00:55:43,640
Speaker 1: you have some conference that used to have a thousand

1154
00:55:43,680 --> 00:55:47,640
Speaker 1: submissions and now it has 60,000 submissions. And some of

1155
00:55:47,680 --> 00:55:51,069
Speaker 1: them might even be very good, but it's like, that's

1156
00:55:51,130 --> 00:55:53,089
Speaker 1: the type of thing that just like, as he said,

1157
00:55:53,150 --> 00:55:56,090
Speaker 1: it like breaks existing systems pretty straightforward.

1158
00:55:56,230 --> 00:55:59,649
Speaker 2: Yeah. It feels like there's a tautology here with AI where,

1159
00:56:00,320 --> 00:56:02,420
Speaker 2: Because of AI, a lot of things are getting easier

1160
00:56:02,520 --> 00:56:05,130
Speaker 2: and more difficult at the same time, right? So you

1161
00:56:05,190 --> 00:56:08,370
Speaker 2: have the ability to generate tons and tons of content,

1162
00:56:08,450 --> 00:56:12,090
Speaker 2: but the ability to verify it doesn't necessarily go up

1163
00:56:12,290 --> 00:56:14,700
Speaker 2: at the same amount of time. Maybe it will one day.

1164
00:56:15,160 --> 00:56:19,140
Speaker 2: I also think the emphasis on questions rather than simple

1165
00:56:19,300 --> 00:56:23,219
Speaker 2: answers and the idea of taste, even in something like mathematics,

1166
00:56:23,280 --> 00:56:25,650
Speaker 2: which is kind of alien to me. You know, you

1167
00:56:25,690 --> 00:56:27,690
Speaker 2: hear about it in code all the time, the ability

1168
00:56:27,730 --> 00:56:30,270
Speaker 2: to write elegant code, and I guess the same applies

1169
00:56:30,310 --> 00:56:34,350
Speaker 2: to mathematics, the ability to elegantly prove a formula or

1170
00:56:34,870 --> 00:56:38,279
Speaker 2: elegantly come up with a theory. That seems to be

1171
00:56:38,380 --> 00:56:44,219
Speaker 2: repeated throughout everything nowadays. If you're able to generate ideas...

1172
00:56:44,360 --> 00:56:48,279
Speaker 2: think outside the box to use a terrible cliche and

1173
00:56:48,520 --> 00:56:53,200
Speaker 2: curate with some taste or write or perform with a

1174
00:56:53,260 --> 00:56:56,110
Speaker 2: particular amount of taste. That seems to be the edge

1175
00:56:56,150 --> 00:56:56,569
Speaker 2: case here.

1176
00:56:56,850 --> 00:56:58,870
Speaker 1: I would also say like, this is actually a pretty

1177
00:56:58,910 --> 00:57:03,250
Speaker 1: high stakes question, even outside of the academic setting sort

1178
00:57:03,290 --> 00:57:06,330
Speaker 1: of research taste, because of course, one of the big

1179
00:57:06,930 --> 00:57:12,710
Speaker 1: questions for AI period is this idea of, recursive self-improvement,

1180
00:57:13,110 --> 00:57:18,150
Speaker 1: can a model build a successor model? And to do

1181
00:57:18,250 --> 00:57:23,950
Speaker 1: that means generating new ideas to then experiment on. And

1182
00:57:24,150 --> 00:57:28,460
Speaker 1: I think it's a pretty big open question still whether

1183
00:57:28,720 --> 00:57:34,640
Speaker 1: the models are good at novel questions, right? So it's like, okay,

1184
00:57:34,960 --> 00:57:38,780
Speaker 1: there's a bunch of outstanding questions. There's reason to think

1185
00:57:38,820 --> 00:57:42,560
Speaker 1: they're solvable. okay, maybe the models get very good at that.

1186
00:57:43,260 --> 00:57:47,960
Speaker 1: Can the model come up with the next Navier-Stokes? Can

1187
00:57:48,000 --> 00:57:50,060
Speaker 1: it come up with the next P equals NP? Can

1188
00:57:50,120 --> 00:57:52,720
Speaker 1: it come up with the next Fermat's last theorem that's

1189
00:57:52,780 --> 00:57:55,760
Speaker 1: in math? But whatever the equivalence of that is in

1190
00:57:55,800 --> 00:57:59,640
Speaker 1: sort of areas like machine learning and so forth, it's

1191
00:57:59,680 --> 00:58:03,070
Speaker 1: a pretty important question for the future of AI. It's like,

1192
00:58:03,630 --> 00:58:08,450
Speaker 1: are the models good at proposing the next research experiment

1193
00:58:08,800 --> 00:58:12,450
Speaker 1: And then, of course, carrying out, conducting the test.

1194
00:58:12,550 --> 00:58:12,690
Speaker 3: Right.

1195
00:58:12,750 --> 00:58:14,690
Speaker 2: And they might be one day. They might be.

1196
00:58:15,050 --> 00:58:19,030
Speaker 1: But right now, there's clearly still a pretty significant role

1197
00:58:19,070 --> 00:58:20,510
Speaker 1: for a human in the lube.

1198
00:58:20,630 --> 00:58:20,910
Speaker 3: Right.

1199
00:58:20,950 --> 00:58:26,280
Speaker 1: To sort of like, I guess that's intuition or whatever. Imagination. Okay,

1200
00:58:26,610 --> 00:58:30,300
Speaker 1: let's try this out, et cetera. And my impression is

1201
00:58:30,360 --> 00:58:35,120
Speaker 1: that the models still aren't great at that particular aspect of,

1202
00:58:35,180 --> 00:58:36,720
Speaker 1: I guess, the scientific method.

1203
00:58:36,920 --> 00:58:40,240
Speaker 2: Yeah, I think that's what Justin said as well. So

1204
00:58:40,680 --> 00:58:42,790
Speaker 2: that gets me back to this idea that the human

1205
00:58:42,910 --> 00:58:47,050
Speaker 2: edge remains the idea generation, out of the box thinking,

1206
00:58:47,110 --> 00:58:50,350
Speaker 2: whatever you want to call it, out of the AI box.

1207
00:58:50,490 --> 00:58:51,410
Speaker 1: Out of the chip thinking.

1208
00:58:51,470 --> 00:58:52,990
Speaker 2: That's out of the chip thinking.

1209
00:58:53,030 --> 00:58:53,840
Speaker 3: That's good. Yeah.

1210
00:58:53,980 --> 00:58:54,660
Speaker 2: Shall we leave it there?

1211
00:58:54,820 --> 00:58:55,320
Speaker 1: Let's leave it there.

1212
00:58:55,540 --> 00:58:55,760
Speaker 3: All right.

1213
00:58:55,800 --> 00:58:58,060
Speaker 2: This has been another episode of the Odd Thoughts Podcast.

1214
00:58:58,080 --> 00:59:00,720
Speaker 2: I'm Traci Allaway. You can follow me at Traci Allaway.

1215
00:59:00,970 --> 00:59:03,710
Speaker 1: And I'm Joe Weisenthal. You can follow me at The Stalwart.

1216
00:59:04,050 --> 00:59:07,590
Speaker 1: Follow our producers, Carmen Rodriguez at Carmen Armit, Dashiell Bennett

1217
00:59:07,660 --> 00:59:10,880
Speaker 1: at Dashbot, Kale Brooks at Kale Brooks, and Kevin Lozano

1218
00:59:11,040 --> 00:59:13,640
Speaker 1: at Kevin Lloyd Lozano. And you can check out all

1219
00:59:13,660 --> 00:59:18,480
Speaker 1: of these topics 24-7 in our Discord, discord.gg slash oddlots.

1220
00:59:18,820 --> 00:59:21,720
Speaker 1: And for more oddlots content, check out our daily newsletter.

1221
00:59:21,940 --> 00:59:24,110
Speaker 1: You can find that at bloomberg.com slash oddlots.

1222
00:59:24,570 --> 00:59:26,510
Speaker 2: And if you enjoyed this conversation, if you like it

1223
00:59:26,570 --> 00:59:29,390
Speaker 2: when we talk about mathematics, then please let us know

1224
00:59:29,530 --> 00:59:32,460
Speaker 2: on your favorite podcast platform. If you're watching on video

1225
00:59:32,620 --> 00:59:34,940
Speaker 2: and you enjoyed the show, then please leave us a

1226
00:59:34,960 --> 00:59:37,460
Speaker 2: comment or like the video or better yet, subscribe.

1227
00:59:37,720 --> 00:59:40,540
Speaker 1: And remember, if you're a Bloomberg subscriber, you can listen

1228
00:59:40,580 --> 00:59:44,230
Speaker 1: to all of our episodes absolutely ad-free. All you need

1229
00:59:44,250 --> 00:59:47,090
Speaker 1: to do is find the Bloomberg channel on Apple Podcasts

1230
00:59:47,270 --> 00:59:48,570
Speaker 1: and follow the instructions there.

1231
00:59:49,010 --> 00:59:49,710
Speaker 2: Thanks for joining.

1232
00:59:59,330 --> 01:00:00,240
Speaker 3: Thank you.