1
00:00:02,720 --> 00:00:13,960
Speaker 1: Bloomberg Audio Studios, Podcasts, Radio News.

2
00:00:18,040 --> 00:00:21,680
Speaker 2: Hello and welcome to another episode of the Odd Lots Podcast.

3
00:00:21,760 --> 00:00:23,239
Speaker 2: I'm Joe Wassenthal and.

4
00:00:23,200 --> 00:00:24,119
Speaker 3: I'm Tracy Alloway.

5
00:00:24,480 --> 00:00:27,520
Speaker 2: Tracy. When we talk about AI, we talk a lot

6
00:00:27,560 --> 00:00:30,880
Speaker 2: about the big labs, the big independent labs, particularly Open

7
00:00:30,920 --> 00:00:34,280
Speaker 2: AI Anthropic. But of course we know that the legacy

8
00:00:34,320 --> 00:00:37,319
Speaker 2: tech companies, the so called hyperscalers, et cetera, they're not

9
00:00:37,400 --> 00:00:41,120
Speaker 2: just going to give up this, this groundbreaking technology without

10
00:00:41,120 --> 00:00:41,360
Speaker 2: a fight.

11
00:00:41,760 --> 00:00:44,839
Speaker 3: No, and we've certainly seen various efforts to I don't

12
00:00:44,840 --> 00:00:48,839
Speaker 3: want to say catch up, but like to keep pace. Yeah.

13
00:00:48,880 --> 00:00:51,720
Speaker 2: So of the major legacy companies, you know, which I

14
00:00:51,720 --> 00:00:57,880
Speaker 2: would say are like Microsoft, Amazon, Meta, and Google or sorry, Alphabet,

15
00:00:58,040 --> 00:01:00,440
Speaker 2: I would say clearly like Alphabet is the one that

16
00:01:00,520 --> 00:01:03,480
Speaker 2: has the model that people are You don't talk about Gemini, right,

17
00:01:03,880 --> 00:01:07,319
Speaker 2: And in a way that's kind of surprising, because one

18
00:01:07,440 --> 00:01:12,160
Speaker 2: common intuition that people have is that companies aren't very

19
00:01:12,160 --> 00:01:15,640
Speaker 2: good at developing the thing that disrupts their own legacy business.

20
00:01:15,720 --> 00:01:15,880
Speaker 4: Right.

21
00:01:15,959 --> 00:01:19,039
Speaker 2: This is just a famous sort of B school thing

22
00:01:19,040 --> 00:01:20,280
Speaker 2: that people talk about all the time.

23
00:01:20,400 --> 00:01:23,760
Speaker 3: Yes, So the issue one would think with AI and

24
00:01:23,800 --> 00:01:27,280
Speaker 3: Google in particular is that Google, you know, famous for

25
00:01:27,319 --> 00:01:29,520
Speaker 3: its search. Yeah, no one believes me. By the way,

26
00:01:29,600 --> 00:01:31,760
Speaker 3: the first time I ever used Google in like I

27
00:01:31,760 --> 00:01:34,199
Speaker 3: guess it was the either late nineteen nineties or early

28
00:01:34,200 --> 00:01:36,319
Speaker 3: two thousands, I told my parents to invest in the

29
00:01:36,319 --> 00:01:38,720
Speaker 3: company and they didn't listen to Oh really, Yeah, they

30
00:01:38,720 --> 00:01:40,840
Speaker 3: don't believe me either, but I swear I did. The

31
00:01:40,840 --> 00:01:42,520
Speaker 3: first time I used it, I was like, oh my god,

32
00:01:42,520 --> 00:01:43,520
Speaker 3: this is so much better.

33
00:01:44,000 --> 00:01:47,039
Speaker 2: So I feel so dumb because I remember the Google

34
00:01:47,080 --> 00:01:50,120
Speaker 2: IPO very well and I thought I was very smart

35
00:01:50,160 --> 00:01:52,280
Speaker 2: because it was like I was twenty. I think it

36
00:01:52,320 --> 00:01:53,720
Speaker 2: was two thousand and four. I was very I thought

37
00:01:53,720 --> 00:01:55,960
Speaker 2: I was very smart back in those days. I don't

38
00:01:55,960 --> 00:01:58,360
Speaker 2: think now that I'm older, I realized I don't know,

39
00:01:58,800 --> 00:02:00,360
Speaker 2: but I thought I was very smart. It's like, oh,

40
00:02:00,400 --> 00:02:03,600
Speaker 2: they sheep, they're just buying. It's overvalued. Should I short

41
00:02:03,600 --> 00:02:05,920
Speaker 2: this ipo? It's like a bubble et cetera. Thank God

42
00:02:06,000 --> 00:02:08,200
Speaker 2: I didn't like put out some short trade and go

43
00:02:08,280 --> 00:02:10,600
Speaker 2: bankru But like, it never occurred to me to go

44
00:02:10,680 --> 00:02:12,720
Speaker 2: long because I don't. And I think there's the journalist

45
00:02:12,760 --> 00:02:14,359
Speaker 2: temperament or all like very cynicals.

46
00:02:14,400 --> 00:02:15,919
Speaker 3: That's right, you have to be an optimist.

47
00:02:15,919 --> 00:02:17,760
Speaker 2: To be an yeah, you have to be an optimist.

48
00:02:17,800 --> 00:02:19,799
Speaker 2: You have to be willing to be part of the crowd.

49
00:02:19,880 --> 00:02:22,920
Speaker 2: You have to ride the wave, like riding waves, et cetera.

50
00:02:23,040 --> 00:02:24,440
Speaker 2: And so like I was like, oh, should I short

51
00:02:24,480 --> 00:02:28,320
Speaker 2: this anyway? I should have bought it? Okay, I held it,

52
00:02:28,639 --> 00:02:30,440
Speaker 2: and you know what, I should have bought it even

53
00:02:30,520 --> 00:02:32,440
Speaker 2: though we don't trade. Like, should have bought it in

54
00:02:32,480 --> 00:02:35,600
Speaker 2: like twenty twenty three when everyone was saying that shat

55
00:02:35,639 --> 00:02:38,280
Speaker 2: GPT was gonna eat its lunch. Should have bought it

56
00:02:38,320 --> 00:02:41,200
Speaker 2: a year later when people were saying, oh, Gemini is

57
00:02:41,240 --> 00:02:44,480
Speaker 2: two woke, et cetera. All these turns there are opportunities.

58
00:02:44,960 --> 00:02:46,800
Speaker 2: But anyway, I'm happy to just talk about.

59
00:02:46,639 --> 00:02:48,120
Speaker 3: It all right now that we've gone on a very

60
00:02:48,120 --> 00:02:50,959
Speaker 3: long segue, what we are getting at, it's an important segue.

61
00:02:51,040 --> 00:02:52,600
Speaker 3: What we are getting at is that in the theory,

62
00:02:52,800 --> 00:02:56,720
Speaker 3: AI would seem to pose a threat to Google's core business,

63
00:02:56,840 --> 00:02:59,000
Speaker 3: which is search. So if you type in a query

64
00:02:59,200 --> 00:03:01,680
Speaker 3: in Google now, now I think I have one open

65
00:03:02,360 --> 00:03:04,519
Speaker 3: from the last time we recorded a podcast. I don't

66
00:03:04,520 --> 00:03:06,040
Speaker 3: know why I was looking the sub It says, can

67
00:03:06,080 --> 00:03:09,360
Speaker 3: you see tankers physically from the Strait of Malacca. Oh yeah,

68
00:03:09,440 --> 00:03:11,520
Speaker 3: in the Strait of Malaca from Singapore. So if you

69
00:03:11,520 --> 00:03:14,239
Speaker 3: type that into Google, yeah, it used to be you

70
00:03:14,280 --> 00:03:15,800
Speaker 3: would just get a bunch of search results.

71
00:03:16,000 --> 00:03:16,480
Speaker 4: That's right.

72
00:03:16,960 --> 00:03:20,360
Speaker 3: Now you get an AI overview, which basically pulls in

73
00:03:20,400 --> 00:03:22,440
Speaker 3: a bunch of results from other pages and gives you

74
00:03:22,480 --> 00:03:24,800
Speaker 3: a sort of summary. And so the question is if

75
00:03:24,800 --> 00:03:27,079
Speaker 3: people are just going to be looking at these summaries

76
00:03:27,680 --> 00:03:31,160
Speaker 3: instead of actually going to the pages that Google Search

77
00:03:31,320 --> 00:03:33,560
Speaker 3: used to turn out as links at the top of

78
00:03:33,600 --> 00:03:37,040
Speaker 3: the page, what does that mean for traffic via Google Search?

79
00:03:37,400 --> 00:03:40,560
Speaker 2: Well, and then the other big element here was that

80
00:03:40,640 --> 00:03:43,600
Speaker 2: you know, people stop like clicking as much on links

81
00:03:43,640 --> 00:03:45,680
Speaker 2: or even seeing links. What does that mean for the

82
00:03:45,720 --> 00:03:49,760
Speaker 2: advertising business? The expectation is that you like see the

83
00:03:49,920 --> 00:03:53,440
Speaker 2: answer right there, et cetera. And one of the questions

84
00:03:53,440 --> 00:03:56,320
Speaker 2: we have about Open Eye in particular, if they're going

85
00:03:56,400 --> 00:03:58,760
Speaker 2: to be able to launch check an advertising big business.

86
00:03:59,000 --> 00:04:01,520
Speaker 2: Can these two things combine.

87
00:04:01,120 --> 00:04:01,600
Speaker 4: Et cetera.

88
00:04:01,920 --> 00:04:04,920
Speaker 2: We know that to this day that Google Search adds

89
00:04:04,920 --> 00:04:07,800
Speaker 2: are like the greatest money printer that's ever been inted

90
00:04:07,880 --> 00:04:10,200
Speaker 2: basically in the history of the world. And so how

91
00:04:10,320 --> 00:04:12,680
Speaker 2: this is going to interact and how Google is thinking

92
00:04:12,720 --> 00:04:15,760
Speaker 2: about these questions. I just say it's a little unclear

93
00:04:15,760 --> 00:04:17,560
Speaker 2: to me, because yeah, it's nice. I could just put

94
00:04:17,600 --> 00:04:19,520
Speaker 2: on the same question, can you see tankers in the

95
00:04:19,520 --> 00:04:21,280
Speaker 2: Strait of Malacca? I don't know if these are right.

96
00:04:21,440 --> 00:04:23,040
Speaker 2: I don't know if the AI, I don't know if

97
00:04:23,040 --> 00:04:24,760
Speaker 2: the aioverview is right, but it's there.

98
00:04:24,960 --> 00:04:28,480
Speaker 3: It's there, so lots of interesting questions here also AI

99
00:04:28,600 --> 00:04:31,560
Speaker 3: slop Right, Yeah, are the search results that are being

100
00:04:31,560 --> 00:04:34,360
Speaker 3: turned up actually of any certain quality?

101
00:04:34,480 --> 00:04:34,760
Speaker 4: Right?

102
00:04:34,920 --> 00:04:37,480
Speaker 2: It's a great point. And one of the reasons perhaps

103
00:04:37,600 --> 00:04:39,520
Speaker 2: that a lot of people, and I would include myself

104
00:04:39,520 --> 00:04:44,600
Speaker 2: in this use AI more and more is because I have,

105
00:04:44,720 --> 00:04:47,400
Speaker 2: like you know, I have some issues, let's say with us.

106
00:04:48,000 --> 00:04:51,000
Speaker 3: So anyway, I'm gonna clip that I have some issues

107
00:04:51,160 --> 00:04:52,000
Speaker 3: Jim Wisenthal.

108
00:04:52,320 --> 00:04:55,799
Speaker 2: Anyway, we really do have the perfect guest to talk

109
00:04:55,880 --> 00:04:59,080
Speaker 2: about this, someone who is like really literally right in

110
00:04:59,120 --> 00:05:01,160
Speaker 2: the middle of all this and can answer all of

111
00:05:01,240 --> 00:05:03,560
Speaker 2: our questions. We're going to be speaking with Liz Reid.

112
00:05:03,760 --> 00:05:06,279
Speaker 2: She is the VP of Search at Google, and she

113
00:05:06,320 --> 00:05:09,520
Speaker 2: has been at the company for over twenty years, has

114
00:05:09,560 --> 00:05:11,760
Speaker 2: been in the current role, has been on the search

115
00:05:11,800 --> 00:05:14,560
Speaker 2: team for a few years, and so we're going to

116
00:05:14,640 --> 00:05:16,720
Speaker 2: end hopefully get answers to all these questions. So, Liz,

117
00:05:16,760 --> 00:05:18,800
Speaker 2: thank you so much for coming on the Outlaws podcast.

118
00:05:19,240 --> 00:05:22,200
Speaker 4: Thank you for having me. Delighted to be here today.

119
00:05:22,360 --> 00:05:25,640
Speaker 2: Absolutely, Why don't you just start by telling us, like,

120
00:05:25,760 --> 00:05:27,440
Speaker 2: what's your role at Google? What does it mean? Okay,

121
00:05:27,440 --> 00:05:29,920
Speaker 2: you're the VP of Search at a company that people

122
00:05:30,080 --> 00:05:33,039
Speaker 2: know for ving a search engine company, But what is

123
00:05:33,080 --> 00:05:34,200
Speaker 2: your title actually entail?

124
00:05:35,200 --> 00:05:37,359
Speaker 4: I lead the search team that you can think of

125
00:05:37,400 --> 00:05:41,120
Speaker 4: as covering the product, the engineering, our user designers, and

126
00:05:41,200 --> 00:05:43,719
Speaker 4: data science, sort of the team that fundamentally builds the

127
00:05:43,720 --> 00:05:44,839
Speaker 4: search product that you use.

128
00:05:45,640 --> 00:05:47,320
Speaker 3: So how much of your day to day is taken

129
00:05:47,400 --> 00:05:51,320
Speaker 3: up thinking about AI nowadays versus like, let's say two

130
00:05:51,440 --> 00:05:52,160
Speaker 3: years ago.

131
00:05:53,240 --> 00:05:56,320
Speaker 4: Well, two years ago, I would say it was also

132
00:05:56,600 --> 00:05:59,359
Speaker 4: still a fairly large about I think AI is a

133
00:05:59,400 --> 00:06:03,240
Speaker 4: deeply tryformative technology in what it opens up, and I

134
00:06:03,279 --> 00:06:06,200
Speaker 4: think AI has been in search for many years in

135
00:06:06,240 --> 00:06:09,400
Speaker 4: different forms. It's much more in the forefront these days

136
00:06:09,400 --> 00:06:12,120
Speaker 4: with things like aio reviews and AI mode, But if

137
00:06:12,160 --> 00:06:15,080
Speaker 4: you go back several years, it was how we transformed

138
00:06:15,080 --> 00:06:17,520
Speaker 4: a bunch of ranking with efforts like Burt and Mum

139
00:06:17,880 --> 00:06:20,480
Speaker 4: that were build on some of the early transformer breakthroughs.

140
00:06:21,000 --> 00:06:22,159
Speaker 4: But at the end of the day, you know, and

141
00:06:22,160 --> 00:06:24,320
Speaker 4: if you go back, once upon a time, AI didn't

142
00:06:24,360 --> 00:06:27,679
Speaker 4: just refer to generative AI referred to general machine learning

143
00:06:27,680 --> 00:06:29,520
Speaker 4: and other things like that. And you know, in the

144
00:06:29,880 --> 00:06:34,000
Speaker 4: early two thousands, Google had spell correction, which felt sort

145
00:06:34,000 --> 00:06:36,080
Speaker 4: of revolutionary at the time that used AI.

146
00:06:36,400 --> 00:06:38,119
Speaker 3: Right, no one calls spell check AI.

147
00:06:38,240 --> 00:06:41,480
Speaker 4: It's all, yeah, nobody calls spell check A but like

148
00:06:41,520 --> 00:06:43,839
Speaker 4: it was at the time, Right, she just shows you

149
00:06:43,880 --> 00:06:46,680
Speaker 4: how far the world has come. But the opportunity to

150
00:06:46,680 --> 00:06:50,400
Speaker 4: really transform search and realize Google's mission at a new

151
00:06:50,520 --> 00:06:53,400
Speaker 4: level is really exciting and an amazing opportunity and very

152
00:06:53,480 --> 00:06:56,320
Speaker 4: humbling to do this for a product that so many

153
00:06:56,360 --> 00:06:56,880
Speaker 4: people use.

154
00:06:57,160 --> 00:07:00,520
Speaker 2: By the way, you mentioned Bert, my little software hobby

155
00:07:00,600 --> 00:07:04,320
Speaker 2: project of training a machine learning model to tell whether

156
00:07:04,400 --> 00:07:07,040
Speaker 2: something is more indicative of the written or spoken word

157
00:07:07,279 --> 00:07:09,560
Speaker 2: is based on Burt. So thank you for developing that

158
00:07:09,680 --> 00:07:12,680
Speaker 2: and thank you for open sourcing it so that someone

159
00:07:12,800 --> 00:07:16,560
Speaker 2: like myself cannot train it. But talk to us about

160
00:07:16,720 --> 00:07:19,360
Speaker 2: just how you're thinking about this core tension that for

161
00:07:19,480 --> 00:07:22,840
Speaker 2: years Google had a business of you go to a

162
00:07:22,960 --> 00:07:25,200
Speaker 2: you put into a term in a search bar, and

163
00:07:25,240 --> 00:07:27,400
Speaker 2: then people click out and some people and some of

164
00:07:27,440 --> 00:07:29,880
Speaker 2: the clicks were to organic results, and some of the

165
00:07:29,920 --> 00:07:34,840
Speaker 2: clicks were to pay results. And now people were entering

166
00:07:34,920 --> 00:07:37,400
Speaker 2: this world in which people expect to get whatever they

167
00:07:37,440 --> 00:07:42,040
Speaker 2: want right there from the query without that impulse for

168
00:07:42,120 --> 00:07:44,280
Speaker 2: a click, and you have a business that is still

169
00:07:44,400 --> 00:07:48,280
Speaker 2: dominated by that clickout one way or another. So just

170
00:07:48,320 --> 00:07:50,560
Speaker 2: like you know, and I want to drive into the details,

171
00:07:50,680 --> 00:07:53,000
Speaker 2: but big picture, is that a real tension.

172
00:07:53,840 --> 00:07:56,720
Speaker 4: I think what's interesting about it is that the space

173
00:07:56,760 --> 00:07:58,840
Speaker 4: of search is very big, and what people are trying

174
00:07:58,840 --> 00:08:01,800
Speaker 4: to do is very big, and sometimes people really want

175
00:08:01,880 --> 00:08:03,720
Speaker 4: quick answers and they want it right in front of them,

176
00:08:03,840 --> 00:08:05,560
Speaker 4: and sometimes they want to go deep or they want

177
00:08:05,560 --> 00:08:09,040
Speaker 4: to hear from particular individuals. Right. I think there's this

178
00:08:09,160 --> 00:08:11,920
Speaker 4: sort of myth that people want AI or the web

179
00:08:13,080 --> 00:08:14,640
Speaker 4: that I actually think what we see is that people

180
00:08:14,680 --> 00:08:17,280
Speaker 4: want AI and the web together. Okay, there are certainly

181
00:08:17,400 --> 00:08:19,560
Speaker 4: questions for which like, you just want the quick answer

182
00:08:19,840 --> 00:08:22,280
Speaker 4: and then you're done, And that's been true in many

183
00:08:22,280 --> 00:08:24,559
Speaker 4: ways for years. Right, we'll talk about this with AI.

184
00:08:24,960 --> 00:08:26,840
Speaker 4: But I bet most of the time you look up

185
00:08:26,840 --> 00:08:28,960
Speaker 4: the weather, you just want to know what the temperature

186
00:08:29,000 --> 00:08:31,280
Speaker 4: is and you're done, right, and you don't once, But

187
00:08:31,320 --> 00:08:33,000
Speaker 4: then you're going to go on a trip, and you're

188
00:08:33,040 --> 00:08:35,360
Speaker 4: going to go and actually dig in more because you're

189
00:08:35,360 --> 00:08:38,319
Speaker 4: going surfing on other pieces. I think if you think

190
00:08:38,320 --> 00:08:40,000
Speaker 4: about this, people like, oh, well I have an answer,

191
00:08:40,080 --> 00:08:42,200
Speaker 4: I don't, so why would I do an ad? Well, like,

192
00:08:42,360 --> 00:08:45,160
Speaker 4: the answer doesn't buy the pair of shoes, You actually

193
00:08:45,200 --> 00:08:48,040
Speaker 4: have to buy the shoes, right, so you still have

194
00:08:48,080 --> 00:08:52,680
Speaker 4: to go pick a merchant for that. People care often

195
00:08:52,760 --> 00:08:55,800
Speaker 4: to hear people's perspectives, right, Like you'll talk about, okay,

196
00:08:55,840 --> 00:08:57,920
Speaker 4: well we want a bunch of answers. And yet this

197
00:08:58,080 --> 00:09:01,440
Speaker 4: is like a golden age for podcast right. So clearly

198
00:09:01,520 --> 00:09:04,320
Speaker 4: people sometimes want to spend a couple of seconds and

199
00:09:04,360 --> 00:09:07,360
Speaker 4: other times they'll spend a whole hour listening to things.

200
00:09:07,840 --> 00:09:09,800
Speaker 4: And so one of the things we see with the

201
00:09:09,840 --> 00:09:13,200
Speaker 4: shift with AI overviews is that you get more of

202
00:09:13,240 --> 00:09:16,080
Speaker 4: this pronouncement with what's your goal? Okay, if all you

203
00:09:16,120 --> 00:09:17,720
Speaker 4: were going to do was go to the web page,

204
00:09:17,800 --> 00:09:19,640
Speaker 4: see the fact, and immediately click back, you're going to

205
00:09:19,640 --> 00:09:21,320
Speaker 4: spend like a half a second on the page. Okay,

206
00:09:21,679 --> 00:09:24,120
Speaker 4: you see those things shift. But if what you were

207
00:09:24,120 --> 00:09:26,280
Speaker 4: going to go and do is read an article for

208
00:09:26,400 --> 00:09:30,000
Speaker 4: five minutes. You're still interested in reading that article for

209
00:09:30,080 --> 00:09:32,880
Speaker 4: five minutes, right, AIO might help you point to the

210
00:09:32,920 --> 00:09:35,640
Speaker 4: right page. So we see fewer bounce clicks where a

211
00:09:35,720 --> 00:09:37,440
Speaker 4: user would sort of go and immediately come back as

212
00:09:37,480 --> 00:09:39,760
Speaker 4: they weren't happy. You see people go though, and they

213
00:09:39,840 --> 00:09:41,920
Speaker 4: want to hear from other people. They want to hear

214
00:09:42,280 --> 00:09:46,280
Speaker 4: their expertise, their perspective, their unique take. I take fashion

215
00:09:46,320 --> 00:09:50,280
Speaker 4: as an interesting example. Sometimes if you hate fashion, then

216
00:09:50,480 --> 00:09:53,679
Speaker 4: like you love using chatbots to replace the need, right,

217
00:09:53,720 --> 00:09:55,960
Speaker 4: Like you didn't really want to spend any time, that's fine.

218
00:09:56,080 --> 00:09:57,800
Speaker 4: But if you were someone who was spending a lot

219
00:09:57,880 --> 00:10:01,280
Speaker 4: of time reading in fluencers and what their interests in

220
00:10:01,360 --> 00:10:03,760
Speaker 4: the fashion, you have not decided to replace that with

221
00:10:03,800 --> 00:10:06,200
Speaker 4: the chatbot, right, You're still going to want to hear

222
00:10:06,280 --> 00:10:10,280
Speaker 4: from those fashion taste makers. And so there's an opportunity

223
00:10:10,320 --> 00:10:13,959
Speaker 4: with aioverviews to help you get started and then make

224
00:10:14,000 --> 00:10:16,839
Speaker 4: it easy for you to dig in and connect. And

225
00:10:16,880 --> 00:10:20,120
Speaker 4: I think people's interest in connecting with other people is

226
00:10:20,679 --> 00:10:22,960
Speaker 4: just as strong these days in many ways.

227
00:10:23,520 --> 00:10:26,320
Speaker 3: So just at a simplistic level, can you tell me, like,

228
00:10:26,559 --> 00:10:29,880
Speaker 3: how does Google determine whether it shows the AI overview

229
00:10:30,240 --> 00:10:34,120
Speaker 3: or not so. If I type in Corgi into Google Search,

230
00:10:34,720 --> 00:10:36,719
Speaker 3: I'm biased because I have two Corkis, But it just

231
00:10:36,760 --> 00:10:38,640
Speaker 3: gives me a bunch of links to like the American

232
00:10:38,720 --> 00:10:41,920
Speaker 3: Kennel Club and like the Corky subreddit and things like that.

233
00:10:42,120 --> 00:10:44,640
Speaker 3: If I type in what is a Corgi question mark,

234
00:10:44,679 --> 00:10:47,360
Speaker 3: it gives me the AI overview. Is it just everything

235
00:10:47,360 --> 00:10:50,120
Speaker 3: with a question mark returns an AI result? Or how

236
00:10:50,160 --> 00:10:53,240
Speaker 3: are you actually deciding what to present to users?

237
00:10:54,520 --> 00:10:57,400
Speaker 4: No, what we try, and an important premise of this

238
00:10:57,760 --> 00:10:59,560
Speaker 4: is that we shouldn't give you AI for the sake

239
00:10:59,600 --> 00:11:02,959
Speaker 4: of giving you AI. Right, The point is for it

240
00:11:03,040 --> 00:11:05,600
Speaker 4: when we think it adds value to people, and so

241
00:11:05,679 --> 00:11:08,000
Speaker 4: it's not really associated with question marks. Question Marks are

242
00:11:08,040 --> 00:11:10,440
Speaker 4: often when people are looking for more of a description.

243
00:11:11,080 --> 00:11:13,559
Speaker 4: They have a harder question which maybe a single web

244
00:11:13,600 --> 00:11:16,760
Speaker 4: page doesn't answer whatever else. But what we're really using

245
00:11:16,880 --> 00:11:19,880
Speaker 4: is looking at signals from users to say does the

246
00:11:19,880 --> 00:11:24,080
Speaker 4: aioverview provide additional value or not? And so most people

247
00:11:24,960 --> 00:11:27,920
Speaker 4: I don't like, I haven't studied Corgi Kray in detail,

248
00:11:28,040 --> 00:11:31,120
Speaker 4: but for Cray like that, probably what we're seeing is

249
00:11:31,400 --> 00:11:33,200
Speaker 4: that most people aren't just trying to figure out what

250
00:11:33,280 --> 00:11:35,280
Speaker 4: is a korky. Maybe they want to see pictures about it.

251
00:11:35,320 --> 00:11:37,760
Speaker 4: They want to click on knowing more about the dog

252
00:11:37,800 --> 00:11:39,760
Speaker 4: breed because they're trying to engage in it. And so

253
00:11:40,320 --> 00:11:43,560
Speaker 4: we basically learn over time based on user signals the

254
00:11:43,559 --> 00:11:45,600
Speaker 4: same way we learn about like when should you show

255
00:11:45,640 --> 00:11:48,080
Speaker 4: the weather one box? And when should you show local results?

256
00:11:48,080 --> 00:11:51,360
Speaker 4: And when should you use sports? That the aioverview provides

257
00:11:51,360 --> 00:11:54,440
Speaker 4: additional value? Great, we show it. It doesn't. Then we

258
00:11:54,440 --> 00:11:56,040
Speaker 4: don't want to get out of the way, Right, you

259
00:11:56,080 --> 00:11:58,880
Speaker 4: don't want your search for Wikipedia. You go and you

260
00:11:58,920 --> 00:12:01,880
Speaker 4: type in Wikipedia. Which people do they want to get

261
00:12:01,880 --> 00:12:06,000
Speaker 4: to Wikipedia? They don't want to go and say, let's

262
00:12:06,000 --> 00:12:08,120
Speaker 4: give me the history of Wikipedia. That's not why they

263
00:12:08,120 --> 00:12:10,480
Speaker 4: search for that, right. If they search for odd lots, right,

264
00:12:10,480 --> 00:12:12,880
Speaker 4: they probably want to quickly get to your podcast. And

265
00:12:12,960 --> 00:12:15,080
Speaker 4: so we have a variety of signals that try and

266
00:12:15,160 --> 00:12:18,040
Speaker 4: help us understand when is an adding value a not

267
00:12:18,400 --> 00:12:22,480
Speaker 4: And we get smarter over time as people both change

268
00:12:22,480 --> 00:12:25,280
Speaker 4: how they ask questions as the models get smarter, right,

269
00:12:25,320 --> 00:12:27,000
Speaker 4: like we don't want to put an air overview if

270
00:12:27,040 --> 00:12:28,880
Speaker 4: we think it's not going to be high quality. So

271
00:12:28,920 --> 00:12:31,000
Speaker 4: as the models have gotten more powerful, we can cover

272
00:12:31,120 --> 00:12:34,120
Speaker 4: more cases and just continue to develop, really with the

273
00:12:34,120 --> 00:12:37,000
Speaker 4: focus being what is the best response to give a

274
00:12:37,080 --> 00:12:38,439
Speaker 4: user for the question they've asked.

275
00:12:54,440 --> 00:12:57,959
Speaker 2: I have a question about what you see among user behavior,

276
00:12:58,080 --> 00:13:01,520
Speaker 2: And my question is do you see the same user

277
00:13:02,480 --> 00:13:05,720
Speaker 2: or a cohort of people who use both Google dot

278
00:13:05,760 --> 00:13:10,959
Speaker 2: com and Gemini Google dot com And do you see

279
00:13:11,280 --> 00:13:15,120
Speaker 2: distinct patterns of queries from the same user but different

280
00:13:15,160 --> 00:13:19,000
Speaker 2: types of searches or do people just sort of throw

281
00:13:19,040 --> 00:13:21,800
Speaker 2: the question mostly in the Google search box and start

282
00:13:21,880 --> 00:13:24,160
Speaker 2: from there, or do you see people who just use

283
00:13:24,240 --> 00:13:26,560
Speaker 2: Gemini and do everything there, Like, what do you see

284
00:13:26,600 --> 00:13:30,640
Speaker 2: in terms of emergent patterns about how an individual chooses

285
00:13:30,679 --> 00:13:33,480
Speaker 2: which of the sites to enter into first and how

286
00:13:33,520 --> 00:13:35,320
Speaker 2: they do different queries in each one.

287
00:13:36,480 --> 00:13:39,160
Speaker 4: Yeah, so maybe just so we're all talked about the

288
00:13:39,200 --> 00:13:41,760
Speaker 4: same thing. There's sort of your main search page, there's

289
00:13:41,840 --> 00:13:44,320
Speaker 4: AI mode that's part of search, and then there's the

290
00:13:44,360 --> 00:13:48,360
Speaker 4: Gemini app right across and I would say, like, there's

291
00:13:48,360 --> 00:13:51,480
Speaker 4: a lot of users, so their behavior varies across all

292
00:13:51,559 --> 00:13:54,480
Speaker 4: of them, but there are some patterns. Okay, there's plenty

293
00:13:54,480 --> 00:13:57,559
Speaker 4: of people who co use across them. There's plenty of

294
00:13:57,600 --> 00:14:00,600
Speaker 4: people that are actually using several AI products right now,

295
00:14:00,640 --> 00:14:03,960
Speaker 4: just in general, right, not even just within Google, across Gemini.

296
00:14:04,040 --> 00:14:09,160
Speaker 4: In search, the more informational ones. If it's an informational query,

297
00:14:09,280 --> 00:14:11,640
Speaker 4: then the probability that they're using search or AI mode

298
00:14:11,679 --> 00:14:14,520
Speaker 4: is going to be higher. If it's a creative query,

299
00:14:14,720 --> 00:14:17,800
Speaker 4: it's a product like more of a productivity question. You know,

300
00:14:17,840 --> 00:14:20,040
Speaker 4: I want to like please rewrite this to make it

301
00:14:20,080 --> 00:14:23,600
Speaker 4: sound more formal. Right, those type questions are going to

302
00:14:23,600 --> 00:14:28,960
Speaker 4: be more Gemini oriented. Between AI mode and search, the

303
00:14:29,000 --> 00:14:32,200
Speaker 4: main short page. Some people use AI moode mostly via

304
00:14:32,280 --> 00:14:34,880
Speaker 4: AO reviews. They start in aio reviews and they transition.

305
00:14:35,280 --> 00:14:38,440
Speaker 4: For those who go direct to AI mode, they tend

306
00:14:38,480 --> 00:14:42,200
Speaker 4: to do that for queries that they consider sort of

307
00:14:42,240 --> 00:14:46,760
Speaker 4: more complex, longer questions, questions where they expect that they're

308
00:14:46,760 --> 00:14:49,400
Speaker 4: going to do more follow ups. Versus if you're doing

309
00:14:49,440 --> 00:14:52,320
Speaker 4: a very browsey querry, you might choose to prefer al

310
00:14:52,560 --> 00:14:54,560
Speaker 4: the syrup. If you know that, like your goal is

311
00:14:54,600 --> 00:14:57,240
Speaker 4: to just get to a particular web page, you're more

312
00:14:57,360 --> 00:15:00,200
Speaker 4: likely to start with the search result page. But you know,

313
00:15:00,200 --> 00:15:04,040
Speaker 4: there's obviously overlap in the use cases. But across search

314
00:15:04,080 --> 00:15:06,720
Speaker 4: and AI Moode tends to be sort of more longer complex,

315
00:15:06,880 --> 00:15:11,640
Speaker 4: more conversational queries versus more traditional queries, and between Gemini

316
00:15:11,840 --> 00:15:14,720
Speaker 4: and Search there's more of a productivity and creativity versus

317
00:15:14,800 --> 00:15:16,080
Speaker 4: information slant on them.

318
00:15:16,480 --> 00:15:19,360
Speaker 3: So, since we're talking about user behavior, one of the

319
00:15:19,360 --> 00:15:21,800
Speaker 3: things that seems to be happening now is people will

320
00:15:21,920 --> 00:15:26,000
Speaker 3: use an LM, it doesn't matter which one. They'll ask

321
00:15:26,040 --> 00:15:29,520
Speaker 3: a question and then they will go and fact check

322
00:15:29,680 --> 00:15:33,040
Speaker 3: the answer that they get on Google. And I'm really

323
00:15:33,080 --> 00:15:35,240
Speaker 3: curious if that's something that you're aware of as a

324
00:15:35,280 --> 00:15:38,640
Speaker 3: sort of user behavior, and if the ideas that maybe

325
00:15:38,680 --> 00:15:43,200
Speaker 3: Google becomes maybe not an AI overviewer per se, but

326
00:15:43,280 --> 00:15:45,960
Speaker 3: maybe the sort of fact checker of last resort for

327
00:15:46,040 --> 00:15:47,000
Speaker 3: other lms.

328
00:15:48,600 --> 00:15:51,960
Speaker 4: I think we're definitely aware that people use Google as

329
00:15:51,960 --> 00:15:54,720
Speaker 4: a fact checker for some of their LM use case.

330
00:15:55,160 --> 00:15:57,560
Speaker 4: I think people have used Google as a place to

331
00:15:57,640 --> 00:16:01,880
Speaker 4: fact check informations for a number of things. A friend

332
00:16:01,920 --> 00:16:05,320
Speaker 4: tells them something you know and sort of come. But

333
00:16:05,400 --> 00:16:07,960
Speaker 4: I think people use Search for a lot more than

334
00:16:08,080 --> 00:16:11,080
Speaker 4: just fact checking, right or even just looking up facts.

335
00:16:11,280 --> 00:16:13,480
Speaker 4: They want to go browse what they're going to go buy,

336
00:16:13,680 --> 00:16:15,600
Speaker 4: They want to go check up the sports score of

337
00:16:15,680 --> 00:16:19,640
Speaker 4: the latest team, and we do see with aio reviews

338
00:16:19,720 --> 00:16:22,000
Speaker 4: that with the presence of AI reviews, people are asking

339
00:16:22,080 --> 00:16:25,360
Speaker 4: more longer questions, They're asking more conversational questions, and so

340
00:16:25,720 --> 00:16:28,000
Speaker 4: some of these questions they started bringing to an LM.

341
00:16:28,440 --> 00:16:31,080
Speaker 4: As we brought aio reviews in, they took on those

342
00:16:31,080 --> 00:16:33,640
Speaker 4: same types of questions and have brought them to search.

343
00:16:34,640 --> 00:16:37,360
Speaker 2: One of the complaints about search, and I would say

344
00:16:37,400 --> 00:16:41,240
Speaker 2: I've complained about this or complained about is how many

345
00:16:41,360 --> 00:16:45,480
Speaker 2: of the results in the serp are like very almost

346
00:16:45,520 --> 00:16:48,920
Speaker 2: like two timely. Let's say someone enters the news, there's

347
00:16:48,960 --> 00:16:51,640
Speaker 2: a headline about so and so, and I'm like, I'm

348
00:16:51,640 --> 00:16:54,640
Speaker 2: really curious about this person and so I like search

349
00:16:54,760 --> 00:16:58,600
Speaker 2: their name and I get like ten results or however

350
00:16:58,600 --> 00:17:01,560
Speaker 2: many results all from the last day since they made

351
00:17:01,600 --> 00:17:04,639
Speaker 2: the news, and it actually is difficult for me to

352
00:17:04,920 --> 00:17:08,680
Speaker 2: find information that didn't have the context of the news.

353
00:17:08,680 --> 00:17:11,440
Speaker 2: So it was like unbiased in some way. Is that

354
00:17:11,640 --> 00:17:14,040
Speaker 2: recognized as an issue, because I certainly feel it as

355
00:17:14,080 --> 00:17:17,600
Speaker 2: a user, But I'm curious from your perspective at Google,

356
00:17:17,920 --> 00:17:20,280
Speaker 2: whether this is something that you think about as a

357
00:17:20,320 --> 00:17:23,159
Speaker 2: way in which search becomes less than ideal.

358
00:17:23,840 --> 00:17:28,760
Speaker 4: I think one of the things that's generally very challenging

359
00:17:28,800 --> 00:17:32,560
Speaker 4: about search is that people enter the same querry with

360
00:17:32,600 --> 00:17:35,560
Speaker 4: a like that's often very short, with a lot of

361
00:17:35,600 --> 00:17:38,600
Speaker 4: different intents in mind. Right, and so in a bunch

362
00:17:38,600 --> 00:17:42,000
Speaker 4: of the examples where you have probably our data says

363
00:17:42,080 --> 00:17:45,119
Speaker 4: that most people just want to see the recent articles

364
00:17:45,160 --> 00:17:46,800
Speaker 4: and those are the ones that get all of the clicks.

365
00:17:46,800 --> 00:17:50,560
Speaker 4: But you didn't, right, And so how do we figure

366
00:17:50,600 --> 00:17:55,040
Speaker 4: out across all of the different intents and match them across?

367
00:17:55,480 --> 00:17:58,600
Speaker 4: And so I think there's this question about both how

368
00:17:58,600 --> 00:18:01,440
Speaker 4: do you get the different facets of a question? How

369
00:18:01,480 --> 00:18:04,760
Speaker 4: can you personalize the results more effectively for people? I

370
00:18:04,840 --> 00:18:08,920
Speaker 4: do see more this is anecdotal as opposed to complete data.

371
00:18:09,000 --> 00:18:12,040
Speaker 4: But to your example, people using AI mode and AIO

372
00:18:12,119 --> 00:18:14,720
Speaker 4: reviews for some of the people queries to like understand

373
00:18:14,720 --> 00:18:17,359
Speaker 4: more about the person independent of the rest of the

374
00:18:17,400 --> 00:18:19,560
Speaker 4: news articles, because you know, if you don't know who

375
00:18:19,800 --> 00:18:21,480
Speaker 4: most of the people searching for it know who the

376
00:18:21,520 --> 00:18:23,600
Speaker 4: person is, but some of the people searching for it don't,

377
00:18:23,920 --> 00:18:26,760
Speaker 4: and so you can see behavior like that. But I

378
00:18:26,800 --> 00:18:32,200
Speaker 4: do think one of the interesting things about the evolution

379
00:18:32,280 --> 00:18:35,840
Speaker 4: with AI is that that people stop talking justin keyword

380
00:18:36,000 --> 00:18:38,280
Speaker 4: es as much and they start expressing more of what

381
00:18:38,320 --> 00:18:40,679
Speaker 4: they want, and then that becomes much easier for us

382
00:18:40,680 --> 00:18:43,440
Speaker 4: to give an answer. Right, So if you say tell

383
00:18:43,480 --> 00:18:47,200
Speaker 4: me about someone versus like, what's new with someone, that's

384
00:18:47,240 --> 00:18:49,359
Speaker 4: actually much easier for us to figure out how to

385
00:18:49,400 --> 00:18:51,439
Speaker 4: give better results than if all we just say is

386
00:18:51,600 --> 00:18:55,680
Speaker 4: someone right. We used to talk about the an example

387
00:18:55,680 --> 00:18:58,000
Speaker 4: createing a different ways is falafful. What do you want

388
00:18:58,000 --> 00:18:59,960
Speaker 4: to know with flaffel? Some people don't know what flafful is.

389
00:19:00,040 --> 00:19:02,680
Speaker 4: They want a definition, some people want recipes, some people

390
00:19:02,680 --> 00:19:05,320
Speaker 4: want to find where to eat, some people want nutritional information.

391
00:19:06,040 --> 00:19:10,879
Speaker 4: They all just use the word falafful, and that's just

392
00:19:10,960 --> 00:19:13,239
Speaker 4: harder to figure out how across all of them we

393
00:19:13,320 --> 00:19:15,760
Speaker 4: do that. We also see that with tensions on things

394
00:19:15,800 --> 00:19:17,800
Speaker 4: like well do you want video results or do you

395
00:19:17,840 --> 00:19:21,600
Speaker 4: want more text based results? People are very opinionated about

396
00:19:21,600 --> 00:19:23,239
Speaker 4: what the right answer is, but they are not very

397
00:19:23,280 --> 00:19:26,080
Speaker 4: opinionated in the same direction, and so we try and meet,

398
00:19:26,359 --> 00:19:28,640
Speaker 4: you know, multiple billions of people's needs at once.

399
00:19:29,160 --> 00:19:32,320
Speaker 2: Just to be clear, the flafful example is great. Would

400
00:19:32,359 --> 00:19:35,840
Speaker 2: you say that today you see a greater diversity of

401
00:19:35,920 --> 00:19:39,160
Speaker 2: falafel related queries, whereas maybe like five or ten years ago,

402
00:19:39,200 --> 00:19:41,880
Speaker 2: you just get falafful, and now people know that they

403
00:19:41,880 --> 00:19:45,440
Speaker 2: could type in where is where does falafel come from?

404
00:19:45,520 --> 00:19:48,480
Speaker 2: What is a falafel, falafel recipe, et cetera. Have people

405
00:19:48,520 --> 00:19:53,760
Speaker 2: gotten more sophisticated over time in their query specifications about falafel?

406
00:19:55,480 --> 00:19:58,679
Speaker 4: I don't know falafful very specifically, but in general, but

407
00:19:58,760 --> 00:20:02,360
Speaker 4: in general, yes, okay. We have seen with AIO reviews

408
00:20:02,440 --> 00:20:06,840
Speaker 4: meaningfully longer queries. We see more natural language queries, but

409
00:20:06,920 --> 00:20:09,439
Speaker 4: it's also not even something as basic of that. It

410
00:20:09,480 --> 00:20:13,119
Speaker 4: can also be like you were searching for restaurants. We

411
00:20:13,240 --> 00:20:16,320
Speaker 4: used to laugh about the like before I worked on search,

412
00:20:16,359 --> 00:20:19,080
Speaker 4: I worked on maps and local some of the intersection

413
00:20:19,160 --> 00:20:21,200
Speaker 4: with search, and people would just be like restaurants in

414
00:20:21,200 --> 00:20:23,280
Speaker 4: New York and you're like, what do you want me

415
00:20:23,320 --> 00:20:26,200
Speaker 4: to do with that querry? Right? Okay, the best restaurants

416
00:20:26,200 --> 00:20:28,159
Speaker 4: in New York are going to take three months in

417
00:20:28,280 --> 00:20:30,879
Speaker 4: ninety nine point nine percent of the population can't afford

418
00:20:31,520 --> 00:20:33,760
Speaker 4: to go to them. Okay, but like, are you picking

419
00:20:33,800 --> 00:20:36,000
Speaker 4: ten random ones? Et cetera. But like, part of why

420
00:20:36,000 --> 00:20:37,440
Speaker 4: people will do that is they had a much more

421
00:20:37,440 --> 00:20:41,200
Speaker 4: compact I want a restaurant in this location for five people.

422
00:20:41,760 --> 00:20:44,320
Speaker 4: It can't be too pricey. I have a vegan member,

423
00:20:44,440 --> 00:20:46,840
Speaker 4: I also have like I have kids. That was the

424
00:20:46,920 --> 00:20:49,520
Speaker 4: question they had in their mind. And in the old

425
00:20:49,520 --> 00:20:52,000
Speaker 4: word of keywordies, none of that information would be kind

426
00:20:52,000 --> 00:20:54,160
Speaker 4: of spread throughout the web, and so you wouldn't feel

427
00:20:54,160 --> 00:20:56,960
Speaker 4: confident you could just put in the question. And now

428
00:20:57,000 --> 00:20:59,720
Speaker 4: with like a using amo, you can start to actually

429
00:21:00,160 --> 00:21:02,520
Speaker 4: and you see people do this, they tell you the

430
00:21:02,600 --> 00:21:05,920
Speaker 4: real problem. Right. They don't take their need and translate

431
00:21:05,920 --> 00:21:08,840
Speaker 4: it to what the computer understands. They try to give

432
00:21:08,960 --> 00:21:12,720
Speaker 4: the computer their actual need and expect us to do

433
00:21:12,800 --> 00:21:16,040
Speaker 4: the translation. And I think that's really exciting to see

434
00:21:16,080 --> 00:21:18,359
Speaker 4: because one we can be more helpful, but also like

435
00:21:18,400 --> 00:21:21,120
Speaker 4: those are real problems people had. If you go back

436
00:21:21,119 --> 00:21:23,520
Speaker 4: to the mission, it was organized the world's information and

437
00:21:23,560 --> 00:21:26,439
Speaker 4: make it universally accessible and useful. Like that useful part, right,

438
00:21:26,440 --> 00:21:28,840
Speaker 4: It's not just that it's organized, is it useful to you?

439
00:21:29,320 --> 00:21:31,080
Speaker 4: And I think one of the most exciting things about

440
00:21:31,119 --> 00:21:33,840
Speaker 4: AI the transformation going on right now, is that you

441
00:21:33,880 --> 00:21:37,000
Speaker 4: can actually make information much more useful to people. And

442
00:21:37,440 --> 00:21:39,920
Speaker 4: that really opens up that makes it so that people

443
00:21:39,960 --> 00:21:41,919
Speaker 4: just ask more questions because we can actually do a

444
00:21:41,920 --> 00:21:43,160
Speaker 4: better job meeting their needs.

445
00:21:43,960 --> 00:21:47,439
Speaker 3: Does that come with any like complications in terms of

446
00:21:47,480 --> 00:21:52,240
Speaker 3: privacy or competition for Google? If people aren't using keywords

447
00:21:52,280 --> 00:21:55,959
Speaker 3: as much anymore, if they're doing basically like query brain

448
00:21:56,160 --> 00:21:59,120
Speaker 3: dumps into the prompt and saying, you know, I am

449
00:21:59,160 --> 00:22:01,639
Speaker 3: so and so, I have a kid, I live here,

450
00:22:01,880 --> 00:22:04,440
Speaker 3: I want to do the following things. This is my issue,

451
00:22:05,240 --> 00:22:07,520
Speaker 3: is that like an added layer of complexity that you

452
00:22:07,600 --> 00:22:10,760
Speaker 3: have to deal with as like a large search company.

453
00:22:11,880 --> 00:22:15,199
Speaker 4: I mean, I think people from a privacy perspective, we

454
00:22:15,240 --> 00:22:17,359
Speaker 4: give people sort of a range of different things. They

455
00:22:17,400 --> 00:22:20,320
Speaker 4: can be sort of an incognito, they can be signed out,

456
00:22:20,400 --> 00:22:24,159
Speaker 4: they can be signed in across so and I think

457
00:22:24,240 --> 00:22:27,440
Speaker 4: Google has a long tradition of really treating people's data

458
00:22:28,000 --> 00:22:31,000
Speaker 4: with a great deal of care and having cutting edge

459
00:22:31,160 --> 00:22:35,240
Speaker 4: security and privacy by design. So I think people are

460
00:22:35,240 --> 00:22:38,480
Speaker 4: seeing the value and they have continued their trust in Google.

461
00:22:39,160 --> 00:22:42,960
Speaker 4: I think it means it's a harder job on quality. Right.

462
00:22:43,040 --> 00:22:45,240
Speaker 4: You have to take this question. There's many parts and

463
00:22:45,280 --> 00:22:46,840
Speaker 4: you have to figure out how you break it apart,

464
00:22:47,600 --> 00:22:50,240
Speaker 4: and you have to do work to think about things

465
00:22:50,280 --> 00:22:53,080
Speaker 4: like latency because you can't just you know, if everyone

466
00:22:53,160 --> 00:22:55,520
Speaker 4: uses the same keyword and it's not personalized, then you

467
00:22:55,520 --> 00:22:57,280
Speaker 4: can cash it all. If all of a sudden the

468
00:22:57,280 --> 00:23:00,919
Speaker 4: carrieres get much more diverse, it has consequences there. But

469
00:23:01,040 --> 00:23:04,000
Speaker 4: I think we just see that it's very empowering people

470
00:23:04,200 --> 00:23:06,440
Speaker 4: right that it takes some of the workout of searching.

471
00:23:07,080 --> 00:23:10,200
Speaker 4: I think sometimes people think, oh, a few years ago

472
00:23:10,200 --> 00:23:11,840
Speaker 4: they said like, oh, well more can you do with

473
00:23:11,880 --> 00:23:14,160
Speaker 4: Google Search? But like, if you actually ask them, Okay,

474
00:23:14,160 --> 00:23:16,240
Speaker 4: when was the last time you spent twenty minutes searching

475
00:23:16,280 --> 00:23:19,560
Speaker 4: when you would have preferred to spend two, it's actually

476
00:23:19,600 --> 00:23:21,479
Speaker 4: not that hard for me. People, Oh, last time I

477
00:23:21,520 --> 00:23:23,680
Speaker 4: was trying to like go find a service. Prior the

478
00:23:23,720 --> 00:23:25,520
Speaker 4: last time I was trying to go, like do these

479
00:23:25,560 --> 00:23:28,240
Speaker 4: bigger tasks in life. And so it's been kind of

480
00:23:28,280 --> 00:23:32,320
Speaker 4: exciting to just make people's lives easier by helping them

481
00:23:32,359 --> 00:23:33,280
Speaker 4: address their real need.

482
00:23:34,880 --> 00:23:37,560
Speaker 2: I have so many different theoretical questions I could ask.

483
00:23:37,640 --> 00:23:39,639
Speaker 2: Here's one, Actually, you know what this is something that

484
00:23:39,760 --> 00:23:42,639
Speaker 2: was inspired by our producer Dash, a conversation that I

485
00:23:42,720 --> 00:23:45,320
Speaker 2: have with ten minutes ago prior to this and related

486
00:23:45,359 --> 00:23:50,879
Speaker 2: to something else. I imagine in your career at Google, spending

487
00:23:51,040 --> 00:23:53,720
Speaker 2: over two decades at this point, you've been involved in

488
00:23:53,960 --> 00:23:58,080
Speaker 2: quite a bit of recruiting and recruiting software engineers in particular,

489
00:23:58,119 --> 00:24:01,680
Speaker 2: and imagine that's particularly important and aspect in somewhere or

490
00:24:01,680 --> 00:24:05,480
Speaker 2: another for the VP of surge, given what we've seen

491
00:24:05,600 --> 00:24:09,440
Speaker 2: with AI coding and so forth. When you're doing one

492
00:24:09,480 --> 00:24:14,679
Speaker 2: of these leak code software developer interviews, et cetera, is

493
00:24:14,720 --> 00:24:17,159
Speaker 2: it different today than five years ago? Do you have

494
00:24:17,200 --> 00:24:21,280
Speaker 2: to think really differently about the battery of technical questions

495
00:24:21,520 --> 00:24:25,439
Speaker 2: that you would propose to as software engineer today, given

496
00:24:25,600 --> 00:24:28,320
Speaker 2: the fund just the restructuring of the nature of the

497
00:24:28,359 --> 00:24:30,320
Speaker 2: job and the world of AI generated code.

498
00:24:30,680 --> 00:24:33,320
Speaker 4: I think the process is definitely evolving. I wouldn't say

499
00:24:33,320 --> 00:24:36,160
Speaker 4: that we have perfected the science yet, Okay, But there's

500
00:24:36,200 --> 00:24:38,280
Speaker 4: two angles in which you're thinking about it right. One

501
00:24:38,400 --> 00:24:41,320
Speaker 4: is you don't want to ask questions for which they

502
00:24:41,400 --> 00:24:43,159
Speaker 4: just go and type in the answer and the chatbot

503
00:24:43,200 --> 00:24:44,920
Speaker 4: and recite it back to you. Okay. So you need

504
00:24:44,960 --> 00:24:47,800
Speaker 4: to make sure to the extent that your goal is

505
00:24:47,840 --> 00:24:50,119
Speaker 4: to understand are they critically thinking? Are they able to

506
00:24:50,160 --> 00:24:52,320
Speaker 4: think through a problem and do that? You want to

507
00:24:52,320 --> 00:24:55,119
Speaker 4: make sure that that's actually what you're assessing, right, And

508
00:24:55,160 --> 00:24:58,399
Speaker 4: so is in person? How are you doing that on

509
00:24:58,480 --> 00:25:01,040
Speaker 4: some basic way? But there's the the other thing that

510
00:25:01,560 --> 00:25:04,439
Speaker 4: I think the tools are powerful. You can use them

511
00:25:04,480 --> 00:25:06,439
Speaker 4: in ways that make you more effective, and you can

512
00:25:06,520 --> 00:25:09,160
Speaker 4: use them in ways that make you less productive. How

513
00:25:09,200 --> 00:25:12,200
Speaker 4: do you think as the fluency is changing with AI,

514
00:25:12,520 --> 00:25:14,440
Speaker 4: the way a software engineer might approach the problem now

515
00:25:14,480 --> 00:25:16,640
Speaker 4: is different than they might have approached the problem five

516
00:25:16,720 --> 00:25:19,600
Speaker 4: years ago without some of these tools. And so I

517
00:25:19,600 --> 00:25:23,360
Speaker 4: think we're all learning how to change asking that question, right,

518
00:25:23,400 --> 00:25:25,359
Speaker 4: are you building up that expertise? Are you building up

519
00:25:25,359 --> 00:25:29,040
Speaker 4: that fluency? And the fluency isn't fixed. What was possible

520
00:25:29,080 --> 00:25:33,080
Speaker 4: with the tools six months ago, let alone two years ago,

521
00:25:33,480 --> 00:25:36,320
Speaker 4: is different than what's possible now, and in six months

522
00:25:36,320 --> 00:25:38,240
Speaker 4: it will be different. So you have to start thinking

523
00:25:38,280 --> 00:25:42,480
Speaker 4: about how part of your interview is thinking about sort

524
00:25:42,480 --> 00:25:44,440
Speaker 4: of fluency with the use of tools in the same

525
00:25:44,480 --> 00:25:48,600
Speaker 4: way that like when IDs became important, or when people

526
00:25:48,600 --> 00:25:51,199
Speaker 4: stop using Assembly language and they started doing in Java. Right,

527
00:25:51,240 --> 00:25:54,119
Speaker 4: you had to sort of evolve the interviews. It's just

528
00:25:54,160 --> 00:25:58,359
Speaker 4: that it's happening very fast, okay, right, and so we

529
00:25:58,440 --> 00:26:00,560
Speaker 4: all have to be on our toes exciting, Like you

530
00:26:00,560 --> 00:26:02,240
Speaker 4: play with this tool that it doesn't work for something,

531
00:26:02,240 --> 00:26:04,199
Speaker 4: and then and then it's not like play with it

532
00:26:04,200 --> 00:26:06,160
Speaker 4: two years later, it's like play with it three months later.

533
00:26:06,520 --> 00:26:08,520
Speaker 4: Maybe the tool will now work for these things.

534
00:26:08,840 --> 00:26:12,280
Speaker 3: Okay, well, speaking of tools, I mean one of the things, Joe,

535
00:26:12,320 --> 00:26:14,600
Speaker 3: I think you've said this. We've been playing around with

536
00:26:14,640 --> 00:26:18,080
Speaker 3: Claude code, this idea that actually, when you start vibe

537
00:26:18,080 --> 00:26:20,879
Speaker 3: coding everything and telling your agent to do everything, it

538
00:26:20,880 --> 00:26:24,240
Speaker 3: feels like you don't even necessarily need like a computer,

539
00:26:24,640 --> 00:26:28,280
Speaker 3: much less a search engine presumably. So I'm just curious,

540
00:26:28,280 --> 00:26:31,040
Speaker 3: if you gaze five or ten years into the future,

541
00:26:31,760 --> 00:26:35,359
Speaker 3: what do you think the default entry point for interacting

542
00:26:35,480 --> 00:26:37,919
Speaker 3: with the web is actually going to be. Is it

543
00:26:37,960 --> 00:26:39,879
Speaker 3: going to be a search engine like Google? Is it

544
00:26:39,920 --> 00:26:42,399
Speaker 3: going to be a specific LM. Is it going to

545
00:26:42,400 --> 00:26:45,480
Speaker 3: be my personal agent that I've vibe coded for all

546
00:26:45,520 --> 00:26:46,560
Speaker 3: of my preferences.

547
00:26:47,320 --> 00:26:49,000
Speaker 2: Can I just add on to this question? This is

548
00:26:49,000 --> 00:26:50,480
Speaker 2: something i'd think about, Like, if I want to send

549
00:26:50,520 --> 00:26:52,919
Speaker 2: an email to Tracy today, then what I have to do,

550
00:26:53,000 --> 00:26:56,000
Speaker 2: like I find the tab and my browser, I scroll

551
00:26:56,000 --> 00:27:00,160
Speaker 2: over there, Okay, that's my Gmail tab, etc. Whatever. Would

552
00:27:00,200 --> 00:27:02,760
Speaker 2: like to just be in my terminal It'd be so

553
00:27:02,840 --> 00:27:05,240
Speaker 2: much easier to say, here, send an email to Tracy

554
00:27:05,320 --> 00:27:08,000
Speaker 2: saying this, there's all these steps that I currently do

555
00:27:08,160 --> 00:27:12,040
Speaker 2: because of the nature of graphical user interfaces that now

556
00:27:12,040 --> 00:27:15,080
Speaker 2: that I've gotten in like clawed coating or whatever.

557
00:27:15,080 --> 00:27:16,000
Speaker 4: Feel a little clunky.

558
00:27:16,040 --> 00:27:19,199
Speaker 2: It feels little yesterday. And so yeah, I'm extremely curious

559
00:27:19,200 --> 00:27:22,560
Speaker 2: about will the web, with these series of boxes that

560
00:27:22,640 --> 00:27:25,600
Speaker 2: we drag and drop, et cetera, is that the future

561
00:27:25,800 --> 00:27:28,840
Speaker 2: or will just be someone talking in English to their computer.

562
00:27:29,960 --> 00:27:31,840
Speaker 4: I don't think like ten years is a long time

563
00:27:31,920 --> 00:27:34,000
Speaker 4: right now where the tech is. That's rights.

564
00:27:34,080 --> 00:27:40,000
Speaker 3: In one year, we still have browsers and pretty much people.

565
00:27:39,880 --> 00:27:42,200
Speaker 4: Like, okay, we believe in ten years will achieve a yeah,

566
00:27:42,200 --> 00:27:44,520
Speaker 4: I will anyone be doing anything the same? Okay. So

567
00:27:44,680 --> 00:27:46,720
Speaker 4: with that aside, I think there are some things I

568
00:27:47,359 --> 00:27:49,080
Speaker 4: believe in in some things I think we don't know.

569
00:27:49,600 --> 00:27:53,800
Speaker 4: I think you already see if you go back ten

570
00:27:54,320 --> 00:27:57,520
Speaker 4: twenty years ago, that the way you interact with the

571
00:27:57,560 --> 00:27:59,359
Speaker 4: tech has evolved a bunch, right. It used to be

572
00:27:59,400 --> 00:28:01,520
Speaker 4: it was just the laptop. Well now it's the phone.

573
00:28:01,520 --> 00:28:03,840
Speaker 4: Well now it's also the watch. Okay, in some cases

574
00:28:03,840 --> 00:28:06,960
Speaker 4: it's the glasses. This sense that it should feel like

575
00:28:07,400 --> 00:28:10,600
Speaker 4: the information is sort of at your fingertips in whatever

576
00:28:10,760 --> 00:28:15,080
Speaker 4: medium is useful, right, And but I don't know that

577
00:28:15,119 --> 00:28:18,639
Speaker 4: this becomes a we haven't so far replaced all of

578
00:28:18,680 --> 00:28:21,119
Speaker 4: the old ones, right, Like you use the phone a

579
00:28:21,160 --> 00:28:24,520
Speaker 4: lot more. But my guess is you're not doing all

580
00:28:24,560 --> 00:28:26,840
Speaker 4: your cloud code work on your phone, and you're doing

581
00:28:26,920 --> 00:28:30,040
Speaker 4: some of it on a desktop. That's true, right. The

582
00:28:30,080 --> 00:28:33,000
Speaker 4: introduction of the watch has supplemented, but it hasn't eliminated

583
00:28:33,000 --> 00:28:35,760
Speaker 4: the desktop, right. So what's been interesting actually is that

584
00:28:35,800 --> 00:28:39,520
Speaker 4: it hasn't gone in the direction of converging to the answer.

585
00:28:40,560 --> 00:28:43,160
Speaker 4: It's actually increased the form factors, and so that you

586
00:28:43,200 --> 00:28:45,960
Speaker 4: want to be able to access this information wherever you are,

587
00:28:46,040 --> 00:28:49,040
Speaker 4: in whatever form factor makes sense. And so will it

588
00:28:49,040 --> 00:28:52,200
Speaker 4: be glasses, will it be something else? Quite possibly, But

589
00:28:52,280 --> 00:28:55,280
Speaker 4: let's say let's even say it's glasses become a big deal.

590
00:28:55,480 --> 00:28:58,720
Speaker 4: Glasses are very small screens even there, you're probably not

591
00:28:58,760 --> 00:29:01,200
Speaker 4: going to do your big productivity on desktop. So I

592
00:29:01,240 --> 00:29:03,960
Speaker 4: think what you'll see is that the access point is

593
00:29:04,000 --> 00:29:07,640
Speaker 4: not confined to one thing, but that the key is

594
00:29:07,680 --> 00:29:10,840
Speaker 4: to eliminate the friction, right, and the toil to your point,

595
00:29:10,880 --> 00:29:13,040
Speaker 4: you had to do six steps. You didn't want to

596
00:29:13,040 --> 00:29:15,720
Speaker 4: do the six steps? Why should you do the six steps? Okay?

597
00:29:16,280 --> 00:29:18,920
Speaker 4: I think you see that, Like some things are much

598
00:29:18,960 --> 00:29:22,200
Speaker 4: easier to do with a chat interface, and then some things,

599
00:29:23,320 --> 00:29:26,600
Speaker 4: actually a chat interface is a super slow way to

600
00:29:26,680 --> 00:29:28,520
Speaker 4: go do right, Like if you have a list and

601
00:29:28,600 --> 00:29:32,520
Speaker 4: you have to go say please remove this long title

602
00:29:32,600 --> 00:29:35,800
Speaker 4: for the tenth item, that's actually much harder to do

603
00:29:35,880 --> 00:29:39,640
Speaker 4: with chat than like an interface that does that, right.

604
00:29:39,880 --> 00:29:42,040
Speaker 4: So I don't think it necessarily converges in a single thing.

605
00:29:42,400 --> 00:29:44,840
Speaker 4: I do think it should feel much more adaptive to

606
00:29:44,880 --> 00:29:46,960
Speaker 4: your point about, well, if this is the way you

607
00:29:47,040 --> 00:29:49,640
Speaker 4: prefer to interact, not just where you are, but how

608
00:29:49,680 --> 00:29:52,240
Speaker 4: you interact, and can you customize and can you create

609
00:29:52,840 --> 00:29:55,360
Speaker 4: to what extent do the user interfaces look designed for

610
00:29:55,440 --> 00:29:57,920
Speaker 4: you versus look designed for general and can you have

611
00:29:57,960 --> 00:30:01,000
Speaker 4: influence in them? I think you'll see that. I do

612
00:30:01,080 --> 00:30:05,840
Speaker 4: think we sometimes like we're very aware of what doesn't

613
00:30:05,880 --> 00:30:08,880
Speaker 4: work well, we're not necessarily aware of what does work well.

614
00:30:08,920 --> 00:30:11,719
Speaker 4: Like companies spend huge efforts working on how do they

615
00:30:11,720 --> 00:30:15,440
Speaker 4: do shopping carts really well? Yeah, Okay, this belief that

616
00:30:15,560 --> 00:30:18,360
Speaker 4: sort of the chatbot will have a more optimized shopping

617
00:30:18,400 --> 00:30:21,520
Speaker 4: cart for every shopping cart place in the world than

618
00:30:21,560 --> 00:30:23,120
Speaker 4: the one you go to every day. I don't know,

619
00:30:23,800 --> 00:30:27,840
Speaker 4: not clear right for those things, but I do think

620
00:30:27,880 --> 00:30:29,880
Speaker 4: it should feel much more personal, It should feel much

621
00:30:29,920 --> 00:30:32,800
Speaker 4: more dynamic, It should feel much more ambient and available

622
00:30:32,840 --> 00:30:34,960
Speaker 4: to you. And I don't think it will be one

623
00:30:34,960 --> 00:30:38,080
Speaker 4: size fits all, either per person or per form factor.

624
00:30:53,920 --> 00:30:56,240
Speaker 2: I've been reading some articles there. I think I saw

625
00:30:56,520 --> 00:30:59,200
Speaker 2: there was a big one in the information recently. Let's

626
00:30:59,200 --> 00:31:02,880
Speaker 2: talk about when you're copetitors meta kind of Yeah, it's

627
00:31:02,920 --> 00:31:06,160
Speaker 2: a competitor and it was like everyone's token maxing there,

628
00:31:06,200 --> 00:31:09,200
Speaker 2: and there's a token leader board and people are competing

629
00:31:09,280 --> 00:31:12,440
Speaker 2: to show that they're using AI more than others. And

630
00:31:12,480 --> 00:31:16,120
Speaker 2: from my perspective, that boggles my mind, because compute is

631
00:31:16,160 --> 00:31:19,239
Speaker 2: a cost, and just using compute per se does not

632
00:31:19,320 --> 00:31:22,280
Speaker 2: strike me as a particularly good way of measuring who

633
00:31:22,320 --> 00:31:25,400
Speaker 2: is productively contributing to the company. I mean, I could

634
00:31:25,840 --> 00:31:28,680
Speaker 2: certainly find an easy, quick, recursive way to burn.

635
00:31:28,480 --> 00:31:32,360
Speaker 3: Tokens, generate a bunch of AI yeah generator.

636
00:31:32,040 --> 00:31:34,360
Speaker 2: And then tell the tell it create one that just

637
00:31:34,440 --> 00:31:37,760
Speaker 2: keeps selling it to improve itself, et cetera. The flip side,

638
00:31:37,760 --> 00:31:40,040
Speaker 2: which some people say is like look it doesn't matter

639
00:31:40,080 --> 00:31:42,960
Speaker 2: at this point because everyone has to figure out how

640
00:31:43,000 --> 00:31:45,280
Speaker 2: they're going to use AI productively in their work. So,

641
00:31:45,360 --> 00:31:48,360
Speaker 2: you know what, don't even worry about metering AI. Tell

642
00:31:48,400 --> 00:31:51,400
Speaker 2: everyone to pedal to the metal and AI use And

643
00:31:51,480 --> 00:31:53,680
Speaker 2: if someone is maxing out on tokens, it means they're

644
00:31:53,680 --> 00:31:56,360
Speaker 2: experimenting with something and then they'll find something that really

645
00:31:56,680 --> 00:32:01,000
Speaker 2: is a productivity enhancer. I'm curious if from your perspective

646
00:32:01,720 --> 00:32:06,760
Speaker 2: it makes sense to like essentially see token consumption or

647
00:32:06,800 --> 00:32:10,560
Speaker 2: compute use as a proxy for like someone who's doing

648
00:32:10,600 --> 00:32:11,640
Speaker 2: their job aggressively.

649
00:32:12,240 --> 00:32:15,360
Speaker 4: Well, I think the thing with all of these proxy

650
00:32:15,440 --> 00:32:18,080
Speaker 4: metrics is if you use them blindly, you're going to

651
00:32:18,160 --> 00:32:20,880
Speaker 4: run yourself into trouble. Yeah, okay, if you, as a

652
00:32:20,960 --> 00:32:23,520
Speaker 4: leader sort of don't use judgment on them, then you

653
00:32:23,680 --> 00:32:26,640
Speaker 4: get the example of like I will just create a

654
00:32:26,720 --> 00:32:29,080
Speaker 4: job that runs in the background and does dumb things

655
00:32:29,160 --> 00:32:33,160
Speaker 4: to do the tokens. Right, So as a leader, your

656
00:32:33,240 --> 00:32:35,560
Speaker 4: job is to sort of use good judgment and not

657
00:32:35,680 --> 00:32:38,280
Speaker 4: just think about the incentives. So if somebody isn't playing

658
00:32:38,320 --> 00:32:41,880
Speaker 4: around at all with the tools when we know that

659
00:32:41,960 --> 00:32:44,840
Speaker 4: they can improve productivity, then we need to figure out

660
00:32:44,880 --> 00:32:47,840
Speaker 4: like why and how we help support, Like maybe there's

661
00:32:47,880 --> 00:32:50,160
Speaker 4: some issue with the part of the system they're working

662
00:32:50,160 --> 00:32:51,760
Speaker 4: on and we should go fix, or maybe we just

663
00:32:51,800 --> 00:32:54,440
Speaker 4: need to help upstill them, or whatever else the case is.

664
00:32:54,960 --> 00:32:58,040
Speaker 4: I do think there is a level of experimentation required, right,

665
00:32:58,080 --> 00:32:59,800
Speaker 4: So I don't think it works if your answer is

666
00:32:59,840 --> 00:33:02,960
Speaker 4: like you need to ensure that all your token use

667
00:33:03,560 --> 00:33:07,000
Speaker 4: is completely optimized, Like, it's not going to work, right.

668
00:33:07,040 --> 00:33:09,920
Speaker 4: People have to learn what's possible. They're doing different jobs.

669
00:33:10,320 --> 00:33:13,840
Speaker 4: The tech is changing, but it can either be like

670
00:33:14,880 --> 00:33:19,360
Speaker 4: don't use the tools or just max your tools blindly.

671
00:33:19,600 --> 00:33:22,560
Speaker 4: It's a it's a noisy signal, but it's a signal, right,

672
00:33:22,600 --> 00:33:25,000
Speaker 4: so go look at it and understand as a place

673
00:33:25,040 --> 00:33:27,600
Speaker 4: of where to look. Don't use it as a final judgment.

674
00:33:28,880 --> 00:33:32,560
Speaker 3: So, speaking of measures and not oversimplifying them, I want

675
00:33:32,600 --> 00:33:34,760
Speaker 3: to go back to the sort of core attention that

676
00:33:34,800 --> 00:33:37,440
Speaker 3: we started the conversation out with, which is the AI

677
00:33:37,600 --> 00:33:42,760
Speaker 3: results versus people actually clicking through to results and generating traffic.

678
00:33:42,920 --> 00:33:46,280
Speaker 3: And I know you were talking about AI being expansionary

679
00:33:46,400 --> 00:33:50,560
Speaker 3: or complementary for Google Search, but I'm very curious how

680
00:33:50,600 --> 00:33:54,960
Speaker 3: you actually measure that, and the more granular you can

681
00:33:55,000 --> 00:33:57,960
Speaker 3: get on this the better, Like what are you specifically

682
00:33:58,000 --> 00:34:02,120
Speaker 3: looking at to say that actually, this is something that's

683
00:34:02,160 --> 00:34:05,720
Speaker 3: good for our business versus something that's detracting from the core.

684
00:34:06,320 --> 00:34:08,759
Speaker 4: So I guess I would say like Google's guidance in

685
00:34:08,800 --> 00:34:10,920
Speaker 4: north Star has always been like focused on the user.

686
00:34:11,160 --> 00:34:13,840
Speaker 4: That's our biggest question at the heart is how do

687
00:34:13,880 --> 00:34:17,520
Speaker 4: we make a great experience for users? And then then

688
00:34:17,560 --> 00:34:21,440
Speaker 4: you want to be thoughtful obviously about other consider Okay,

689
00:34:21,520 --> 00:34:23,600
Speaker 4: you like, if you don't have a healthy ecosystem, you

690
00:34:23,640 --> 00:34:26,360
Speaker 4: can't build a service ongoing, so you need to make

691
00:34:26,400 --> 00:34:28,960
Speaker 4: sure you're nurturing a health ecosystem. If you have make

692
00:34:29,080 --> 00:34:32,280
Speaker 4: no money, then you can't fund right this wonderful service.

693
00:34:32,320 --> 00:34:33,840
Speaker 4: So you have to be thoughtful about those, Okay, But

694
00:34:33,960 --> 00:34:36,439
Speaker 4: the place you start with is try and build something

695
00:34:36,480 --> 00:34:40,800
Speaker 4: amazing for users. One of the things we've seen again

696
00:34:40,800 --> 00:34:45,400
Speaker 4: and again with Google Search is if you're doing a

697
00:34:45,440 --> 00:34:49,839
Speaker 4: really great job, like people will not just do another career.

698
00:34:49,880 --> 00:34:52,200
Speaker 4: They will come back to you more often. Okay, they

699
00:34:52,200 --> 00:34:55,720
Speaker 4: will take their phone out of their pocket and extra time. Okay,

700
00:34:55,960 --> 00:34:59,160
Speaker 4: that's a high bar. Right. It's one thing to go

701
00:34:59,200 --> 00:35:01,120
Speaker 4: and say I've showed you something, can you do one

702
00:35:01,120 --> 00:35:02,560
Speaker 4: more thing while I'm showing it to you, is it

703
00:35:02,560 --> 00:35:05,279
Speaker 4: another thing to get you to decide you're going to

704
00:35:05,360 --> 00:35:07,239
Speaker 4: bother to unlock your phone, you're going to boot up

705
00:35:07,280 --> 00:35:09,759
Speaker 4: your desktop, you're going to navigate in the browser. And

706
00:35:09,800 --> 00:35:12,000
Speaker 4: so one of the things we really look for is

707
00:35:12,000 --> 00:35:15,400
Speaker 4: when we're doing these changes, does it cause people to

708
00:35:15,520 --> 00:35:18,360
Speaker 4: come to search more often? Not just use search more often,

709
00:35:18,400 --> 00:35:21,600
Speaker 4: but come more often. We also do various like ux

710
00:35:22,080 --> 00:35:25,400
Speaker 4: research studies and try and understand whether people happy about

711
00:35:25,520 --> 00:35:27,920
Speaker 4: or not, what are the things they find are frustrating?

712
00:35:28,560 --> 00:35:31,480
Speaker 4: Are more users more users adopting it, not just how

713
00:35:31,560 --> 00:35:33,080
Speaker 4: much are they using it? So we look at a

714
00:35:33,080 --> 00:35:35,759
Speaker 4: bunch of different metrics, but one of the biggest is

715
00:35:35,800 --> 00:35:38,480
Speaker 4: really like do you choose to come and ask Google?

716
00:35:38,560 --> 00:35:42,000
Speaker 4: Like do you essentially hire Google more often? Right? For

717
00:35:42,120 --> 00:35:44,879
Speaker 4: things you need? And one of the things that's very

718
00:35:44,920 --> 00:35:49,840
Speaker 4: surprising to people at times is they think they come

719
00:35:49,960 --> 00:35:53,359
Speaker 4: somewhere for all the questions they have already today, right,

720
00:35:53,440 --> 00:35:55,319
Speaker 4: Like maybe they think they come to Google all the time,

721
00:35:55,400 --> 00:35:57,160
Speaker 4: or they think they go to Google plus LMS or

722
00:35:57,200 --> 00:35:59,719
Speaker 4: Google plus LMS plus, Ticktalk plus whatever. Okay, okay, they

723
00:35:59,719 --> 00:36:02,040
Speaker 4: think that ask all the questions they have, but that's

724
00:36:02,080 --> 00:36:05,719
Speaker 4: not true. You actually make a calculation when questions go

725
00:36:05,760 --> 00:36:07,759
Speaker 4: through your mind of is it worth spending any time

726
00:36:07,760 --> 00:36:10,399
Speaker 4: to figure out the answer to this question? Okay? And

727
00:36:10,520 --> 00:36:13,239
Speaker 4: if the answer is no, then you just don't ask

728
00:36:13,280 --> 00:36:16,960
Speaker 4: the question, right. And so when we talk about as

729
00:36:16,960 --> 00:36:20,360
Speaker 4: an expansionary moment, what we really mean is there's a

730
00:36:20,360 --> 00:36:22,680
Speaker 4: whole bunch of questions people have. There's a whole bunch

731
00:36:22,719 --> 00:36:28,040
Speaker 4: of curiosity. That is, people are not exploring, and they're

732
00:36:28,040 --> 00:36:30,600
Speaker 4: not exploring because they view it as too difficult or

733
00:36:30,640 --> 00:36:33,160
Speaker 4: too much time, or not sure that it will be

734
00:36:33,200 --> 00:36:36,399
Speaker 4: worth it. And AI lowers that barrier, and it can

735
00:36:36,440 --> 00:36:40,239
Speaker 4: lower that barrier in ways that are sometimes for us

736
00:36:40,320 --> 00:36:43,799
Speaker 4: English speaker are surprising, which is like actually, in a

737
00:36:43,800 --> 00:36:45,920
Speaker 4: bunch of countries, there's not all the content in the

738
00:36:45,920 --> 00:36:49,319
Speaker 4: web and the language you speak. Okay, llms can help

739
00:36:49,400 --> 00:36:53,240
Speaker 4: unlock that content. AI overviews because it's using an LLM,

740
00:36:53,280 --> 00:36:56,080
Speaker 4: can be more multi lingual than just the web corpuses

741
00:36:56,120 --> 00:36:58,840
Speaker 4: by default. Right, Okay, So suddenly information that wasn't available

742
00:36:58,840 --> 00:37:01,120
Speaker 4: to you as a Hindey speakers now available. It can

743
00:37:01,160 --> 00:37:03,799
Speaker 4: be visual. You had a question about this flower, You

744
00:37:03,800 --> 00:37:06,279
Speaker 4: had a question about that cool purse, you saw like

745
00:37:06,320 --> 00:37:07,560
Speaker 4: where can you buy it? But you know how to

746
00:37:07,600 --> 00:37:10,879
Speaker 4: describe it as possible. You can also just be like

747
00:37:11,239 --> 00:37:14,040
Speaker 4: my kid has a question. Do I say like I

748
00:37:14,120 --> 00:37:16,640
Speaker 4: don't know, or do I go ask the question? Right?

749
00:37:16,960 --> 00:37:19,520
Speaker 4: You see, with young kids they ask questions all the time, right,

750
00:37:19,520 --> 00:37:20,239
Speaker 4: they go why why? Why?

751
00:37:20,280 --> 00:37:20,400
Speaker 1: Why?

752
00:37:20,400 --> 00:37:22,319
Speaker 4: Why? Why? And like at some point parents are like

753
00:37:22,840 --> 00:37:28,479
Speaker 4: because or stop bothering, They go, Okay. They do that

754
00:37:28,640 --> 00:37:33,640
Speaker 4: because from a kid's perspective, they assume adults know everything right,

755
00:37:34,000 --> 00:37:37,319
Speaker 4: and it is no cost to them, right, They're not

756
00:37:37,360 --> 00:37:39,920
Speaker 4: worried about their time and other things as an adult.

757
00:37:40,719 --> 00:37:44,080
Speaker 4: It's not that you're not curious, you just don't think

758
00:37:44,120 --> 00:37:46,759
Speaker 4: everything is known and you don't have the time, right.

759
00:37:46,840 --> 00:37:49,400
Speaker 4: And so if you lower that barrier, it allows you

760
00:37:49,440 --> 00:37:52,520
Speaker 4: to be that kid again. That just sort of explores

761
00:37:52,560 --> 00:37:55,560
Speaker 4: all these things or get started on those projects that

762
00:37:55,640 --> 00:37:58,880
Speaker 4: felt daunting or enables you to save or learn a

763
00:37:58,920 --> 00:38:00,920
Speaker 4: new skill or whatever else. And that's really exciting.

764
00:38:01,080 --> 00:38:04,120
Speaker 3: But I don't want to dismiss the wonder of being

765
00:38:04,120 --> 00:38:06,719
Speaker 3: a kid and learning about the world. But I'm going

766
00:38:06,800 --> 00:38:09,520
Speaker 3: to sound very callous in a second, But how do

767
00:38:09,560 --> 00:38:12,040
Speaker 3: you make money off of that? Like, how do you

768
00:38:12,080 --> 00:38:14,719
Speaker 3: make money off of the AI overviews. Is it just

769
00:38:14,800 --> 00:38:17,719
Speaker 3: customer retention? Is that what we're basically boiling it down to.

770
00:38:17,800 --> 00:38:20,160
Speaker 3: And then if it is customer retention, then you could

771
00:38:20,200 --> 00:38:22,480
Speaker 3: have a strong argument for saying that, like everything can

772
00:38:22,560 --> 00:38:24,359
Speaker 3: just go through Gemini instead of Search.

773
00:38:25,239 --> 00:38:27,240
Speaker 4: I think I would say a couple of things. Search

774
00:38:27,360 --> 00:38:30,640
Speaker 4: only shows ads on a subset of queries. Right, You're

775
00:38:30,680 --> 00:38:33,000
Speaker 4: like less than a quarter of queries, right, So there's

776
00:38:33,000 --> 00:38:37,480
Speaker 4: a whole bunch of queries pre a overviews, right, that

777
00:38:37,480 --> 00:38:40,120
Speaker 4: that you don't make money on because many of them

778
00:38:40,160 --> 00:38:43,399
Speaker 4: are not of commercial need. Right. And so you asked

779
00:38:43,400 --> 00:38:47,040
Speaker 4: a question about tankers earlier, probably pre aio that quarry

780
00:38:47,040 --> 00:38:49,439
Speaker 4: would have shown ads anyway, it doesn't show ads.

781
00:38:50,280 --> 00:38:51,520
Speaker 2: Yeah, I jecked this too.

782
00:38:53,080 --> 00:38:55,080
Speaker 4: It doesn't show ads before. There are no ads in

783
00:38:55,080 --> 00:38:57,520
Speaker 4: that cry right, nobody's trying to advertise something on that. Okay,

784
00:38:57,640 --> 00:39:02,759
Speaker 4: so those craries avers like it doesn't disrupt, right.

785
00:39:03,160 --> 00:39:03,680
Speaker 2: Then there's a.

786
00:39:03,640 --> 00:39:08,000
Speaker 4: Mounchal class of quarries where like you're shopping the presence

787
00:39:08,040 --> 00:39:11,680
Speaker 4: of an A review or Geminal answers, it doesn't preclude

788
00:39:11,719 --> 00:39:15,799
Speaker 4: the need to still buy the item. So there's still

789
00:39:15,800 --> 00:39:17,920
Speaker 4: this huge opportunity with the ads because there's all of

790
00:39:17,960 --> 00:39:21,160
Speaker 4: this choice that's going I think you also see that

791
00:39:21,200 --> 00:39:24,000
Speaker 4: there's an expansion of queries right to this point. So

792
00:39:24,040 --> 00:39:26,640
Speaker 4: you get more queries and so some of those queries

793
00:39:26,640 --> 00:39:28,200
Speaker 4: are more commercial, some of them are not, but some

794
00:39:28,239 --> 00:39:30,560
Speaker 4: of them are more commercial, and so those become new

795
00:39:30,600 --> 00:39:33,640
Speaker 4: opportunities for ads. And there can also be things like

796
00:39:34,600 --> 00:39:38,120
Speaker 4: when the query is under specified or it's a single query,

797
00:39:38,160 --> 00:39:40,160
Speaker 4: you actually don't know as much, so you can't maybe

798
00:39:40,239 --> 00:39:43,680
Speaker 4: target the ads as well. If people start expressing more

799
00:39:43,760 --> 00:39:46,120
Speaker 4: of their need, if it's more of a conversation, they're

800
00:39:46,120 --> 00:39:49,040
Speaker 4: going more down funnel, you can actually create better ads, right,

801
00:39:49,080 --> 00:39:52,040
Speaker 4: And so you can think about new opportunities for ads formats. Right,

802
00:39:52,480 --> 00:39:54,560
Speaker 4: some number of years ago people have said, like, how

803
00:39:54,560 --> 00:39:57,040
Speaker 4: can you make money from a feed? Okay, well Instagram

804
00:39:57,040 --> 00:40:00,640
Speaker 4: ads are very popular, yeah, right, and so there's new

805
00:40:00,680 --> 00:40:04,879
Speaker 4: ads formats as you recognize new technology and new opportunities.

806
00:40:04,960 --> 00:40:07,200
Speaker 4: But the commercial needs are still often there, and the

807
00:40:07,239 --> 00:40:10,319
Speaker 4: desire for user choice is still often there, and so

808
00:40:11,040 --> 00:40:13,080
Speaker 4: there's still a lot of possibility going forward. And so

809
00:40:13,120 --> 00:40:14,840
Speaker 4: it's worked out very well right now with us in

810
00:40:14,880 --> 00:40:15,320
Speaker 4: the balance.

811
00:40:15,960 --> 00:40:19,280
Speaker 2: So I realized that I am a user of both

812
00:40:19,600 --> 00:40:23,360
Speaker 2: the Gemini app or Gemini dot Google dot com and

813
00:40:23,800 --> 00:40:25,719
Speaker 2: just Google dot com. And I have like a sort

814
00:40:25,760 --> 00:40:27,759
Speaker 2: of really an intuition of which one I go to

815
00:40:27,800 --> 00:40:29,319
Speaker 2: for which purpose. So if I want to look up

816
00:40:29,560 --> 00:40:32,640
Speaker 2: the capital of Moldova, I'm not going to just search

817
00:40:32,680 --> 00:40:35,360
Speaker 2: capital of Moldova, which I just got. Do you know

818
00:40:35,400 --> 00:40:36,080
Speaker 2: that what it is?

819
00:40:36,120 --> 00:40:36,480
Speaker 4: Tracing?

820
00:40:36,960 --> 00:40:38,279
Speaker 2: Sorry, I'm not trying to stump you.

821
00:40:38,880 --> 00:40:41,959
Speaker 3: I know, I feel like you're gonna say it it's Kish.

822
00:40:42,239 --> 00:40:43,359
Speaker 3: I'd never heard. No, I didn't.

823
00:40:43,440 --> 00:40:45,799
Speaker 2: Yeah, I didn't know that either. But anyway, but if

824
00:40:45,840 --> 00:40:49,000
Speaker 2: I you know, if I want to understand what are

825
00:40:49,239 --> 00:40:53,879
Speaker 2: some academic papers that have been written about why it

826
00:40:54,120 --> 00:40:57,640
Speaker 2: is that high frequency trading firms tend to not have

827
00:40:57,719 --> 00:41:00,759
Speaker 2: outside capital, et cetera. And what was the theory for this,

828
00:41:01,040 --> 00:41:03,040
Speaker 2: I think at this point I would use Gemini for that,

829
00:41:03,080 --> 00:41:06,319
Speaker 2: and hope to I'd use Gemini or some other ones,

830
00:41:06,320 --> 00:41:08,560
Speaker 2: But within the context of this conversation, that's more of

831
00:41:08,560 --> 00:41:11,279
Speaker 2: a Gemini query for me than a Google one. At

832
00:41:11,280 --> 00:41:14,800
Speaker 2: this point, Well, there always be two boxes, or do

833
00:41:14,840 --> 00:41:19,239
Speaker 2: you foresee eventually there is just one box and it

834
00:41:19,280 --> 00:41:22,640
Speaker 2: will just know this is like, why do we need

835
00:41:22,719 --> 00:41:23,360
Speaker 2: two boxes?

836
00:41:24,480 --> 00:41:27,040
Speaker 4: I don't know what life will be like in five years.

837
00:41:27,560 --> 00:41:32,759
Speaker 4: I think it's very Sometimes people want sort of an experience.

838
00:41:33,080 --> 00:41:36,640
Speaker 4: Although the information need seems this like similar, they actually

839
00:41:36,680 --> 00:41:39,680
Speaker 4: want different experiences across okay, And so if you take

840
00:41:39,880 --> 00:41:44,759
Speaker 4: a preleum example, people use YouTube for search. Some in

841
00:41:44,800 --> 00:41:47,120
Speaker 4: the US they use it, some in India they use

842
00:41:47,160 --> 00:41:50,320
Speaker 4: it a huge amount. They bring a bunch of queries

843
00:41:50,320 --> 00:41:53,439
Speaker 4: that you would bring to Google Search in the US. Right,

844
00:41:53,920 --> 00:41:55,640
Speaker 4: you could say, okay, well we haven't Why haven't we

845
00:41:55,680 --> 00:41:59,200
Speaker 4: collapsed YouTube and search search box into one search box? Right,

846
00:41:59,280 --> 00:42:01,520
Speaker 4: and do that and it has been slave In the case,

847
00:42:01,680 --> 00:42:04,799
Speaker 4: we have the Google app and we have Chrome. They

848
00:42:04,920 --> 00:42:06,920
Speaker 4: both allow you to search, and they both allow you

849
00:42:06,920 --> 00:42:09,160
Speaker 4: to browse the web. You have a set of people

850
00:42:09,160 --> 00:42:10,600
Speaker 4: that love the Google app, and you have a set

851
00:42:10,640 --> 00:42:11,879
Speaker 4: of people that love Chrome, and you have a set

852
00:42:11,880 --> 00:42:14,239
Speaker 4: of people that use both on a phone. But you

853
00:42:14,280 --> 00:42:18,520
Speaker 4: can't necessarily convince either population that they want to stop

854
00:42:18,640 --> 00:42:20,520
Speaker 4: using one app and just switch to the other app.

855
00:42:20,560 --> 00:42:24,279
Speaker 4: So I don't think the space is so huge and

856
00:42:24,320 --> 00:42:27,520
Speaker 4: it's changing so quickly right now that to sort of

857
00:42:27,840 --> 00:42:30,360
Speaker 4: be able to know for sure whether or not you

858
00:42:30,400 --> 00:42:38,000
Speaker 4: can create one sufficiently dynamic personalized experience that one app

859
00:42:38,040 --> 00:42:40,839
Speaker 4: when entry point can truly do it all. I don't

860
00:42:40,840 --> 00:42:44,560
Speaker 4: think we know yet on that people come for restaurant searches,

861
00:42:44,600 --> 00:42:46,800
Speaker 4: they come to Maps, and they come to Google Search.

862
00:42:47,840 --> 00:42:50,759
Speaker 4: We have not collapsed the Maps app and the search app.

863
00:42:51,080 --> 00:42:53,440
Speaker 4: At some point it becomes big. You know, you're putting

864
00:42:53,480 --> 00:42:55,759
Speaker 4: all this directions coded into the Google Search app. Is

865
00:42:55,760 --> 00:42:59,000
Speaker 4: that actually useful even if you had a full Maps view?

866
00:42:59,120 --> 00:43:00,520
Speaker 4: So I think we're just going to have to learn

867
00:43:00,560 --> 00:43:03,480
Speaker 4: over time about what's good. But the space is really giant,

868
00:43:03,680 --> 00:43:07,120
Speaker 4: and they do have different emphasis right now on what

869
00:43:07,200 --> 00:43:09,440
Speaker 4: they try to excel at. And you want to make

870
00:43:09,480 --> 00:43:11,719
Speaker 4: sure that in the attempt to bring things together, you

871
00:43:11,760 --> 00:43:15,440
Speaker 4: don't become only okay at everything, and you want to

872
00:43:15,440 --> 00:43:17,399
Speaker 4: make sure that you can shine at all the use

873
00:43:17,440 --> 00:43:19,640
Speaker 4: cases people need. And that may mean two products, or

874
00:43:19,640 --> 00:43:21,600
Speaker 4: that may not, or maybe a third product, right like,

875
00:43:21,640 --> 00:43:22,880
Speaker 4: I don't know, in five years there may be a

876
00:43:22,920 --> 00:43:25,560
Speaker 4: third product that replaces all the products. You have. Your

877
00:43:25,600 --> 00:43:27,600
Speaker 4: personal agent you don't talk to areout. I don't know.

878
00:43:27,880 --> 00:43:29,959
Speaker 3: So I realized we kind of promised to talk about

879
00:43:30,000 --> 00:43:33,080
Speaker 3: AI slop a little bit in this conversation. So one

880
00:43:33,120 --> 00:43:35,160
Speaker 3: of the things that's happening with AI is not just

881
00:43:35,239 --> 00:43:37,520
Speaker 3: that I can ask a bunch of questions I might

882
00:43:37,719 --> 00:43:41,680
Speaker 3: not otherwise have had time or the inclination to ask,

883
00:43:41,800 --> 00:43:45,640
Speaker 3: but also AI is being used to generate vast amounts

884
00:43:45,680 --> 00:43:49,640
Speaker 3: of content that are aimed at potentially answering any silly

885
00:43:49,719 --> 00:43:52,719
Speaker 3: question I or anyone else on Earth might have. Yeah,

886
00:43:53,120 --> 00:43:56,240
Speaker 3: just churning it out. And I'm very curious how search

887
00:43:56,520 --> 00:43:59,840
Speaker 3: is weighing I guess the quality of its results in

888
00:44:00,120 --> 00:44:02,839
Speaker 3: the new slop era of the Internet.

889
00:44:03,280 --> 00:44:05,959
Speaker 4: I think there's a tendency at times sometimes think about

890
00:44:05,960 --> 00:44:08,480
Speaker 4: AI slop as if it's before AI slop.

891
00:44:08,520 --> 00:44:12,520
Speaker 3: There was slop, yes, right, human generated slop.

892
00:44:12,560 --> 00:44:16,760
Speaker 4: There there is human generated slap. Now there's AI generated slap.

893
00:44:16,840 --> 00:44:19,919
Speaker 4: So there has always been slop on the web. And

894
00:44:20,000 --> 00:44:23,680
Speaker 4: so what doesn't really matter at some level is how

895
00:44:23,760 --> 00:44:26,840
Speaker 4: much slop is on the web so much as is

896
00:44:26,880 --> 00:44:29,400
Speaker 4: there great content on the web and can you surface

897
00:44:29,440 --> 00:44:33,080
Speaker 4: it right? And this is Google's bread and butter, and

898
00:44:33,160 --> 00:44:37,360
Speaker 4: ranking is and has a long history of looking for

899
00:44:37,480 --> 00:44:41,759
Speaker 4: spam and trying to drop it and make sure it

900
00:44:41,760 --> 00:44:44,560
Speaker 4: doesn't show. And like we crawl many many more pages

901
00:44:44,600 --> 00:44:46,480
Speaker 4: than we even put in our index there's pages we

902
00:44:46,480 --> 00:44:48,640
Speaker 4: put in the index that we never surface, right, so

903
00:44:48,640 --> 00:44:51,320
Speaker 4: that we can keep that rate of spam and slop

904
00:44:51,560 --> 00:44:55,839
Speaker 4: at a very low rate. And it is a constant effort, right, Like,

905
00:44:55,920 --> 00:44:59,160
Speaker 4: it's not a problem you solve because some of the

906
00:44:59,160 --> 00:45:02,080
Speaker 4: people generating this, right, there's a lot of financial incentives

907
00:45:02,640 --> 00:45:07,200
Speaker 4: associated with it. But that is what like what people

908
00:45:07,280 --> 00:45:09,280
Speaker 4: have come to trust Google is that it will show

909
00:45:09,400 --> 00:45:11,600
Speaker 4: great information and it's the thing that we will continue

910
00:45:11,640 --> 00:45:14,080
Speaker 4: to put a huge amount of effort in. And so

911
00:45:14,080 --> 00:45:15,520
Speaker 4: that's the way I would think about It's not like

912
00:45:15,560 --> 00:45:18,760
Speaker 4: how much AI slop or non or human generated slop

913
00:45:18,960 --> 00:45:23,120
Speaker 4: or whatever automated slop pre AI post human generated there is,

914
00:45:23,640 --> 00:45:27,800
Speaker 4: but making sure that the information you do see is trusted.

915
00:45:28,920 --> 00:45:31,440
Speaker 2: Liz read, thank you so much for coming on of

916
00:45:31,640 --> 00:45:34,880
Speaker 2: last there was a fascinating conversation. I have like a

917
00:45:34,920 --> 00:45:38,120
Speaker 2: billion more questions, but man, we'll have you on in

918
00:45:38,160 --> 00:45:40,880
Speaker 2: a three months when the entire world has changed and

919
00:45:40,880 --> 00:45:41,439
Speaker 2: we'll get an.

920
00:45:41,360 --> 00:45:43,840
Speaker 4: Update for Thank you very much. It's a pleasure to be.

921
00:45:43,880 --> 00:45:44,359
Speaker 3: On the JAM.

922
00:45:57,120 --> 00:45:59,520
Speaker 2: I like the point about human generated slap.

923
00:45:59,800 --> 00:46:02,040
Speaker 3: You remember, yeah, but the difference is the volume.

924
00:46:03,160 --> 00:46:06,000
Speaker 2: But but like, do you remember Jason keller cannas to

925
00:46:06,040 --> 00:46:09,960
Speaker 2: startup Mahallow. No, so Jason keller can who's been on

926
00:46:10,000 --> 00:46:12,120
Speaker 2: the podcast. Yeah, he hit this startup.

927
00:46:12,160 --> 00:46:12,560
Speaker 1: For a while.

928
00:46:12,880 --> 00:46:15,120
Speaker 2: It was like the biggest piece of garbage in the world.

929
00:46:15,239 --> 00:46:17,200
Speaker 2: It was no for real go, people need to go.

930
00:46:17,600 --> 00:46:19,480
Speaker 2: It was called Mahallow and the idea was like they

931
00:46:19,480 --> 00:46:21,360
Speaker 2: were just going to hire a lot of people to

932
00:46:21,400 --> 00:46:24,759
Speaker 2: like write articles that were not very good to like

933
00:46:24,800 --> 00:46:27,840
Speaker 2: appear in Google like sea.

934
00:46:28,360 --> 00:46:28,520
Speaker 4: Yeah.

935
00:46:28,560 --> 00:46:30,640
Speaker 2: There was a famous one that my old colleague and

936
00:46:30,760 --> 00:46:34,279
Speaker 2: I think the business insider N Carlson discovered and it

937
00:46:34,320 --> 00:46:37,440
Speaker 2: was like, if you search like how to play the xylophone,

938
00:46:38,640 --> 00:46:40,880
Speaker 2: there was a Mahallow article for that, and it was

939
00:46:41,360 --> 00:46:44,880
Speaker 2: I swear to god, okay, it was step one, decide

940
00:46:44,880 --> 00:46:46,520
Speaker 2: if you want to play a xylophone.

941
00:46:46,840 --> 00:46:48,240
Speaker 3: Well that's an important step.

942
00:46:48,640 --> 00:46:52,640
Speaker 2: Step two, get a xylophone. Step three, learn to read

943
00:46:52,760 --> 00:46:57,799
Speaker 2: sheet music. Step four practice reading sheet music and plate

944
00:46:58,320 --> 00:47:01,680
Speaker 2: so like this was actually like this, I just remembering,

945
00:47:01,760 --> 00:47:04,440
Speaker 2: like it is people have been trying to like stuff

946
00:47:04,520 --> 00:47:07,959
Speaker 2: complete garbage into the search results for a very long

947
00:47:08,040 --> 00:47:11,560
Speaker 2: time and I always get a chuckle thinking about that example,

948
00:47:11,600 --> 00:47:14,600
Speaker 2: and you should go look for it.

949
00:47:13,360 --> 00:47:19,920
Speaker 3: I'm very distracted. It was so bad. Wait, we're just

950
00:47:19,920 --> 00:47:22,480
Speaker 3: gonna like laugh about this article for look at this.

951
00:47:22,680 --> 00:47:26,200
Speaker 2: Just search it mahallow how to play the Xylophone?

952
00:47:26,960 --> 00:47:30,440
Speaker 3: Yeah, I see something from mahallow dot com on YouTube

953
00:47:30,440 --> 00:47:31,120
Speaker 3: that can see it.

954
00:47:31,600 --> 00:47:34,240
Speaker 2: Yeah, yeah, just search So yeah, business inside of February

955
00:47:34,560 --> 00:47:38,480
Speaker 2: line hilariously useless Mallow guide to playing the Xylophone. Did

956
00:47:38,520 --> 00:47:43,759
Speaker 2: you write that, no, Nick Carlson? Yeah yeah, Unfortunately now

957
00:47:44,040 --> 00:47:47,160
Speaker 2: it's behind the paywall and itself has covered a sloppy ad,

958
00:47:47,280 --> 00:47:47,880
Speaker 2: so I guess.

959
00:47:48,239 --> 00:47:52,400
Speaker 3: But anyway, sorry, decide whether you want to buy a

960
00:47:52,560 --> 00:47:57,080
Speaker 3: used or new sylophone. Metal sylophones are less expensive than

961
00:47:57,120 --> 00:48:00,960
Speaker 3: wouldn't one. That's useful. That's a useful But.

962
00:48:00,880 --> 00:48:03,600
Speaker 2: When you see that, it's like, please AI, like save

963
00:48:03,719 --> 00:48:06,520
Speaker 2: us from this human generated garbage they were trying to

964
00:48:06,520 --> 00:48:07,920
Speaker 2: clog search results before.

965
00:48:08,120 --> 00:48:10,839
Speaker 3: All Right, on a serious note, Yeah, I did think

966
00:48:10,880 --> 00:48:15,120
Speaker 3: the point about not customer retention, but like expanding the

967
00:48:15,280 --> 00:48:19,040
Speaker 3: volume of like user queries on the platform made a

968
00:48:19,040 --> 00:48:22,080
Speaker 3: lot of sense, to which I hadn't like really considered

969
00:48:22,239 --> 00:48:24,360
Speaker 3: that much before. So even if you do get a

970
00:48:24,360 --> 00:48:28,279
Speaker 3: bunch of no click users, they are more inclined to

971
00:48:28,320 --> 00:48:30,720
Speaker 3: come back to the platform in the future. And maybe

972
00:48:30,760 --> 00:48:33,360
Speaker 3: some of that eventually lands in clicks.

973
00:48:33,520 --> 00:48:35,080
Speaker 2: The other thing I hadn't thought of it, and I thought

974
00:48:35,120 --> 00:48:38,120
Speaker 2: it was a great point, which is that there are

975
00:48:38,239 --> 00:48:42,160
Speaker 2: multiple Google currently runs multiple search boxes. There's the YouTube

976
00:48:42,320 --> 00:48:44,160
Speaker 2: you look at the Mahallow article and cracking up.

977
00:48:45,040 --> 00:48:52,360
Speaker 3: Its Step four is experiment with different mallets. I just

978
00:48:52,400 --> 00:48:53,759
Speaker 3: really want to play the sylophone.

979
00:48:54,040 --> 00:48:58,200
Speaker 2: It's so good. It's so good.

980
00:48:58,440 --> 00:49:00,239
Speaker 3: Step five is practice regular.

981
00:49:01,760 --> 00:49:04,319
Speaker 2: It's crazy, like I think AI is much better, like

982
00:49:04,560 --> 00:49:07,200
Speaker 2: if I you know, it's so good, isn't it. It's

983
00:49:07,239 --> 00:49:10,560
Speaker 2: like the biggest steaming pile of garbage I've ever seen

984
00:49:10,680 --> 00:49:13,680
Speaker 2: on the internet fifteen years or ten years before anyone

985
00:49:13,719 --> 00:49:14,280
Speaker 2: has ever.

986
00:49:14,440 --> 00:49:17,759
Speaker 3: Can we actually, can we have just to talk about?

987
00:49:17,840 --> 00:49:20,880
Speaker 2: Should have Jason back on the podcast, just to like

988
00:49:21,040 --> 00:49:24,560
Speaker 2: grill him about what exactly he was thinking and his

989
00:49:24,680 --> 00:49:27,399
Speaker 2: sins against the Internet for having put this out there.

990
00:49:27,480 --> 00:49:30,560
Speaker 3: Well he was an early adopter of non AI slob.

991
00:49:31,520 --> 00:49:35,560
Speaker 2: But anyway, that point about there are multiple search boxes already, right,

992
00:49:35,560 --> 00:49:38,200
Speaker 2: there's the YouTube search box. You're still left.

993
00:49:38,320 --> 00:49:40,080
Speaker 3: I'm sorry, I was trying to make eye contact with

994
00:49:40,120 --> 00:49:47,600
Speaker 3: you and not look at my computer. Yeah, okay, all right, yes,

995
00:49:47,719 --> 00:49:48,440
Speaker 3: Shall we leave it there?

996
00:49:48,480 --> 00:49:49,200
Speaker 1: Let's leave it there.

997
00:49:49,360 --> 00:49:51,879
Speaker 3: This has been another episode of the Authoughts podcast. I'm

998
00:49:51,880 --> 00:49:54,760
Speaker 3: Tracy Alloway. You can follow me at Tracy Alloway.

999
00:49:54,440 --> 00:49:56,880
Speaker 2: And I'm Joe Wisnhal. You can follow me at the

1000
00:49:56,920 --> 00:50:00,120
Speaker 2: Stalwart Fellow or producers Cormon, Rod Vegas at Kerr and

1001
00:50:00,160 --> 00:50:03,600
Speaker 2: our Man Dash, I'll Bennett at Dashbot and Kilbrooks Kilbrooks

1002
00:50:03,800 --> 00:50:06,040
Speaker 2: And for more odd Lots content, go to Bloomberg dot

1003
00:50:06,040 --> 00:50:08,440
Speaker 2: com slash odd Lots. We have a daily newsletter and

1004
00:50:08,520 --> 00:50:10,640
Speaker 2: all of our episodes, and you can chat about all

1005
00:50:10,680 --> 00:50:13,040
Speaker 2: of these topics twenty four to seven in our discord

1006
00:50:13,360 --> 00:50:15,600
Speaker 2: Discord dot gg slash odlines.

1007
00:50:15,840 --> 00:50:17,600
Speaker 3: And if you enjoy odd Lots, if you like it

1008
00:50:17,640 --> 00:50:20,600
Speaker 3: when we talk about AI and human generated Slot, then

1009
00:50:20,600 --> 00:50:23,880
Speaker 3: please leave us a positive review on your favorite podcast platform.

1010
00:50:24,080 --> 00:50:26,400
Speaker 3: And remember, if you are a Bloomberg subscriber, you can

1011
00:50:26,440 --> 00:50:29,560
Speaker 3: listen to all of our episodes absolutely ad free. All

1012
00:50:29,560 --> 00:50:31,640
Speaker 3: you need to do is find the Bloomberg channel on

1013
00:50:31,719 --> 00:50:35,080
Speaker 3: Apple Podcasts and follow the instructions there. Thanks for listening,

1014
00:51:01,000 --> 00:51:01,120
Speaker 3: fo