1 00:00:15,316 --> 00:00:23,156 Speaker 1: Pushkin. It feels like searching the web is a problem 2 00:00:23,236 --> 00:00:26,756 Speaker 1: that's been solved. You know, it's ridiculously easy for me 3 00:00:26,796 --> 00:00:30,036 Speaker 1: to say, find out when Alexander Hamilton was shot eighteen 4 00:00:30,076 --> 00:00:33,796 Speaker 1: oh four, or whether they are making Sing three. Not yet, 5 00:00:34,156 --> 00:00:38,876 Speaker 1: but Matthew McConaughey has expressed interest and yet. And maybe 6 00:00:38,876 --> 00:00:42,556 Speaker 1: this is not surprising. The people who spend their lives 7 00:00:42,716 --> 00:00:46,876 Speaker 1: working on search do not think search is solved. This 8 00:00:46,956 --> 00:00:50,116 Speaker 1: is partly because the people at the frontier of search 9 00:00:50,596 --> 00:00:53,796 Speaker 1: don't just want to search the web. They want to 10 00:00:53,796 --> 00:00:57,876 Speaker 1: answer every question that might cross your mind, even questions 11 00:00:57,996 --> 00:01:06,796 Speaker 1: you can't put into words. I'm Jacob Goldstein, and this 12 00:01:06,916 --> 00:01:09,636 Speaker 1: is What's your problem? The show where on entrepreneurs and 13 00:01:09,676 --> 00:01:11,916 Speaker 1: engineers talk about how they're going to change the world 14 00:01:12,236 --> 00:01:15,156 Speaker 1: once they solve a few problems. My guest today is 15 00:01:15,316 --> 00:01:19,396 Speaker 1: Kathy Edwards, vice president and GM of Search at Google. 16 00:01:20,516 --> 00:01:23,956 Speaker 1: Cathy's problem is this, how do you teach computers to 17 00:01:23,956 --> 00:01:26,676 Speaker 1: tell people what they want to know, even if they 18 00:01:26,716 --> 00:01:30,676 Speaker 1: don't know how to ask. Later in the conversation we 19 00:01:30,756 --> 00:01:33,356 Speaker 1: get to the frontier of what Kathy and Google are 20 00:01:33,396 --> 00:01:36,356 Speaker 1: working on now, but we started with the problem they 21 00:01:36,396 --> 00:01:39,196 Speaker 1: have largely solved in the six years Kathy has been 22 00:01:39,236 --> 00:01:42,916 Speaker 1: at Google, the jump from search results based on keywords 23 00:01:43,316 --> 00:01:46,676 Speaker 1: to search results based on natural language, the way people 24 00:01:46,716 --> 00:01:51,316 Speaker 1: talk in everyday life. So one of the problems that 25 00:01:51,356 --> 00:01:54,556 Speaker 1: we were working on around six years ago is this 26 00:01:54,716 --> 00:01:58,276 Speaker 1: problem of natural language queries. So, if you're old enough 27 00:01:58,316 --> 00:02:01,756 Speaker 1: to remember the early days of search on the Internet, 28 00:02:02,116 --> 00:02:05,836 Speaker 1: there was this idea of keywordies, right, that you had 29 00:02:05,876 --> 00:02:09,236 Speaker 1: to sort of take this idea you had in your 30 00:02:09,236 --> 00:02:11,756 Speaker 1: mind of what you what you needed to know and 31 00:02:11,916 --> 00:02:15,436 Speaker 1: figure out what were the exact right keywords to enter 32 00:02:15,476 --> 00:02:17,756 Speaker 1: into the search engine to get your results back right. 33 00:02:17,796 --> 00:02:20,716 Speaker 1: I mean an example as an example, very early in 34 00:02:20,716 --> 00:02:23,436 Speaker 1: the you know, I remember being taught how to query 35 00:02:23,876 --> 00:02:28,716 Speaker 1: back in you know, nineteen ninety nine and being told 36 00:02:28,796 --> 00:02:32,356 Speaker 1: never used the word and never used the word because 37 00:02:32,716 --> 00:02:35,676 Speaker 1: the word and or the word there is in almost 38 00:02:35,796 --> 00:02:38,516 Speaker 1: every document on the Internet. And the way it worked 39 00:02:38,556 --> 00:02:41,796 Speaker 1: back then is you did this word matching, right, and 40 00:02:41,836 --> 00:02:44,436 Speaker 1: so if you had a word that was in your 41 00:02:44,516 --> 00:02:47,076 Speaker 1: query and there was that same word in the document, 42 00:02:47,476 --> 00:02:50,876 Speaker 1: then that document would be returned and potentially scored right. 43 00:02:51,276 --> 00:02:54,436 Speaker 1: And that was very helpful if it was a word 44 00:02:54,476 --> 00:02:58,716 Speaker 1: like genetics, right, which was highly specific and wasn't in 45 00:02:58,796 --> 00:03:00,996 Speaker 1: a heap of documents on the internet. But the word 46 00:03:00,996 --> 00:03:04,196 Speaker 1: and not very specific. And you know, in the very 47 00:03:04,236 --> 00:03:08,276 Speaker 1: early days the Internet, these words weren't even weighted particularly right, 48 00:03:08,316 --> 00:03:11,956 Speaker 1: The word and count for as much as the word genetics, 49 00:03:11,956 --> 00:03:14,716 Speaker 1: and so a document might have a ton of the 50 00:03:14,796 --> 00:03:17,276 Speaker 1: uses of the word and and one use of the 51 00:03:17,276 --> 00:03:20,116 Speaker 1: word genetics, and it would score really highly, even though 52 00:03:20,116 --> 00:03:22,436 Speaker 1: it wasn't particularly genetics. Folk. Now, by the time you 53 00:03:22,436 --> 00:03:25,556 Speaker 1: get to Google, that part is solved, right Google buy 54 00:03:25,716 --> 00:03:29,156 Speaker 1: that part is years ago? Is waiting genetics more more 55 00:03:29,196 --> 00:03:33,556 Speaker 1: heavily than it's waiting the But but what what part 56 00:03:33,636 --> 00:03:36,516 Speaker 1: six years ago was not solved? That's solved now or 57 00:03:36,556 --> 00:03:40,436 Speaker 1: solved ish now. But we were still seeing people do 58 00:03:40,596 --> 00:03:44,596 Speaker 1: these very keyword oriented queries. So they weren't saying things 59 00:03:44,676 --> 00:03:48,516 Speaker 1: like what wine pairs best with chicken? Or if they were, 60 00:03:48,876 --> 00:03:53,036 Speaker 1: they were doing those queries and getting not the best results, 61 00:03:53,076 --> 00:03:56,116 Speaker 1: because not only is there a question of word matching 62 00:03:56,116 --> 00:03:59,156 Speaker 1: and how much each word counts for, there's also the 63 00:03:59,276 --> 00:04:03,556 Speaker 1: question of like does the word what appear at all? Right? 64 00:04:03,676 --> 00:04:07,276 Speaker 1: Like are the answers to that question actually just documents 65 00:04:07,356 --> 00:04:09,716 Speaker 1: that talk about the best wine to pair with chicken 66 00:04:10,076 --> 00:04:16,716 Speaker 1: is you know, chardonnay, right, and not so much talking. 67 00:04:16,916 --> 00:04:18,596 Speaker 1: You know, they didn't include the question, and so we 68 00:04:18,676 --> 00:04:23,756 Speaker 1: sort of saw these like SEO documents that would spring 69 00:04:23,876 --> 00:04:26,236 Speaker 1: up that would have the questions kind of baked in 70 00:04:26,276 --> 00:04:29,116 Speaker 1: and an attempt to match. But those documents weren't necessarily 71 00:04:29,156 --> 00:04:33,436 Speaker 1: the best answers. And so this is when we started 72 00:04:33,436 --> 00:04:36,356 Speaker 1: to go just that next level deeper in our language 73 00:04:36,476 --> 00:04:41,956 Speaker 1: understanding with these AI models, their language models that really 74 00:04:42,036 --> 00:04:46,196 Speaker 1: can start to map out in a concept space, things 75 00:04:46,316 --> 00:04:49,356 Speaker 1: like this sort of translation between how you might ask 76 00:04:49,396 --> 00:04:51,356 Speaker 1: a query and then what that might look like in 77 00:04:51,396 --> 00:04:53,916 Speaker 1: the document. So, to take the example you gave of 78 00:04:54,196 --> 00:04:57,916 Speaker 1: what wine pairs best with chicken, even as late as 79 00:04:57,956 --> 00:04:59,956 Speaker 1: six years ago when you got to Google, you're saying 80 00:05:00,636 --> 00:05:05,316 Speaker 1: Google wasn't great at delivering the best results to a 81 00:05:05,396 --> 00:05:08,956 Speaker 1: query like that because it was written as speech, not 82 00:05:09,156 --> 00:05:11,396 Speaker 1: written as a series of keywords. So six years ago, 83 00:05:11,516 --> 00:05:16,596 Speaker 1: I would have been better off typing chicken wine pairing. 84 00:05:17,356 --> 00:05:19,116 Speaker 1: I would have got better results if I did that, 85 00:05:19,156 --> 00:05:21,636 Speaker 1: you're saying, because that's kind of the way. That's the 86 00:05:21,636 --> 00:05:23,476 Speaker 1: way Google had mapped the web. It was like a 87 00:05:23,516 --> 00:05:26,076 Speaker 1: series of important words and what sites are reliable and 88 00:05:26,116 --> 00:05:29,876 Speaker 1: they it just the technology wasn't there to actually try 89 00:05:29,876 --> 00:05:35,316 Speaker 1: and understand the way people ask questions in real life. Absolutely, 90 00:05:35,716 --> 00:05:40,636 Speaker 1: And it was this idea of bringing AI into search 91 00:05:41,036 --> 00:05:44,596 Speaker 1: and having these like large scale language models. That first 92 00:05:44,676 --> 00:05:48,036 Speaker 1: one was called Bert. We now use one called mom, 93 00:05:48,076 --> 00:05:50,796 Speaker 1: which is get to mom. But let's talk about Let's 94 00:05:50,796 --> 00:05:53,836 Speaker 1: try talking about Bert for a second. So how do 95 00:05:53,916 --> 00:05:58,996 Speaker 1: you get from search results that are fundamentally keyword based 96 00:05:59,476 --> 00:06:03,636 Speaker 1: to search results that are fundamentally you know, answering questions 97 00:06:03,676 --> 00:06:06,396 Speaker 1: that are posed in a more natural way, like how 98 00:06:06,396 --> 00:06:11,076 Speaker 1: do you make that leap? So the fundamental insight is 99 00:06:11,476 --> 00:06:14,236 Speaker 1: you go from looking at these words as tokens that 100 00:06:14,716 --> 00:06:18,556 Speaker 1: get matched against each other to suddenly you look at 101 00:06:18,596 --> 00:06:21,756 Speaker 1: all the words in all the documents on the Internet 102 00:06:22,076 --> 00:06:25,156 Speaker 1: and you create what's code an embedding space, which is 103 00:06:25,276 --> 00:06:27,436 Speaker 1: essentially you can think of it as a map of 104 00:06:27,476 --> 00:06:32,676 Speaker 1: the concepts that these documents know about. And suddenly, by 105 00:06:32,716 --> 00:06:34,636 Speaker 1: being able to say, okay, you can take a query, 106 00:06:34,796 --> 00:06:39,396 Speaker 1: map that into this concept embedding space. You'd take these documents, 107 00:06:39,396 --> 00:06:42,476 Speaker 1: map that into the content embedding space. You can start 108 00:06:42,516 --> 00:06:47,116 Speaker 1: to actually match together not these words, but what people 109 00:06:47,236 --> 00:06:50,716 Speaker 1: actually mean what they actually mean when they ask these questions, 110 00:06:50,716 --> 00:06:53,676 Speaker 1: and what they actually mean when they write these web 111 00:06:53,716 --> 00:06:56,836 Speaker 1: pages on the Internet. That seems I mean, A, it 112 00:06:56,836 --> 00:07:00,196 Speaker 1: seems super hard, right, and B As I'm parsing that, 113 00:07:00,356 --> 00:07:03,916 Speaker 1: I'm tempted to use a lot of anthropomorphic language, right, 114 00:07:03,956 --> 00:07:06,676 Speaker 1: I'm tempted to say, like, you have to go from 115 00:07:06,716 --> 00:07:09,076 Speaker 1: the computer just sort of having a list of words 116 00:07:09,316 --> 00:07:11,556 Speaker 1: and kind of weights around those words to a computer 117 00:07:12,076 --> 00:07:16,316 Speaker 1: understanding what people mean. Like, am I right to say that? 118 00:07:16,436 --> 00:07:19,196 Speaker 1: Or is that just my like layperson intuition getting in 119 00:07:19,196 --> 00:07:22,716 Speaker 1: the way of what's going on? I Mean, the first 120 00:07:22,716 --> 00:07:25,836 Speaker 1: thing I'll say is I think we're very far away 121 00:07:25,876 --> 00:07:29,996 Speaker 1: from the computer having any sort of sentience and truly understanding. 122 00:07:30,356 --> 00:07:32,396 Speaker 1: But I think it is true. It is fair to 123 00:07:32,476 --> 00:07:37,236 Speaker 1: say that there is a level of deeper understanding that 124 00:07:37,276 --> 00:07:40,956 Speaker 1: you're not just looking at these words as as you know, 125 00:07:41,036 --> 00:07:44,076 Speaker 1: bits in a computer, but you're actually starting to model 126 00:07:44,236 --> 00:07:46,676 Speaker 1: in a way that a human might, a brain might 127 00:07:46,756 --> 00:07:50,956 Speaker 1: model what the concepts are. And I do think that's 128 00:07:50,956 --> 00:07:53,636 Speaker 1: a first step of getting closer to this sort of 129 00:07:53,796 --> 00:07:59,356 Speaker 1: natural human understanding. So is there a way to talk 130 00:07:59,396 --> 00:08:04,316 Speaker 1: about how that works? It's it's pattern matching effectively right, 131 00:08:04,356 --> 00:08:07,916 Speaker 1: And it just so happens that if you magnify pattern 132 00:08:07,956 --> 00:08:10,316 Speaker 1: matching on a very large scale, that can be a 133 00:08:10,476 --> 00:08:15,396 Speaker 1: pretty compelling understanding. And so that's the sort of big idea, 134 00:08:15,636 --> 00:08:19,036 Speaker 1: the theory of how it works. I'm sure in actually 135 00:08:19,236 --> 00:08:23,116 Speaker 1: building the thing in building Burt, which was this big 136 00:08:23,636 --> 00:08:28,716 Speaker 1: model that did work, it wasn't that easy, right, I mean, 137 00:08:28,836 --> 00:08:33,076 Speaker 1: is there a is there a story version of how 138 00:08:33,116 --> 00:08:36,276 Speaker 1: you built it? So I think there were two hard 139 00:08:36,316 --> 00:08:39,796 Speaker 1: points along the journey. The first hard point was just 140 00:08:40,996 --> 00:08:43,956 Speaker 1: these models were being built at a scale that was 141 00:08:44,076 --> 00:08:48,516 Speaker 1: unprecedented the amount of information. You know, traditional neural networks 142 00:08:48,556 --> 00:08:53,756 Speaker 1: would run on thousands, maybe millions of training examples. Suddenly 143 00:08:53,796 --> 00:08:58,076 Speaker 1: you're trying to model all the words on the Internet 144 00:08:58,116 --> 00:09:01,956 Speaker 1: and this scale. Firstly, this scale is what gets you 145 00:09:02,036 --> 00:09:04,756 Speaker 1: the amount of training to actually get the concepts model 146 00:09:04,836 --> 00:09:09,436 Speaker 1: to be compelling. But frankly, the computers just couldn't process. 147 00:09:09,476 --> 00:09:12,236 Speaker 1: So you're you're building this model and saying, okay, now 148 00:09:12,236 --> 00:09:15,596 Speaker 1: to learn what you need to learn, read literally every 149 00:09:15,596 --> 00:09:18,476 Speaker 1: word on the Internet, is that right, yes, and not 150 00:09:18,516 --> 00:09:22,076 Speaker 1: read at once, because every layer of the neuronet needs 151 00:09:22,116 --> 00:09:25,236 Speaker 1: to read it and reprocess it. Right. So you're reading 152 00:09:25,396 --> 00:09:29,196 Speaker 1: every word, you know, a massive number of times. And 153 00:09:29,236 --> 00:09:31,956 Speaker 1: at the time we didn't really have the compute power. 154 00:09:32,236 --> 00:09:36,836 Speaker 1: You just needed more more computers, essentially more and more chips, 155 00:09:37,036 --> 00:09:40,756 Speaker 1: more more engines to just process and process and process. 156 00:09:41,996 --> 00:09:47,236 Speaker 1: So our research team had developed these these chips that 157 00:09:47,836 --> 00:09:51,516 Speaker 1: these processes that were really optimized for doing a sort 158 00:09:51,556 --> 00:09:55,036 Speaker 1: of deep learning work. And it was that these chips 159 00:09:55,076 --> 00:09:56,876 Speaker 1: and the way that we could sort of put all 160 00:09:56,916 --> 00:09:59,236 Speaker 1: the chips together at a work in concert to solve 161 00:09:59,316 --> 00:10:03,356 Speaker 1: this problem that really unlocked the amount of processing power 162 00:10:03,356 --> 00:10:05,756 Speaker 1: and needed to even build these models in the festival. 163 00:10:05,836 --> 00:10:09,196 Speaker 1: So the binding constraint wasn't like the theory or the 164 00:10:09,236 --> 00:10:10,916 Speaker 1: ideas of it, like you knew how to do it, 165 00:10:10,956 --> 00:10:14,956 Speaker 1: you just didn't have enough enough horsepower to actually make 166 00:10:14,996 --> 00:10:18,396 Speaker 1: it happen. Well, we knew that we could do it, 167 00:10:18,436 --> 00:10:21,356 Speaker 1: we didn't know offer to be any good, right, it 168 00:10:20,916 --> 00:10:24,676 Speaker 1: wasn't it, and you couldn't even try, right, yeah? Right? 169 00:10:25,236 --> 00:10:28,636 Speaker 1: And so then we tried it and we found out, actually, 170 00:10:28,676 --> 00:10:32,476 Speaker 1: this thing is pretty compelling. It can understand things that 171 00:10:32,516 --> 00:10:36,676 Speaker 1: our models previously have never understood. You know. But I 172 00:10:36,716 --> 00:10:39,716 Speaker 1: will say the second and this gets to the second 173 00:10:39,796 --> 00:10:44,956 Speaker 1: hard part. We once we had these large scale language models, 174 00:10:44,956 --> 00:10:49,436 Speaker 1: we didn't quite know how to put them into search ranking. 175 00:10:49,676 --> 00:10:52,876 Speaker 1: This was not something that had been done before. So 176 00:10:52,956 --> 00:10:57,796 Speaker 1: we have in search this incredibly rigorous methodology for testing 177 00:10:57,956 --> 00:11:01,676 Speaker 1: any given change to our algorithm, and it's it's based 178 00:11:01,676 --> 00:11:05,556 Speaker 1: in statistics, and it's statistically samples queries, and we look 179 00:11:05,556 --> 00:11:08,556 Speaker 1: at the before and they after, and there's a scoring 180 00:11:08,596 --> 00:11:11,716 Speaker 1: system to say is it better or not? And I 181 00:11:11,796 --> 00:11:17,756 Speaker 1: remember looking at the early experiments from this burst integration 182 00:11:17,876 --> 00:11:22,236 Speaker 1: into our search engine, and the queries that it was 183 00:11:22,596 --> 00:11:27,916 Speaker 1: impacting were just queries that, honestly before we would have said, 184 00:11:28,396 --> 00:11:30,796 Speaker 1: we don't know how we can solve this query. And 185 00:11:30,956 --> 00:11:35,796 Speaker 1: suddenly the model was just able to figure out these 186 00:11:35,836 --> 00:11:41,596 Speaker 1: sort of unspoken concepts that just our previous technology just 187 00:11:41,716 --> 00:11:43,956 Speaker 1: would not have even been able to come close to. 188 00:11:44,516 --> 00:11:46,356 Speaker 1: Like give me an example, like what kind of thing? 189 00:11:47,836 --> 00:11:50,956 Speaker 1: So here's a really great example. This is directly from 190 00:11:51,556 --> 00:11:56,956 Speaker 1: the one of the very first bit evaluations that we 191 00:11:56,996 --> 00:12:01,796 Speaker 1: did internally, and the query is can you get medicine 192 00:12:01,876 --> 00:12:05,836 Speaker 1: for someone? Pharmacy? Right? And so what's interesting about this 193 00:12:05,916 --> 00:12:09,876 Speaker 1: question is the users looking for something very specific. They're 194 00:12:09,916 --> 00:12:13,356 Speaker 1: looking like maybe my partner is sick. Can I go 195 00:12:13,356 --> 00:12:16,156 Speaker 1: and pick up their prescription at the pharmacy for them? 196 00:12:16,316 --> 00:12:18,516 Speaker 1: Or do they have to go and get it? Right? 197 00:12:18,676 --> 00:12:22,636 Speaker 1: It's also a goodly jankie where it's half in natural 198 00:12:22,716 --> 00:12:25,676 Speaker 1: language can you get medicine for someone? And half in 199 00:12:25,756 --> 00:12:28,676 Speaker 1: like keyword ease, they're just typing pharmacy at the end, right, 200 00:12:28,716 --> 00:12:35,156 Speaker 1: it's a weird exactly yeah. And so previously we didn't 201 00:12:35,196 --> 00:12:38,676 Speaker 1: know how to pause out this intent right, this idea. 202 00:12:38,796 --> 00:12:42,156 Speaker 1: You know, we could tell that it was about getting 203 00:12:42,156 --> 00:12:45,916 Speaker 1: a prescription from a pharmacy, but this notion of force 204 00:12:46,116 --> 00:12:53,236 Speaker 1: someone was a concept that was just slightly too complex. Oh, 205 00:12:53,556 --> 00:12:55,636 Speaker 1: I didn't even understand it until now. What they mean 206 00:12:55,756 --> 00:12:58,596 Speaker 1: is can I pick up someone else's prescription? That's what 207 00:12:58,636 --> 00:13:02,076 Speaker 1: they're actually asking, But it's very it's poorly worded, frankly, 208 00:13:02,116 --> 00:13:06,876 Speaker 1: and they'refore hard to figure out exactly right. And so previously, 209 00:13:06,956 --> 00:13:11,876 Speaker 1: before Bert, we would turn these wonderful web pages saying 210 00:13:11,956 --> 00:13:14,996 Speaker 1: this is how you get a prescription filled, which you 211 00:13:15,036 --> 00:13:19,116 Speaker 1: can imagine if you're this user doing this query, you're like, yeah, 212 00:13:19,156 --> 00:13:21,316 Speaker 1: I already know how to get a prescription that filled. 213 00:13:21,436 --> 00:13:24,956 Speaker 1: Thanks for me. What I need is filled for somebody 214 00:13:24,956 --> 00:13:29,396 Speaker 1: else exactly, and with Bert we were able to understand 215 00:13:29,476 --> 00:13:32,556 Speaker 1: pick up this idea of the force someone and put 216 00:13:32,596 --> 00:13:35,396 Speaker 1: the appropriate weight on it, that that was the sort 217 00:13:35,396 --> 00:13:38,956 Speaker 1: of you know, discriminating thing in the query, that that 218 00:13:39,036 --> 00:13:41,356 Speaker 1: was the key thing that the query turned on. And 219 00:13:42,196 --> 00:13:46,636 Speaker 1: then we were able to show this this web page 220 00:13:46,676 --> 00:13:48,556 Speaker 1: that talked about can I have a friend or family 221 00:13:48,596 --> 00:13:51,436 Speaker 1: member pick up a prescription for me? And that was 222 00:13:51,476 --> 00:13:54,196 Speaker 1: the sort of like aha moment where we could all 223 00:13:54,236 --> 00:13:56,316 Speaker 1: just sit around and be like, Wow, this is a 224 00:13:56,436 --> 00:13:59,876 Speaker 1: new level of understanding that we haven't got to previously. 225 00:14:02,516 --> 00:14:05,156 Speaker 1: So with birth, Google got to the point where it 226 00:14:05,236 --> 00:14:08,996 Speaker 1: was very very good at dealing with words in a deep, 227 00:14:09,236 --> 00:14:12,316 Speaker 1: complex way. But words make up less and less of 228 00:14:12,356 --> 00:14:16,156 Speaker 1: the Internet. Pictures and videos are a whole other story 229 00:14:16,716 --> 00:14:25,916 Speaker 1: that's coming up in a minute. Now back to the show. 230 00:14:26,396 --> 00:14:29,156 Speaker 1: So you have got to this place now where you 231 00:14:28,396 --> 00:14:32,876 Speaker 1: have you Google have made the leap from keyword based 232 00:14:32,876 --> 00:14:35,876 Speaker 1: searches to intention based search is what do people mean? Right? 233 00:14:35,876 --> 00:14:39,436 Speaker 1: Which is this big interesting leap? And so I'm interested 234 00:14:39,476 --> 00:14:42,436 Speaker 1: in kind of the next leap, like what's the next 235 00:14:42,636 --> 00:14:47,116 Speaker 1: big hard problem you're trying to solve? What's really interesting 236 00:14:47,196 --> 00:14:50,876 Speaker 1: to me is this idea of how many questions you 237 00:14:50,916 --> 00:14:54,556 Speaker 1: don't ask because you don't even know the words, right, 238 00:14:54,996 --> 00:14:58,316 Speaker 1: Like this is a bit of a sad story. But 239 00:14:58,436 --> 00:15:01,676 Speaker 1: I have at my house this oak tree, and the 240 00:15:01,716 --> 00:15:04,756 Speaker 1: oak tree I think is dead, and it's very sad 241 00:15:04,756 --> 00:15:08,196 Speaker 1: for me because a very beautiful oak tree. And what's 242 00:15:08,236 --> 00:15:11,556 Speaker 1: interesting is, you know, I looked at the oak tree, 243 00:15:11,596 --> 00:15:14,716 Speaker 1: I'm like, wow, those leaves are kind of brown, Like, 244 00:15:14,796 --> 00:15:17,276 Speaker 1: that's not it doesn't seem right to me. I wonder 245 00:15:17,316 --> 00:15:20,676 Speaker 1: if there's something's wrong with the oak tree, right, But 246 00:15:20,756 --> 00:15:24,676 Speaker 1: I can't necessarily right now really articulate that to a 247 00:15:24,716 --> 00:15:27,596 Speaker 1: computer this fundamental question of is this oak tree dead? 248 00:15:27,636 --> 00:15:30,156 Speaker 1: And if not, what can I do to save it? Right? 249 00:15:30,676 --> 00:15:32,436 Speaker 1: So what I do is I go and type in 250 00:15:32,476 --> 00:15:36,356 Speaker 1: some queries, I say, you know, oak tree dead? How 251 00:15:36,356 --> 00:15:38,956 Speaker 1: do I know if my oak tree is dead? You know? 252 00:15:39,036 --> 00:15:42,556 Speaker 1: And I get back results. But those results aren't necessarily 253 00:15:42,636 --> 00:15:45,876 Speaker 1: taking what they're not taking in any context of this 254 00:15:46,036 --> 00:15:49,196 Speaker 1: particular tree and what do the leaves look like? And 255 00:15:49,236 --> 00:15:53,756 Speaker 1: so this idea of how can you start to ask 256 00:15:53,836 --> 00:15:58,796 Speaker 1: these questions using all of the information around you, using 257 00:15:59,036 --> 00:16:03,396 Speaker 1: your camera to actually capture this particular oak tree, using 258 00:16:03,396 --> 00:16:07,716 Speaker 1: your location to know, you know, what are the native 259 00:16:07,716 --> 00:16:11,436 Speaker 1: oaks in this area? And what's the current incidence of 260 00:16:12,196 --> 00:16:14,476 Speaker 1: sudden oak death syndrome, which is a thing that I 261 00:16:14,516 --> 00:16:17,116 Speaker 1: have recently learned exists. Okay, so I get why this 262 00:16:17,196 --> 00:16:22,116 Speaker 1: is a hard thing to search in a text box, right, 263 00:16:22,236 --> 00:16:24,556 Speaker 1: And so the thing that's interesting to me is how 264 00:16:24,596 --> 00:16:29,436 Speaker 1: can we facilitate asking those types of questions where it's 265 00:16:29,476 --> 00:16:33,876 Speaker 1: a mix of here's something that you're looking at, Here's 266 00:16:33,916 --> 00:16:37,156 Speaker 1: something that you're saying with your words that adds to 267 00:16:37,236 --> 00:16:40,876 Speaker 1: the picture. You know, here's a lemon tree that's got 268 00:16:40,956 --> 00:16:43,156 Speaker 1: some black spots on it? What's wrong with it? Like? 269 00:16:43,196 --> 00:16:46,156 Speaker 1: How can you help me understand what I should do 270 00:16:46,236 --> 00:16:49,996 Speaker 1: about this? You know, these sorts of questions I think 271 00:16:50,156 --> 00:16:54,156 Speaker 1: are right now. We have to do a tremendous amount 272 00:16:54,156 --> 00:16:57,676 Speaker 1: of work to try and translate these questions into text 273 00:16:57,876 --> 00:17:01,276 Speaker 1: that we would issue to a search engine. And yeah, 274 00:17:01,516 --> 00:17:05,316 Speaker 1: we use that. Yeah, yeah, normal people. Yes, we're all 275 00:17:05,356 --> 00:17:08,196 Speaker 1: doing it. And when you think it's you know, sometimes 276 00:17:08,196 --> 00:17:12,876 Speaker 1: it's very easy, right, but sometimes you're like really having 277 00:17:12,916 --> 00:17:14,836 Speaker 1: to work hard to come up with a query that 278 00:17:14,836 --> 00:17:16,636 Speaker 1: will actually get you the answers that you need. And 279 00:17:16,676 --> 00:17:19,276 Speaker 1: I think that's really the next frontier for us is 280 00:17:19,716 --> 00:17:23,196 Speaker 1: how do we on the query side help users just 281 00:17:23,436 --> 00:17:30,156 Speaker 1: naturally intuitively express whatever information need they have. And then 282 00:17:30,276 --> 00:17:34,276 Speaker 1: how do we understand the whole universe of information, not 283 00:17:34,396 --> 00:17:38,356 Speaker 1: just the web pages, that all the images and video 284 00:17:38,516 --> 00:17:43,636 Speaker 1: and audio out there, and take that next level of 285 00:17:43,676 --> 00:17:47,756 Speaker 1: like concept understanding to match those together so that we 286 00:17:47,796 --> 00:17:53,716 Speaker 1: can get users even more precise answers that really help them. Great. 287 00:17:53,836 --> 00:18:00,356 Speaker 1: So that's the like vast dream slash big problem. Can 288 00:18:00,396 --> 00:18:02,356 Speaker 1: we reduce it a little bit so we can talk 289 00:18:02,396 --> 00:18:05,036 Speaker 1: in sort of practical terms about what you're working on. 290 00:18:05,076 --> 00:18:08,556 Speaker 1: I mean, I know there's this new AI model that 291 00:18:09,156 --> 00:18:12,516 Speaker 1: integrates images, like you can, you know, whatever, take a 292 00:18:12,556 --> 00:18:14,876 Speaker 1: picture with the camera on your phone and put in text. 293 00:18:14,996 --> 00:18:17,156 Speaker 1: So like, well, you have this new model, and it, 294 00:18:17,356 --> 00:18:20,676 Speaker 1: like the old one, has this worm fuzzy acronym. Right, 295 00:18:20,716 --> 00:18:23,516 Speaker 1: it's called MUM, which stands for hold on, I gotta 296 00:18:23,516 --> 00:18:27,756 Speaker 1: look at my notes, the multitask unified model. So like, 297 00:18:28,076 --> 00:18:33,036 Speaker 1: tell me about MUM. So MOM is our next level 298 00:18:33,076 --> 00:18:36,676 Speaker 1: model that you know Bert was about language. MOM is 299 00:18:36,716 --> 00:18:41,316 Speaker 1: about all these different modalities of information coming together, particular 300 00:18:41,516 --> 00:18:44,636 Speaker 1: images and language. I mean, is that if really images 301 00:18:44,676 --> 00:18:49,596 Speaker 1: and language and we've got some limited applications of it 302 00:18:49,636 --> 00:18:53,876 Speaker 1: in search today. So for example, you can take the 303 00:18:54,276 --> 00:18:58,756 Speaker 1: take the photo of somebody's handbag and say you want 304 00:18:58,756 --> 00:19:01,396 Speaker 1: to shop it, and that will work today. And that 305 00:19:01,556 --> 00:19:04,036 Speaker 1: is like we were not able to do this previously, 306 00:19:04,116 --> 00:19:06,076 Speaker 1: and that in and of itself is a big breakthrough. 307 00:19:06,516 --> 00:19:09,516 Speaker 1: But there's still so much headroom, right like, this, still 308 00:19:09,556 --> 00:19:13,516 Speaker 1: so much ability to say, you know, you can add 309 00:19:13,556 --> 00:19:18,476 Speaker 1: sort of I would I would classify our current ability 310 00:19:18,516 --> 00:19:22,236 Speaker 1: to process words in this multimodal context as you know, 311 00:19:22,396 --> 00:19:24,396 Speaker 1: kind of like back in the early days days of 312 00:19:24,436 --> 00:19:26,636 Speaker 1: the internet, you can say near me to find where 313 00:19:26,636 --> 00:19:29,196 Speaker 1: you can buy it. Near me, you can say buy, 314 00:19:29,396 --> 00:19:34,396 Speaker 1: but you can't necessarily like ask an incredibly complicated question 315 00:19:34,956 --> 00:19:38,116 Speaker 1: about a picture, right like, so we're kind of back 316 00:19:38,156 --> 00:19:42,276 Speaker 1: to keywords in this new pictures plus words universe. Let 317 00:19:42,316 --> 00:19:45,356 Speaker 1: me ask a dumb question, why why can't you just 318 00:19:45,436 --> 00:19:51,036 Speaker 1: take all of your brilliant intent AI and copy and 319 00:19:51,116 --> 00:19:56,516 Speaker 1: paste it to fit with the image AI. So a 320 00:19:56,556 --> 00:20:01,716 Speaker 1: couple of things. The first is that anytime we develop 321 00:20:02,236 --> 00:20:04,876 Speaker 1: sort of this new technology, we also need to see 322 00:20:04,916 --> 00:20:07,916 Speaker 1: how users start using it, right And so I think 323 00:20:07,956 --> 00:20:12,156 Speaker 1: it's also fed say that we don't have. You know, 324 00:20:12,196 --> 00:20:14,196 Speaker 1: we have a ton of people using this, but we 325 00:20:15,036 --> 00:20:18,156 Speaker 1: haven't yet. There hasn't been time for that new technology 326 00:20:18,196 --> 00:20:21,156 Speaker 1: to really be accepted by the world. And then we 327 00:20:21,196 --> 00:20:25,396 Speaker 1: have this vast set of queries that we're doing poorly on, right. 328 00:20:25,436 --> 00:20:27,276 Speaker 1: So that's the other thing you should know about Google. 329 00:20:27,516 --> 00:20:29,556 Speaker 1: We spend a lot of time looking at the queries 330 00:20:29,596 --> 00:20:31,356 Speaker 1: where we're failing. That's one of the other reasons we 331 00:20:31,396 --> 00:20:33,876 Speaker 1: have a deep appreciation of how search is an unsolved 332 00:20:33,916 --> 00:20:37,236 Speaker 1: problem because we're just constantly looking at queries whether the 333 00:20:37,316 --> 00:20:40,716 Speaker 1: users clearly not getting what they're looking for. And I'll 334 00:20:41,116 --> 00:20:45,156 Speaker 1: use that as a as a siege to figure out 335 00:20:45,196 --> 00:20:47,836 Speaker 1: how to make things better. So do I understand you 336 00:20:47,916 --> 00:20:50,196 Speaker 1: that the fundamental thing you need now is just lots 337 00:20:50,236 --> 00:20:52,716 Speaker 1: of people to use this thing so you can see 338 00:20:53,436 --> 00:20:56,156 Speaker 1: the weird ways people search and the things they sort 339 00:20:56,196 --> 00:21:00,236 Speaker 1: of do that are hard to understand. That's certainly one 340 00:21:00,276 --> 00:21:03,756 Speaker 1: of the things we need. I mean, it is fundamentally 341 00:21:04,316 --> 00:21:07,636 Speaker 1: search works in service of our users, right, and so 342 00:21:07,876 --> 00:21:13,196 Speaker 1: understanding the the failures is critical to how we get better. 343 00:21:13,396 --> 00:21:15,476 Speaker 1: I think there are also just things that we know 344 00:21:15,556 --> 00:21:17,956 Speaker 1: that we need to do on the AI and the 345 00:21:17,996 --> 00:21:22,396 Speaker 1: model side that we'll continue working through, right, the ability 346 00:21:22,476 --> 00:21:28,156 Speaker 1: to really bring together more of the two step process 347 00:21:28,196 --> 00:21:31,636 Speaker 1: of how do you conceptually understand the words, conceptually understand 348 00:21:31,636 --> 00:21:35,076 Speaker 1: the image, and then bring those two things together and 349 00:21:35,156 --> 00:21:37,556 Speaker 1: have that be a bit deeper on both sides rather 350 00:21:37,636 --> 00:21:40,716 Speaker 1: than just the combination together and those sorts of things. 351 00:21:41,676 --> 00:21:44,236 Speaker 1: But yeah, I mean people coming in and using it 352 00:21:44,276 --> 00:21:46,636 Speaker 1: and then having a bad time, we'll then make it better. 353 00:21:48,276 --> 00:21:53,316 Speaker 1: It seems like there have been two main threads of 354 00:21:53,396 --> 00:21:57,476 Speaker 1: AI research. One is basically language and the other is 355 00:21:57,876 --> 00:22:02,796 Speaker 1: basically vision and images. I mean, it is it right 356 00:22:02,836 --> 00:22:05,196 Speaker 1: to think of what you're trying to do as the 357 00:22:05,236 --> 00:22:10,756 Speaker 1: synthesis of those two sort of main AI traditions. Yeah, 358 00:22:10,796 --> 00:22:15,956 Speaker 1: I think so. I think it is clearly the case 359 00:22:16,076 --> 00:22:20,196 Speaker 1: that just like uh, you know, with Bert, we took 360 00:22:20,196 --> 00:22:23,236 Speaker 1: all these words and we got down to concepts. Right, 361 00:22:23,476 --> 00:22:26,236 Speaker 1: It is clearly the case that human beings understand the 362 00:22:26,276 --> 00:22:28,756 Speaker 1: world through concepts, and they do that visually, and they 363 00:22:28,796 --> 00:22:32,876 Speaker 1: do that with language, and ultimately the concepts are the same. Right, 364 00:22:32,956 --> 00:22:36,516 Speaker 1: So being able being able to say, okay, here's here's 365 00:22:36,516 --> 00:22:39,516 Speaker 1: a concept, and we can attach to that what that 366 00:22:39,596 --> 00:22:43,076 Speaker 1: concept looks like or that visual representation of that concept 367 00:22:43,116 --> 00:22:45,876 Speaker 1: as much as it has one and the words surrounding 368 00:22:45,916 --> 00:22:50,196 Speaker 1: that concept. That's when we can really unlock this true 369 00:22:51,596 --> 00:22:56,076 Speaker 1: natural way of understanding the world that we think is 370 00:22:56,436 --> 00:22:58,836 Speaker 1: going to enable people to ask all those questions that 371 00:22:58,876 --> 00:23:02,356 Speaker 1: they have that they're not asking right now. Are there 372 00:23:03,276 --> 00:23:08,756 Speaker 1: applications that go beyond search that come to mind if 373 00:23:08,756 --> 00:23:13,316 Speaker 1: you figure this out? Yeah, I mean I think that 374 00:23:16,316 --> 00:23:20,596 Speaker 1: search has this connotation of kind of find what's out there. 375 00:23:22,316 --> 00:23:26,476 Speaker 1: I think there's something, you know, we're thinking about what 376 00:23:26,516 --> 00:23:31,156 Speaker 1: this looks like in the generative space. So for example, 377 00:23:31,196 --> 00:23:35,836 Speaker 1: if you're looking for you know, I bake birthday cakes, 378 00:23:36,356 --> 00:23:40,836 Speaker 1: and sometimes I for my kids, and sometimes what my 379 00:23:40,916 --> 00:23:43,636 Speaker 1: kids want, and a birthday cake just actually doesn't exist 380 00:23:43,796 --> 00:23:46,956 Speaker 1: on the internet right or there's like only one or two, 381 00:23:47,156 --> 00:23:48,956 Speaker 1: So then I have to come up with it myself. 382 00:23:48,996 --> 00:23:54,716 Speaker 1: And like, what if AI could help us generate as 383 00:23:54,756 --> 00:23:59,156 Speaker 1: sample image just based on these concepts that I could 384 00:23:59,196 --> 00:24:02,716 Speaker 1: then use for inspiration. I think that's a pretty interesting concept. 385 00:24:03,476 --> 00:24:05,356 Speaker 1: There's obviously a lot of things that we need to 386 00:24:05,356 --> 00:24:07,996 Speaker 1: be very thoughtful about in this space as we do it, 387 00:24:09,396 --> 00:24:12,996 Speaker 1: But I think this idea of extending search past the 388 00:24:13,076 --> 00:24:15,916 Speaker 1: notion of connecting you with the information that's out there. 389 00:24:15,956 --> 00:24:21,676 Speaker 1: To actually synthesizing new information for you is pretty interesting 390 00:24:21,876 --> 00:24:25,156 Speaker 1: and something we're talking about a lot. You know. One 391 00:24:25,196 --> 00:24:27,316 Speaker 1: of the things that has become clear to me talking 392 00:24:27,356 --> 00:24:32,716 Speaker 1: with you is clearly, I think too narrowly about search. Right. 393 00:24:32,756 --> 00:24:35,756 Speaker 1: I have this very kind of twenty years ago idea 394 00:24:35,836 --> 00:24:41,636 Speaker 1: of like searching text on the web, and the web 395 00:24:41,716 --> 00:24:45,516 Speaker 1: has become much less text based in that time. Right, 396 00:24:46,876 --> 00:24:50,196 Speaker 1: the web includes Instagram, the web includes TikTok, and those 397 00:24:50,196 --> 00:24:54,196 Speaker 1: are places where, weirdly to me, lots of people go 398 00:24:54,316 --> 00:24:56,956 Speaker 1: to search like people go on TikTok to find whatever, 399 00:24:56,996 --> 00:24:58,716 Speaker 1: where to go out to eat, which would never occur 400 00:24:58,796 --> 00:25:01,796 Speaker 1: to me. So I mean it's that part of the 401 00:25:01,916 --> 00:25:06,596 Speaker 1: sort of motivation on some level for you to figure out, Oh, right, text, 402 00:25:06,636 --> 00:25:08,516 Speaker 1: that's not enough, clue, we got to figure out how 403 00:25:08,556 --> 00:25:10,596 Speaker 1: to search in video and what does that even mean. 404 00:25:12,156 --> 00:25:15,596 Speaker 1: I think we're really driven by what our users are 405 00:25:15,596 --> 00:25:19,396 Speaker 1: telling us, and we just have really robust mechanisms for 406 00:25:20,156 --> 00:25:23,556 Speaker 1: understanding what our users are doing. And it's pretty clear 407 00:25:23,756 --> 00:25:29,316 Speaker 1: that people around the world find image and video content 408 00:25:29,396 --> 00:25:34,276 Speaker 1: to be pretty compelling, right, I Mean that's sort of 409 00:25:34,316 --> 00:25:38,916 Speaker 1: a very obvious statement, but you know the Internet in 410 00:25:38,956 --> 00:25:41,796 Speaker 1: the early days, it was banned with constrained. It was 411 00:25:41,876 --> 00:25:45,276 Speaker 1: technology constrained. It had to be words because that's what 412 00:25:45,396 --> 00:25:50,556 Speaker 1: the technology enabled, not necessarily because that's what human beings 413 00:25:50,596 --> 00:25:54,276 Speaker 1: most enjoy in terms of an information consumption experience. And 414 00:25:54,356 --> 00:25:57,876 Speaker 1: so we really are driven by what we're seeing in 415 00:25:57,916 --> 00:26:00,676 Speaker 1: the user trends, and we're really driven by just this 416 00:26:00,716 --> 00:26:03,076 Speaker 1: mission of how do we keep helping people get the 417 00:26:03,116 --> 00:26:10,996 Speaker 1: best answers to their questions that we can give them. 418 00:26:11,036 --> 00:26:14,036 Speaker 1: In a minute, the Lightning round, including what you learn 419 00:26:14,116 --> 00:26:16,756 Speaker 1: about the Internet when you spend six years working at 420 00:26:16,756 --> 00:26:27,196 Speaker 1: Google Search. Now, let's get back to the show. I 421 00:26:27,236 --> 00:26:29,316 Speaker 1: want to do a lightning round. We close usually with 422 00:26:29,356 --> 00:26:32,316 Speaker 1: a lightning round on this show, just a bunch of 423 00:26:32,556 --> 00:26:37,876 Speaker 1: relatively quick questions. So in this instance, I googled best 424 00:26:37,956 --> 00:26:41,196 Speaker 1: Lightning Round questions, and right at the top of the 425 00:26:41,196 --> 00:26:43,076 Speaker 1: search results page I didn't even have to click through, 426 00:26:43,236 --> 00:26:45,036 Speaker 1: is this bulleted list. I'm just going to give you 427 00:26:45,076 --> 00:26:48,716 Speaker 1: a few from there. Sounds good. Favorite day of the week, 428 00:26:50,476 --> 00:26:57,076 Speaker 1: oh Monday, because I get to go to work and 429 00:26:57,516 --> 00:26:59,716 Speaker 1: not deal with my kids all day. Who I love 430 00:26:59,916 --> 00:27:06,116 Speaker 1: very daily. Good favorite city in US besides the one 431 00:27:06,156 --> 00:27:10,716 Speaker 1: you live in just reading here New York's City. Thank you. 432 00:27:12,156 --> 00:27:14,956 Speaker 1: Would you rather be able to speak every language in 433 00:27:14,956 --> 00:27:19,196 Speaker 1: the world or be able to talk to animals? Speak 434 00:27:19,236 --> 00:27:22,636 Speaker 1: every language in the world. I'm shocked, to be honest, 435 00:27:22,676 --> 00:27:24,796 Speaker 1: Although I get that like a Google might actually figure 436 00:27:24,796 --> 00:27:27,276 Speaker 1: that out. Does it not seem like you can already 437 00:27:27,316 --> 00:27:30,596 Speaker 1: get a translator for every language. Talking that animals would 438 00:27:30,636 --> 00:27:34,236 Speaker 1: be like a revolution and human understanding of the natural world, 439 00:27:35,076 --> 00:27:38,676 Speaker 1: I guess. But I do not speak any language as really, 440 00:27:38,916 --> 00:27:42,316 Speaker 1: and I constantly feel bad about it. So maybe that 441 00:27:42,396 --> 00:27:48,676 Speaker 1: was just a fair personal feeling of weakness. So okay, 442 00:27:48,836 --> 00:27:51,116 Speaker 1: so we're now we're pivoting out of the Google lightning 443 00:27:51,156 --> 00:27:57,116 Speaker 1: round questions into my own bespoke lightning round questions. What's 444 00:27:57,156 --> 00:28:02,196 Speaker 1: your favorite kind of cake to bake? Oh? Well, so 445 00:28:03,636 --> 00:28:09,476 Speaker 1: I really make these, like quite elaborate cakes for my 446 00:28:09,596 --> 00:28:13,156 Speaker 1: children because I want them to be able to grow 447 00:28:13,196 --> 00:28:15,476 Speaker 1: up and say, wow, I remember you making great cakes 448 00:28:15,476 --> 00:28:20,396 Speaker 1: for us. Mostly so I recently made in one of 449 00:28:20,396 --> 00:28:23,196 Speaker 1: my kids plays Minecraft a lot, and there is a 450 00:28:23,276 --> 00:28:26,276 Speaker 1: character called a slime, which is a sort of jelly 451 00:28:26,356 --> 00:28:28,316 Speaker 1: blob that kind of jumps on top of you and 452 00:28:28,436 --> 00:28:30,556 Speaker 1: kills you if you don't fight it off. And so 453 00:28:30,596 --> 00:28:33,556 Speaker 1: I made a slime cake with a cake embedded in 454 00:28:34,316 --> 00:28:38,396 Speaker 1: the jelly. So big idea one here? What do you 455 00:28:38,436 --> 00:28:43,876 Speaker 1: think you understand about the Internet that most people don't understand? Oh? 456 00:28:43,916 --> 00:28:47,356 Speaker 1: I like this one. I think most people don't understand 457 00:28:47,396 --> 00:28:50,876 Speaker 1: how much it changes every day. And you know, so 458 00:28:50,916 --> 00:28:54,076 Speaker 1: we have this astonishing stat that even I didn't believe 459 00:28:54,116 --> 00:28:56,116 Speaker 1: when I heard it, which is that fifteen percent of 460 00:28:56,116 --> 00:28:59,196 Speaker 1: the queries Google sees every day we have never seen before. 461 00:28:59,396 --> 00:29:01,916 Speaker 1: And that happens every day. That there's fifteen percent, I 462 00:29:02,036 --> 00:29:06,356 Speaker 1: just completely new. And the same happens on the internet side. 463 00:29:06,396 --> 00:29:09,396 Speaker 1: Every day we index a ton of new content we've 464 00:29:09,436 --> 00:29:13,116 Speaker 1: never seen before about ideas that are completely new to 465 00:29:13,236 --> 00:29:18,036 Speaker 1: humanity at that time, right, And you know, we have 466 00:29:18,116 --> 00:29:22,116 Speaker 1: to be able to continually understand that and keep up. 467 00:29:22,156 --> 00:29:25,036 Speaker 1: And I think that people sort of have this idea 468 00:29:25,116 --> 00:29:27,196 Speaker 1: that there's a fixed amount of information out there, but 469 00:29:27,236 --> 00:29:31,756 Speaker 1: actually human beings are astonishingly productive and are constantly coming 470 00:29:31,836 --> 00:29:35,476 Speaker 1: up with new ideas. If everything goes well, what problem 471 00:29:35,476 --> 00:29:38,556 Speaker 1: will you be trying to solve in five years, I 472 00:29:38,596 --> 00:29:41,636 Speaker 1: will still be working on making Google Search better for 473 00:29:41,716 --> 00:29:44,396 Speaker 1: all our users. I think we will I think we 474 00:29:44,436 --> 00:29:48,156 Speaker 1: will be working on this for the next hundred years. 475 00:29:48,236 --> 00:29:51,836 Speaker 1: Is there a narrower answer, like this particular problem you're 476 00:29:51,836 --> 00:29:57,076 Speaker 1: working on now of integrating image and words basically like 477 00:29:57,516 --> 00:30:00,036 Speaker 1: you think you'll obviously it won't be completely solved, but 478 00:30:00,076 --> 00:30:02,076 Speaker 1: you think that'll basically work. And if so, is there 479 00:30:02,116 --> 00:30:09,916 Speaker 1: a next thing? I think the problem of a video 480 00:30:10,316 --> 00:30:12,916 Speaker 1: I think will continue to be hard because there's just 481 00:30:13,036 --> 00:30:15,756 Speaker 1: such a large amount of information in a given video. 482 00:30:17,276 --> 00:30:21,956 Speaker 1: The other problem that I'm really interested in is helping 483 00:30:22,676 --> 00:30:27,516 Speaker 1: people pause information with helpful context. So, like, how you 484 00:30:27,556 --> 00:30:31,676 Speaker 1: know we've unleashed the all of the world's information on people, 485 00:30:32,356 --> 00:30:36,156 Speaker 1: how do you actually help help them sift through that 486 00:30:36,236 --> 00:30:39,756 Speaker 1: and make good decisions, whether it's choosing a reliable merchant 487 00:30:39,796 --> 00:30:43,676 Speaker 1: to buy from or finding reliable medical information? How do 488 00:30:43,716 --> 00:30:46,196 Speaker 1: you help people make those decisions for themselves and be 489 00:30:46,316 --> 00:30:49,476 Speaker 1: literate with their information choices. What's one piece of advice 490 00:30:49,476 --> 00:30:54,996 Speaker 1: you'd give to someone trying to solve a hard problem. 491 00:30:55,036 --> 00:30:58,596 Speaker 1: I would say, find a really great group of people 492 00:30:58,716 --> 00:31:02,556 Speaker 1: to help solve up with you, because generally trying to 493 00:31:02,556 --> 00:31:05,836 Speaker 1: solve hard things by yourself enser being an active fustriction. 494 00:31:09,436 --> 00:31:12,356 Speaker 1: Kathy Edwards is vice president and g M of Search 495 00:31:12,596 --> 00:31:18,236 Speaker 1: at Google. Today's show was edited by Robert Smith, produced 496 00:31:18,276 --> 00:31:22,596 Speaker 1: by Edith Russelo, and engineered by Amanda k Waugh. I'm 497 00:31:22,676 --> 00:31:25,116 Speaker 1: Jacob Goldstein, and we'll be back next week with another 498 00:31:25,116 --> 00:31:31,636 Speaker 1: episode of What's Your Problem.