1 00:00:15,356 --> 00:00:21,956 Speaker 1: Pushkin. I'm Jacob Goldstein and this is What's Your Problem. 2 00:00:21,996 --> 00:00:25,076 Speaker 1: Two quick things before we start the show today. One, 3 00:00:25,716 --> 00:00:27,716 Speaker 1: we're going to take a little break of a few 4 00:00:27,716 --> 00:00:29,956 Speaker 1: weeks after this week's episode, but we'll be back soon. 5 00:00:30,436 --> 00:00:33,116 Speaker 1: And two, I want to say thanks for listening. I 6 00:00:33,196 --> 00:00:35,996 Speaker 1: really appreciate it. I want to keep making a show 7 00:00:36,036 --> 00:00:39,116 Speaker 1: that you like, so please feel free to email us 8 00:00:39,276 --> 00:00:42,476 Speaker 1: at problem at pushkin dot fm. You can also find 9 00:00:42,476 --> 00:00:45,836 Speaker 1: me on LinkedIn or on x formerly known as Twitter 10 00:00:46,076 --> 00:00:50,036 Speaker 1: at Jacob Goldstein. Okay, that's the housekeeping. Now here's today's show. 11 00:01:01,796 --> 00:01:06,076 Speaker 1: My guest today is Brian McClendon. He's the chief technology 12 00:01:06,076 --> 00:01:10,396 Speaker 1: officer at a company called Niantic Spatial, and his problem 13 00:01:10,516 --> 00:01:15,756 Speaker 1: is this, how do you make maps better? Specifically, how 14 00:01:15,756 --> 00:01:17,956 Speaker 1: do you take the two dimensional maps that we have 15 00:01:17,996 --> 00:01:21,836 Speaker 1: today and turn them into three dimensional maps that change 16 00:01:21,836 --> 00:01:24,676 Speaker 1: over time to keep up with changes in the real world. 17 00:01:25,236 --> 00:01:28,676 Speaker 1: Maps like this would be very useful for robots. For 18 00:01:28,756 --> 00:01:31,756 Speaker 1: one thing, Brian's company is in fact doing a pilot 19 00:01:31,796 --> 00:01:35,236 Speaker 1: with a robot delivery company, those little robots that drive 20 00:01:35,316 --> 00:01:37,636 Speaker 1: down the sidewalk to bring you a pizza or whatever. 21 00:01:38,196 --> 00:01:41,076 Speaker 1: But the maps could also be useful for people. For example, 22 00:01:41,116 --> 00:01:43,516 Speaker 1: the company is working on a project with the US 23 00:01:43,596 --> 00:01:47,436 Speaker 1: Coast Guard to build better flight simulators for helicopter pilots, 24 00:01:47,756 --> 00:01:50,436 Speaker 1: and the kind of maps that the company is building 25 00:01:50,596 --> 00:01:54,036 Speaker 1: could also be useful for augmented reality, and in fact, 26 00:01:54,476 --> 00:01:58,396 Speaker 1: Niantic Spatial spun out of the company that made Pokemon Go, 27 00:01:58,836 --> 00:02:02,956 Speaker 1: the augmented reality game. Brian's been working on maps for 28 00:02:02,996 --> 00:02:06,276 Speaker 1: a long time. In the early two thousands, he worked 29 00:02:06,316 --> 00:02:09,196 Speaker 1: at a mapping startup that got acquired by Google in 30 00:02:09,196 --> 00:02:11,796 Speaker 1: two thousand and four, and Google turned the project he 31 00:02:11,876 --> 00:02:15,396 Speaker 1: was working on into Google Earth, and after that, Brian 32 00:02:15,636 --> 00:02:19,596 Speaker 1: kept working on mapping at Google until twenty fifteen. In 33 00:02:19,636 --> 00:02:22,636 Speaker 1: our conversation, Brian and I discussed the problems he's working 34 00:02:22,636 --> 00:02:26,116 Speaker 1: on now trying to build better maps, including why trees 35 00:02:26,236 --> 00:02:29,476 Speaker 1: are his nemesis. Also, we talked about what it will 36 00:02:29,476 --> 00:02:33,436 Speaker 1: take to teach Ai to understand the physical world, but 37 00:02:33,516 --> 00:02:36,436 Speaker 1: to start. Brian told me the biggest lesson that he 38 00:02:36,596 --> 00:02:38,916 Speaker 1: learned about mapping during his time at Google. 39 00:02:39,236 --> 00:02:41,676 Speaker 2: The most interesting thing is that map data in two 40 00:02:41,716 --> 00:02:44,636 Speaker 2: thousand and four, you know, was already digital. We had 41 00:02:44,996 --> 00:02:47,796 Speaker 2: suppliers that you could buy map data from, and you know, 42 00:02:47,876 --> 00:02:50,596 Speaker 2: map quest was out there showing you a little little 43 00:02:50,596 --> 00:02:53,436 Speaker 2: map on the screen on your website, on their website. 44 00:02:53,836 --> 00:02:57,396 Speaker 2: But what we discovered when we started build Google Maps, 45 00:02:57,476 --> 00:02:59,636 Speaker 2: we used that same data and we showed it to 46 00:02:59,676 --> 00:03:03,636 Speaker 2: our users and launched in two thousand and five. We 47 00:03:03,676 --> 00:03:05,476 Speaker 2: came up with this idea of street view, where we'd 48 00:03:05,516 --> 00:03:08,196 Speaker 2: go take pictures of a lot of places around around 49 00:03:08,276 --> 00:03:10,316 Speaker 2: the city and publish them. 50 00:03:10,796 --> 00:03:12,916 Speaker 1: Can I say, by the way, because I remember when 51 00:03:12,956 --> 00:03:16,676 Speaker 1: that happened and when I heard of that, and I 52 00:03:16,716 --> 00:03:18,676 Speaker 1: when I first heard of it, I was like, surely 53 00:03:18,716 --> 00:03:22,276 Speaker 1: they're not going to drive cars with cameras on every 54 00:03:22,356 --> 00:03:26,356 Speaker 1: street and take pictures, Like, surely that's too much. But no, 55 00:03:26,596 --> 00:03:30,156 Speaker 1: that's what it was like so wildly ambitious, just on 56 00:03:30,196 --> 00:03:31,476 Speaker 1: a logistical level. 57 00:03:31,636 --> 00:03:34,316 Speaker 3: Absolutely, that was It was a crazy project. 58 00:03:34,636 --> 00:03:36,796 Speaker 2: You know, Larry Page gets credit for coming up with 59 00:03:36,876 --> 00:03:39,556 Speaker 2: the idea and then following through and supporting us as 60 00:03:39,876 --> 00:03:40,436 Speaker 2: we built it. 61 00:03:40,956 --> 00:03:44,156 Speaker 1: What did you say when someone was it Larry Page 62 00:03:44,196 --> 00:03:46,116 Speaker 1: who said to you, hey, let's do this, Who told 63 00:03:46,156 --> 00:03:46,556 Speaker 1: you about it? 64 00:03:47,116 --> 00:03:49,996 Speaker 2: You know, Larry, Larry had actually taken a video camera 65 00:03:50,076 --> 00:03:53,156 Speaker 2: and driven around the Stanford campus, you know, you know, 66 00:03:53,356 --> 00:03:56,036 Speaker 2: sort of systematically and said, you know, why can't we 67 00:03:56,076 --> 00:03:57,116 Speaker 2: do this for the whole world? 68 00:03:57,676 --> 00:04:00,156 Speaker 1: What did you say when he said, why can't we 69 00:04:00,156 --> 00:04:01,876 Speaker 1: do this for the whole world? What did you say? 70 00:04:02,276 --> 00:04:04,516 Speaker 3: I said, he's Larry Page and it's his money. Man. 71 00:04:04,516 --> 00:04:06,716 Speaker 3: If he wants to do it, we'll give it a shot. 72 00:04:06,756 --> 00:04:10,596 Speaker 2: And and honestly that's you know, the support that Google 73 00:04:10,636 --> 00:04:14,476 Speaker 2: had for organizing the world's information was incredibly strong. 74 00:04:14,516 --> 00:04:16,036 Speaker 3: I mean, Larry was serious about this. 75 00:04:16,196 --> 00:04:19,876 Speaker 2: And we initially built a very expensive data collection van 76 00:04:20,156 --> 00:04:23,356 Speaker 2: that drove around a few cities. Then we started to 77 00:04:23,476 --> 00:04:28,516 Speaker 2: drive other cars that were cheaper but still okay, and 78 00:04:28,556 --> 00:04:31,716 Speaker 2: then that's when we launched the service. And then we 79 00:04:31,796 --> 00:04:33,956 Speaker 2: eventually said we need to do this for the entire 80 00:04:34,036 --> 00:04:37,836 Speaker 2: United States. And the reason we did this was that 81 00:04:37,956 --> 00:04:41,276 Speaker 2: when we launched the first cities, the first thing that 82 00:04:41,356 --> 00:04:44,516 Speaker 2: people said was they looked at the pictures on the screen, 83 00:04:44,716 --> 00:04:47,316 Speaker 2: and then they looked at their map data, and they 84 00:04:47,316 --> 00:04:49,756 Speaker 2: started complaining about the map data. How can you get 85 00:04:49,796 --> 00:04:52,116 Speaker 2: this wrong? I have a picture right here, this shows 86 00:04:52,116 --> 00:04:54,436 Speaker 2: a street sign and a street number, and you have 87 00:04:54,516 --> 00:04:57,436 Speaker 2: it completely in a different location. So a picture is 88 00:04:57,476 --> 00:05:01,356 Speaker 2: worth a thousand words and probably megabytes of actual useful data. 89 00:05:01,996 --> 00:05:06,436 Speaker 2: And there hadn't been any systematic sort of extraction of 90 00:05:06,476 --> 00:05:08,876 Speaker 2: the value from those pictures, because that's not how maps 91 00:05:08,876 --> 00:05:11,516 Speaker 2: had been made in the past. And so the big 92 00:05:11,556 --> 00:05:14,556 Speaker 2: insight we ever had was that map data to that 93 00:05:14,716 --> 00:05:16,316 Speaker 2: moment was not good. 94 00:05:16,676 --> 00:05:20,116 Speaker 1: And by taking pictures for the first time, people could 95 00:05:20,196 --> 00:05:25,476 Speaker 1: in a scaled way look at a photo right next 96 00:05:25,516 --> 00:05:28,436 Speaker 1: to right on top of the map data and see like, oh, 97 00:05:28,476 --> 00:05:30,756 Speaker 1: the map is wrong, and this photo proves that. Like 98 00:05:30,796 --> 00:05:33,716 Speaker 1: it was the moment when people understood how wrong maps 99 00:05:33,716 --> 00:05:35,436 Speaker 1: in general were exactly. 100 00:05:35,716 --> 00:05:38,156 Speaker 2: And people always kind of knew this because they always 101 00:05:38,156 --> 00:05:40,156 Speaker 2: got frustrated every now and then when they'd use the 102 00:05:40,196 --> 00:05:43,516 Speaker 2: maps and they would fail. But now we had this 103 00:05:43,796 --> 00:05:48,196 Speaker 2: much larger complaint body, and we knew that the data 104 00:05:48,196 --> 00:05:50,516 Speaker 2: that we'd been buying from third parties wasn't good, and 105 00:05:50,596 --> 00:05:52,996 Speaker 2: so we decided to make our own maps. And the 106 00:05:53,036 --> 00:05:56,276 Speaker 2: biggest basis for that was the street view pictures themselves. 107 00:05:56,916 --> 00:05:59,876 Speaker 2: And we drove all of the US, Canada, Mexico in 108 00:05:59,996 --> 00:06:03,476 Speaker 2: two thousand and eight, and by two thousand and thirteen 109 00:06:03,516 --> 00:06:04,796 Speaker 2: we had driven fifty countries. 110 00:06:05,876 --> 00:06:08,156 Speaker 1: So the point of street view was not just for 111 00:06:08,236 --> 00:06:11,436 Speaker 1: the like, hey, you can look at pictures, it's to 112 00:06:11,516 --> 00:06:14,196 Speaker 1: get better data as an input to the sort of 113 00:06:14,236 --> 00:06:19,236 Speaker 1: more traditional maps. Absolutely, I never knew that. Is that 114 00:06:19,276 --> 00:06:23,276 Speaker 1: why Apple Maps sucked when it first came out, because 115 00:06:23,316 --> 00:06:23,996 Speaker 1: they didn't have them. 116 00:06:24,116 --> 00:06:26,556 Speaker 3: I'm so glad you said. It keeps me from saying that. 117 00:06:26,996 --> 00:06:29,756 Speaker 1: I remember when it happened. I remember they fired the guy. 118 00:06:29,876 --> 00:06:33,236 Speaker 1: Remember they fired the guy. But is that why? Because 119 00:06:33,236 --> 00:06:37,116 Speaker 1: they just were buying the normal map data and everybody 120 00:06:37,236 --> 00:06:39,156 Speaker 1: was comparing it to Google and you guys had literally 121 00:06:39,236 --> 00:06:42,236 Speaker 1: driven cars on every street in America to figure it out. 122 00:06:42,316 --> 00:06:45,476 Speaker 2: That's exactly right, and they eventually got their Apple Apple 123 00:06:45,556 --> 00:06:49,316 Speaker 2: has driven you know, many millions of miles themselves, But 124 00:06:49,396 --> 00:06:51,556 Speaker 2: if you look at it worldwide, I would say that 125 00:06:51,596 --> 00:06:52,916 Speaker 2: Google is still probably better. 126 00:06:55,596 --> 00:06:57,556 Speaker 1: I don't have a dog in that fight. It is 127 00:06:57,716 --> 00:07:02,796 Speaker 1: interesting how both Google and Apple had these We're just 128 00:07:02,876 --> 00:07:05,916 Speaker 1: throwing off crazy amounts of money, right each for their 129 00:07:05,956 --> 00:07:09,436 Speaker 1: own special reasons, but we're reinvesting it in an interesting 130 00:07:09,516 --> 00:07:13,036 Speaker 1: way that like x Ante, I would not have guessed. Right, 131 00:07:13,156 --> 00:07:16,636 Speaker 1: Like this sort of search giant and the fancy phone 132 00:07:16,676 --> 00:07:20,396 Speaker 1: maker becoming kind of going to the vanguard of mapping 133 00:07:21,316 --> 00:07:22,156 Speaker 1: is not obvious. 134 00:07:22,996 --> 00:07:26,436 Speaker 2: So I think when Steve Jobs added Google Maps to 135 00:07:26,836 --> 00:07:30,476 Speaker 2: the iPhone very very quickly. He realized it was strategic 136 00:07:31,156 --> 00:07:33,756 Speaker 2: and you know, he knew that because there was a 137 00:07:33,796 --> 00:07:36,636 Speaker 2: GPS on the phone and because he now had a 138 00:07:36,716 --> 00:07:40,916 Speaker 2: you know, pan and zoom capable touch screen, that he 139 00:07:41,076 --> 00:07:44,276 Speaker 2: had a you know, an amazing new method for helping 140 00:07:44,316 --> 00:07:45,596 Speaker 2: people understand the world. 141 00:07:45,676 --> 00:07:49,916 Speaker 3: And so getting that map data on there was very important. 142 00:07:49,916 --> 00:07:52,916 Speaker 2: And Android and iOS both had it very quickly. But 143 00:07:53,076 --> 00:07:55,196 Speaker 2: the moment you do that, you have the same problem 144 00:07:55,196 --> 00:07:57,556 Speaker 2: with street view showed was that people walk around, they 145 00:07:57,596 --> 00:07:59,116 Speaker 2: look at their map and they look up at the 146 00:07:59,116 --> 00:08:02,836 Speaker 2: street around them, and it's wrong. And so they knew that, 147 00:08:02,956 --> 00:08:05,596 Speaker 2: you know, the biggest complaint they would be getting was 148 00:08:05,636 --> 00:08:09,076 Speaker 2: this factual failure from their users about their maps. 149 00:08:09,596 --> 00:08:14,596 Speaker 1: Yeah, well a map that's wrong is worse than no map, arguably. 150 00:08:14,756 --> 00:08:14,996 Speaker 3: Yes. 151 00:08:16,236 --> 00:08:19,116 Speaker 1: I mean, were there surprises to you about, you know, 152 00:08:19,356 --> 00:08:22,356 Speaker 1: sort of what what developed, what didn't developed, how people 153 00:08:22,476 --> 00:08:25,836 Speaker 1: used Google Maps, things you thought might happen that didn't happen. 154 00:08:25,916 --> 00:08:29,196 Speaker 2: I think one of the things that you know, we 155 00:08:29,836 --> 00:08:34,756 Speaker 2: probably speculated about. But if you realized that Google Earth 156 00:08:34,876 --> 00:08:37,756 Speaker 2: and Maps launched in February and June of two thousand 157 00:08:37,796 --> 00:08:38,156 Speaker 2: and five. 158 00:08:38,996 --> 00:08:43,356 Speaker 3: In August, Katrina happened, and. 159 00:08:43,236 --> 00:08:45,676 Speaker 1: Katrina was Hurricane Katrina in New Orleans. 160 00:08:45,716 --> 00:08:49,636 Speaker 2: Yeah, yeah, it was an incredible hurricane, did huge amounts 161 00:08:49,676 --> 00:08:52,836 Speaker 2: of damage and the city of New Orleans was flooded, 162 00:08:52,956 --> 00:08:57,636 Speaker 2: you know, completely, and this turned into a challenge for 163 00:08:57,876 --> 00:09:00,596 Speaker 2: you know, everybody involved, you know, both for just escaping 164 00:09:00,636 --> 00:09:02,916 Speaker 2: the city and going to the right places, but then 165 00:09:02,956 --> 00:09:06,436 Speaker 2: it became about rescuing people on rooftops. And the problem 166 00:09:06,556 --> 00:09:09,716 Speaker 2: now was the streets were gone and they couldn't figure 167 00:09:09,716 --> 00:09:12,236 Speaker 2: out where things were. And so it turned out that 168 00:09:12,316 --> 00:09:16,636 Speaker 2: the helicopter pilots at the time used Google Earth to 169 00:09:17,436 --> 00:09:21,476 Speaker 2: find out where the address was on the picture and 170 00:09:21,516 --> 00:09:24,516 Speaker 2: then use the picture to navigate their helicopter to find 171 00:09:24,516 --> 00:09:28,196 Speaker 2: the people waiting on the roof. And there were literally 172 00:09:28,316 --> 00:09:32,676 Speaker 2: hundreds of rescues done using Google Earth's bipilots, you know, 173 00:09:32,756 --> 00:09:34,876 Speaker 2: a few months after we'd released it, and that was 174 00:09:34,956 --> 00:09:36,556 Speaker 2: that was a big surprise for us. 175 00:09:37,476 --> 00:09:41,036 Speaker 1: It is extremely interesting to me that the company right 176 00:09:41,076 --> 00:09:44,876 Speaker 1: now spun out of the company that made Pokemon Go, 177 00:09:45,036 --> 00:09:46,916 Speaker 1: and that there was this kind of kernel of an 178 00:09:46,956 --> 00:09:51,076 Speaker 1: idea which was like, oh, hey, people you have playing 179 00:09:51,116 --> 00:09:54,596 Speaker 1: Pokemon Go have taken a billion photos in the world, 180 00:09:54,676 --> 00:09:58,116 Speaker 1: like truly a billion photos. This is an interesting data set. 181 00:09:58,196 --> 00:10:00,476 Speaker 1: Maybe we can do something with it. I mean, if 182 00:10:00,516 --> 00:10:02,876 Speaker 1: I understand right, that was before you got there, But like, 183 00:10:03,236 --> 00:10:05,036 Speaker 1: how did that playoff? Did they know when they were 184 00:10:05,116 --> 00:10:07,396 Speaker 1: launching Pokemon Go that like, oh, we're going to get 185 00:10:07,436 --> 00:10:08,996 Speaker 1: this data? Did somebody realize it? 186 00:10:09,756 --> 00:10:13,556 Speaker 2: So interestingly, I think the uh, you know, John John 187 00:10:13,556 --> 00:10:16,756 Speaker 2: Hankey definitely had a very long term vision. He was 188 00:10:16,796 --> 00:10:19,596 Speaker 2: thinking about not just the game itself, but about games 189 00:10:19,596 --> 00:10:22,596 Speaker 2: in the real world. Now, the photos that were taken 190 00:10:22,716 --> 00:10:25,556 Speaker 2: initially by Pokemon Go players were you know, taking a 191 00:10:25,556 --> 00:10:28,596 Speaker 2: picture of a Pokemon into location. You know that those 192 00:10:28,596 --> 00:10:34,716 Speaker 2: photos stayed private. What happened was that John, seeing the future, said, 193 00:10:34,836 --> 00:10:38,116 Speaker 2: can we create a game mode where players consciously go 194 00:10:38,156 --> 00:10:41,156 Speaker 2: out and record videos for us of poke stops and 195 00:10:41,236 --> 00:10:44,516 Speaker 2: help us map these locations by recording videos. So it 196 00:10:44,556 --> 00:10:47,676 Speaker 2: was a conscious gameplay activity where you would get in 197 00:10:47,836 --> 00:10:51,916 Speaker 2: game rewards if you would upload a thirty second video 198 00:10:52,916 --> 00:10:53,996 Speaker 2: to the to the game. 199 00:10:54,036 --> 00:10:55,396 Speaker 3: And that was where that data came from. 200 00:10:55,476 --> 00:10:58,556 Speaker 2: So it was consciously contributed by the by the players 201 00:10:58,636 --> 00:11:00,716 Speaker 2: rather than sort of background collected. 202 00:11:00,996 --> 00:11:05,036 Speaker 1: And and what was what was the point of that 203 00:11:05,076 --> 00:11:06,796 Speaker 1: From the point of view of the company, so that 204 00:11:06,836 --> 00:11:09,876 Speaker 1: we wanted to create more ar gaming experiences. 205 00:11:10,396 --> 00:11:14,516 Speaker 2: And the Pokemon Go already used augmented reality to create 206 00:11:14,596 --> 00:11:17,036 Speaker 2: those photos that you were talking about. Put a Pokemon 207 00:11:17,116 --> 00:11:18,916 Speaker 2: sitting on a chair and take a picture of it, 208 00:11:19,036 --> 00:11:21,996 Speaker 2: or so it was already using augmented reality. What we 209 00:11:22,076 --> 00:11:24,396 Speaker 2: wanted to do is to be able to augment the 210 00:11:24,476 --> 00:11:28,076 Speaker 2: poke stops in the outdoors, and so to do that accurately, 211 00:11:28,076 --> 00:11:30,396 Speaker 2: you actually need to have a very accurate map of 212 00:11:30,436 --> 00:11:32,636 Speaker 2: those pokey stops, not just where they are with a 213 00:11:32,676 --> 00:11:35,716 Speaker 2: blue dot, but actually the three D model of the 214 00:11:35,716 --> 00:11:39,596 Speaker 2: benches around it and the statue itself, and so creating 215 00:11:39,636 --> 00:11:43,156 Speaker 2: that three D model and then being able to point 216 00:11:43,196 --> 00:11:45,676 Speaker 2: your phone at it and say, I know exactly where 217 00:11:45,676 --> 00:11:49,796 Speaker 2: you're standing relative to that statue. Now the Pokemon that 218 00:11:49,836 --> 00:11:52,356 Speaker 2: was placed by the last person is exactly the same 219 00:11:52,356 --> 00:11:53,956 Speaker 2: place that it is for you. 220 00:11:54,476 --> 00:11:57,676 Speaker 1: So you need a really sophisticated model of the physical 221 00:11:57,676 --> 00:12:00,196 Speaker 1: world for that to work, otherwise it'll be like floating 222 00:12:00,196 --> 00:12:02,156 Speaker 1: in space a foot from where it's supposed to be. 223 00:12:02,196 --> 00:12:05,356 Speaker 2: That it is exactly the problem we spent several years solving, 224 00:12:05,516 --> 00:12:08,756 Speaker 2: and we only launched the product that used it. I 225 00:12:08,796 --> 00:12:13,356 Speaker 2: think in twenty twenty three, and that was part of 226 00:12:13,596 --> 00:12:17,396 Speaker 2: Pokemon Go and it's called Pokemon Go playgrounds, and it's 227 00:12:17,396 --> 00:12:21,916 Speaker 2: the ability for players to leave Pokemon at locations in 228 00:12:21,916 --> 00:12:25,876 Speaker 2: interesting formations and interesting combinations and then have other players 229 00:12:25,916 --> 00:12:27,396 Speaker 2: discover them or add to them. 230 00:12:28,556 --> 00:12:32,196 Speaker 1: And at what point did somebody think, oh, we can 231 00:12:32,436 --> 00:12:35,836 Speaker 1: do something wholly other than gaming here, we can build 232 00:12:35,876 --> 00:12:37,916 Speaker 1: a new kind of map that is, you know, more 233 00:12:37,996 --> 00:12:39,396 Speaker 1: legible to a robot. 234 00:12:39,596 --> 00:12:43,036 Speaker 2: I think that in some people's minds, you know, that 235 00:12:43,156 --> 00:12:45,716 Speaker 2: was the thought from day one. I think John would 236 00:12:45,836 --> 00:12:48,836 Speaker 2: argue in his mind he was he was really thinking 237 00:12:48,876 --> 00:12:52,196 Speaker 2: in that direction. But I think, you know, from a 238 00:12:52,356 --> 00:12:56,116 Speaker 2: product perspective, in a business perspective, being able to allow 239 00:12:57,116 --> 00:13:01,316 Speaker 2: developers of applications to go scan their own places, not 240 00:13:01,356 --> 00:13:04,316 Speaker 2: Poke stops, but any location around the world and then 241 00:13:04,396 --> 00:13:07,676 Speaker 2: create these experiences. That was our first step. I think 242 00:13:07,716 --> 00:13:11,316 Speaker 2: the robotics ass inspect came into play, you know, more recently, 243 00:13:11,356 --> 00:13:13,916 Speaker 2: probably in the last eighteen to twenty four months, as 244 00:13:13,996 --> 00:13:18,556 Speaker 2: we realize that, you know, robots struggle with finding themselves 245 00:13:18,636 --> 00:13:21,436 Speaker 2: in the real world, and so they need a form 246 00:13:21,476 --> 00:13:23,196 Speaker 2: of maps, don't we all. 247 00:13:26,076 --> 00:13:37,676 Speaker 1: And we'll be back in just a minute. So let's 248 00:13:37,676 --> 00:13:40,756 Speaker 1: talk about where you are now, like what's in the 249 00:13:40,796 --> 00:13:43,236 Speaker 1: world now that you're making. 250 00:13:43,756 --> 00:13:46,716 Speaker 2: I think the uh, you know, the experience of working 251 00:13:46,796 --> 00:13:50,396 Speaker 2: with the data from the phones of the players taught 252 00:13:50,476 --> 00:13:53,596 Speaker 2: us a lot about how to reconstruct and create these 253 00:13:53,636 --> 00:13:57,596 Speaker 2: precise maps, not with super fancy survey tools but with 254 00:13:57,796 --> 00:13:59,036 Speaker 2: just consumer devices. 255 00:13:59,036 --> 00:14:01,916 Speaker 3: And what we got good at was using. 256 00:14:01,956 --> 00:14:05,116 Speaker 2: Low quality data, which I mean, it's still very good, 257 00:14:05,116 --> 00:14:08,316 Speaker 2: but it's you know, it's not professional great anything, and 258 00:14:08,516 --> 00:14:11,876 Speaker 2: being able to turn that into professional grade data. And 259 00:14:11,956 --> 00:14:15,676 Speaker 2: so I think that was the skill that we're bringing 260 00:14:15,676 --> 00:14:18,196 Speaker 2: into the enterprise applications that you hear now. 261 00:14:18,676 --> 00:14:20,396 Speaker 3: And we can do it not just at the scale 262 00:14:20,436 --> 00:14:22,716 Speaker 3: of a. 263 00:14:21,676 --> 00:14:24,476 Speaker 2: Statue or a single location, but we can do it 264 00:14:24,516 --> 00:14:28,036 Speaker 2: at city scale. We can collect an entire city and 265 00:14:28,316 --> 00:14:29,316 Speaker 2: help you find yourself. 266 00:14:29,956 --> 00:14:31,676 Speaker 1: Have you collected an entire city? 267 00:14:31,796 --> 00:14:32,116 Speaker 3: Yes? 268 00:14:33,036 --> 00:14:33,596 Speaker 1: What city? 269 00:14:34,756 --> 00:14:37,756 Speaker 2: We have all of San Francisco and pieces of other cities, 270 00:14:38,796 --> 00:14:42,676 Speaker 2: and they're usually collected from drone data. Basically commissioning and 271 00:14:42,716 --> 00:14:46,356 Speaker 2: saying we wanted a drone based camera, you know, follow 272 00:14:46,356 --> 00:14:50,076 Speaker 2: a path, take a picture every several meters and take 273 00:14:50,116 --> 00:14:55,596 Speaker 2: it from multiple angles, and then use photogrammetry to reconstruct 274 00:14:55,716 --> 00:14:58,716 Speaker 2: that city at a level of detail, you know, beyond 275 00:14:58,756 --> 00:14:59,756 Speaker 2: what Google can do it. 276 00:15:00,196 --> 00:15:02,676 Speaker 1: And when you say drone, you mean like a flying drone, 277 00:15:02,676 --> 00:15:05,036 Speaker 1: like a little quad copter what exactly. 278 00:15:04,636 --> 00:15:08,316 Speaker 2: A very lightweight, you know, very limited under two hundred 279 00:15:08,316 --> 00:15:09,836 Speaker 2: and fifty grams smallest ones. 280 00:15:09,876 --> 00:15:12,796 Speaker 1: And so what are you doing now with your three 281 00:15:12,876 --> 00:15:15,156 Speaker 1: D robot legible map of San Francisco. 282 00:15:15,636 --> 00:15:19,156 Speaker 2: We are combining all of that interesting data collected on 283 00:15:19,196 --> 00:15:22,076 Speaker 2: the ground with phones and with three sixty cameras with 284 00:15:22,196 --> 00:15:26,756 Speaker 2: this drone data and also with satellite data and proving 285 00:15:26,796 --> 00:15:31,396 Speaker 2: out that we can help robots and humans find themselves, 286 00:15:31,396 --> 00:15:35,196 Speaker 2: localize themselves, you know, anywhere in the city without having 287 00:15:35,236 --> 00:15:37,676 Speaker 2: to pre map it, you know, specifically at that location. 288 00:15:38,716 --> 00:15:41,716 Speaker 1: So you mean like you a robot, It's just like 289 00:15:41,756 --> 00:15:44,836 Speaker 1: you blindfold the robot for lack of a better word, 290 00:15:44,876 --> 00:15:47,356 Speaker 1: and then take the blindfold off, and it knows right 291 00:15:47,396 --> 00:15:48,916 Speaker 1: away where it is and which way it's looking. 292 00:15:49,156 --> 00:15:50,316 Speaker 3: That's absolutely the goal. 293 00:15:50,436 --> 00:15:54,436 Speaker 1: Yes, what do you have to do to get there? 294 00:15:55,676 --> 00:15:58,276 Speaker 2: I think the challenge is that you know, the world 295 00:15:58,356 --> 00:16:01,196 Speaker 2: is a complicated and varied place, and you know, trees. 296 00:16:01,596 --> 00:16:05,676 Speaker 2: Trees are my nemesis in my career, and you know 297 00:16:05,716 --> 00:16:06,716 Speaker 2: they are they are absence. 298 00:16:06,756 --> 00:16:09,036 Speaker 1: I'm proachry. For the record, I'm approached. 299 00:16:09,156 --> 00:16:13,836 Speaker 2: I understand they are visually amazing, but they are in 300 00:16:13,876 --> 00:16:17,796 Speaker 2: fact fractally based. Right they have very high detail, and 301 00:16:17,836 --> 00:16:20,356 Speaker 2: worse than that, they're leaves. They flutter in the wind. 302 00:16:20,396 --> 00:16:24,596 Speaker 2: It's deeply inconvenient, and so when you take multiple pictures 303 00:16:24,636 --> 00:16:27,076 Speaker 2: of a tree, no two pictures have the tree in 304 00:16:27,116 --> 00:16:28,116 Speaker 2: the same spot. 305 00:16:28,516 --> 00:16:30,716 Speaker 1: When they change rapidly with tom. 306 00:16:30,596 --> 00:16:34,556 Speaker 2: Absolutely so another problem. The old way of doing this 307 00:16:34,796 --> 00:16:37,396 Speaker 2: was that, you know, satellites and airplanes would take pictures 308 00:16:37,396 --> 00:16:40,036 Speaker 2: of cities in the winter because the leaves were off, 309 00:16:40,436 --> 00:16:43,356 Speaker 2: and that gave them more information. But if you're trying 310 00:16:43,356 --> 00:16:45,596 Speaker 2: to build a model the city and localize, you actually 311 00:16:45,676 --> 00:16:48,276 Speaker 2: need to see them in all their states, not just 312 00:16:48,316 --> 00:16:49,156 Speaker 2: the leaf off state. 313 00:16:49,876 --> 00:16:52,956 Speaker 1: So is the short answer to how you solve that problem? AI? 314 00:16:53,076 --> 00:16:56,236 Speaker 2: How do you solve the tree problem? You try to 315 00:16:56,276 --> 00:16:59,396 Speaker 2: look beyond the trees. You know, trees other than the 316 00:16:59,436 --> 00:17:02,316 Speaker 2: tree trunk. There's not a lot of stability there, but 317 00:17:02,436 --> 00:17:05,036 Speaker 2: there are many things behind the tree or next to 318 00:17:05,076 --> 00:17:08,756 Speaker 2: the tree that are useful and being able to effectively 319 00:17:09,236 --> 00:17:12,676 Speaker 2: in an AI. To ignore the tree is usually your 320 00:17:12,716 --> 00:17:17,676 Speaker 2: best bet because there's enough remaining features in view that 321 00:17:17,716 --> 00:17:20,316 Speaker 2: you can find enough to connect yourself to that location. 322 00:17:23,076 --> 00:17:25,796 Speaker 1: I know you just made a deal with a delivery 323 00:17:25,916 --> 00:17:29,156 Speaker 1: robot company. Like, is that the first use case? Like, 324 00:17:29,156 --> 00:17:31,556 Speaker 1: what do you think the first use cases are going 325 00:17:31,636 --> 00:17:31,836 Speaker 1: to be? 326 00:17:32,156 --> 00:17:35,316 Speaker 2: I mean, definitely is the first use case they struggle with, 327 00:17:35,356 --> 00:17:39,716 Speaker 2: effectively the blindfold problem that sometimes they don't don't have 328 00:17:39,836 --> 00:17:41,596 Speaker 2: enough context to know where they are, and they need 329 00:17:41,596 --> 00:17:45,036 Speaker 2: to rediscover their location, and they need to need do 330 00:17:45,116 --> 00:17:49,076 Speaker 2: so accurately enough that either the autonomous controller or some 331 00:17:49,276 --> 00:17:51,356 Speaker 2: human who drops in to look at where it is 332 00:17:51,836 --> 00:17:55,076 Speaker 2: can know where they are in the city because city 333 00:17:55,076 --> 00:17:56,436 Speaker 2: blocks tend to look the same. 334 00:17:56,796 --> 00:17:58,516 Speaker 3: Unless you have that information. 335 00:17:58,636 --> 00:18:01,196 Speaker 1: How does the robot get lost in the first place? Like, 336 00:18:01,236 --> 00:18:03,356 Speaker 1: how does the robot find itself in that situation? 337 00:18:03,836 --> 00:18:05,356 Speaker 3: There's two ways. 338 00:18:05,396 --> 00:18:07,596 Speaker 2: One is I mean they do it they hit a reboot, 339 00:18:07,636 --> 00:18:12,636 Speaker 2: and they literally lose connection if they are cover if 340 00:18:12,636 --> 00:18:16,036 Speaker 2: for any reason enough of the world is blocked from 341 00:18:16,036 --> 00:18:20,556 Speaker 2: them for long enough, they lose lose control. GPS by 342 00:18:20,596 --> 00:18:24,156 Speaker 2: itself is rarely good enough because, especially in cities, the 343 00:18:24,236 --> 00:18:27,196 Speaker 2: GPS has you know, reflections off of buildings, and you 344 00:18:27,196 --> 00:18:30,636 Speaker 2: can be a city block away and the GPS will 345 00:18:30,636 --> 00:18:35,556 Speaker 2: happily tell you bigger there so. 346 00:18:33,676 --> 00:18:37,516 Speaker 1: Hu So it's insufficiently precise, particularly in cities. 347 00:18:37,596 --> 00:18:41,156 Speaker 2: Yes, and big cities with big, tall buildings are absolutely 348 00:18:41,196 --> 00:18:44,916 Speaker 2: the hardest challenge other than tunnels and parking critches. 349 00:18:45,716 --> 00:18:48,996 Speaker 1: But that's good for you. Like that that weakness is 350 00:18:49,036 --> 00:18:57,116 Speaker 1: your opportunity exactly. What tell me about the broader industry 351 00:18:57,196 --> 00:18:59,956 Speaker 1: is the right word, but the broader context, you know, 352 00:18:59,996 --> 00:19:02,836 Speaker 1: who else is working on things like you're working on. 353 00:19:03,596 --> 00:19:06,516 Speaker 2: I mean, I think we've had, you know, companies that 354 00:19:06,556 --> 00:19:09,396 Speaker 2: have worked on building digital twins of the world. They're 355 00:19:09,436 --> 00:19:12,516 Speaker 2: in the micro or the macro for you know, since 356 00:19:12,556 --> 00:19:17,276 Speaker 2: before Google, and Google certainly is the biggest wholesale attempt 357 00:19:17,276 --> 00:19:21,836 Speaker 2: at this, but there's a need for more resolution and 358 00:19:22,556 --> 00:19:26,156 Speaker 2: also to have it from suppliers other than Google, because 359 00:19:26,196 --> 00:19:28,516 Speaker 2: one of the areas that I think is hard is 360 00:19:28,556 --> 00:19:31,396 Speaker 2: that Google is all about recreating the public world, but 361 00:19:31,716 --> 00:19:34,556 Speaker 2: they don't really have any way for a company to 362 00:19:34,676 --> 00:19:38,876 Speaker 2: do their own location and add it to the real world. 363 00:19:39,916 --> 00:19:43,316 Speaker 1: Oh interesting, the hospital did I read hospital as an example. 364 00:19:43,316 --> 00:19:44,996 Speaker 1: We were talking about that when we're doing the prep 365 00:19:45,036 --> 00:19:47,836 Speaker 1: of like how hard it is to find your way 366 00:19:47,876 --> 00:19:48,796 Speaker 1: around a hospital? 367 00:19:49,196 --> 00:19:52,036 Speaker 3: It is, and I just experienced that yesterday at Stanford. 368 00:19:52,996 --> 00:19:53,436 Speaker 3: I think the. 369 00:19:54,916 --> 00:19:58,036 Speaker 2: Challenges building a map and with the hospital, there's actually 370 00:19:58,076 --> 00:20:00,516 Speaker 2: a lot of change that happens, you know, in the 371 00:20:00,516 --> 00:20:03,116 Speaker 2: corridors of a hospital at any given time. The big 372 00:20:03,196 --> 00:20:07,236 Speaker 2: quarriors are stable, but most of the actual patient areas 373 00:20:07,316 --> 00:20:11,116 Speaker 2: are you know, constantly changeing their visual appearance, and so 374 00:20:11,196 --> 00:20:15,036 Speaker 2: getting robots that are able to navigate that space accurately 375 00:20:15,036 --> 00:20:18,436 Speaker 2: and reliably is certainly one of our potential use cases. 376 00:20:19,796 --> 00:20:23,356 Speaker 1: So there's an app called scan a Verse that's you right, 377 00:20:23,956 --> 00:20:27,516 Speaker 1: the app scan a Verse. So I downloaded that this 378 00:20:27,596 --> 00:20:33,316 Speaker 1: week and astonishingly, they're estonishingly to me. Within one hundred 379 00:20:33,396 --> 00:20:37,676 Speaker 1: yards of my house, there are two things, two objects 380 00:20:37,676 --> 00:20:40,036 Speaker 1: that have been scanned in three D and uploaded to 381 00:20:40,076 --> 00:20:44,076 Speaker 1: this app, which is interesting cool. I had no idea 382 00:20:44,156 --> 00:20:45,996 Speaker 1: what's going on there? What is it? And why are 383 00:20:46,036 --> 00:20:49,556 Speaker 1: people on my block taking pictures of things. 384 00:20:49,716 --> 00:20:52,876 Speaker 2: Scan Versus is a great app, and we actually acquired 385 00:20:52,916 --> 00:20:53,916 Speaker 2: the one person. 386 00:20:53,596 --> 00:20:56,516 Speaker 3: Who wrote it, a brilliant programmer. 387 00:20:56,596 --> 00:21:00,956 Speaker 2: And turned it into this product that allows capture of 388 00:21:00,996 --> 00:21:03,796 Speaker 2: three D objects. And what you're seeing there is that 389 00:21:03,916 --> 00:21:05,716 Speaker 2: a lot of users love to be able to go 390 00:21:05,756 --> 00:21:08,476 Speaker 2: out and scan their area their location or a statue 391 00:21:08,596 --> 00:21:13,596 Speaker 2: or whatever, and recreated in three D visually almost completely 392 00:21:13,636 --> 00:21:16,516 Speaker 2: photo realistically, and then share that with the rest of 393 00:21:16,516 --> 00:21:19,636 Speaker 2: the world. And so you're experiencing sort of the map 394 00:21:19,676 --> 00:21:21,756 Speaker 2: that scanners puts up at the front of the front 395 00:21:21,796 --> 00:21:22,076 Speaker 2: of them. 396 00:21:23,236 --> 00:21:25,996 Speaker 1: So it's in the same way that people like to 397 00:21:26,036 --> 00:21:30,116 Speaker 1: do whatever geokashig and geo guessing and whatever. It's some 398 00:21:30,396 --> 00:21:33,636 Speaker 1: version of kind of sharing the real world in a 399 00:21:33,756 --> 00:21:34,436 Speaker 1: digital way. 400 00:21:34,556 --> 00:21:35,996 Speaker 3: Exactly is it big? 401 00:21:36,036 --> 00:21:39,116 Speaker 1: I infer from the fact that they're too on my 402 00:21:39,196 --> 00:21:40,876 Speaker 1: block or one on my block and one around the 403 00:21:40,876 --> 00:21:43,516 Speaker 1: corner that lots of people are using it? Is that true? 404 00:21:43,996 --> 00:21:45,956 Speaker 1: Do I live in a weirdly dense area? 405 00:21:46,636 --> 00:21:48,316 Speaker 2: There are a lot of people using We have hundreds 406 00:21:48,316 --> 00:21:53,956 Speaker 2: of thousands of users and they are passionate about this ability, 407 00:21:54,516 --> 00:21:58,596 Speaker 2: and we have used we've recently, very recently added the 408 00:21:58,756 --> 00:22:02,196 Speaker 2: enterprise capabilities to that app. People want to create a 409 00:22:02,236 --> 00:22:04,836 Speaker 2: digital twin of a tiny little place, they can do 410 00:22:04,876 --> 00:22:06,916 Speaker 2: it on their phone and upload it like you saw. 411 00:22:07,236 --> 00:22:10,756 Speaker 2: If they want to recreate their factor or their business, 412 00:22:11,196 --> 00:22:13,796 Speaker 2: you know, hundreds of square meters, they can do that 413 00:22:13,836 --> 00:22:18,316 Speaker 2: with our enterprise version and then allow for modeling and 414 00:22:18,356 --> 00:22:21,996 Speaker 2: allow for robotic training within their site using that tool. 415 00:22:22,996 --> 00:22:27,196 Speaker 1: So you can use your phone and walk around your factory, 416 00:22:27,276 --> 00:22:29,676 Speaker 1: say you're a little, you know, small factory or your 417 00:22:29,716 --> 00:22:33,996 Speaker 1: construction site or whatever, and the three D rendering it 418 00:22:34,036 --> 00:22:37,796 Speaker 1: generates is like robot legible. Then you can train your 419 00:22:37,916 --> 00:22:40,356 Speaker 1: robot on the thing you made with your iPhone. 420 00:22:40,396 --> 00:22:41,276 Speaker 3: That's correct. 421 00:22:41,996 --> 00:22:47,396 Speaker 1: So there's this phrase you hear an AI world model, right, 422 00:22:47,476 --> 00:22:50,916 Speaker 1: And it's sort of kind of a complement to a 423 00:22:50,996 --> 00:22:51,636 Speaker 1: language model. 424 00:22:51,676 --> 00:22:51,836 Speaker 3: Right. 425 00:22:51,836 --> 00:22:54,476 Speaker 1: Instead of AI that's trained on language, it's AI in 426 00:22:54,516 --> 00:22:57,596 Speaker 1: some way that's trained on the physical world. Like, tell 427 00:22:57,636 --> 00:22:59,556 Speaker 1: me about that and how your work fits in with that. 428 00:23:00,316 --> 00:23:03,796 Speaker 2: So the world models have been mostly focused to date 429 00:23:03,876 --> 00:23:06,756 Speaker 2: on solving a sort of very specific problem, which is 430 00:23:06,756 --> 00:23:12,276 Speaker 2: how to translate video, which they're mostly programmed with, into 431 00:23:12,956 --> 00:23:15,036 Speaker 2: understanding of the world that's good enough that it can 432 00:23:15,076 --> 00:23:15,956 Speaker 2: predict the future. 433 00:23:16,476 --> 00:23:18,236 Speaker 1: Yeah, and when you say predict the future, you don't 434 00:23:18,276 --> 00:23:20,516 Speaker 1: mean any like weird oracle thing, right, you mean like 435 00:23:20,796 --> 00:23:23,076 Speaker 1: predict the future as well as a human can that. Like, 436 00:23:23,116 --> 00:23:26,476 Speaker 1: if a fork starts to fall off the table, it's 437 00:23:26,476 --> 00:23:27,276 Speaker 1: going to fall all the way to. 438 00:23:27,276 --> 00:23:28,916 Speaker 3: The grounds, that's exactly right. 439 00:23:28,996 --> 00:23:32,716 Speaker 2: So if something is initiated, how would it play out 440 00:23:33,076 --> 00:23:35,796 Speaker 2: with the objects. And if you've seen enough forks fall 441 00:23:35,836 --> 00:23:38,556 Speaker 2: and videos, if you've seen enough objects fall, then you 442 00:23:38,556 --> 00:23:39,676 Speaker 2: can start to predict it. 443 00:23:39,716 --> 00:23:42,676 Speaker 1: And we were so used to now AI being so 444 00:23:42,716 --> 00:23:45,156 Speaker 1: good at language, it's easy to forget that it has 445 00:23:45,276 --> 00:23:49,716 Speaker 1: no understanding of the physical world. I mean except in language. 446 00:23:49,956 --> 00:23:52,556 Speaker 3: That's right, so weird. Yeah, it can, it can think 447 00:23:52,556 --> 00:23:53,116 Speaker 3: in its brain. 448 00:23:53,236 --> 00:23:55,596 Speaker 2: I know what the rules of Newton's law are, but 449 00:23:55,716 --> 00:23:57,876 Speaker 2: you know, turning that into pixels on the screen is 450 00:23:57,876 --> 00:24:01,916 Speaker 2: actually pretty hard. But if you train enough video models 451 00:24:01,916 --> 00:24:04,516 Speaker 2: on it, then then it can figure it out. And 452 00:24:04,556 --> 00:24:09,316 Speaker 2: so the I think the connecting the language model to 453 00:24:09,436 --> 00:24:13,796 Speaker 2: the physical world is in some ways. There's two ways 454 00:24:13,836 --> 00:24:15,996 Speaker 2: to do it, and one is to have the model 455 00:24:16,076 --> 00:24:19,276 Speaker 2: understand the physics video. The other is to turn the 456 00:24:19,316 --> 00:24:24,356 Speaker 2: world itself into words or into meaning. And this is 457 00:24:24,436 --> 00:24:28,076 Speaker 2: you know, typically called semantics. And there's all sorts of 458 00:24:28,756 --> 00:24:33,436 Speaker 2: you know, technologies that try to label information about a picture. 459 00:24:33,796 --> 00:24:37,076 Speaker 2: And if you train enough of models on the labeling 460 00:24:37,116 --> 00:24:39,516 Speaker 2: of the pictures, you can then label any but any 461 00:24:39,556 --> 00:24:41,116 Speaker 2: picture of the real world. 462 00:24:41,196 --> 00:24:44,276 Speaker 1: And this is kind of the first big breakthrough of 463 00:24:44,636 --> 00:24:46,596 Speaker 1: neural networks, right, I mean, it sounds like. 464 00:24:46,556 --> 00:24:48,556 Speaker 3: That cats, cats and dogs, cats and dogs. 465 00:24:48,636 --> 00:24:50,516 Speaker 1: Yeah, and image net right. 466 00:24:50,996 --> 00:24:54,516 Speaker 2: So if you give the information about the embeddings, it's 467 00:24:54,596 --> 00:24:57,076 Speaker 2: it's called and you do that not just in cat 468 00:24:57,116 --> 00:24:59,076 Speaker 2: and dog dimension, but you do it in hundreds of 469 00:24:59,116 --> 00:25:03,796 Speaker 2: dimensions of objects. You can give these language models a 470 00:25:03,916 --> 00:25:07,156 Speaker 2: the equivalent of a language understanding of the real world, 471 00:25:07,276 --> 00:25:09,636 Speaker 2: and then they can do amazing things with it. 472 00:25:09,916 --> 00:25:12,196 Speaker 1: Huh. And so how does all that relate to the 473 00:25:12,236 --> 00:25:12,876 Speaker 1: work you're doing. 474 00:25:13,636 --> 00:25:16,756 Speaker 2: We believe that to do that successfully you need to 475 00:25:16,796 --> 00:25:20,556 Speaker 2: start with three D data because the ability to label 476 00:25:20,636 --> 00:25:23,956 Speaker 2: something in three dimensions is much more accurate and with 477 00:25:24,076 --> 00:25:26,956 Speaker 2: and argue with much more information than if all you 478 00:25:26,996 --> 00:25:29,476 Speaker 2: have is a two D picture of it. Because in 479 00:25:29,516 --> 00:25:34,076 Speaker 2: two D pictures there's always, uh, you know, obscuration of data, 480 00:25:34,276 --> 00:25:37,516 Speaker 2: and things are behind things and you can't see the 481 00:25:37,516 --> 00:25:40,276 Speaker 2: whole picture. But in three D you eventually see everything 482 00:25:40,316 --> 00:25:41,036 Speaker 2: that is seeable. 483 00:25:41,876 --> 00:25:44,356 Speaker 1: I mean is that the fun not to be reductive, 484 00:25:44,396 --> 00:25:47,356 Speaker 1: but kind of to be reductive, is that the fundamental 485 00:25:47,476 --> 00:25:50,156 Speaker 1: thing you were doing that is novel is making a 486 00:25:50,196 --> 00:25:53,116 Speaker 1: three dimensional map and world where maps have always been 487 00:25:53,156 --> 00:25:53,836 Speaker 1: two dimensions. 488 00:25:53,836 --> 00:25:57,436 Speaker 2: Before I would say yes, I'm there, there are there 489 00:25:57,436 --> 00:26:00,996 Speaker 2: are three D visual maps. There are you know, rare 490 00:26:01,476 --> 00:26:04,436 Speaker 2: one off three maps, but trying to do a fully 491 00:26:05,156 --> 00:26:08,476 Speaker 2: language embedded three D semantic map of the world so 492 00:26:08,556 --> 00:26:12,396 Speaker 2: that a language, models and robots and others can understand 493 00:26:12,396 --> 00:26:13,396 Speaker 2: it is what we're working on. 494 00:26:15,276 --> 00:26:20,436 Speaker 1: So for zoom out like, well, two things, I guess 495 00:26:20,436 --> 00:26:21,956 Speaker 1: there's two things we'll do, the sad one and the 496 00:26:21,956 --> 00:26:24,436 Speaker 1: happy one. Like what might go wrong for you? Like 497 00:26:24,516 --> 00:26:26,476 Speaker 1: what has to go Yeah, what might go wrong for 498 00:26:26,516 --> 00:26:28,116 Speaker 1: you over the next years. It seems like you're at 499 00:26:28,156 --> 00:26:30,756 Speaker 1: this moment when you have this interesting technology, you're sort 500 00:26:30,756 --> 00:26:33,116 Speaker 1: of proving it out. It's not widely used yet. What 501 00:26:33,156 --> 00:26:34,196 Speaker 1: are the sort of pitfalls? 502 00:26:34,636 --> 00:26:37,196 Speaker 2: I mean, it was hard hard mapping the world in 503 00:26:37,236 --> 00:26:39,516 Speaker 2: two D for humans. Mapping the world in three D 504 00:26:39,716 --> 00:26:42,876 Speaker 2: four robots requires a level of detail and precision that 505 00:26:43,236 --> 00:26:46,676 Speaker 2: nobody's ever done before. And the only way to get 506 00:26:46,716 --> 00:26:49,316 Speaker 2: it done is to apply AI to the problem, to 507 00:26:49,316 --> 00:26:51,916 Speaker 2: figure out how to make it more accurate, more quickly 508 00:26:52,796 --> 00:26:56,756 Speaker 2: and more cheaply than has been done in the past, because, 509 00:26:57,476 --> 00:27:02,036 Speaker 2: you know, carefully reconstructing the world, every square meter of it. 510 00:27:02,156 --> 00:27:06,036 Speaker 2: The old way is you know, billions, if not trillion dollars, 511 00:27:06,476 --> 00:27:09,716 Speaker 2: and this can't be that nobody's going to pay that. 512 00:27:10,556 --> 00:27:16,996 Speaker 1: Yeah, billions maybe, but trillions definitely not. And so if 513 00:27:17,036 --> 00:27:22,636 Speaker 1: you if you get it right, like give me the 514 00:27:22,796 --> 00:27:25,756 Speaker 1: like the big exciting outcome, you know what I mean, 515 00:27:25,796 --> 00:27:27,916 Speaker 1: like if it goes, well, what's the world look like? 516 00:27:27,996 --> 00:27:28,916 Speaker 1: How is the world better? 517 00:27:29,836 --> 00:27:31,796 Speaker 2: I think that if you know, as we're seeing, the 518 00:27:32,196 --> 00:27:35,356 Speaker 2: ais have been able to dig into human knowledge and 519 00:27:35,516 --> 00:27:37,916 Speaker 2: you know ingest all of the text of all of 520 00:27:37,956 --> 00:27:41,196 Speaker 2: our scientific papers and all of our blog posts and 521 00:27:41,236 --> 00:27:44,756 Speaker 2: all of the Instagram and they've synthesized this in many 522 00:27:44,756 --> 00:27:48,636 Speaker 2: cases into some very interesting information that answers at least 523 00:27:48,676 --> 00:27:51,436 Speaker 2: you know, personally relevant questions today. And if you talk 524 00:27:51,476 --> 00:27:54,036 Speaker 2: to some mathematicians, you know they're they're they're doing pretty 525 00:27:54,036 --> 00:27:57,036 Speaker 2: well in math as well. But the questions about the 526 00:27:57,036 --> 00:28:00,236 Speaker 2: real world are are are simply not there yet. And 527 00:28:01,076 --> 00:28:04,436 Speaker 2: but the the advantage of an AI is it doesn't 528 00:28:04,756 --> 00:28:08,476 Speaker 2: stop and it doesn't stumble on too much data, and 529 00:28:08,596 --> 00:28:11,796 Speaker 2: if you can give it enough information about the world, 530 00:28:11,876 --> 00:28:15,036 Speaker 2: it can start to answer like really interesting questions about 531 00:28:15,196 --> 00:28:18,956 Speaker 2: the city, about how the city works, about which intersections 532 00:28:18,996 --> 00:28:23,236 Speaker 2: are you know too busy, and about you know, you know, 533 00:28:23,636 --> 00:28:26,716 Speaker 2: flow of people and you know I think things that 534 00:28:26,876 --> 00:28:29,316 Speaker 2: things that could make the city more efficient because you 535 00:28:29,436 --> 00:28:33,756 Speaker 2: have a more complete vision of it, you know, especially 536 00:28:34,236 --> 00:28:36,276 Speaker 2: in you know, three dimensions. You know, the two D 537 00:28:36,436 --> 00:28:38,396 Speaker 2: maps for a street are fine, but the moment you 538 00:28:38,396 --> 00:28:40,676 Speaker 2: go down to the subways in New York, some of 539 00:28:40,716 --> 00:28:43,316 Speaker 2: those are you know, three four levels deep, and some 540 00:28:43,356 --> 00:28:45,836 Speaker 2: of them are extremely popular in others are ghost towns. 541 00:28:45,916 --> 00:28:50,556 Speaker 2: And and you know, I think there's opportunities to improve 542 00:28:51,516 --> 00:28:55,396 Speaker 2: the infrastructure that we already have built without having to 543 00:28:55,556 --> 00:28:56,756 Speaker 2: like rebuild it from scrap. 544 00:29:00,996 --> 00:29:02,996 Speaker 1: We'll be back in a minute with the lightning round. 545 00:29:14,076 --> 00:29:17,236 Speaker 1: We're gonna finish with the lightning round. Tell me about 546 00:29:17,236 --> 00:29:19,596 Speaker 1: the Meadowbrook apartments in Lawrence, Kansas. 547 00:29:20,636 --> 00:29:24,036 Speaker 2: I grew up in Meadowbrook. It was a fine apartment complex. 548 00:29:24,716 --> 00:29:28,036 Speaker 2: I moved there when I was four and moved out 549 00:29:28,076 --> 00:29:34,676 Speaker 2: when I was eighteen. And so the center of Google 550 00:29:34,716 --> 00:29:38,596 Speaker 2: Earth on the windows PC in particular, will zoom you 551 00:29:38,636 --> 00:29:42,236 Speaker 2: into the bedroom of the apartment of my apartment building. 552 00:29:42,996 --> 00:29:48,436 Speaker 1: Still still yeah, that's a robust easter egg. 553 00:29:49,276 --> 00:29:51,716 Speaker 2: And there's another one in that category, which is that 554 00:29:51,796 --> 00:29:53,996 Speaker 2: I a friend of mine who I went to the 555 00:29:54,116 --> 00:29:57,636 Speaker 2: University of Kansas with, I hired him in two thousand 556 00:29:57,676 --> 00:30:00,636 Speaker 2: and five to port Google Earth to the Macintosh, and 557 00:30:00,756 --> 00:30:03,156 Speaker 2: he secretly changed the center of Google Earth on the 558 00:30:03,196 --> 00:30:07,396 Speaker 2: mac to be Shanook, Kansas, where he's from. But he 559 00:30:07,476 --> 00:30:11,436 Speaker 2: did it to the mains main intersection there in Chanut 560 00:30:11,916 --> 00:30:13,476 Speaker 2: and the city is she who loved it so much. 561 00:30:13,516 --> 00:30:16,236 Speaker 2: They actually built a mural on the street at that location. 562 00:30:16,436 --> 00:30:17,876 Speaker 2: So if you ever run a mac give it a 563 00:30:17,916 --> 00:30:19,076 Speaker 2: try and you'll see what it says. 564 00:30:22,276 --> 00:30:24,596 Speaker 1: What's one tip for becoming a pac Man champion? 565 00:30:27,036 --> 00:30:29,116 Speaker 3: The old old machines have patterns. 566 00:30:29,316 --> 00:30:31,996 Speaker 2: And when I was, you know, the very first pac 567 00:30:32,036 --> 00:30:35,956 Speaker 2: Man machine, if you knew the five key pattern and above, 568 00:30:36,556 --> 00:30:39,716 Speaker 2: you know I was able to break a million. They're 569 00:30:39,756 --> 00:30:41,916 Speaker 2: really professional people. I think the high score of the 570 00:30:41,916 --> 00:30:45,396 Speaker 2: Infinite Perfect Score is three point three million, but I 571 00:30:45,476 --> 00:30:48,076 Speaker 2: was able to get a million. But these days, you know, 572 00:30:48,196 --> 00:30:51,116 Speaker 2: I still play Miss pac Man at home. I have 573 00:30:51,116 --> 00:30:55,396 Speaker 2: a home machine, and you know, it's really about reaction, 574 00:30:55,836 --> 00:30:58,636 Speaker 2: and you know it's if you play it on fast mode, 575 00:30:58,676 --> 00:30:59,516 Speaker 2: it's all about speed. 576 00:31:01,276 --> 00:31:04,156 Speaker 1: Do I recall that Miss pac Man was more deterministic 577 00:31:04,196 --> 00:31:05,276 Speaker 1: than pac Man? Is that right? 578 00:31:05,396 --> 00:31:06,516 Speaker 3: It's it's the other way around. 579 00:31:06,516 --> 00:31:08,876 Speaker 2: When they launched Miss pac Man, they surprised people because 580 00:31:08,876 --> 00:31:12,276 Speaker 2: they had extra logic and they actually randomized it, so 581 00:31:12,516 --> 00:31:14,876 Speaker 2: literally no pattern really works even from the. 582 00:31:15,076 --> 00:31:17,716 Speaker 1: Oh okay, so it's a it's a kind of more 583 00:31:17,756 --> 00:31:19,876 Speaker 1: elegant game maybe at some level, or you could play 584 00:31:19,916 --> 00:31:22,276 Speaker 1: it for longer. Anyways, how are your skills? 585 00:31:23,796 --> 00:31:26,356 Speaker 2: My my I score is turned seventy nine thousand, My 586 00:31:26,356 --> 00:31:28,516 Speaker 2: friend's high score is turned in seventy five thousand and 587 00:31:28,756 --> 00:31:30,476 Speaker 2: and and that's where we're at. 588 00:31:32,636 --> 00:31:35,996 Speaker 1: Are you currently training to win any other video game championships? No? 589 00:31:36,156 --> 00:31:39,036 Speaker 3: I I my my only my only games. 590 00:31:40,716 --> 00:31:42,396 Speaker 2: I've played a lot of video games in the early 591 00:31:42,436 --> 00:31:45,236 Speaker 2: eighties and some Missile Command, miss pac Man, pac Man, 592 00:31:45,516 --> 00:31:47,436 Speaker 2: and Galaga. 593 00:31:47,756 --> 00:31:49,716 Speaker 3: I think we're one of my main games. 594 00:31:50,476 --> 00:31:53,516 Speaker 1: I liked the Big Ball on Missile Command, the Arcade 595 00:31:53,556 --> 00:31:56,636 Speaker 1: Missile Command, remember had that giant ball. 596 00:31:56,396 --> 00:31:59,556 Speaker 3: Almost a bowling ball, and it had had momentum. 597 00:32:00,716 --> 00:32:02,756 Speaker 1: But the the only games I. 598 00:32:02,716 --> 00:32:06,196 Speaker 2: Really play now are are the New York Times word games, 599 00:32:06,396 --> 00:32:09,476 Speaker 2: and I still play pokem let Go. 600 00:32:12,716 --> 00:32:15,956 Speaker 1: Do you have any like favorite maps from history or 601 00:32:15,996 --> 00:32:19,996 Speaker 1: whatever their maps you love, or like genius cartographers. 602 00:32:19,676 --> 00:32:22,396 Speaker 3: There's there's one map. 603 00:32:22,436 --> 00:32:25,516 Speaker 2: There's more of a sort of a data science presentation. 604 00:32:25,636 --> 00:32:27,956 Speaker 2: There was a tough tie was a guy who wrote 605 00:32:28,116 --> 00:32:31,916 Speaker 2: some really great books about data visualization, and he highlighted 606 00:32:31,916 --> 00:32:33,836 Speaker 2: this map which has always stuck with me. 607 00:32:33,876 --> 00:32:35,716 Speaker 3: And it was a single. 608 00:32:35,396 --> 00:32:39,276 Speaker 2: Sheet map that showed Napoleon's you know, march to Russia, 609 00:32:39,756 --> 00:32:43,196 Speaker 2: including the size of the army as it progressed, and 610 00:32:43,236 --> 00:32:46,436 Speaker 2: then it decimated itself in Russia and then shrunk and 611 00:32:46,516 --> 00:32:49,076 Speaker 2: shrunk and shrunk as it returned because they were, you know, 612 00:32:49,196 --> 00:32:52,676 Speaker 2: going through winter with no food and just the visual 613 00:32:52,756 --> 00:32:56,236 Speaker 2: The amount of information in that map, beyond the map 614 00:32:56,236 --> 00:32:57,436 Speaker 2: itself is incredible. 615 00:32:59,036 --> 00:33:00,636 Speaker 1: Anything you think shouldn't be mapped. 616 00:33:04,316 --> 00:33:08,236 Speaker 3: The inside of people's homes, you know, you have to 617 00:33:08,316 --> 00:33:09,076 Speaker 3: have a private space. 618 00:33:10,676 --> 00:33:11,956 Speaker 1: What was the last time you got lost? 619 00:33:15,156 --> 00:33:15,956 Speaker 3: It's a good question. 620 00:33:17,996 --> 00:33:22,676 Speaker 2: Never for very long, you know, I don't remember that. 621 00:33:22,716 --> 00:33:26,716 Speaker 2: Blue dot's pretty powerful. Tell me about catalog in your life. 622 00:33:28,276 --> 00:33:30,396 Speaker 2: So I started doing it in nineteen ninety three. I 623 00:33:30,516 --> 00:33:33,556 Speaker 2: started recording books, I read, movies I watched, and you know, 624 00:33:33,636 --> 00:33:36,716 Speaker 2: various other events in my life. And I actually backpropagated 625 00:33:36,876 --> 00:33:39,916 Speaker 2: and re remembered movies I'd seen before nineteen ninety three. 626 00:33:40,436 --> 00:33:42,956 Speaker 2: So I have a list of every book, every movie, 627 00:33:43,076 --> 00:33:47,116 Speaker 2: as well as many other life events all in a spreadsheet, 628 00:33:47,156 --> 00:33:51,076 Speaker 2: and it's you know, as I grow older, it is 629 00:33:51,116 --> 00:33:55,636 Speaker 2: you know, it becomes my memory because it's not super detailed, 630 00:33:56,156 --> 00:33:58,916 Speaker 2: but you know, in many cases just the event itself 631 00:33:58,956 --> 00:34:01,316 Speaker 2: existing reminds me of what I was doing that year, 632 00:34:01,396 --> 00:34:02,356 Speaker 2: that month, that day. 633 00:34:03,716 --> 00:34:05,996 Speaker 1: What like do you look back on it, like in 634 00:34:06,036 --> 00:34:08,436 Speaker 1: the way people might look at old photos or something like, 635 00:34:08,476 --> 00:34:10,396 Speaker 1: how do you how do you kind of use it 636 00:34:10,516 --> 00:34:11,196 Speaker 1: or engage with it? 637 00:34:11,236 --> 00:34:11,516 Speaker 3: I do. 638 00:34:11,556 --> 00:34:13,996 Speaker 2: I looked back at like what movies was I watching 639 00:34:14,036 --> 00:34:16,996 Speaker 2: in the last semester before I graduated high school? You know, 640 00:34:17,076 --> 00:34:20,276 Speaker 2: how did I spend my time that year? And was 641 00:34:20,316 --> 00:34:22,076 Speaker 2: what was available? And what's going on? 642 00:34:22,876 --> 00:34:26,276 Speaker 1: And you know you have qualitative notes? Do you have 643 00:34:26,436 --> 00:34:28,636 Speaker 1: ranking like are there interesting columns in those? 644 00:34:29,596 --> 00:34:31,916 Speaker 2: I've gone back through and I have my top forty 645 00:34:31,996 --> 00:34:34,836 Speaker 2: for let's see, what do I have my top forty 646 00:34:34,876 --> 00:34:37,356 Speaker 2: for books? And I have my top two hundred for movies, 647 00:34:37,396 --> 00:34:39,316 Speaker 2: and I have them somewhat ranked within. 648 00:34:39,196 --> 00:34:41,676 Speaker 1: That top movie, top book. 649 00:34:42,836 --> 00:34:47,276 Speaker 2: Let's see. I'll still say Contact for reasons. On the movie, 650 00:34:47,636 --> 00:34:51,556 Speaker 2: I just find it very compelling, And on the book, 651 00:34:53,116 --> 00:34:59,396 Speaker 2: I love Scholzie and Hugh Howie, So I almost want 652 00:34:59,436 --> 00:35:01,716 Speaker 2: to say Silo but I'll still say three body problem. 653 00:35:02,676 --> 00:35:04,836 Speaker 1: So so both both science fiction. 654 00:35:05,756 --> 00:35:09,036 Speaker 2: I am unapassionately almost entirely science fiction. 655 00:35:10,396 --> 00:35:13,476 Speaker 1: It's really I guess it shouldn't be surprising, but it's 656 00:35:13,516 --> 00:35:18,916 Speaker 1: particularly it's persistently notable to me how influential science fiction 657 00:35:19,196 --> 00:35:23,916 Speaker 1: is in the world. Like you know, the world we 658 00:35:23,956 --> 00:35:27,356 Speaker 1: live in, people built it in some significant degree because 659 00:35:27,396 --> 00:35:30,196 Speaker 1: they were inspired by science fiction in really specific ways. 660 00:35:30,516 --> 00:35:33,476 Speaker 2: Absolutely, Carl Sagan wrote Contact, and you know it was 661 00:35:33,596 --> 00:35:40,156 Speaker 2: hugely influential that way. But Neil Stephenson wrote snow Crash 662 00:35:40,236 --> 00:35:43,236 Speaker 2: and described a version of Google Earth before we built it, 663 00:35:43,316 --> 00:35:45,716 Speaker 2: and both John and I had read that book and 664 00:35:47,476 --> 00:35:49,956 Speaker 2: feel at least partially inspired by that. There were other 665 00:35:50,036 --> 00:35:53,636 Speaker 2: things that inspired us too, but that very specific description 666 00:35:53,756 --> 00:35:57,596 Speaker 2: of a visualizable, holographic three D model of the planet 667 00:35:57,596 --> 00:35:59,436 Speaker 2: that you could zoom into is very powerful. 668 00:36:00,876 --> 00:36:04,556 Speaker 1: Thank you so much for your time, was very interesting 669 00:36:04,596 --> 00:36:05,116 Speaker 1: to talk with you. 670 00:36:05,756 --> 00:36:06,076 Speaker 3: Thank you. 671 00:36:13,836 --> 00:36:17,956 Speaker 1: Brian McLendon is the chief technology officer at Niantic Spatial. 672 00:36:18,596 --> 00:36:21,876 Speaker 1: Our show Today was produced by Gabriel Hunter Chang. It 673 00:36:22,076 --> 00:36:25,276 Speaker 1: was edited by Lydia Jean Kott and engineered by Hansdale. 674 00:36:25,356 --> 00:36:27,836 Speaker 1: She I'm Jacob Goldstein and We'll be back in a 675 00:36:27,836 --> 00:36:30,996 Speaker 1: couple weeks with more episodes of What's Your Problem. Thanks 676 00:36:31,036 --> 00:36:31,556 Speaker 1: for listening.