WEBVTT - Using Pokémon Go to Map the World 0:00:15.356 --> 0:00:21.956 Pushkin. I'm Jacob Goldstein and this is What's Your Problem. 0:00:21.996 --> 0:00:25.076 Two quick things before we start the show today. One, 0:00:25.716 --> 0:00:27.716 we're going to take a little break of a few 0:00:27.716 --> 0:00:29.956 weeks after this week's episode, but we'll be back soon. 0:00:30.436 --> 0:00:33.116 And two, I want to say thanks for listening. I 0:00:33.196 --> 0:00:35.996 really appreciate it. I want to keep making a show 0:00:36.036 --> 0:00:39.116 that you like, so please feel free to email us 0:00:39.276 --> 0:00:42.476 at problem at pushkin dot fm. You can also find 0:00:42.476 --> 0:00:45.836 me on LinkedIn or on x formerly known as Twitter 0:00:46.076 --> 0:00:50.036 at Jacob Goldstein. Okay, that's the housekeeping. Now here's today's show. 0:01:01.796 --> 0:01:06.076 My guest today is Brian McClendon. He's the chief technology 0:01:06.076 --> 0:01:10.396 officer at a company called Niantic Spatial, and his problem 0:01:10.516 --> 0:01:15.756 is this, how do you make maps better? Specifically, how 0:01:15.756 --> 0:01:17.956 do you take the two dimensional maps that we have 0:01:17.996 --> 0:01:21.836 today and turn them into three dimensional maps that change 0:01:21.836 --> 0:01:24.676 over time to keep up with changes in the real world. 0:01:25.236 --> 0:01:28.676 Maps like this would be very useful for robots. For 0:01:28.756 --> 0:01:31.756 one thing, Brian's company is in fact doing a pilot 0:01:31.796 --> 0:01:35.236 with a robot delivery company, those little robots that drive 0:01:35.316 --> 0:01:37.636 down the sidewalk to bring you a pizza or whatever. 0:01:38.196 --> 0:01:41.076 But the maps could also be useful for people. For example, 0:01:41.116 --> 0:01:43.516 the company is working on a project with the US 0:01:43.596 --> 0:01:47.436 Coast Guard to build better flight simulators for helicopter pilots, 0:01:47.756 --> 0:01:50.436 and the kind of maps that the company is building 0:01:50.596 --> 0:01:54.036 could also be useful for augmented reality, and in fact, 0:01:54.476 --> 0:01:58.396 Niantic Spatial spun out of the company that made Pokemon Go, 0:01:58.836 --> 0:02:02.956 the augmented reality game. Brian's been working on maps for 0:02:02.996 --> 0:02:06.276 a long time. In the early two thousands, he worked 0:02:06.316 --> 0:02:09.196 at a mapping startup that got acquired by Google in 0:02:09.196 --> 0:02:11.796 two thousand and four, and Google turned the project he 0:02:11.876 --> 0:02:15.396 was working on into Google Earth, and after that, Brian 0:02:15.636 --> 0:02:19.596 kept working on mapping at Google until twenty fifteen. In 0:02:19.636 --> 0:02:22.636 our conversation, Brian and I discussed the problems he's working 0:02:22.636 --> 0:02:26.116 on now trying to build better maps, including why trees 0:02:26.236 --> 0:02:29.476 are his nemesis. Also, we talked about what it will 0:02:29.476 --> 0:02:33.436 take to teach Ai to understand the physical world, but 0:02:33.516 --> 0:02:36.436 to start. Brian told me the biggest lesson that he 0:02:36.596 --> 0:02:38.916 learned about mapping during his time at Google. 0:02:39.236 --> 0:02:41.676 The most interesting thing is that map data in two 0:02:41.716 --> 0:02:44.636 thousand and four, you know, was already digital. We had 0:02:44.996 --> 0:02:47.796 suppliers that you could buy map data from, and you know, 0:02:47.876 --> 0:02:50.596 map quest was out there showing you a little little 0:02:50.596 --> 0:02:53.436 map on the screen on your website, on their website. 0:02:53.836 --> 0:02:57.396 But what we discovered when we started build Google Maps, 0:02:57.476 --> 0:02:59.636 we used that same data and we showed it to 0:02:59.676 --> 0:03:03.636 our users and launched in two thousand and five. We 0:03:03.676 --> 0:03:05.476 came up with this idea of street view, where we'd 0:03:05.516 --> 0:03:08.196 go take pictures of a lot of places around around 0:03:08.276 --> 0:03:10.316 the city and publish them. 0:03:10.796 --> 0:03:12.916 Can I say, by the way, because I remember when 0:03:12.956 --> 0:03:16.676 that happened and when I heard of that, and I 0:03:16.716 --> 0:03:18.676 when I first heard of it, I was like, surely 0:03:18.716 --> 0:03:22.276 they're not going to drive cars with cameras on every 0:03:22.356 --> 0:03:26.356 street and take pictures, Like, surely that's too much. But no, 0:03:26.596 --> 0:03:30.156 that's what it was like so wildly ambitious, just on 0:03:30.196 --> 0:03:31.476 a logistical level. 0:03:31.636 --> 0:03:34.316 Absolutely, that was It was a crazy project. 0:03:34.636 --> 0:03:36.796 You know, Larry Page gets credit for coming up with 0:03:36.876 --> 0:03:39.556 the idea and then following through and supporting us as 0:03:39.876 --> 0:03:40.436 we built it. 0:03:40.956 --> 0:03:44.156 What did you say when someone was it Larry Page 0:03:44.196 --> 0:03:46.116 who said to you, hey, let's do this, Who told 0:03:46.156 --> 0:03:46.556 you about it? 0:03:47.116 --> 0:03:49.996 You know, Larry, Larry had actually taken a video camera 0:03:50.076 --> 0:03:53.156 and driven around the Stanford campus, you know, you know, 0:03:53.356 --> 0:03:56.036 sort of systematically and said, you know, why can't we 0:03:56.076 --> 0:03:57.116 do this for the whole world? 0:03:57.676 --> 0:04:00.156 What did you say when he said, why can't we 0:04:00.156 --> 0:04:01.876 do this for the whole world? What did you say? 0:04:02.276 --> 0:04:04.516 I said, he's Larry Page and it's his money. Man. 0:04:04.516 --> 0:04:06.716 If he wants to do it, we'll give it a shot. 0:04:06.756 --> 0:04:10.596 And and honestly that's you know, the support that Google 0:04:10.636 --> 0:04:14.476 had for organizing the world's information was incredibly strong. 0:04:14.516 --> 0:04:16.036 I mean, Larry was serious about this. 0:04:16.196 --> 0:04:19.876 And we initially built a very expensive data collection van 0:04:20.156 --> 0:04:23.356 that drove around a few cities. Then we started to 0:04:23.476 --> 0:04:28.516 drive other cars that were cheaper but still okay, and 0:04:28.556 --> 0:04:31.716 then that's when we launched the service. And then we 0:04:31.796 --> 0:04:33.956 eventually said we need to do this for the entire 0:04:34.036 --> 0:04:37.836 United States. And the reason we did this was that 0:04:37.956 --> 0:04:41.276 when we launched the first cities, the first thing that 0:04:41.356 --> 0:04:44.516 people said was they looked at the pictures on the screen, 0:04:44.716 --> 0:04:47.316 and then they looked at their map data, and they 0:04:47.316 --> 0:04:49.756 started complaining about the map data. How can you get 0:04:49.796 --> 0:04:52.116 this wrong? I have a picture right here, this shows 0:04:52.116 --> 0:04:54.436 a street sign and a street number, and you have 0:04:54.516 --> 0:04:57.436 it completely in a different location. So a picture is 0:04:57.476 --> 0:05:01.356 worth a thousand words and probably megabytes of actual useful data. 0:05:01.996 --> 0:05:06.436 And there hadn't been any systematic sort of extraction of 0:05:06.476 --> 0:05:08.876 the value from those pictures, because that's not how maps 0:05:08.876 --> 0:05:11.516 had been made in the past. And so the big 0:05:11.556 --> 0:05:14.556 insight we ever had was that map data to that 0:05:14.716 --> 0:05:16.316 moment was not good. 0:05:16.676 --> 0:05:20.116 And by taking pictures for the first time, people could 0:05:20.196 --> 0:05:25.476 in a scaled way look at a photo right next 0:05:25.516 --> 0:05:28.436 to right on top of the map data and see like, oh, 0:05:28.476 --> 0:05:30.756 the map is wrong, and this photo proves that. Like 0:05:30.796 --> 0:05:33.716 it was the moment when people understood how wrong maps 0:05:33.716 --> 0:05:35.436 in general were exactly. 0:05:35.716 --> 0:05:38.156 And people always kind of knew this because they always 0:05:38.156 --> 0:05:40.156 got frustrated every now and then when they'd use the 0:05:40.196 --> 0:05:43.516 maps and they would fail. But now we had this 0:05:43.796 --> 0:05:48.196 much larger complaint body, and we knew that the data 0:05:48.196 --> 0:05:50.516 that we'd been buying from third parties wasn't good, and 0:05:50.596 --> 0:05:52.996 so we decided to make our own maps. And the 0:05:53.036 --> 0:05:56.276 biggest basis for that was the street view pictures themselves. 0:05:56.916 --> 0:05:59.876 And we drove all of the US, Canada, Mexico in 0:05:59.996 --> 0:06:03.476 two thousand and eight, and by two thousand and thirteen 0:06:03.516 --> 0:06:04.796 we had driven fifty countries. 0:06:05.876 --> 0:06:08.156 So the point of street view was not just for 0:06:08.236 --> 0:06:11.436 the like, hey, you can look at pictures, it's to 0:06:11.516 --> 0:06:14.196 get better data as an input to the sort of 0:06:14.236 --> 0:06:19.236 more traditional maps. Absolutely, I never knew that. Is that 0:06:19.276 --> 0:06:23.276 why Apple Maps sucked when it first came out, because 0:06:23.316 --> 0:06:23.996 they didn't have them. 0:06:24.116 --> 0:06:26.556 I'm so glad you said. It keeps me from saying that. 0:06:26.996 --> 0:06:29.756 I remember when it happened. I remember they fired the guy. 0:06:29.876 --> 0:06:33.236 Remember they fired the guy. But is that why? Because 0:06:33.236 --> 0:06:37.116 they just were buying the normal map data and everybody 0:06:37.236 --> 0:06:39.156 was comparing it to Google and you guys had literally 0:06:39.236 --> 0:06:42.236 driven cars on every street in America to figure it out. 0:06:42.316 --> 0:06:45.476 That's exactly right, and they eventually got their Apple Apple 0:06:45.556 --> 0:06:49.316 has driven you know, many millions of miles themselves, But 0:06:49.396 --> 0:06:51.556 if you look at it worldwide, I would say that 0:06:51.596 --> 0:06:52.916 Google is still probably better. 0:06:55.596 --> 0:06:57.556 I don't have a dog in that fight. It is 0:06:57.716 --> 0:07:02.796 interesting how both Google and Apple had these We're just 0:07:02.876 --> 0:07:05.916 throwing off crazy amounts of money, right each for their 0:07:05.956 --> 0:07:09.436 own special reasons, but we're reinvesting it in an interesting 0:07:09.516 --> 0:07:13.036 way that like x Ante, I would not have guessed. Right, 0:07:13.156 --> 0:07:16.636 Like this sort of search giant and the fancy phone 0:07:16.676 --> 0:07:20.396 maker becoming kind of going to the vanguard of mapping 0:07:21.316 --> 0:07:22.156 is not obvious. 0:07:22.996 --> 0:07:26.436 So I think when Steve Jobs added Google Maps to 0:07:26.836 --> 0:07:30.476 the iPhone very very quickly. He realized it was strategic 0:07:31.156 --> 0:07:33.756 and you know, he knew that because there was a 0:07:33.796 --> 0:07:36.636 GPS on the phone and because he now had a 0:07:36.716 --> 0:07:40.916 you know, pan and zoom capable touch screen, that he 0:07:41.076 --> 0:07:44.276 had a you know, an amazing new method for helping 0:07:44.316 --> 0:07:45.596 people understand the world. 0:07:45.676 --> 0:07:49.916 And so getting that map data on there was very important. 0:07:49.916 --> 0:07:52.916 And Android and iOS both had it very quickly. But 0:07:53.076 --> 0:07:55.196 the moment you do that, you have the same problem 0:07:55.196 --> 0:07:57.556 with street view showed was that people walk around, they 0:07:57.596 --> 0:07:59.116 look at their map and they look up at the 0:07:59.116 --> 0:08:02.836 street around them, and it's wrong. And so they knew that, 0:08:02.956 --> 0:08:05.596 you know, the biggest complaint they would be getting was 0:08:05.636 --> 0:08:09.076 this factual failure from their users about their maps. 0:08:09.596 --> 0:08:14.596 Yeah, well a map that's wrong is worse than no map, arguably. 0:08:14.756 --> 0:08:14.996 Yes. 0:08:16.236 --> 0:08:19.116 I mean, were there surprises to you about, you know, 0:08:19.356 --> 0:08:22.356 sort of what what developed, what didn't developed, how people 0:08:22.476 --> 0:08:25.836 used Google Maps, things you thought might happen that didn't happen. 0:08:25.916 --> 0:08:29.196 I think one of the things that you know, we 0:08:29.836 --> 0:08:34.756 probably speculated about. But if you realized that Google Earth 0:08:34.876 --> 0:08:37.756 and Maps launched in February and June of two thousand 0:08:37.796 --> 0:08:38.156 and five. 0:08:38.996 --> 0:08:43.356 In August, Katrina happened, and. 0:08:43.236 --> 0:08:45.676 Katrina was Hurricane Katrina in New Orleans. 0:08:45.716 --> 0:08:49.636 Yeah, yeah, it was an incredible hurricane, did huge amounts 0:08:49.676 --> 0:08:52.836 of damage and the city of New Orleans was flooded, 0:08:52.956 --> 0:08:57.636 you know, completely, and this turned into a challenge for 0:08:57.876 --> 0:09:00.596 you know, everybody involved, you know, both for just escaping 0:09:00.636 --> 0:09:02.916 the city and going to the right places, but then 0:09:02.956 --> 0:09:06.436 it became about rescuing people on rooftops. And the problem 0:09:06.556 --> 0:09:09.716 now was the streets were gone and they couldn't figure 0:09:09.716 --> 0:09:12.236 out where things were. And so it turned out that 0:09:12.316 --> 0:09:16.636 the helicopter pilots at the time used Google Earth to 0:09:17.436 --> 0:09:21.476 find out where the address was on the picture and 0:09:21.516 --> 0:09:24.516 then use the picture to navigate their helicopter to find 0:09:24.516 --> 0:09:28.196 the people waiting on the roof. And there were literally 0:09:28.316 --> 0:09:32.676 hundreds of rescues done using Google Earth's bipilots, you know, 0:09:32.756 --> 0:09:34.876 a few months after we'd released it, and that was 0:09:34.956 --> 0:09:36.556 that was a big surprise for us. 0:09:37.476 --> 0:09:41.036 It is extremely interesting to me that the company right 0:09:41.076 --> 0:09:44.876 now spun out of the company that made Pokemon Go, 0:09:45.036 --> 0:09:46.916 and that there was this kind of kernel of an 0:09:46.956 --> 0:09:51.076 idea which was like, oh, hey, people you have playing 0:09:51.116 --> 0:09:54.596 Pokemon Go have taken a billion photos in the world, 0:09:54.676 --> 0:09:58.116 like truly a billion photos. This is an interesting data set. 0:09:58.196 --> 0:10:00.476 Maybe we can do something with it. I mean, if 0:10:00.516 --> 0:10:02.876 I understand right, that was before you got there, But like, 0:10:03.236 --> 0:10:05.036 how did that playoff? Did they know when they were 0:10:05.116 --> 0:10:07.396 launching Pokemon Go that like, oh, we're going to get 0:10:07.436 --> 0:10:08.996 this data? Did somebody realize it? 0:10:09.756 --> 0:10:13.556 So interestingly, I think the uh, you know, John John 0:10:13.556 --> 0:10:16.756 Hankey definitely had a very long term vision. He was 0:10:16.796 --> 0:10:19.596 thinking about not just the game itself, but about games 0:10:19.596 --> 0:10:22.596 in the real world. Now, the photos that were taken 0:10:22.716 --> 0:10:25.556 initially by Pokemon Go players were you know, taking a 0:10:25.556 --> 0:10:28.596 picture of a Pokemon into location. You know that those 0:10:28.596 --> 0:10:34.716 photos stayed private. What happened was that John, seeing the future, said, 0:10:34.836 --> 0:10:38.116 can we create a game mode where players consciously go 0:10:38.156 --> 0:10:41.156 out and record videos for us of poke stops and 0:10:41.236 --> 0:10:44.516 help us map these locations by recording videos. So it 0:10:44.556 --> 0:10:47.676 was a conscious gameplay activity where you would get in 0:10:47.836 --> 0:10:51.916 game rewards if you would upload a thirty second video 0:10:52.916 --> 0:10:53.996 to the to the game. 0:10:54.036 --> 0:10:55.396 And that was where that data came from. 0:10:55.476 --> 0:10:58.556 So it was consciously contributed by the by the players 0:10:58.636 --> 0:11:00.716 rather than sort of background collected. 0:11:00.996 --> 0:11:05.036 And and what was what was the point of that 0:11:05.076 --> 0:11:06.796 From the point of view of the company, so that 0:11:06.836 --> 0:11:09.876 we wanted to create more ar gaming experiences. 0:11:10.396 --> 0:11:14.516 And the Pokemon Go already used augmented reality to create 0:11:14.596 --> 0:11:17.036 those photos that you were talking about. Put a Pokemon 0:11:17.116 --> 0:11:18.916 sitting on a chair and take a picture of it, 0:11:19.036 --> 0:11:21.996 or so it was already using augmented reality. What we 0:11:22.076 --> 0:11:24.396 wanted to do is to be able to augment the 0:11:24.476 --> 0:11:28.076 poke stops in the outdoors, and so to do that accurately, 0:11:28.076 --> 0:11:30.396 you actually need to have a very accurate map of 0:11:30.436 --> 0:11:32.636 those pokey stops, not just where they are with a 0:11:32.676 --> 0:11:35.716 blue dot, but actually the three D model of the 0:11:35.716 --> 0:11:39.596 benches around it and the statue itself, and so creating 0:11:39.636 --> 0:11:43.156 that three D model and then being able to point 0:11:43.196 --> 0:11:45.676 your phone at it and say, I know exactly where 0:11:45.676 --> 0:11:49.796 you're standing relative to that statue. Now the Pokemon that 0:11:49.836 --> 0:11:52.356 was placed by the last person is exactly the same 0:11:52.356 --> 0:11:53.956 place that it is for you. 0:11:54.476 --> 0:11:57.676 So you need a really sophisticated model of the physical 0:11:57.676 --> 0:12:00.196 world for that to work, otherwise it'll be like floating 0:12:00.196 --> 0:12:02.156 in space a foot from where it's supposed to be. 0:12:02.196 --> 0:12:05.356 That it is exactly the problem we spent several years solving, 0:12:05.516 --> 0:12:08.756 and we only launched the product that used it. I 0:12:08.796 --> 0:12:13.356 think in twenty twenty three, and that was part of 0:12:13.596 --> 0:12:17.396 Pokemon Go and it's called Pokemon Go playgrounds, and it's 0:12:17.396 --> 0:12:21.916 the ability for players to leave Pokemon at locations in 0:12:21.916 --> 0:12:25.876 interesting formations and interesting combinations and then have other players 0:12:25.916 --> 0:12:27.396 discover them or add to them. 0:12:28.556 --> 0:12:32.196 And at what point did somebody think, oh, we can 0:12:32.436 --> 0:12:35.836 do something wholly other than gaming here, we can build 0:12:35.876 --> 0:12:37.916 a new kind of map that is, you know, more 0:12:37.996 --> 0:12:39.396 legible to a robot. 0:12:39.596 --> 0:12:43.036 I think that in some people's minds, you know, that 0:12:43.156 --> 0:12:45.716 was the thought from day one. I think John would 0:12:45.836 --> 0:12:48.836 argue in his mind he was he was really thinking 0:12:48.876 --> 0:12:52.196 in that direction. But I think, you know, from a 0:12:52.356 --> 0:12:56.116 product perspective, in a business perspective, being able to allow 0:12:57.116 --> 0:13:01.316 developers of applications to go scan their own places, not 0:13:01.356 --> 0:13:04.316 Poke stops, but any location around the world and then 0:13:04.396 --> 0:13:07.676 create these experiences. That was our first step. I think 0:13:07.716 --> 0:13:11.316 the robotics ass inspect came into play, you know, more recently, 0:13:11.356 --> 0:13:13.916 probably in the last eighteen to twenty four months, as 0:13:13.996 --> 0:13:18.556 we realize that, you know, robots struggle with finding themselves 0:13:18.636 --> 0:13:21.436 in the real world, and so they need a form 0:13:21.476 --> 0:13:23.196 of maps, don't we all. 0:13:26.076 --> 0:13:37.676 And we'll be back in just a minute. So let's 0:13:37.676 --> 0:13:40.756 talk about where you are now, like what's in the 0:13:40.796 --> 0:13:43.236 world now that you're making. 0:13:43.756 --> 0:13:46.716 I think the uh, you know, the experience of working 0:13:46.796 --> 0:13:50.396 with the data from the phones of the players taught 0:13:50.476 --> 0:13:53.596 us a lot about how to reconstruct and create these 0:13:53.636 --> 0:13:57.596 precise maps, not with super fancy survey tools but with 0:13:57.796 --> 0:13:59.036 just consumer devices. 0:13:59.036 --> 0:14:01.916 And what we got good at was using. 0:14:01.956 --> 0:14:05.116 Low quality data, which I mean, it's still very good, 0:14:05.116 --> 0:14:08.316 but it's you know, it's not professional great anything, and 0:14:08.516 --> 0:14:11.876 being able to turn that into professional grade data. And 0:14:11.956 --> 0:14:15.676 so I think that was the skill that we're bringing 0:14:15.676 --> 0:14:18.196 into the enterprise applications that you hear now. 0:14:18.676 --> 0:14:20.396 And we can do it not just at the scale 0:14:20.436 --> 0:14:22.716 of a. 0:14:21.676 --> 0:14:24.476 Statue or a single location, but we can do it 0:14:24.516 --> 0:14:28.036 at city scale. We can collect an entire city and 0:14:28.316 --> 0:14:29.316 help you find yourself. 0:14:29.956 --> 0:14:31.676 Have you collected an entire city? 0:14:31.796 --> 0:14:32.116 Yes? 0:14:33.036 --> 0:14:33.596 What city? 0:14:34.756 --> 0:14:37.756 We have all of San Francisco and pieces of other cities, 0:14:38.796 --> 0:14:42.676 and they're usually collected from drone data. Basically commissioning and 0:14:42.716 --> 0:14:46.356 saying we wanted a drone based camera, you know, follow 0:14:46.356 --> 0:14:50.076 a path, take a picture every several meters and take 0:14:50.116 --> 0:14:55.596 it from multiple angles, and then use photogrammetry to reconstruct 0:14:55.716 --> 0:14:58.716 that city at a level of detail, you know, beyond 0:14:58.756 --> 0:14:59.756 what Google can do it. 0:15:00.196 --> 0:15:02.676 And when you say drone, you mean like a flying drone, 0:15:02.676 --> 0:15:05.036 like a little quad copter what exactly. 0:15:04.636 --> 0:15:08.316 A very lightweight, you know, very limited under two hundred 0:15:08.316 --> 0:15:09.836 and fifty grams smallest ones. 0:15:09.876 --> 0:15:12.796 And so what are you doing now with your three 0:15:12.876 --> 0:15:15.156 D robot legible map of San Francisco. 0:15:15.636 --> 0:15:19.156 We are combining all of that interesting data collected on 0:15:19.196 --> 0:15:22.076 the ground with phones and with three sixty cameras with 0:15:22.196 --> 0:15:26.756 this drone data and also with satellite data and proving 0:15:26.796 --> 0:15:31.396 out that we can help robots and humans find themselves, 0:15:31.396 --> 0:15:35.196 localize themselves, you know, anywhere in the city without having 0:15:35.236 --> 0:15:37.676 to pre map it, you know, specifically at that location. 0:15:38.716 --> 0:15:41.716 So you mean like you a robot, It's just like 0:15:41.756 --> 0:15:44.836 you blindfold the robot for lack of a better word, 0:15:44.876 --> 0:15:47.356 and then take the blindfold off, and it knows right 0:15:47.396 --> 0:15:48.916 away where it is and which way it's looking. 0:15:49.156 --> 0:15:50.316 That's absolutely the goal. 0:15:50.436 --> 0:15:54.436 Yes, what do you have to do to get there? 0:15:55.676 --> 0:15:58.276 I think the challenge is that you know, the world 0:15:58.356 --> 0:16:01.196 is a complicated and varied place, and you know, trees. 0:16:01.596 --> 0:16:05.676 Trees are my nemesis in my career, and you know 0:16:05.716 --> 0:16:06.716 they are they are absence. 0:16:06.756 --> 0:16:09.036 I'm proachry. For the record, I'm approached. 0:16:09.156 --> 0:16:13.836 I understand they are visually amazing, but they are in 0:16:13.876 --> 0:16:17.796 fact fractally based. Right they have very high detail, and 0:16:17.836 --> 0:16:20.356 worse than that, they're leaves. They flutter in the wind. 0:16:20.396 --> 0:16:24.596 It's deeply inconvenient, and so when you take multiple pictures 0:16:24.636 --> 0:16:27.076 of a tree, no two pictures have the tree in 0:16:27.116 --> 0:16:28.116 the same spot. 0:16:28.516 --> 0:16:30.716 When they change rapidly with tom. 0:16:30.596 --> 0:16:34.556 Absolutely so another problem. The old way of doing this 0:16:34.796 --> 0:16:37.396 was that, you know, satellites and airplanes would take pictures 0:16:37.396 --> 0:16:40.036 of cities in the winter because the leaves were off, 0:16:40.436 --> 0:16:43.356 and that gave them more information. But if you're trying 0:16:43.356 --> 0:16:45.596 to build a model the city and localize, you actually 0:16:45.676 --> 0:16:48.276 need to see them in all their states, not just 0:16:48.316 --> 0:16:49.156 the leaf off state. 0:16:49.876 --> 0:16:52.956 So is the short answer to how you solve that problem? AI? 0:16:53.076 --> 0:16:56.236 How do you solve the tree problem? You try to 0:16:56.276 --> 0:16:59.396 look beyond the trees. You know, trees other than the 0:16:59.436 --> 0:17:02.316 tree trunk. There's not a lot of stability there, but 0:17:02.436 --> 0:17:05.036 there are many things behind the tree or next to 0:17:05.076 --> 0:17:08.756 the tree that are useful and being able to effectively 0:17:09.236 --> 0:17:12.676 in an AI. To ignore the tree is usually your 0:17:12.716 --> 0:17:17.676 best bet because there's enough remaining features in view that 0:17:17.716 --> 0:17:20.316 you can find enough to connect yourself to that location. 0:17:23.076 --> 0:17:25.796 I know you just made a deal with a delivery 0:17:25.916 --> 0:17:29.156 robot company. Like, is that the first use case? Like, 0:17:29.156 --> 0:17:31.556 what do you think the first use cases are going 0:17:31.636 --> 0:17:31.836 to be? 0:17:32.156 --> 0:17:35.316 I mean, definitely is the first use case they struggle with, 0:17:35.356 --> 0:17:39.716 effectively the blindfold problem that sometimes they don't don't have 0:17:39.836 --> 0:17:41.596 enough context to know where they are, and they need 0:17:41.596 --> 0:17:45.036 to rediscover their location, and they need to need do 0:17:45.116 --> 0:17:49.076 so accurately enough that either the autonomous controller or some 0:17:49.276 --> 0:17:51.356 human who drops in to look at where it is 0:17:51.836 --> 0:17:55.076 can know where they are in the city because city 0:17:55.076 --> 0:17:56.436 blocks tend to look the same. 0:17:56.796 --> 0:17:58.516 Unless you have that information. 0:17:58.636 --> 0:18:01.196 How does the robot get lost in the first place? Like, 0:18:01.236 --> 0:18:03.356 how does the robot find itself in that situation? 0:18:03.836 --> 0:18:05.356 There's two ways. 0:18:05.396 --> 0:18:07.596 One is I mean they do it they hit a reboot, 0:18:07.636 --> 0:18:12.636 and they literally lose connection if they are cover if 0:18:12.636 --> 0:18:16.036 for any reason enough of the world is blocked from 0:18:16.036 --> 0:18:20.556 them for long enough, they lose lose control. GPS by 0:18:20.596 --> 0:18:24.156 itself is rarely good enough because, especially in cities, the 0:18:24.236 --> 0:18:27.196 GPS has you know, reflections off of buildings, and you 0:18:27.196 --> 0:18:30.636 can be a city block away and the GPS will 0:18:30.636 --> 0:18:35.556 happily tell you bigger there so. 0:18:33.676 --> 0:18:37.516 Hu So it's insufficiently precise, particularly in cities. 0:18:37.596 --> 0:18:41.156 Yes, and big cities with big, tall buildings are absolutely 0:18:41.196 --> 0:18:44.916 the hardest challenge other than tunnels and parking critches. 0:18:45.716 --> 0:18:48.996 But that's good for you. Like that that weakness is 0:18:49.036 --> 0:18:57.116 your opportunity exactly. What tell me about the broader industry 0:18:57.196 --> 0:18:59.956 is the right word, but the broader context, you know, 0:18:59.996 --> 0:19:02.836 who else is working on things like you're working on. 0:19:03.596 --> 0:19:06.516 I mean, I think we've had, you know, companies that 0:19:06.556 --> 0:19:09.396 have worked on building digital twins of the world. They're 0:19:09.436 --> 0:19:12.516 in the micro or the macro for you know, since 0:19:12.556 --> 0:19:17.276 before Google, and Google certainly is the biggest wholesale attempt 0:19:17.276 --> 0:19:21.836 at this, but there's a need for more resolution and 0:19:22.556 --> 0:19:26.156 also to have it from suppliers other than Google, because 0:19:26.196 --> 0:19:28.516 one of the areas that I think is hard is 0:19:28.556 --> 0:19:31.396 that Google is all about recreating the public world, but 0:19:31.716 --> 0:19:34.556 they don't really have any way for a company to 0:19:34.676 --> 0:19:38.876 do their own location and add it to the real world. 0:19:39.916 --> 0:19:43.316 Oh interesting, the hospital did I read hospital as an example. 0:19:43.316 --> 0:19:44.996 We were talking about that when we're doing the prep 0:19:45.036 --> 0:19:47.836 of like how hard it is to find your way 0:19:47.876 --> 0:19:48.796 around a hospital? 0:19:49.196 --> 0:19:52.036 It is, and I just experienced that yesterday at Stanford. 0:19:52.996 --> 0:19:53.436 I think the. 0:19:54.916 --> 0:19:58.036 Challenges building a map and with the hospital, there's actually 0:19:58.076 --> 0:20:00.516 a lot of change that happens, you know, in the 0:20:00.516 --> 0:20:03.116 corridors of a hospital at any given time. The big 0:20:03.196 --> 0:20:07.236 quarriors are stable, but most of the actual patient areas 0:20:07.316 --> 0:20:11.116 are you know, constantly changeing their visual appearance, and so 0:20:11.196 --> 0:20:15.036 getting robots that are able to navigate that space accurately 0:20:15.036 --> 0:20:18.436 and reliably is certainly one of our potential use cases. 0:20:19.796 --> 0:20:23.356 So there's an app called scan a Verse that's you right, 0:20:23.956 --> 0:20:27.516 the app scan a Verse. So I downloaded that this 0:20:27.596 --> 0:20:33.316 week and astonishingly, they're estonishingly to me. Within one hundred 0:20:33.396 --> 0:20:37.676 yards of my house, there are two things, two objects 0:20:37.676 --> 0:20:40.036 that have been scanned in three D and uploaded to 0:20:40.076 --> 0:20:44.076 this app, which is interesting cool. I had no idea 0:20:44.156 --> 0:20:45.996 what's going on there? What is it? And why are 0:20:46.036 --> 0:20:49.556 people on my block taking pictures of things. 0:20:49.716 --> 0:20:52.876 Scan Versus is a great app, and we actually acquired 0:20:52.916 --> 0:20:53.916 the one person. 0:20:53.596 --> 0:20:56.516 Who wrote it, a brilliant programmer. 0:20:56.596 --> 0:21:00.956 And turned it into this product that allows capture of 0:21:00.996 --> 0:21:03.796 three D objects. And what you're seeing there is that 0:21:03.916 --> 0:21:05.716 a lot of users love to be able to go 0:21:05.756 --> 0:21:08.476 out and scan their area their location or a statue 0:21:08.596 --> 0:21:13.596 or whatever, and recreated in three D visually almost completely 0:21:13.636 --> 0:21:16.516 photo realistically, and then share that with the rest of 0:21:16.516 --> 0:21:19.636 the world. And so you're experiencing sort of the map 0:21:19.676 --> 0:21:21.756 that scanners puts up at the front of the front 0:21:21.796 --> 0:21:22.076 of them. 0:21:23.236 --> 0:21:25.996 So it's in the same way that people like to 0:21:26.036 --> 0:21:30.116 do whatever geokashig and geo guessing and whatever. It's some 0:21:30.396 --> 0:21:33.636 version of kind of sharing the real world in a 0:21:33.756 --> 0:21:34.436 digital way. 0:21:34.556 --> 0:21:35.996 Exactly is it big? 0:21:36.036 --> 0:21:39.116