WEBVTT - Using AI to Build Better Robots 0:00:15.356 --> 0:00:21.876 Pushkin for a long time. Now, we've had a lot 0:00:21.876 --> 0:00:25.876 of technological innovation in virtual things in bits, you know, 0:00:26.316 --> 0:00:30.636 the Internet, digital images, large language models, etc. We have 0:00:30.716 --> 0:00:36.076 had noticeably less innovation in actual things and things made 0:00:36.116 --> 0:00:39.116 of atoms, things that would hurt if you dropped them 0:00:39.116 --> 0:00:42.956 on your foot. Now that seems to be changing. People 0:00:43.036 --> 0:00:47.316 are using innovations in bits, improvements in computing and communications 0:00:47.316 --> 0:00:51.916 and AI to drive innovation in actual things, everything from 0:00:51.956 --> 0:01:03.236 batteries to garbage cans to airplanes. Next up robots. I'm 0:01:03.316 --> 0:01:05.836 Jacob Boldstein and this is What's Your Problem, the show 0:01:05.876 --> 0:01:08.156 where I talk to people who are trying to make 0:01:08.276 --> 0:01:12.796 technological progress. My guest today is Peter Chi. He's the 0:01:12.836 --> 0:01:17.236 co founder and CEO of Covariant. Peter's work at Covariant 0:01:17.276 --> 0:01:20.356 was partly inspired by the work of Fafe Lead, who 0:01:20.396 --> 0:01:24.756 coincidentally is the AI researcher I interviewed just last week 0:01:24.876 --> 0:01:28.556 on the show. Peter's problem is this, how do you 0:01:28.596 --> 0:01:31.476 take the AI breakthroughs of the past decade or so 0:01:32.236 --> 0:01:34.596 and make them work in robots. 0:01:39.116 --> 0:01:44.316 So to really tell the story of robotics, like we 0:01:44.396 --> 0:01:47.956 have to tell the story of robotics even without AI 0:01:48.476 --> 0:01:51.276 like robotics for a very long time. It's a field 0:01:51.476 --> 0:01:56.196 that you would actually find in mechanical engineering departments of universities. 0:01:56.236 --> 0:01:59.556 Like it's largely a hardware problem. It's a control problem, 0:01:59.596 --> 0:02:02.196 like how can you design the moti well, how can 0:02:02.236 --> 0:02:03.636 you design the gearbox? Well? 0:02:03.756 --> 0:02:04.116 Yeah, right? 0:02:04.236 --> 0:02:07.636 Can you design like the control algorithm so that you 0:02:07.676 --> 0:02:11.476 can get the robot to a exact xyz location in 0:02:11.556 --> 0:02:15.036 the three D physical world like without oscillating around and 0:02:15.076 --> 0:02:15.516 you can. 0:02:15.436 --> 0:02:17.756 Making the thing move? How do you build the parts 0:02:17.756 --> 0:02:19.396 that make the thing move the way we. 0:02:19.396 --> 0:02:22.956 Wanted to move exactly? Like it's all about like telling 0:02:23.196 --> 0:02:26.156 this piece of machinery that we call robot to do 0:02:26.196 --> 0:02:28.596 the thing that's exactly what we tell it to do, 0:02:29.036 --> 0:02:32.236 which turned out to be like obviously a fairly difficult 0:02:32.236 --> 0:02:34.916 engineering problems, and that's why people have woke on it 0:02:34.996 --> 0:02:38.396 for many decades. But it has gotten really good. 0:02:38.476 --> 0:02:40.876 And so this is like this is like the classic 0:02:40.996 --> 0:02:43.996 kind of image you see from a car assembly line 0:02:44.036 --> 0:02:48.676 of like a robot arm you know whatever, welding a 0:02:48.796 --> 0:02:52.356 part onto the body of a cart again and again 0:02:52.516 --> 0:02:53.396 again all day long. 0:02:53.476 --> 0:02:54.796 Yeah, exactly right. 0:02:54.636 --> 0:02:57.516 So they're good at robots are clearly good at welding 0:02:57.516 --> 0:02:59.996 the same part onto the same car a million times? 0:03:00.356 --> 0:03:02.716 What are the limits of that approach? What were the 0:03:02.836 --> 0:03:05.236 problems people were bumping up against. 0:03:05.556 --> 0:03:08.516 Yeah, so the problem is that in order to use 0:03:08.596 --> 0:03:13.316 that kind of robotics, it has a really big limitations 0:03:13.356 --> 0:03:15.916 on your environment. Right. You basically need to be able 0:03:15.916 --> 0:03:22.076 to reduce your task to be solvable by repeated motion. Right, 0:03:22.236 --> 0:03:24.876 And so if you look at like, how like these 0:03:24.956 --> 0:03:28.356 kind of assembly lines that use classical robots, they always 0:03:28.516 --> 0:03:32.436 feed the material into exactly the same place. So no 0:03:32.476 --> 0:03:34.476 matter how how the way that they came in from 0:03:34.516 --> 0:03:37.236 their suppliers and whatnot, you always need to load them 0:03:37.316 --> 0:03:39.956 up in exactly the same way because like there's really 0:03:39.996 --> 0:03:43.476 no adaptivity at all that these robots have because they're 0:03:43.516 --> 0:03:45.236 just executing the same thing again, and it. 0:03:45.436 --> 0:03:47.876 Just it has to be they're very precise, but their 0:03:47.916 --> 0:03:51.916 whole environment has to be super homogeneous, the same every 0:03:51.956 --> 0:03:52.996 time exactly. 0:03:53.076 --> 0:03:56.836 Like, So, like that's the problem one, but that's very difficult, Like, 0:03:56.876 --> 0:03:59.956 not everything can be reduced that way. The second problem 0:03:59.956 --> 0:04:02.356 with it is even in the case that you can 0:04:02.396 --> 0:04:07.156 reduce the problem to that kind of pure mechanical repeated motion, 0:04:07.716 --> 0:04:10.796 it's still very expensive because you still need to program 0:04:10.836 --> 0:04:14.116 a robot to do that one specific task, and if 0:04:14.156 --> 0:04:17.356 you change your task slightly, you need to reprogram everything, 0:04:17.796 --> 0:04:21.676 typically from scratch, and that means like robot are not 0:04:21.756 --> 0:04:26.196 just extremely rigid, which limits like the range of capabilities 0:04:26.236 --> 0:04:30.396 that they can rich and do. And it's also very 0:04:30.476 --> 0:04:34.196 expensive even on the very fixed rigid capability will. 0:04:34.116 --> 0:04:37.156 Right, And so you need something that's the same every time, 0:04:37.196 --> 0:04:38.996 and you need to be doing a lot of it 0:04:39.116 --> 0:04:42.996 because otherwise the economy of scale just doesn't work out. 0:04:42.996 --> 0:04:45.036 It's too expensive to try and get the robot to 0:04:45.076 --> 0:04:45.716 do something else. 0:04:46.076 --> 0:04:46.516 Exactly. 0:04:47.036 --> 0:04:50.796 So I know that you as a as a student, 0:04:50.876 --> 0:04:53.396 as an undergrad and a grad student, if I have 0:04:53.436 --> 0:04:56.436 it right, you worked in the lab of this professor 0:04:56.436 --> 0:04:59.196 at Berkeley who for a long time had been trying 0:04:59.236 --> 0:05:02.996 to teach robots to fold towels. Yes, which is an 0:05:03.036 --> 0:05:06.116 amazing problem because it's one of those ones that seems 0:05:06.156 --> 0:05:09.836 so simple, right, it seems like way easier than riveting 0:05:09.916 --> 0:05:12.636 parts onto a car or whatever, but turned out to 0:05:12.636 --> 0:05:16.876 be in fact much harder for robots, right. And I 0:05:16.876 --> 0:05:19.596 feel like that's telling like why was that so hard 0:05:19.636 --> 0:05:20.156 for robots? 0:05:20.196 --> 0:05:22.276 And what do we learn from that when you do 0:05:22.836 --> 0:05:25.476 welding on a car body, like as we have discussed, 0:05:25.476 --> 0:05:28.716 like you can reduce the problem to just simple mechanical 0:05:28.956 --> 0:05:33.276 repeated motion. But because like these a piece of fabric 0:05:33.396 --> 0:05:37.596 is flexible, is deformable, like it can come in many 0:05:37.596 --> 0:05:40.836 many different kinds of shapes. It has many different possibilities, right, and. 0:05:40.756 --> 0:05:43.676 It's much more complex than a than a car body. 0:05:43.716 --> 0:05:46.596 Weirdly not into exactly, but when you think about it's like, 0:05:46.676 --> 0:05:49.516 oh it could have a little fast possibilities. 0:05:50.196 --> 0:05:52.436 It has a lot more possibility and how it can 0:05:52.476 --> 0:05:55.916 present itself to you. And because exactly of that, like 0:05:56.036 --> 0:06:00.916 recall like the first limitation of traditional robotics, which is 0:06:00.956 --> 0:06:03.716 it can only work with problems that can be reduced 0:06:03.716 --> 0:06:09.916 to repeated motion like and towel folding and folding apparel 0:06:09.956 --> 0:06:12.876 items is exactly one of those things that cannot be 0:06:12.956 --> 0:06:14.076 reduced because. 0:06:13.836 --> 0:06:16.676 It's just a little bit different every time it tells 0:06:16.676 --> 0:06:18.116 a little bit of a different shape. It might be 0:06:18.156 --> 0:06:21.116 sitting on the table and it'll folded over some. 0:06:20.996 --> 0:06:24.076 Weird exactly, like if it's folded onto itself, like how 0:06:24.156 --> 0:06:26.436 much is it folded onto yourself? Like how much wrinkle 0:06:26.556 --> 0:06:28.756 does it have? Like all of those make a big 0:06:28.796 --> 0:06:32.316 difference in terms of what the robots should do with it, 0:06:32.356 --> 0:06:35.036 and so that like it's a really good example of 0:06:35.316 --> 0:06:40.116 something that's traditional robotics cannot solve and you really need 0:06:40.156 --> 0:06:41.956 AI to solve it. 0:06:42.076 --> 0:06:45.596 And when your co founder started working on the problem, 0:06:45.676 --> 0:06:49.556 it was sort of before the kind of modern era 0:06:49.596 --> 0:06:52.116 of AI that we're living in now, right, And I 0:06:53.116 --> 0:06:56.716 read that there was this big moment in kind of 0:06:56.756 --> 0:07:03.636 the origin of the company was when this database essentially 0:07:03.716 --> 0:07:08.156 of labeled images called image net was released, Which was 0:07:08.196 --> 0:07:10.316 interesting to me because I just talk to Faye Lee 0:07:10.436 --> 0:07:10.716 for this. 0:07:10.756 --> 0:07:16.396 So we have is actually one of the advisor investors. 0:07:16.436 --> 0:07:18.556 Just the coincidence. For the record, she didn't put me 0:07:18.596 --> 0:07:21.796 in touch with you, but tell me about sort of 0:07:21.956 --> 0:07:26.596 why why image net was meaningful in the in the 0:07:26.636 --> 0:07:28.076 birth of your company. 0:07:28.476 --> 0:07:33.196 Yeah, there was actually a bigger lesson here than just robotics. 0:07:33.556 --> 0:07:38.676 The bigger lesson is that artificial intelligence is actually becoming 0:07:38.796 --> 0:07:42.396 simpler and simpler, Like when you look at the field 0:07:42.396 --> 0:07:46.516 of artificial intelligence fifteen twenty years ago, like it used 0:07:46.516 --> 0:07:48.676 to be many different sub fields. Like the people that 0:07:48.716 --> 0:07:51.676 work on robotics have a completely different tool sets than 0:07:51.716 --> 0:07:54.036 the people that work on computer vision, and the people 0:07:54.036 --> 0:07:56.316 that are working on computer vision had a different two 0:07:56.396 --> 0:07:58.756 sets from people that are working on natural language processing. 0:07:59.116 --> 0:08:02.356 Well, and they feel different, Right, Like, teaching computers to 0:08:02.556 --> 0:08:07.316 understand an image, you know, to sort of deal with 0:08:07.436 --> 0:08:09.716 an image of the world and understand what it means, 0:08:09.916 --> 0:08:14.196 feels quite different than teaching a computer to have a conversation. 0:08:14.396 --> 0:08:16.436 It doesn't. It's not obvious that you would use the 0:08:16.436 --> 0:08:17.996 same tools to do exactly. 0:08:17.636 --> 0:08:21.356 Those and that definitely was the consensus, Like why. 0:08:21.236 --> 0:08:22.116 Would it be the same? 0:08:22.396 --> 0:08:23.876 Yeah, what would it be the same? Like it feels 0:08:23.956 --> 0:08:25.556 very different, Like it feels like you need different kind 0:08:25.596 --> 0:08:29.356 of data, and I would just like say it. Essentially, 0:08:29.396 --> 0:08:32.516 the field of AI is becoming more and more unified, 0:08:32.836 --> 0:08:35.756 Like the methodology, the model that you would use is 0:08:35.756 --> 0:08:39.236 actually becoming similar and similar and sometimes it's even the 0:08:39.276 --> 0:08:43.876 same across these very different fields of robotics, computer vision, language. 0:08:43.996 --> 0:08:46.676 So it's basically, you build a neural network and then 0:08:46.716 --> 0:08:49.116 you just train it on a bunch of images or 0:08:49.196 --> 0:08:52.036 train it on a bunch of documents, and what you 0:08:52.156 --> 0:08:55.036 train it on is what determines sort of what it's 0:08:55.076 --> 0:08:55.476 good for. 0:08:55.996 --> 0:08:59.596 Is it like that that that basically is it like 0:08:59.636 --> 0:09:05.436 it's it's like it instead of each sub feel coming 0:09:05.556 --> 0:09:08.956 up with different and think about it as like hand 0:09:09.156 --> 0:09:13.276 programmed intelligence, right, like let's try to break down what's 0:09:13.316 --> 0:09:16.716 a sentence means, like break into different parts and when 0:09:16.756 --> 0:09:18.996 you approach a computer vision problem, or let's try to 0:09:19.036 --> 0:09:21.996 come up with different features and some features represent an edge, 0:09:22.076 --> 0:09:23.836 some features represent the background. 0:09:24.236 --> 0:09:27.956 Instead of like trying to manually program the kind of 0:09:28.036 --> 0:09:32.316 quite human intelligence into AI, you're basically taking a step 0:09:32.356 --> 0:09:34.876 back and say, I'm just going to create a very 0:09:34.916 --> 0:09:39.516 flexible learning mechanism, which is an artificial new net, and 0:09:39.556 --> 0:09:41.436 then we're just going to feed it a lot of data. 0:09:41.916 --> 0:09:44.796 And if you have different types of problems that you're solving, 0:09:45.036 --> 0:09:48.316 you're just feeding like this new net different types of data, 0:09:49.156 --> 0:09:51.476 but they're still like really the same kind of mechanism. 0:09:51.516 --> 0:09:55.036 Like then that is a drastic departure from how artificial 0:09:55.076 --> 0:09:56.836 intelligence used to be done. 0:09:57.196 --> 0:10:00.316 And so in that world, then the differentiation is just 0:10:00.436 --> 0:10:03.916 in the data set that you are feeding the totally model. 0:10:03.916 --> 0:10:07.076 Totally and in fact, like I mean, this is like 0:10:07.196 --> 0:10:10.156 jumping way forward in time. We were still talking about 0:10:10.156 --> 0:10:13.156 image net. But if you look at really the most 0:10:13.236 --> 0:10:17.196 popular technologies today, these large language models. When you use 0:10:17.276 --> 0:10:22.236 different companies large language models, it's really what you're saying, 0:10:22.276 --> 0:10:26.596 like as if I'm using GPT chat, gipt four versus 0:10:26.676 --> 0:10:32.836 using Google's bought backed by Germini versus Anthpics, Claude or 0:10:33.036 --> 0:10:34.836 coheres command model. 0:10:34.636 --> 0:10:37.836 You're just naming all the different big language large language models. Now. 0:10:37.916 --> 0:10:39.836 Yeah, and like when you think about these different big 0:10:39.916 --> 0:10:43.236 language models, you're just really referring to the different data 0:10:43.236 --> 0:10:44.956 sets that's see behind. 0:10:44.636 --> 0:10:47.236 That they were trained on. So this is interesting kind 0:10:47.276 --> 0:10:50.316 of abstract big picture talk. I want to kind of 0:10:50.436 --> 0:10:54.596 map this now onto the story of Covariate, the story 0:10:54.676 --> 0:10:57.756 of your company. Right, so we're going back in time. Now, 0:10:58.956 --> 0:11:01.196 tell me about the moment when you decided to start 0:11:01.196 --> 0:11:02.316 the company. What's going on? 0:11:02.916 --> 0:11:04.876 Yeah, so the moment that we decide to start a 0:11:04.956 --> 0:11:09.636 company is you'll refer to this image net mode like 0:11:09.716 --> 0:11:14.036 this like large data set that actually first time like 0:11:14.156 --> 0:11:18.796 really taught people that you can train a network to 0:11:18.916 --> 0:11:21.116 solve one specific task really work. 0:11:21.636 --> 0:11:27.916 So this is twenty twelve. There's when a neural net 0:11:28.436 --> 0:11:32.516 trains on image net and does a really good job, 0:11:32.676 --> 0:11:35.436 way better than anybody has ever done. Any Then any 0:11:35.476 --> 0:11:39.316 model has ever done of identifying objects. 0:11:38.716 --> 0:11:42.516 In exactly right, exactly. That was really significant because that 0:11:42.676 --> 0:11:45.716 means if you can collect a lot of data for 0:11:45.996 --> 0:11:48.756 a single task, and if you can get a group 0:11:48.796 --> 0:11:51.996 of PhDs to work on a model for that single task, 0:11:52.356 --> 0:11:55.076 you basically have AI. Like you can solve that task 0:11:55.196 --> 0:11:58.756 really well. I mean like you might need to iterate 0:11:58.796 --> 0:12:00.956 on your data, you might need to iterate on your algorithm, 0:12:01.076 --> 0:12:03.276 but ultimately you can solve that task really well. 0:12:03.436 --> 0:12:07.356 You're basically saying, if you can gather the data, you 0:12:07.396 --> 0:12:11.316 can gather a shitload of data of whatever kind, then 0:12:11.396 --> 0:12:14.356 you can get AI around. 0:12:14.156 --> 0:12:16.756 For that for that kind, right, And which is like 0:12:16.876 --> 0:12:20.596 why you saw artificial intelligence like really started working like 0:12:20.676 --> 0:12:24.116 after twenty twelve, Like like Google, Facebook, all of these 0:12:24.116 --> 0:12:27.476 companies like have a lot of AI based applications, but 0:12:27.516 --> 0:12:31.236 they are largely not democratized because like in order to 0:12:31.316 --> 0:12:33.956 get any single AI working, you still need a lot 0:12:33.996 --> 0:12:36.636 of data and you still need a team of PhDs 0:12:36.676 --> 0:12:39.996 to work on it. So it was a huge breakthrough 0:12:40.076 --> 0:12:43.556 in AI, but it was not sufficient for really widespread 0:12:43.636 --> 0:12:46.276 usage just because the barrier to create one AI is 0:12:46.316 --> 0:12:51.596 so high. Okay, and then comes the second inflection point 0:12:51.916 --> 0:12:54.596 in the history of AI, which is really the start 0:12:54.796 --> 0:12:58.876 of foundation models. So like I'm talking about the most 0:12:58.876 --> 0:13:01.796 initial version of GPT, right, Like I'm talking about like 0:13:02.236 --> 0:13:05.716 these large language models that are trained on multiple tasks 0:13:06.036 --> 0:13:09.396 so that they're incredibly generalizable, Like you can ask to 0:13:09.396 --> 0:13:11.556 do something new and it can do it really well. 0:13:12.076 --> 0:13:16.796 And also it performed better at single task than specialized model. 0:13:16.916 --> 0:13:19.116 And so just to be clear, like until a few 0:13:19.236 --> 0:13:23.036 years ago, people thought reasonably that if you want to 0:13:23.076 --> 0:13:28.516 build AI to whatever translate language right, you would work 0:13:28.556 --> 0:13:31.316 really hard on that. You would try and build an 0:13:31.356 --> 0:13:36.916 AI specifically designed to be really good at translating text 0:13:36.996 --> 0:13:41.996 from one language to another. But this really surprising result, 0:13:42.076 --> 0:13:45.236 this really surprising thing that emerged from just work people 0:13:45.236 --> 0:13:47.796 were doing, was, in fact, that's not the best way 0:13:48.116 --> 0:13:52.636 to get AI to translate language. It's just throw everything 0:13:52.716 --> 0:13:55.316 you can, all the words on the internet at an 0:13:55.356 --> 0:13:58.876 AI model and just say, figure out everything about language, 0:13:58.876 --> 0:14:02.316 figure out how to answer questions about history, and figure 0:14:02.356 --> 0:14:05.076 out how to translate, and figure out how to give 0:14:05.116 --> 0:14:08.396 me a recipe for you know, pasta. And it turns 0:14:08.396 --> 0:14:12.316 out that LAD technique gets you better results at each 0:14:12.356 --> 0:14:15.556 specific thing than trying to build specialized model exactly. 0:14:15.596 --> 0:14:18.916 And that is really the magic of foundation models. And 0:14:18.956 --> 0:14:22.756 that's the thing that we're not obvious to people outside 0:14:22.756 --> 0:14:26.196 of open Ai for a very long time. And because 0:14:26.236 --> 0:14:28.236 we came from open Ai, a lot of the founding 0:14:28.276 --> 0:14:30.676 team at Coverin came from open Ai. We saw that 0:14:30.756 --> 0:14:36.796 inside earlier, and that inside allow us to start Covariant 0:14:36.796 --> 0:14:40.916 to build foundation models for robotics way before other people 0:14:40.956 --> 0:14:42.236 even believed in the. 0:14:42.236 --> 0:14:45.796 Approach, even people in the field, even people in the field. 0:14:45.916 --> 0:14:47.916 So so, yeah, so you were when did you go 0:14:47.956 --> 0:14:49.876 to open Ai? You were you went to work at 0:14:49.876 --> 0:14:50.396 open Ai. 0:14:50.756 --> 0:14:53.876 We went to open Ai when it was about ten 0:14:54.036 --> 0:14:57.356 ish people like like sometime in twenty sixteen. 0:14:57.036 --> 0:15:03.116 Okay, And and when do you sort of personally have 0:15:03.276 --> 0:15:06.836 this realization and you're not the only one to have it, 0:15:06.876 --> 0:15:11.236 but when do you see that the power of foundation models? 0:15:12.916 --> 0:15:15.156 There are two things on it, Like, so the first 0:15:15.196 --> 0:15:20.436 thing is that early on at open Ai, we believed 0:15:20.516 --> 0:15:24.476 in the idea of scaling, like really scaling up the 0:15:24.476 --> 0:15:27.236 model and scaling up the data sets. And you actually 0:15:27.276 --> 0:15:31.796 see models getting like getting increasingly smarter as you actually 0:15:31.796 --> 0:15:34.196 scale them up. So one is that, and then the 0:15:34.236 --> 0:15:39.396 other one is I would say we we had conviction 0:15:39.596 --> 0:15:44.156 in foundation model for robotics probably earlier than foundation model 0:15:44.196 --> 0:15:46.556 for language. And this is like the one key thing 0:15:46.636 --> 0:15:51.276 is that if you think about building a large language 0:15:51.316 --> 0:15:54.796 model that tried to compress the whole internet of knowledge, 0:15:55.036 --> 0:15:58.276 you still need to compress many things that are not 0:15:58.396 --> 0:16:01.396 quite related to each other. Right, Like maybe you're browsing 0:16:01.436 --> 0:16:05.716 on Wikipedia and you have to recite the composition of 0:16:05.836 --> 0:16:09.876 materials of soil on the moon, and you also need 0:16:09.916 --> 0:16:12.436 to learn how to play chess. What there is there's 0:16:12.476 --> 0:16:15.476 really nothing in common with these two things. Yeah, these 0:16:15.476 --> 0:16:17.556 two parts of the knowledge, but you are asking one 0:16:17.596 --> 0:16:21.036 AI model to learn all of these. And the place 0:16:21.116 --> 0:16:23.676 that make a lot of sense to us is that 0:16:24.476 --> 0:16:27.356 there's only one physical world. Like even when you have 0:16:27.716 --> 0:16:30.956 many different robots that need to do different things in 0:16:31.036 --> 0:16:34.556 different factories, different warehouses, they are still interacting in the 0:16:34.596 --> 0:16:37.956 same physical world. And so building a foundation model for 0:16:38.076 --> 0:16:42.676 robotics like has this amazing property of grounding that like, 0:16:42.756 --> 0:16:45.516 no matter what kind of tasks that you're asking this 0:16:45.596 --> 0:16:48.396 foundation model to learn, well, it's just learning with the 0:16:48.396 --> 0:16:50.596 same sets of physics, Like the same surrounding. 0:16:50.756 --> 0:16:52.876 Is the literal ground with the models. 0:16:52.636 --> 0:16:54.596 The literal ground exactly, and. 0:16:54.516 --> 0:16:57.916 The models have to understand just how the physical world works, 0:16:57.956 --> 0:17:01.596 and that you drop a thing, it will fall and etc. 0:17:02.116 --> 0:17:05.556 And it's if it's something that is like deformable, you 0:17:05.596 --> 0:17:07.316 push it, it would move a little bit. If something 0:17:07.436 --> 0:17:10.916 is rigid like you would sly, if something is rollable, 0:17:10.956 --> 0:17:11.716 it would roll away. 0:17:11.796 --> 0:17:11.916 Like. 0:17:12.116 --> 0:17:13.996 These are the type of things that like, no matter 0:17:14.436 --> 0:17:17.076 where you are on Earth and what type of robots 0:17:17.116 --> 0:17:19.836 like body you're using, is the same, right, and so 0:17:19.956 --> 0:17:22.196 like if you can be one single foundation model that 0:17:22.276 --> 0:17:25.076 can learn from all of these different data, it would 0:17:25.076 --> 0:17:26.236 be incredibly powerful. 0:17:26.676 --> 0:17:29.756 So just to just to state it clearly, So you're 0:17:29.836 --> 0:17:33.036 at open AI, you're seeing the power of foundation models, 0:17:33.636 --> 0:17:36.476 you decide to leave and start the company that is 0:17:36.556 --> 0:17:39.676 now Covariant, Like what are you setting out to do 0:17:39.876 --> 0:17:41.756 when you when you start the company? 0:17:42.276 --> 0:17:46.596 Yeah, So when we started Covariant, we had this really 0:17:46.636 --> 0:17:51.556 strong conviction that like, there should be a future that 0:17:51.756 --> 0:17:54.996 has a lot of autonomous robots doing all the things 0:17:55.036 --> 0:18:00.716 that are repetitive, injury prone, dangerous, like and so that 0:18:00.876 --> 0:18:04.316 can really revolutionize the physical world, make it a lot 0:18:04.356 --> 0:18:09.676 more abundant and to really enable that future of autonomous robots, 0:18:09.836 --> 0:18:12.716 you need really smart AI, and like because of the 0:18:12.756 --> 0:18:15.316 inside that we just talk about, like we believe that 0:18:15.356 --> 0:18:17.596 AI had to be a foundation model. We believe like 0:18:17.636 --> 0:18:20.356 you should have single model that learn from all these 0:18:20.356 --> 0:18:23.036 two different robots together and become smarter together. 0:18:23.596 --> 0:18:28.116 So I mean, so the basic idea is, the dream 0:18:28.196 --> 0:18:35.476 is to build one AI foundation model for robots. Basically, yeah, 0:18:35.636 --> 0:18:37.756 in the same way that you can ask chat GBT 0:18:37.956 --> 0:18:40.196 anything in language and it can answer you in language 0:18:40.196 --> 0:18:42.796 about any different thing. You have a model where you 0:18:42.996 --> 0:18:45.876 just sort of make it be the brain of any 0:18:45.996 --> 0:18:48.716 robot and that robot can sort of see the world 0:18:48.796 --> 0:18:51.356 and move and pick things up and behave in the 0:18:51.396 --> 0:18:52.436 world exactly. 0:18:53.076 --> 0:18:56.396 And there's one key problem, Like the key problem is 0:18:56.436 --> 0:19:01.636 that unlike foundation model for language, where you can scrape 0:19:01.676 --> 0:19:04.596 the whole Internet of text as your pre training data, 0:19:05.036 --> 0:19:09.036 there's nothing that is equivalent in the case of robotic 0:19:09.436 --> 0:19:11.236 I mean, there are some images online, there are some 0:19:11.316 --> 0:19:14.556 YouTube videos online, but by and large, like they don't 0:19:14.596 --> 0:19:20.036 really give you the same type of data that are 0:19:20.036 --> 0:19:22.316 in the form of robots interacting with the world. And 0:19:22.356 --> 0:19:25.116 the big problem is because they're just not that many robots, 0:19:25.116 --> 0:19:28.556 they're just doing interesting things in the world. And so 0:19:28.636 --> 0:19:31.436 a big chunk of what we set out to build 0:19:31.436 --> 0:19:34.156 as a company is recognizing that we need to build 0:19:34.276 --> 0:19:37.436 fundation model for robotics. And in order to be a 0:19:37.476 --> 0:19:40.916 foundation model for robotics, you need to have large data sets. 0:19:41.196 --> 0:19:43.676 And in order to create large data sets, you have 0:19:43.836 --> 0:19:47.836 to have robots that are actually creating value for customers 0:19:47.876 --> 0:19:51.476 in production at scale, because if you're only collecting data 0:19:51.516 --> 0:19:53.996 in your own lab, there's only so much data. And 0:19:54.436 --> 0:19:59.076 so I would say the last six years of Covariance 0:19:59.076 --> 0:20:04.956 is largely focused on really building autonomous robot systems that 0:20:05.076 --> 0:20:08.476 work really well for customers, and they're doing interesting things 0:20:08.516 --> 0:20:11.996 to a level of autonomy and reliability that have not 0:20:12.076 --> 0:20:13.236 been hit before. 0:20:16.196 --> 0:20:19.036 In other words, Peter and his colleagues at Covariant built 0:20:19.276 --> 0:20:22.756 robot arms that businesses are paying to use out in 0:20:22.796 --> 0:20:26.476 the world. But to some extent, those robot arms are 0:20:26.516 --> 0:20:29.316 just a means to an end because they're not just 0:20:29.396 --> 0:20:32.356 doing warehouse work. They're collecting more and more. 0:20:32.276 --> 0:20:35.436 Data that Peter and his colleagues are feeding back into 0:20:35.476 --> 0:20:38.276 their AI model to try and make it get better 0:20:38.476 --> 0:20:43.076 and better. In a minute, what those robot arms are 0:20:43.156 --> 0:20:46.876 actually doing out in the world and what they're learning. 0:20:56.916 --> 0:21:02.036 So let's talk about what the robots you have built 0:21:02.436 --> 0:21:05.356 are doing out in the world. Besides generating the data 0:21:05.396 --> 0:21:08.116 that you will use to train the next generation of robots, 0:21:08.116 --> 0:21:11.196 what are they actually doing in the world right now today. 0:21:11.356 --> 0:21:14.636 Yeah, So when we started the company, we soveyed the 0:21:14.716 --> 0:21:19.156 landscape pretty carefully and then we selected warehouse and logics 0:21:19.156 --> 0:21:22.836 as the primary sector that we focus on today. When 0:21:22.876 --> 0:21:26.196 you are stopping online shops, like when you click a 0:21:26.196 --> 0:21:30.436 button and something shows up next day, there's tremendous amount 0:21:30.636 --> 0:21:34.676 of complexities behind that back end logistics of getting things 0:21:34.716 --> 0:21:38.196 to you. And typically it's estimated that each item is 0:21:38.276 --> 0:21:41.476 touched fifteen to twenty times between when you click a 0:21:41.516 --> 0:21:43.396 butttom to buy and when it shows up. 0:21:44.716 --> 0:21:48.476 In contrast, that fifteen to twenty touches of getting a 0:21:48.516 --> 0:21:52.596 thing from the warehouse to your door is your opportunity. 0:21:52.316 --> 0:21:56.556 Exactly right, And like combining with that, people don't want 0:21:56.596 --> 0:21:59.596 to drive in the middle of the night two hours 0:21:59.596 --> 0:22:03.116 into a suburb to work in a warehouse, Like it's 0:22:03.196 --> 0:22:06.156 just a kind of job that has extremely high turnover rate, 0:22:06.236 --> 0:22:08.156 Like not the kind of job that people stay there 0:22:08.196 --> 0:22:11.196 for a very long time, and so there's a tremendous 0:22:11.196 --> 0:22:15.756 amount of desire for more robotics and more automations in 0:22:15.796 --> 0:22:20.596 those environment to do picking up objects, sorting them into 0:22:20.636 --> 0:22:24.636 the right compartment of boxes, and then packing it nicely 0:22:24.716 --> 0:22:27.996 and then shipping it out to you as a customers. 0:22:28.196 --> 0:22:30.556 Tell me a little bit more specifically, I mean, what's 0:22:30.636 --> 0:22:34.236 one thing one of your robots is doing today in 0:22:35.036 --> 0:22:36.916 a warehouse somewhere on the earth. 0:22:37.236 --> 0:22:41.756 Yeah, so we would get like pretty detail and giky here, 0:22:41.796 --> 0:22:43.756 and so we'd actually tell you a little bit of 0:22:43.836 --> 0:22:48.996 like the needy, greitty details of like warehouse. Like, so 0:22:49.076 --> 0:22:50.916 let me describe what the robot is doing. Like the 0:22:50.996 --> 0:22:55.316