Speaker 1: Pushkin for a long time. Now, we've had a lot
Speaker 1: of technological innovation in virtual things in bits, you know,
Speaker 1: the Internet, digital images, large language models, etc. We have
Speaker 1: had noticeably less innovation in actual things and things made
Speaker 1: of atoms, things that would hurt if you dropped them
Speaker 1: on your foot. Now that seems to be changing. People
Speaker 1: are using innovations in bits, improvements in computing and communications
Speaker 1: and AI to drive innovation in actual things, everything from
Speaker 1: batteries to garbage cans to airplanes. Next up robots. I'm
Speaker 1: Jacob Boldstein and this is What's Your Problem, the show
Speaker 1: where I talk to people who are trying to make
Speaker 1: technological progress. My guest today is Peter Chi. He's the
Speaker 1: co founder and CEO of Covariant. Peter's work at Covariant
Speaker 1: was partly inspired by the work of Fafe Lead, who
Speaker 1: coincidentally is the AI researcher I interviewed just last week
Speaker 1: on the show. Peter's problem is this, how do you
Speaker 1: take the AI breakthroughs of the past decade or so
Speaker 1: and make them work in robots.
Speaker 2: So to really tell the story of robotics, like we
Speaker 2: have to tell the story of robotics even without AI
Speaker 2: like robotics for a very long time. It's a field
Speaker 2: that you would actually find in mechanical engineering departments of universities.
Speaker 2: Like it's largely a hardware problem. It's a control problem,
Speaker 2: like how can you design the moti well, how can
Speaker 2: you design the gearbox? Well?
Speaker 1: Yeah, right?
Speaker 2: Can you design like the control algorithm so that you
Speaker 2: can get the robot to a exact xyz location in
Speaker 2: the three D physical world like without oscillating around and
Speaker 2: you can.
Speaker 1: Making the thing move? How do you build the parts
Speaker 1: that make the thing move the way we.
Speaker 2: Wanted to move exactly? Like it's all about like telling
Speaker 2: this piece of machinery that we call robot to do
Speaker 2: the thing that's exactly what we tell it to do,
Speaker 2: which turned out to be like obviously a fairly difficult
Speaker 2: engineering problems, and that's why people have woke on it
Speaker 2: for many decades. But it has gotten really good.
Speaker 1: And so this is like this is like the classic
Speaker 1: kind of image you see from a car assembly line
Speaker 1: of like a robot arm you know whatever, welding a
Speaker 1: part onto the body of a cart again and again
Speaker 1: again all day long.
Speaker 2: Yeah, exactly right.
Speaker 1: So they're good at robots are clearly good at welding
Speaker 1: the same part onto the same car a million times?
Speaker 1: What are the limits of that approach? What were the
Speaker 1: problems people were bumping up against.
Speaker 2: Yeah, so the problem is that in order to use
Speaker 2: that kind of robotics, it has a really big limitations
Speaker 2: on your environment. Right. You basically need to be able
Speaker 2: to reduce your task to be solvable by repeated motion. Right,
Speaker 2: And so if you look at like, how like these
Speaker 2: kind of assembly lines that use classical robots, they always
Speaker 2: feed the material into exactly the same place. So no
Speaker 2: matter how how the way that they came in from
Speaker 2: their suppliers and whatnot, you always need to load them
Speaker 2: up in exactly the same way because like there's really
Speaker 2: no adaptivity at all that these robots have because they're
Speaker 2: just executing the same thing again, and it.
Speaker 1: Just it has to be they're very precise, but their
Speaker 1: whole environment has to be super homogeneous, the same every
Speaker 1: time exactly.
Speaker 2: Like, So, like that's the problem one, but that's very difficult, Like,
Speaker 2: not everything can be reduced that way. The second problem
Speaker 2: with it is even in the case that you can
Speaker 2: reduce the problem to that kind of pure mechanical repeated motion,
Speaker 2: it's still very expensive because you still need to program
Speaker 2: a robot to do that one specific task, and if
Speaker 2: you change your task slightly, you need to reprogram everything,
Speaker 2: typically from scratch, and that means like robot are not
Speaker 2: just extremely rigid, which limits like the range of capabilities
Speaker 2: that they can rich and do. And it's also very
Speaker 2: expensive even on the very fixed rigid capability will.
Speaker 1: Right, And so you need something that's the same every time,
Speaker 1: and you need to be doing a lot of it
Speaker 1: because otherwise the economy of scale just doesn't work out.
Speaker 1: It's too expensive to try and get the robot to
Speaker 1: do something else.
Speaker 2: Exactly.
Speaker 1: So I know that you as a as a student,
Speaker 1: as an undergrad and a grad student, if I have
Speaker 1: it right, you worked in the lab of this professor
Speaker 1: at Berkeley who for a long time had been trying
Speaker 1: to teach robots to fold towels. Yes, which is an
Speaker 1: amazing problem because it's one of those ones that seems
Speaker 1: so simple, right, it seems like way easier than riveting
Speaker 1: parts onto a car or whatever, but turned out to
Speaker 1: be in fact much harder for robots, right. And I
Speaker 1: feel like that's telling like why was that so hard
Speaker 1: for robots?
Speaker 2: And what do we learn from that when you do
Speaker 2: welding on a car body, like as we have discussed,
Speaker 2: like you can reduce the problem to just simple mechanical
Speaker 2: repeated motion. But because like these a piece of fabric
Speaker 2: is flexible, is deformable, like it can come in many
Speaker 2: many different kinds of shapes. It has many different possibilities, right, and.
Speaker 1: It's much more complex than a than a car body.
Speaker 1: Weirdly not into exactly, but when you think about it's like,
Speaker 1: oh it could have a little fast possibilities.
Speaker 2: It has a lot more possibility and how it can
Speaker 2: present itself to you. And because exactly of that, like
Speaker 2: recall like the first limitation of traditional robotics, which is
Speaker 2: it can only work with problems that can be reduced
Speaker 2: to repeated motion like and towel folding and folding apparel
Speaker 2: items is exactly one of those things that cannot be
Speaker 2: reduced because.
Speaker 1: It's just a little bit different every time it tells
Speaker 1: a little bit of a different shape. It might be
Speaker 1: sitting on the table and it'll folded over some.
Speaker 2: Weird exactly, like if it's folded onto itself, like how
Speaker 2: much is it folded onto yourself? Like how much wrinkle
Speaker 2: does it have? Like all of those make a big
Speaker 2: difference in terms of what the robots should do with it,
Speaker 2: and so that like it's a really good example of
Speaker 2: something that's traditional robotics cannot solve and you really need
Speaker 2: AI to solve it.
Speaker 1: And when your co founder started working on the problem,
Speaker 1: it was sort of before the kind of modern era
Speaker 1: of AI that we're living in now, right, And I
Speaker 1: read that there was this big moment in kind of
Speaker 1: the origin of the company was when this database essentially
Speaker 1: of labeled images called image net was released, Which was
Speaker 1: interesting to me because I just talk to Faye Lee
Speaker 1: for this.
Speaker 2: So we have is actually one of the advisor investors.
Speaker 1: Just the coincidence. For the record, she didn't put me
Speaker 1: in touch with you, but tell me about sort of
Speaker 1: why why image net was meaningful in the in the
Speaker 1: birth of your company.
Speaker 2: Yeah, there was actually a bigger lesson here than just robotics.
Speaker 2: The bigger lesson is that artificial intelligence is actually becoming
Speaker 2: simpler and simpler, Like when you look at the field
Speaker 2: of artificial intelligence fifteen twenty years ago, like it used
Speaker 2: to be many different sub fields. Like the people that
Speaker 2: work on robotics have a completely different tool sets than
Speaker 2: the people that work on computer vision, and the people
Speaker 2: that are working on computer vision had a different two
Speaker 2: sets from people that are working on natural language processing.
Speaker 1: Well, and they feel different, Right, Like, teaching computers to
Speaker 1: understand an image, you know, to sort of deal with
Speaker 1: an image of the world and understand what it means,
Speaker 1: feels quite different than teaching a computer to have a conversation.
Speaker 1: It doesn't. It's not obvious that you would use the
Speaker 1: same tools to do exactly.
Speaker 2: Those and that definitely was the consensus, Like why.
Speaker 1: Would it be the same?
Speaker 2: Yeah, what would it be the same? Like it feels
Speaker 2: very different, Like it feels like you need different kind
Speaker 2: of data, and I would just like say it. Essentially,
Speaker 2: the field of AI is becoming more and more unified,
Speaker 2: Like the methodology, the model that you would use is
Speaker 2: actually becoming similar and similar and sometimes it's even the
Speaker 2: same across these very different fields of robotics, computer vision, language.
Speaker 1: So it's basically, you build a neural network and then
Speaker 1: you just train it on a bunch of images or
Speaker 1: train it on a bunch of documents, and what you
Speaker 1: train it on is what determines sort of what it's
Speaker 1: good for.
Speaker 3: Is it like that that that basically is it like
Speaker 3: it's it's like it instead of each sub feel coming
Speaker 3: up with different and think about it as like hand
Speaker 3: programmed intelligence, right, like let's try to break down what's
Speaker 3: a sentence means, like break into different parts and when
Speaker 3: you approach a computer vision problem, or let's try to
Speaker 3: come up with different features and some features represent an edge,
Speaker 3: some features represent the background.
Speaker 2: Instead of like trying to manually program the kind of
Speaker 2: quite human intelligence into AI, you're basically taking a step
Speaker 2: back and say, I'm just going to create a very
Speaker 2: flexible learning mechanism, which is an artificial new net, and
Speaker 2: then we're just going to feed it a lot of data.
Speaker 2: And if you have different types of problems that you're solving,
Speaker 2: you're just feeding like this new net different types of data,
Speaker 2: but they're still like really the same kind of mechanism.
Speaker 2: Like then that is a drastic departure from how artificial
Speaker 2: intelligence used to be done.
Speaker 1: And so in that world, then the differentiation is just
Speaker 1: in the data set that you are feeding the totally model.
Speaker 2: Totally and in fact, like I mean, this is like
Speaker 2: jumping way forward in time. We were still talking about
Speaker 2: image net. But if you look at really the most
Speaker 2: popular technologies today, these large language models. When you use
Speaker 2: different companies large language models, it's really what you're saying,
Speaker 2: like as if I'm using GPT chat, gipt four versus
Speaker 2: using Google's bought backed by Germini versus Anthpics, Claude or
Speaker 2: coheres command model.
Speaker 1: You're just naming all the different big language large language models. Now.
Speaker 2: Yeah, and like when you think about these different big
Speaker 2: language models, you're just really referring to the different data
Speaker 2: sets that's see behind.
Speaker 1: That they were trained on. So this is interesting kind
Speaker 1: of abstract big picture talk. I want to kind of
Speaker 1: map this now onto the story of Covariate, the story
Speaker 1: of your company. Right, so we're going back in time. Now,
Speaker 1: tell me about the moment when you decided to start
Speaker 1: the company. What's going on?
Speaker 2: Yeah, so the moment that we decide to start a
Speaker 2: company is you'll refer to this image net mode like
Speaker 2: this like large data set that actually first time like
Speaker 2: really taught people that you can train a network to
Speaker 2: solve one specific task really work.
Speaker 1: So this is twenty twelve. There's when a neural net
Speaker 1: trains on image net and does a really good job,
Speaker 1: way better than anybody has ever done. Any Then any
Speaker 1: model has ever done of identifying objects.
Speaker 2: In exactly right, exactly. That was really significant because that
Speaker 2: means if you can collect a lot of data for
Speaker 2: a single task, and if you can get a group
Speaker 2: of PhDs to work on a model for that single task,
Speaker 2: you basically have AI. Like you can solve that task
Speaker 2: really well. I mean like you might need to iterate
Speaker 2: on your data, you might need to iterate on your algorithm,
Speaker 2: but ultimately you can solve that task really well.
Speaker 1: You're basically saying, if you can gather the data, you
Speaker 1: can gather a shitload of data of whatever kind, then
Speaker 1: you can get AI around.
Speaker 2: For that for that kind, right, And which is like
Speaker 2: why you saw artificial intelligence like really started working like
Speaker 2: after twenty twelve, Like like Google, Facebook, all of these
Speaker 2: companies like have a lot of AI based applications, but
Speaker 2: they are largely not democratized because like in order to
Speaker 2: get any single AI working, you still need a lot
Speaker 2: of data and you still need a team of PhDs
Speaker 2: to work on it. So it was a huge breakthrough
Speaker 2: in AI, but it was not sufficient for really widespread
Speaker 2: usage just because the barrier to create one AI is
Speaker 2: so high. Okay, and then comes the second inflection point
Speaker 2: in the history of AI, which is really the start
Speaker 2: of foundation models. So like I'm talking about the most
Speaker 2: initial version of GPT, right, Like I'm talking about like
Speaker 2: these large language models that are trained on multiple tasks
Speaker 2: so that they're incredibly generalizable, Like you can ask to
Speaker 2: do something new and it can do it really well.
Speaker 2: And also it performed better at single task than specialized model.
Speaker 1: And so just to be clear, like until a few
Speaker 1: years ago, people thought reasonably that if you want to
Speaker 1: build AI to whatever translate language right, you would work
Speaker 1: really hard on that. You would try and build an
Speaker 1: AI specifically designed to be really good at translating text
Speaker 1: from one language to another. But this really surprising result,
Speaker 1: this really surprising thing that emerged from just work people
Speaker 1: were doing, was, in fact, that's not the best way
Speaker 1: to get AI to translate language. It's just throw everything
Speaker 1: you can, all the words on the internet at an
Speaker 1: AI model and just say, figure out everything about language,
Speaker 1: figure out how to answer questions about history, and figure
Speaker 1: out how to translate, and figure out how to give
Speaker 1: me a recipe for you know, pasta. And it turns
Speaker 1: out that LAD technique gets you better results at each
Speaker 1: specific thing than trying to build specialized model exactly.
Speaker 2: And that is really the magic of foundation models. And
Speaker 2: that's the thing that we're not obvious to people outside
Speaker 2: of open Ai for a very long time. And because
Speaker 2: we came from open Ai, a lot of the founding
Speaker 2: team at Coverin came from open Ai. We saw that
Speaker 2: inside earlier, and that inside allow us to start Covariant
Speaker 2: to build foundation models for robotics way before other people
Speaker 2: even believed in the.
Speaker 1: Approach, even people in the field, even people in the field.
Speaker 1: So so, yeah, so you were when did you go
Speaker 1: to open Ai? You were you went to work at
Speaker 1: open Ai.
Speaker 2: We went to open Ai when it was about ten
Speaker 2: ish people like like sometime in twenty sixteen.
Speaker 1: Okay, And and when do you sort of personally have
Speaker 1: this realization and you're not the only one to have it,
Speaker 1: but when do you see that the power of foundation models?
Speaker 2: There are two things on it, Like, so the first
Speaker 2: thing is that early on at open Ai, we believed
Speaker 2: in the idea of scaling, like really scaling up the
Speaker 2: model and scaling up the data sets. And you actually
Speaker 2: see models getting like getting increasingly smarter as you actually
Speaker 2: scale them up. So one is that, and then the
Speaker 2: other one is I would say we we had conviction
Speaker 2: in foundation model for robotics probably earlier than foundation model
Speaker 2: for language. And this is like the one key thing
Speaker 2: is that if you think about building a large language
Speaker 2: model that tried to compress the whole internet of knowledge,
Speaker 2: you still need to compress many things that are not
Speaker 2: quite related to each other. Right, Like maybe you're browsing
Speaker 2: on Wikipedia and you have to recite the composition of
Speaker 2: materials of soil on the moon, and you also need
Speaker 2: to learn how to play chess. What there is there's
Speaker 2: really nothing in common with these two things. Yeah, these
Speaker 2: two parts of the knowledge, but you are asking one
Speaker 2: AI model to learn all of these. And the place
Speaker 2: that make a lot of sense to us is that
Speaker 2: there's only one physical world. Like even when you have
Speaker 2: many different robots that need to do different things in
Speaker 2: different factories, different warehouses, they are still interacting in the
Speaker 2: same physical world. And so building a foundation model for
Speaker 2: robotics like has this amazing property of grounding that like,
Speaker 2: no matter what kind of tasks that you're asking this
Speaker 2: foundation model to learn, well, it's just learning with the
Speaker 2: same sets of physics, Like the same surrounding.
Speaker 1: Is the literal ground with the models.
Speaker 2: The literal ground exactly, and.
Speaker 1: The models have to understand just how the physical world works,
Speaker 1: and that you drop a thing, it will fall and etc.
Speaker 2: And it's if it's something that is like deformable, you
Speaker 2: push it, it would move a little bit. If something
Speaker 2: is rigid like you would sly, if something is rollable,
Speaker 2: it would roll away.
Speaker 1: Like.
Speaker 2: These are the type of things that like, no matter
Speaker 2: where you are on Earth and what type of robots
Speaker 2: like body you're using, is the same, right, and so
Speaker 2: like if you can be one single foundation model that
Speaker 2: can learn from all of these different data, it would
Speaker 2: be incredibly powerful.
Speaker 1: So just to just to state it clearly, So you're
Speaker 1: at open AI, you're seeing the power of foundation models,
Speaker 1: you decide to leave and start the company that is
Speaker 1: now Covariant, Like what are you setting out to do
Speaker 1: when you when you start the company?
Speaker 2: Yeah, So when we started Covariant, we had this really
Speaker 2: strong conviction that like, there should be a future that
Speaker 2: has a lot of autonomous robots doing all the things
Speaker 2: that are repetitive, injury prone, dangerous, like and so that
Speaker 2: can really revolutionize the physical world, make it a lot
Speaker 2: more abundant and to really enable that future of autonomous robots,
Speaker 2: you need really smart AI, and like because of the
Speaker 2: inside that we just talk about, like we believe that
Speaker 2: AI had to be a foundation model. We believe like
Speaker 2: you should have single model that learn from all these
Speaker 2: two different robots together and become smarter together.
Speaker 1: So I mean, so the basic idea is, the dream
Speaker 1: is to build one AI foundation model for robots. Basically, yeah,
Speaker 1: in the same way that you can ask chat GBT
Speaker 1: anything in language and it can answer you in language
Speaker 1: about any different thing. You have a model where you
Speaker 1: just sort of make it be the brain of any
Speaker 1: robot and that robot can sort of see the world
Speaker 1: and move and pick things up and behave in the
Speaker 1: world exactly.
Speaker 2: And there's one key problem, Like the key problem is
Speaker 2: that unlike foundation model for language, where you can scrape
Speaker 2: the whole Internet of text as your pre training data,
Speaker 2: there's nothing that is equivalent in the case of robotic
Speaker 2: I mean, there are some images online, there are some
Speaker 2: YouTube videos online, but by and large, like they don't
Speaker 2: really give you the same type of data that are
Speaker 2: in the form of robots interacting with the world. And
Speaker 2: the big problem is because they're just not that many robots,
Speaker 2: they're just doing interesting things in the world. And so
Speaker 2: a big chunk of what we set out to build
Speaker 2: as a company is recognizing that we need to build
Speaker 2: fundation model for robotics. And in order to be a
Speaker 2: foundation model for robotics, you need to have large data sets.
Speaker 2: And in order to create large data sets, you have
Speaker 2: to have robots that are actually creating value for customers
Speaker 2: in production at scale, because if you're only collecting data
Speaker 2: in your own lab, there's only so much data. And
Speaker 2: so I would say the last six years of Covariance
Speaker 2: is largely focused on really building autonomous robot systems that
Speaker 2: work really well for customers, and they're doing interesting things
Speaker 2: to a level of autonomy and reliability that have not
Speaker 2: been hit before.
Speaker 4: In other words, Peter and his colleagues at Covariant built
Speaker 4: robot arms that businesses are paying to use out in
Speaker 4: the world. But to some extent, those robot arms are
Speaker 4: just a means to an end because they're not just
Speaker 4: doing warehouse work. They're collecting more and more.
Speaker 1: Data that Peter and his colleagues are feeding back into
Speaker 1: their AI model to try and make it get better
Speaker 1: and better. In a minute, what those robot arms are
Speaker 1: actually doing out in the world and what they're learning.
Speaker 1: So let's talk about what the robots you have built
Speaker 1: are doing out in the world. Besides generating the data
Speaker 1: that you will use to train the next generation of robots,
Speaker 1: what are they actually doing in the world right now today.
Speaker 2: Yeah, So when we started the company, we soveyed the
Speaker 2: landscape pretty carefully and then we selected warehouse and logics
Speaker 2: as the primary sector that we focus on today. When
Speaker 2: you are stopping online shops, like when you click a
Speaker 2: button and something shows up next day, there's tremendous amount
Speaker 2: of complexities behind that back end logistics of getting things
Speaker 2: to you. And typically it's estimated that each item is
Speaker 2: touched fifteen to twenty times between when you click a
Speaker 2: butttom to buy and when it shows up.
Speaker 1: In contrast, that fifteen to twenty touches of getting a
Speaker 1: thing from the warehouse to your door is your opportunity.
Speaker 2: Exactly right, And like combining with that, people don't want
Speaker 2: to drive in the middle of the night two hours
Speaker 2: into a suburb to work in a warehouse, Like it's
Speaker 2: just a kind of job that has extremely high turnover rate,
Speaker 2: Like not the kind of job that people stay there
Speaker 2: for a very long time, and so there's a tremendous
Speaker 2: amount of desire for more robotics and more automations in
Speaker 2: those environment to do picking up objects, sorting them into
Speaker 2: the right compartment of boxes, and then packing it nicely
Speaker 2: and then shipping it out to you as a customers.
Speaker 1: Tell me a little bit more specifically, I mean, what's
Speaker 1: one thing one of your robots is doing today in
Speaker 1: a warehouse somewhere on the earth.
Speaker 2: Yeah, so we would get like pretty detail and giky here,
Speaker 2: and so we'd actually tell you a little bit of
Speaker 2: like the needy, greitty details of like warehouse. Like, so
Speaker 2: let me describe what the robot is doing. Like the
Speaker 2: robot is doing, is I have a toad full of
Speaker 2: items that come up to the robots, and then I
Speaker 2: would need to grab one thing at a time, like we're.
Speaker 1: Is like a tote bag with a bunch of different stuff.
Speaker 2: And a bunch of different stuff in it, and like
Speaker 2: because like these stuffs are all layout in a chaotic
Speaker 2: way and like they're overlapping with each other. If you're
Speaker 2: not careful, you might drag out multiple items at the
Speaker 2: same time. And these items all have different shapes like
Speaker 2: they might be transparent, they might be reflective, they might
Speaker 2: be hard to see.
Speaker 1: This is hard, like to go back to our old
Speaker 1: Like riveting parts on a car is easy, Folding a
Speaker 1: towel is hard. This is hard because it's heterogeneous. Things
Speaker 1: look different, They come differently every time. This is hard
Speaker 1: for robots. Hard for a sort of classical robot to do.
Speaker 2: Impossible for a classical impossible.
Speaker 1: Not just hard, impossible. Can your robots do it?
Speaker 2: Yeah? They can do it? How extremely well?
Speaker 1: How did you solve How did you solve it?
Speaker 2: How does it work in the end of the day?
Speaker 2: Like it the way that it operates us very similar
Speaker 2: to how like humans vision system work, Like we have
Speaker 2: two eyes and then like by two eyes looking at
Speaker 2: something like we like we can figure out what's the
Speaker 2: depth of a certain items, like because our two eyes
Speaker 2: can triangulate a single point the three D world and
Speaker 2: it's the same kind of mechanisms and so like you
Speaker 2: can just use multiple regular cameras, just like the one
Speaker 2: that you have on your iPhone, and by having multiple
Speaker 2: ones of those like you give the new net the
Speaker 2: ability to triangulate what's happening.
Speaker 1: Just the way our two eyes allow us to see
Speaker 1: depth essentially exactly right, And are there other things like
Speaker 1: like weight, I mean the arm is going to be
Speaker 1: picking things up like weight or whether you know, presumably
Speaker 1: there's like could be a shirt in a plastic bag,
Speaker 1: could be a box, or something's are rigid, some things
Speaker 1: are deformable.
Speaker 2: Yeah, So what we have found is that if you
Speaker 2: just have a visual understanding of the world that is
Speaker 2: as robust as human, you go a really long way. Right,
Speaker 2: Like so when I when I pick up a cup, right, like,
Speaker 2: I'm not doing a lot of calculations on my how
Speaker 2: is my fingers faced? Like exactly, it gets translated to
Speaker 2: the cup and making sure it holds right.
Speaker 1: It's part of the miracle of being a person though, Right,
Speaker 1: it's like a really hard problem to pick.
Speaker 2: It's a very hard problem. But then like your brains
Speaker 2: subconsciously solve it for you, like your your system one
Speaker 2: thinking somewhat solved that.
Speaker 1: For thinking, you don't have to think about yeahm.
Speaker 2: Hm, exactly, And and you can like imagine like when
Speaker 2: you do this, like in fact, like even if my
Speaker 2: fingers are numb, I can still do this perfectly like
Speaker 2: just because like so it acquires this intuitive understanding of
Speaker 2: interaction with physical will so well that you can do it.
Speaker 1: So basically vision, vision gets you most of the way there.
Speaker 2: I would say, vision and then the ability to intuit
Speaker 2: physics from your visual input that you'll get.
Speaker 1: That told me that second. What is wild though, Like
Speaker 1: I mean into it, I mean into it in the
Speaker 1: in the context of it, I mean sort of make inferences.
Speaker 1: I mean intuit is yeah, okay, yeah.
Speaker 2: But like by by into it, I mean like it's
Speaker 2: not doing some kind of detail physical calculation.
Speaker 1: It's not doing math. It's not doing math.
Speaker 2: Yeah, it's doing a kind of high level pattern matching
Speaker 2: of well, like based on how these things looks, this
Speaker 2: is likely going to be a successful way to approach
Speaker 2: the item and interact with it.
Speaker 1: What what's next?
Speaker 2: So what is next immediately is very exciting? Right. We
Speaker 2: are now getting to a place that by we, I
Speaker 2: mean we as in AI community has gotten to a
Speaker 2: place that we have enough computation power and algorithmic and
Speaker 2: modeling understanding that can allow us to extract a lot
Speaker 2: out of data.
Speaker 1: Right, So from any given amount of data, you can get.
Speaker 2: More, you can get more out of it. Right.
Speaker 1: It's exciting for you because data is such a constraint
Speaker 1: on what you're trying to do exactly.
Speaker 2: And then like we're building up this large robotic data
Speaker 2: set by tapping into a lot of these events, is
Speaker 2: it gives us the ability to even get more out
Speaker 2: of the data sets that we're building and allow us
Speaker 2: to build smarter than better robotics foundation models that perform
Speaker 2: better at the current tasks that they're supposed to do
Speaker 2: and also power more robots.
Speaker 1: When people talk about concerns around AI, they often kind
Speaker 1: of have jokingly use the phrase killer robots, which is
Speaker 1: usually like a metaphor or something. But in your instance,
Speaker 1: because you are building robots, and because you are building
Speaker 1: by design a model that is supposed to be used
Speaker 1: for lots of different purposes, I can in fact very
Speaker 1: easily imagine killer robot applications of your work. Like that
Speaker 1: seems like a very plausible thing someone could do with it, Like,
Speaker 1: is that something you think about worry about?
Speaker 2: I would say very Fortunately, in the very near to
Speaker 2: medium term use cases, we are very safe because like
Speaker 2: all of these industrial robots are very much confined by
Speaker 2: the stations that they're designed in. Curse like industrial robots
Speaker 2: heavy machineries that are subject to regulations and are very
Speaker 2: carefully They are like careful design guidelines and compliance requirements
Speaker 2: for them. They are already by design safe.
Speaker 1: You're saying your model is built for robots basically for
Speaker 1: robot arms. I mean, is that essentially what the model
Speaker 1: you're building is really a foundation model for robot arms
Speaker 1: that are built to just be in one place, pick
Speaker 1: things up, put them down, that sort of thing. It's
Speaker 1: not a foundation model that you could have map onto
Speaker 1: a car or something, or even to a robot that
Speaker 1: walks around. It wouldn't work for that.
Speaker 2: That would not be the near term use cases. Like
Speaker 2: so like because the near term use cases are more
Speaker 2: in this safe by construction setting, it allows us to
Speaker 2: not more about that problem and in fact, like have
Speaker 2: basically no way to misuse the technology in the way
Speaker 2: we need to. But I do agree with you, like
Speaker 2: as we actually unleash this model to a set of
Speaker 2: use cases, like when these robots can actually interact with
Speaker 2: the world in a lot more freeform way those other cases,
Speaker 2: that the safety considerations become a lot more important and
Speaker 2: there's definitely a lot more work that need to be
Speaker 2: done for that to be reliable.
Speaker 1: We'll be back in a minute with the lightning round.
Speaker 1: M there is a lightning round that we're gonna do
Speaker 1: now for the end of the interview, what household chore
Speaker 1: do you wish that a robot could do?
Speaker 2: Cleaning up kitchen? Yeah, but I don't like the cleanup
Speaker 2: of it.
Speaker 1: That seems like a really hard one, like putting stuff away.
Speaker 1: Basically I'm wiping the counter maybe less hard, but like
Speaker 1: putting stuff away seems like a really hard job for
Speaker 1: a robot.
Speaker 2: It is. And these are the type of jobs that
Speaker 2: start to get to how we are limitation, Like, these
Speaker 2: are the type of jobs that start to get to
Speaker 2: like you probably do want a humanoid robot, like you
Speaker 2: probably do want something that kind of moves and conform
Speaker 2: to the human standard of interacting with the world.
Speaker 1: Because the kitchen is optimized, right, or you would have
Speaker 1: to redesign the kitchen for a robot, and but then
Speaker 1: that would suck because then you couldn't get your plates
Speaker 1: because they'd be in some random spot or whatever. Okay,
Speaker 1: so you left open ai in twenty seventeen. In the
Speaker 1: past year, open ai became like this household word, GPT
Speaker 1: became a household word. Were you surprised as a you know,
Speaker 1: old school former open AI guy, were you surprised by
Speaker 1: how how wild the world went for GPT or by
Speaker 1: how good it was how soon?
Speaker 2: I was definitely surprised by the speed of it. I
Speaker 2: was surprised by the speed of bull of the technologists
Speaker 2: development and the speed of adoption. But I was not
Speaker 2: surprised by the fact that it could be dispig and
Speaker 2: it could be bigger.
Speaker 1: You know, when you were talking about sort of warehouse
Speaker 1: and getting data from you know, picking and packing basically,
Speaker 1: I thought, of course, as anyone would, of Amazon, and
Speaker 1: I've read that they're working on some kind of robot
Speaker 1: arm I feel like they would just have so much
Speaker 1: data that they could gather if they wanted, just because
Speaker 1: they're so big, they have so many warehouse I mean
Speaker 1: the same way that say Google just gets tons of
Speaker 1: data every day with every Google search and the way
Speaker 1: people like I feel like that would be very hard
Speaker 1: to compete with, But.
Speaker 2: We also don't need to compete with them. They are
Speaker 2: also a very large role that Amazon is not serving,
Speaker 2: and there are a lot of customers that don't have
Speaker 2: the same degree of engineering team data access as Amazon, and.
Speaker 1: You could be the shop by of you could be
Speaker 1: the Shopify of warehouse robots.
Speaker 2: There are all of these people that still need help,
Speaker 2: and we very gladly help them.
Speaker 1: What was the first robot you personally built.
Speaker 2: I think it's probably one of the first pick and
Speaker 2: place robot e Covariant.
Speaker 1: You didn't build them when you were kid there.
Speaker 2: I didn't build them when I was kid.
Speaker 1: What made you get into robots?
Speaker 2: I would say I'm an AI person first and robot
Speaker 2: person set, and a big part of the interest in
Speaker 2: robotics is probably driven by my interests in AI. Like
Speaker 2: it just like we have just not make as much
Speaker 2: progress for AI in the physical world. That's AI in
Speaker 2: the digital world, and to a large degree, like I
Speaker 2: think we have to make progress there because like ultimately
Speaker 2: we live in the physical world. Like you're creating all
Speaker 2: these intelligence and amazing things in the digital world. That
Speaker 2: is all great, but where's AI in the physical world?
Speaker 2: Like this remarkably little progress there despite how much AI
Speaker 2: has moved forward. And so to some degree, like it's
Speaker 2: a it's driven by a conviction that like AI has
Speaker 2: to progress forward and AI will have a large impact
Speaker 2: in the physical world.
Speaker 1: What do you understand about robots that most people don't.
Speaker 2: I think the most interesting thing about robots that I
Speaker 2: would say, like we understand a covariant that maybe outside
Speaker 2: of the company don't is making one robot work is
Speaker 2: obviously hard and fun, but making a lot of robots
Speaker 2: work as scale for a lot of customers take a
Speaker 2: lot of operational discipline. Like it's about like doing many
Speaker 2: many things right, like before robots go into a facility,
Speaker 2: like how should they prep the site so the robots
Speaker 2: actually work. To ship robots at scale, it's a competencies
Speaker 2: that requires a lot of operational excellence, and that is
Speaker 2: something that I would say, like when most people think
Speaker 2: about robots like big thing about is like this sexy,
Speaker 2: interesting technology, they don't think about having to nail a
Speaker 2: thousand steps, a thousand small steps well in order to
Speaker 2: have robots actually have an impact in the world at scale.
Speaker 1: That's the leap from the academic lab to being a
Speaker 1: real company selling real products in the world. Great, anything
Speaker 1: else do you want to talk about?
Speaker 2: No? I enjoyed this conversation.
Speaker 1: Yeah, likewise, thank you. Theater Chen is the co founder
Speaker 1: and CEO of Covariate. Today's show was produced by Edith
Speaker 1: Russolo and Gabriel Hunter Chang. It was edited by Karen
Speaker 1: Chakerji and engineered by Sarah Bugueer. You can email us
Speaker 1: at at Pushkin dot f M. I'm Jacob Goldstein, and
Speaker 1: we'll be back next week with another episode of What's
Speaker 1: Your Problem