WEBVTT - Cerebras Built a New Way to Make Sand Think - The Story 0:00:16.160 --> 0:00:19.160 Welcome to tech Stuff. I'm os Voloscian. There are many 0:00:19.160 --> 0:00:22.040 ways to think about AI. The one that recently struck 0:00:22.079 --> 0:00:24.320 me quite a lot comes from Demis Hassebus, founder of 0:00:24.400 --> 0:00:27.160 Google's Deep Mind, who said, if you stop to think 0:00:27.200 --> 0:00:30.960 about it, we've essentially found a way to make sand think. 0:00:31.600 --> 0:00:34.479 He was talking about silicon, the element that is melted 0:00:34.520 --> 0:00:38.040 and sliced into the chips that power AI, and today 0:00:38.360 --> 0:00:41.040 I want to talk about those chips. Our guest today 0:00:41.120 --> 0:00:44.360 is Steve Vassalo, a venture capitalist who was the first 0:00:44.520 --> 0:00:47.880 backer of Cerebras, a chip company that went public earlier 0:00:47.880 --> 0:00:52.920 this year in the largest ever IPO for a semiconductor company. Steve, 0:00:53.200 --> 0:00:54.160 Welcome to tech Stuff. 0:00:54.560 --> 0:00:56.720 Thanks Arles. It's great to be here, great. 0:00:56.480 --> 0:00:59.360 To see I'm curious what's your reaction to that, To 0:00:59.440 --> 0:01:01.760 that framing of demis making sand think. 0:01:02.800 --> 0:01:05.320 I mean, it's such a great reminder of what it 0:01:05.400 --> 0:01:08.520 is that we're doing and how fundamental it is, both 0:01:08.520 --> 0:01:11.080 from a physics perspective and all the hard work that 0:01:11.280 --> 0:01:13.600 is required to take a piece of sand and turn 0:01:13.640 --> 0:01:16.640 it into ultimately intelligence. And then there's also something that 0:01:17.120 --> 0:01:19.600 feels sort of I don't know, almost I would say 0:01:19.720 --> 0:01:21.520 sort of you know, just sort of scary and fun 0:01:21.560 --> 0:01:24.360 in a humanity sense, like, Wow, what are we building 0:01:24.400 --> 0:01:26.440 right now? What are we doing to be able to 0:01:26.920 --> 0:01:30.520 sort of turn this, you know, very basic material that 0:01:30.560 --> 0:01:34.360 surrounds us everywhere into something that is super intelligent. 0:01:35.240 --> 0:01:37.880 I want to ask you all about the story of 0:01:37.920 --> 0:01:41.200 Cerebrest and and also frankly, what wayfer scale means. But 0:01:41.280 --> 0:01:43.440 before we get there, can you just zoom out a 0:01:43.440 --> 0:01:46.399 little bit and give us the wider context of what's 0:01:46.440 --> 0:01:49.160 going on with chips right now? Because I think we 0:01:49.200 --> 0:01:51.440 will see the headlines. I mean, I'm going to read 0:01:51.440 --> 0:01:53.800 a few, but the New York Times recently had it's 0:01:53.840 --> 0:01:56.560 not just in Nvidia. The AI boom has ignited Asia's 0:01:56.600 --> 0:01:59.400 chip companies. Dull Street Journal had a couple of months 0:01:59.400 --> 0:02:02.600 ago inside Apple's push to build an all American Chip 0:02:02.880 --> 0:02:06.960 axios in July, Chip craze for the layman, what's going 0:02:06.960 --> 0:02:07.360 on here? 0:02:07.760 --> 0:02:08.760 Yeah? I love that question. 0:02:08.880 --> 0:02:13.320 So the way I think about it is really sort 0:02:13.320 --> 0:02:16.440 of starts with what I'll call workloads, And the workloads are, 0:02:16.480 --> 0:02:18.560 you know, what is the work of computing, And if 0:02:18.600 --> 0:02:21.760 you go back to the nineteen seventies and eighties, the 0:02:21.840 --> 0:02:25.520 workloads at that time were very serialized. They were kind 0:02:25.520 --> 0:02:28.120 of diverse if you think about the invention of things 0:02:28.160 --> 0:02:30.240 like the spreadsheet, or you know, as a kid, I 0:02:30.240 --> 0:02:33.000 wrote my college essays on word Star and an eighty 0:02:33.040 --> 0:02:34.320 eighty six IBM computer. 0:02:35.000 --> 0:02:38.360 Diverse in a sense that different computers for different applications, 0:02:38.639 --> 0:02:40.119 or diverse in what sense. 0:02:40.160 --> 0:02:43.440 Diverse in that you had one computing platform, which turned 0:02:43.440 --> 0:02:46.280 out to be Intel's X eighty six platform that turned 0:02:46.320 --> 0:02:50.079 out to be a great general purpose computing platform for 0:02:50.720 --> 0:02:53.639 a wide variety of applications. And then sometime in the 0:02:53.680 --> 0:02:56.320 kind of mid nineteen eighties, we began to see a 0:02:56.360 --> 0:02:59.720 new workload emerge. And that workload really came from two 0:02:59.760 --> 0:03:02.760 areas as one was from gaming and the rendering of 0:03:02.760 --> 0:03:06.760 pixels and a need for paralyzation of the compute. And 0:03:06.919 --> 0:03:09.600 then also in believe it or not, the CAD tools, 0:03:09.639 --> 0:03:13.440 so you know, building new products, whether its futures exactly. 0:03:13.480 --> 0:03:14.480 So computer aided. 0:03:14.280 --> 0:03:18.760 Design pushed the perimeter of what was possible on traditional 0:03:18.800 --> 0:03:22.080 computing CPU general purpose systems. So that was really the 0:03:22.120 --> 0:03:24.640 dawn of the GPU, and really that was Nvidia. And 0:03:24.680 --> 0:03:26.680 by the way, at the time of Nvidia's founding, there 0:03:26.720 --> 0:03:29.639 were like we're seeing today thirty other companies working on 0:03:30.000 --> 0:03:33.360 graphics processing units. And I say graphics processing units. We 0:03:33.680 --> 0:03:35.840 many people, you know, see the term GPU. They don't 0:03:36.240 --> 0:03:39.160 know or remember that, like graphics is the g It 0:03:39.240 --> 0:03:42.000 was purpose built for that workload. And then the next 0:03:42.080 --> 0:03:45.800 real shift was a few decades later, but was really 0:03:46.120 --> 0:03:49.280 a constraint a workload driven by mobile devices, and the 0:03:49.320 --> 0:03:52.760 constraints there are totally different, right. This is form factor, 0:03:53.000 --> 0:03:56.720 this is power consumption, battery life of course related to that, 0:03:57.040 --> 0:03:59.400 and so you've got a totally different workload. And then 0:03:59.400 --> 0:04:01.600 what we began to see this is now zooming forward 0:04:01.600 --> 0:04:05.280 to kind of the twenty fifteen timeframe, really starting in 0:04:05.320 --> 0:04:08.600 the twenty twelve timeframes, but the workloads began to really 0:04:08.640 --> 0:04:11.640 build in the mid twenty tens. And what you saw was, 0:04:11.800 --> 0:04:14.560 oh wait a minute, statistical inference over large data sets. 0:04:14.840 --> 0:04:17.960 Our portfolio companies were beginning to hire data science teams 0:04:18.400 --> 0:04:22.359 business models that weren't possible before you had sort of 0:04:22.360 --> 0:04:25.919 a real awareness of kind of the data underlying your This. 0:04:26.720 --> 0:04:30.159 Was the era when phrases like big data es out 0:04:30.160 --> 0:04:33.719 computing mark andresen saying software eats the world. Is this 0:04:33.760 --> 0:04:35.040 the area referring to sort. 0:04:34.920 --> 0:04:37.000 Of about five years later than software into the world, 0:04:37.000 --> 0:04:40.360 but kind of roughly contemporaneous with that. But really what 0:04:40.400 --> 0:04:43.000 you're seeing big data is absolutely right and in a 0:04:43.120 --> 0:04:47.680 sense that, wow, there are signals inside of our businesses 0:04:47.720 --> 0:04:50.320 that if we're more attuned to we could make better decisions. 0:04:50.720 --> 0:04:53.080 And what was interesting at that time, and this kind 0:04:53.080 --> 0:04:55.400 of ties back to the sort of the GPU wave 0:04:55.960 --> 0:04:59.960 was those kinds of workloads. AI workloads are very data 0:05:00.120 --> 0:05:02.559 io intensive, so that just means the data is moving 0:05:02.600 --> 0:05:06.440 through the system with with a lot of bandwidth, memory 0:05:06.480 --> 0:05:08.960 bandwidth and with high high frequency. 0:05:08.520 --> 0:05:11.000 Back and forth, back and forth. We had somebody who 0:05:11.040 --> 0:05:13.760 I'm sure you know, Nick McEwen from course on the 0:05:13.760 --> 0:05:16.359 show recently, who was at Cisco in the days of 0:05:16.400 --> 0:05:18.400 building the kind of you know dominat net Round. 0:05:18.440 --> 0:05:20.880 He was building networks, networking systems in that company was 0:05:20.880 --> 0:05:21.640 acquired by Cisco. 0:05:21.680 --> 0:05:22.120 Exactly. 0:05:22.240 --> 0:05:25.839 He was saying, today's data center, a single data center, 0:05:25.880 --> 0:05:28.760 has more back and forth connections than the whole Internet 0:05:28.760 --> 0:05:30.120 did in the nineties. 0:05:29.680 --> 0:05:31.560 Exactly, exactly. In fact, was fun. 0:05:31.600 --> 0:05:35.520 There was an announcement yesterday and videos new vera Rubin 0:05:36.160 --> 0:05:37.600 and you looked at I mean I sort of laughed 0:05:37.600 --> 0:05:40.559 when I looked at the photo of the launch because 0:05:40.560 --> 0:05:42.840 you're staring at this thing and it was just basically 0:05:42.880 --> 0:05:48.240 this medusa of basically cables and connectors behind all of that, 0:05:48.400 --> 0:05:51.560 you know, literally probably miles of AMFONYL cables. 0:05:52.160 --> 0:05:54.240 Was of course some graphics processing. 0:05:53.960 --> 0:05:56.400 Units, but what they're having to do is connect all 0:05:56.400 --> 0:05:59.839 that stuff back together. And so this ties to the 0:06:00.000 --> 0:06:02.440 point around wafer scale, which is, if you are going 0:06:02.520 --> 0:06:06.920 to build something starting from scratch, not using platforms that 0:06:07.000 --> 0:06:11.320 already exist, not relying on CPUs or GPUs or ARM architectures, 0:06:11.600 --> 0:06:13.719 and you were to say, what would you build that's 0:06:13.960 --> 0:06:15.719 really designed for these kinds of workloads? 0:06:15.920 --> 0:06:17.520 You build what we built at Cerebras. 0:06:17.800 --> 0:06:20.560 And that's the journey that we started in twenty fifteen, 0:06:21.000 --> 0:06:23.240 and really it's born of this notion of big, big 0:06:23.279 --> 0:06:27.120 workloads deserve custom purpose built silicon, and that has been 0:06:27.200 --> 0:06:30.320 really the story of every silicon wave over the last 0:06:30.320 --> 0:06:31.000 five decades. 0:06:31.560 --> 0:06:35.040 And talk about this concept of wafers scale because I can. 0:06:35.320 --> 0:06:38.799 I mean, if you look at the Cerebras chip, it's 0:06:38.839 --> 0:06:44.159 fifty times bigger than an Nvidia GPU. And as you 0:06:44.240 --> 0:06:46.640 mentioned with with Jensen holding up with verrub and it 0:06:46.720 --> 0:06:49.040 is this kind of like, you know, sort of different 0:06:49.160 --> 0:06:52.520 silicon chips connected together with cables and stuff, whereas your 0:06:52.600 --> 0:06:55.360 chip is is kind of a single huge piece of silicon. 0:06:55.520 --> 0:06:56.960 Why do they do that and why do you keep 0:06:56.960 --> 0:06:57.919 it as a single piece. 0:06:58.200 --> 0:07:00.599 The way to think about sort of silicon, MANUF is 0:07:00.640 --> 0:07:03.800 it's grown as you started this conversation with sand, you know, 0:07:03.880 --> 0:07:08.400 it's grown from sand into single crystalline columns, and then 0:07:08.440 --> 0:07:11.720 those columns are sliced very thinly that is then cut 0:07:11.760 --> 0:07:13.840 to basically turn it into a nine inch roughly nine 0:07:13.880 --> 0:07:17.120 inch square, and that nine inch square has trillions of 0:07:17.120 --> 0:07:20.880 transistors on it. And the way Nvidia or ARM or 0:07:20.960 --> 0:07:22.840 others build their chips is they might start with a 0:07:22.840 --> 0:07:25.600 twelve in wafer, but then they cut it. They cut 0:07:25.600 --> 0:07:28.520 that wafer into many smaller components that are, as you said, 0:07:28.560 --> 0:07:31.280 sort of fifty or fifty eight times smaller than ours, 0:07:31.640 --> 0:07:33.679 and then they cut it and then in many cases 0:07:33.680 --> 0:07:36.560 they're actually literally putting them back together. And you ask 0:07:36.640 --> 0:07:39.640 a question, which is why cut it? That's exactly the 0:07:39.720 --> 0:07:42.160 question that we asked ourselves, which is you shouldn't cut it. 0:07:43.000 --> 0:07:44.120 You should keep it all together. 0:07:44.160 --> 0:07:46.840 Because as soon as you as soon as you take 0:07:46.880 --> 0:07:51.080 communications from a chip off of the chip. So once 0:07:51.600 --> 0:07:54.200 once a bit needs to move from one chip to another, 0:07:54.640 --> 0:07:58.680 you basically have other components involved, and everything you add 0:07:58.760 --> 0:08:00.440 as you would imagine with any sytem them. As soon 0:08:00.480 --> 0:08:03.840 as you add another component between two pieces of silicon, 0:08:04.000 --> 0:08:07.200 you've slowed it down, You've created new constraints. And so 0:08:07.360 --> 0:08:10.120 the big idea here is don't do any of that. 0:08:10.400 --> 0:08:12.400 I'm sure you're familiar. I mean, Elon, you know he's 0:08:12.440 --> 0:08:14.880 working on other stuff in this domain, but you know 0:08:14.920 --> 0:08:17.040 he has this, this sort of rule of thumb. When 0:08:17.080 --> 0:08:19.960 you work at SpaceX or one of his companies, you 0:08:20.000 --> 0:08:22.160 hear this all the time, which is the best part 0:08:22.240 --> 0:08:25.600 is no part. And the point of that is you'd 0:08:25.720 --> 0:08:28.280 like to reduce all of your components down to their 0:08:28.320 --> 0:08:30.640 simplest sort of platonic ideal form. 0:08:30.920 --> 0:08:31.920 And that is exactly what. 0:08:31.880 --> 0:08:34.120 We decided to do with Cerebus, which is, instead of 0:08:34.280 --> 0:08:38.040 taking a twelve inch column of silicon, slicing it into 0:08:38.200 --> 0:08:41.920 hundreds of smaller pieces, then putting those on a motherboard 0:08:41.920 --> 0:08:42.800 and reconnecting them. 0:08:42.679 --> 0:08:45.240 With copper wires or copper traces. 0:08:45.000 --> 0:08:47.400 Let's keep it all on one piece so that the 0:08:47.559 --> 0:08:51.600 data can move as fast as possible throughout that single 0:08:51.640 --> 0:08:54.679 piece of silicon, and we'll get a thousand x improvement 0:08:54.800 --> 0:08:58.160 in performance. And that's from a memory bandwidth perspective. That's 0:08:58.160 --> 0:09:00.839 exactly what we delivered, and so's I were now call 0:09:00.880 --> 0:09:03.480 it roughly fifteen in some cases even more than that 0:09:03.760 --> 0:09:06.720 times faster than any other inference platform on the planet. 0:09:07.280 --> 0:09:09.400 What do you think the biggest misconception is about the 0:09:10.040 --> 0:09:12.520 quote unquote chip craze today. 0:09:13.480 --> 0:09:16.400 I would say the biggest misconception is how hard it 0:09:16.480 --> 0:09:20.360 is to build a new platform to yield those systems, 0:09:20.640 --> 0:09:22.839 and the yield it's basically for everyone you make, how 0:09:22.840 --> 0:09:25.960 many of those can you use? And that that is 0:09:26.000 --> 0:09:28.760 sort of one of those fundamental drivers in the semiconductor industry. 0:09:29.040 --> 0:09:32.240 But in the case of large semiconductor systems powering it 0:09:32.640 --> 0:09:35.000 so you know, most of these systems are now taking 0:09:35.040 --> 0:09:38.719 tens of kilowatts. Back when I was building startups a 0:09:38.760 --> 0:09:41.640 couple decades ago, you know, a rack was somewhere on 0:09:41.679 --> 0:09:43.600 the other of two to three kilowatts. We now have 0:09:43.679 --> 0:09:47.400 racks that are sixty kilowatts, and so you're trying to 0:09:47.440 --> 0:09:49.120 power this system now. Of course, once you're powering it. 0:09:49.200 --> 0:09:50.360 Guess what else you have to do. You have to 0:09:50.360 --> 0:09:52.319 cool it. You have to get that all the all 0:09:52.360 --> 0:09:54.679 the heat out of that system. And then of course, 0:09:54.800 --> 0:09:57.120 once if that's that's now working, now you're going to 0:09:57.200 --> 0:09:59.240 program this thing that has you know, two and a 0:09:59.240 --> 0:10:02.920 half trillion trans and then get you know, modern models 0:10:02.960 --> 0:10:04.720 to work on that and get them up and running 0:10:04.840 --> 0:10:07.040 in a matter of days or weeks instead of months 0:10:07.160 --> 0:10:10.920 or years. And so you're compounding many, many hard problems. 0:10:11.240 --> 0:10:15.600 And in most businesses, when you compound hard problems, what 0:10:15.679 --> 0:10:18.040 comes out the other end often doesn't work. And so, 0:10:18.640 --> 0:10:21.040 you know, the single greatest challenge really is can you 0:10:21.120 --> 0:10:22.520 actually make this thing at scale? 0:10:23.160 --> 0:10:23.319 Yeah? 0:10:23.360 --> 0:10:25.920 I think I think you said somewhere, you know, I 0:10:25.960 --> 0:10:28.520 sometimes joke that Cerebras was five startups in one. You've 0:10:28.520 --> 0:10:31.520 sold wafer scale, great nive power system, cool it, package it, 0:10:31.520 --> 0:10:34.080 deploy code onto it, integrated with an existing frameworks, get 0:10:34.120 --> 0:10:36.840 into data centers. Each one of those challenges was quite 0:10:36.880 --> 0:10:39.800 literally its own company. So I mean, let's go back 0:10:39.840 --> 0:10:43.640 to twenty fifteen. I mean, you had this insight, you know, 0:10:43.720 --> 0:10:46.520 every new wave of computing demands and new type of 0:10:46.520 --> 0:10:49.760 silicon right and We talked about the computing of the eighties, 0:10:49.800 --> 0:10:53.040 one type of chips and the GPUs for gaming, which 0:10:53.280 --> 0:10:56.559 ended up by sort of accident essentially powering the air revolution, 0:10:57.080 --> 0:10:59.960 the arm chips that power cell phones, and then these 0:11:00.080 --> 0:11:04.240 these new chips that cerebreast design for data centers. Essentially 0:11:04.480 --> 0:11:07.200 exactly who were you in twenty fifteen, and we've got 0:11:07.200 --> 0:11:10.320 the kind of we've got the global context, the industry context, 0:11:10.320 --> 0:11:14.080 what's the what was the Steve context? And what made you? 0:11:14.720 --> 0:11:17.120 I mean nowadays twenty twenty six. When we think about 0:11:17.360 --> 0:11:20.280 venture capital and investing, you know, we often think about 0:11:20.320 --> 0:11:23.839 like momentum based, like how do I get into anthropic 0:11:23.880 --> 0:11:26.800 before it goes public right? Versus like how do I 0:11:26.880 --> 0:11:29.720 make a ten year plus bet on something which which 0:11:29.920 --> 0:11:33.880 you know is not only requires all these macro things 0:11:33.920 --> 0:11:36.600 to break my way, but also has like just layer 0:11:36.679 --> 0:11:39.200 upon layer, up on layer of complexity. And you know, 0:11:39.240 --> 0:11:41.199 these these all these different elements we talked about in 0:11:41.280 --> 0:11:45.160 terms of cooling and packaging and integration. What what made 0:11:45.200 --> 0:11:49.160 you want to do this? And how much of your career, 0:11:49.240 --> 0:11:52.240 shall we say, or your capital, your financial capital, your 0:11:52.240 --> 0:11:54.240 relation capital did you stake on this? 0:11:54.760 --> 0:11:57.679 I love that, Yeah, so I mean the quick bounce 0:11:57.720 --> 0:12:00.680 on me is you know, I'm trained as a roboticist. 0:12:01.600 --> 0:12:03.880 I lived for the first ten years of my career 0:12:03.960 --> 0:12:07.280 at the intersection of electrical and mechanical systems. I worked 0:12:07.280 --> 0:12:09.160 at a company called Ideo, designing products. 0:12:10.280 --> 0:12:10.840 Some of those. 0:12:10.720 --> 0:12:13.400 Products brought those two worlds together as well as sort 0:12:13.400 --> 0:12:16.080 of the design world, sort of really answering the question 0:12:16.120 --> 0:12:19.000 of once you've solved those hard technical problems at inter 0:12:19.000 --> 0:12:22.079 section of electrical and mechanical systems, does anyone actually care? 0:12:22.120 --> 0:12:24.000 Do people love it and want it? So I spent 0:12:24.040 --> 0:12:26.800 really a decade building products. Came to Foundation actually as 0:12:26.800 --> 0:12:30.040 an entrepreneur in residence on the path to starting another company. 0:12:30.040 --> 0:12:31.920 And this is now in the spring of two thousand 0:12:31.960 --> 0:12:36.760 and seven. And so I'm a builder by nature, if 0:12:36.800 --> 0:12:39.400 I would say, sort of more of an accidental venture 0:12:39.400 --> 0:12:42.960 capitalist than one who sort of, you know, thought to 0:12:43.000 --> 0:12:45.400 do this from when I was a teenager. I didn't 0:12:45.400 --> 0:12:48.000 I literally didn't know the term venture capital until I 0:12:48.040 --> 0:12:50.600 moved to Silicon Valley back in the mid nineteen nineties. 0:12:51.040 --> 0:12:53.920 And the specific context and I love the question around 0:12:53.960 --> 0:12:56.240 sort of what was required to get an investment like this. 0:12:57.000 --> 0:12:59.640 Approved as it were, more than a decade ago. 0:13:00.679 --> 0:13:02.840 The answer to that question is is really sort of 0:13:02.880 --> 0:13:05.120 central to how we work at Foundation Capital, which is 0:13:05.679 --> 0:13:08.880 we go really deep in opportunity space, and we're not 0:13:08.880 --> 0:13:12.079 trying to be everything to everyone, and when we go deep, 0:13:12.640 --> 0:13:15.120 we try to kind of build what we call our 0:13:15.160 --> 0:13:17.559 points of view, a sense of what would need to 0:13:17.600 --> 0:13:19.520 be true for us to be excited about a new 0:13:19.520 --> 0:13:21.760 investment in this area. What are the attributes of the 0:13:21.760 --> 0:13:25.120 founders that would make them successful or not, What is 0:13:25.160 --> 0:13:27.720 the timing of the market opportunity, because of course many 0:13:27.800 --> 0:13:30.560 venture capital investments, you know, have great ideas, but are 0:13:30.559 --> 0:13:32.680 maybe a decade too early or in some cases multiple 0:13:32.679 --> 0:13:35.640 dedcades too early. And so for me, I had been 0:13:35.640 --> 0:13:39.160 studying this space, and I'd actually met Andrew Feldman and 0:13:39.200 --> 0:13:42.480 his co founder Gary Back no joke, in October of 0:13:42.480 --> 0:13:45.480 two thousand and seven, seven years before we really re 0:13:45.559 --> 0:13:49.079 engaged around the opportunity at Cerebras, and they were actually 0:13:49.080 --> 0:13:51.400 at that time just starting their prior company, a company 0:13:51.440 --> 0:13:53.840 called c Micro, which was kind of also in the 0:13:53.960 --> 0:13:57.840 kind of data center think sort of warehouse scale computing concept, 0:13:57.960 --> 0:14:02.400 and we didn't actually invest in that company. I hit 0:14:02.440 --> 0:14:05.640 it off with them and they were acquired by AMD 0:14:05.760 --> 0:14:07.800 maybe five years later, and a couple of years into that, 0:14:08.280 --> 0:14:10.920 I reached back out to Andrew and I could tell immediately, 0:14:10.960 --> 0:14:13.440 I mean, I had this intuition going into the conversation 0:14:13.679 --> 0:14:15.160 that he was not going to stay long for AMD. 0:14:15.280 --> 0:14:18.800 He had another startup in him. And so we basically, 0:14:18.800 --> 0:14:21.160 over the course of the next two years riffed on 0:14:21.240 --> 0:14:24.160 a whole bunch of ideas that at that time were 0:14:24.280 --> 0:14:26.080 kind of around that big data concept that you and 0:14:26.080 --> 0:14:28.440 I talked about earlier, and we were looking at a 0:14:28.440 --> 0:14:31.040 whole bunch of companies were using Andrew as a sounding 0:14:31.080 --> 0:14:34.000 board for the opportunity space. And then he and Gary 0:14:34.000 --> 0:14:37.240 and then Sean and then Michael and JP started to 0:14:37.280 --> 0:14:40.600 basically converge on this idea around Hey, wait a minute, 0:14:40.640 --> 0:14:43.280 there's this new workload, and this new workload we got 0:14:43.320 --> 0:14:46.160 to pay attention to, and this new workload while as 0:14:46.160 --> 0:14:47.520 you said, sort of NVIDIA is a bit of a 0:14:47.520 --> 0:14:50.080 happy accident that it happens to be better than CPU's, 0:14:50.360 --> 0:14:53.200 but it's not what you build. And so the you know, 0:14:53.240 --> 0:14:55.840 these five founders basically set out to go build something 0:14:55.920 --> 0:14:58.800 that was purpose built, and I had this prepared mind 0:14:58.880 --> 0:15:00.880 I'd been working with Andrew on kind of a range 0:15:00.880 --> 0:15:03.760 of concepts for two years. So when he finally said 0:15:03.760 --> 0:15:06.280 this is now kind of March timeframe of twenty sixteen, 0:15:06.440 --> 0:15:08.680 it's like, I think we're ready. I told him, I said, look, 0:15:08.720 --> 0:15:10.640 we want to be your first term sheet and so 0:15:10.680 --> 0:15:14.200 we It was no joke on April first, so April 0:15:14.240 --> 0:15:16.760 Fool's Day, I said, Andrew, you know, we presented it 0:15:16.840 --> 0:15:18.800 with an offer, and over the course of the next 0:15:18.840 --> 0:15:22.320 few weeks we basically converged on that initial financing, which 0:15:22.360 --> 0:15:25.119 we which we did in partnership with Eric at Benchmark 0:15:25.160 --> 0:15:28.640 and Pierre and Leora at Eclipse, And yeah, that was 0:15:28.680 --> 0:15:31.880 the beginning of a cerebras back in May of twenty sixteen. 0:15:33.040 --> 0:15:36.520 What was the closest moment between twenty sixteen and ringing 0:15:36.520 --> 0:15:39.400 the bet On in the New York Sulck Exchange to failure? 0:15:39.840 --> 0:15:44.080 Oh many, We had many, so many white knuckle moments. 0:15:44.400 --> 0:15:46.520 We didn't ship the first product for three and a 0:15:46.520 --> 0:15:49.760 half years from that May twenty sixteen. I'd say there 0:15:49.760 --> 0:15:52.760 were lots of challenges around getting that first semiconductor system 0:15:52.800 --> 0:15:55.840 to work. You know, we'd spent multiple millions building the 0:15:55.840 --> 0:15:58.600 first one and no joke. The very first system we 0:15:58.680 --> 0:16:02.040 plugged it in and it blew up. I mean we 0:16:02.440 --> 0:16:04.560 joke about like we call those thermal events because you 0:16:04.560 --> 0:16:05.920 never want to tell your landlord you've. 0:16:05.840 --> 0:16:08.440 Had a fire. But we had an event. 0:16:08.880 --> 0:16:11.120 We had a small thermal event around our first system. 0:16:11.440 --> 0:16:13.440 And what you're trusting in those cases is, wow, that 0:16:13.520 --> 0:16:16.040 was very expensive. That one system you could you could 0:16:16.040 --> 0:16:18.400 as certain cost you know, millions of dollars with all 0:16:18.440 --> 0:16:20.640 the R and D that was invested in it. And 0:16:20.720 --> 0:16:22.240 then you've got to just trust that we're going to 0:16:22.320 --> 0:16:25.520 come back, you know, the next day and and dig 0:16:25.600 --> 0:16:28.320 through what went wrong and figure it out. 0:16:28.400 --> 0:16:30.680 And and we did. And that team is just. 0:16:30.680 --> 0:16:34.560 Born of so much grit around sort of getting a 0:16:34.560 --> 0:16:37.280 system that again there were no there were no playbooks 0:16:37.280 --> 0:16:39.560 for this. TSMC had never done anything like what we 0:16:39.560 --> 0:16:41.600 were asking them to do. And then the other piece 0:16:41.600 --> 0:16:43.720 I would say that was a very scary moment was 0:16:43.760 --> 0:16:45.800 when we realized, you know, we were initially actually going 0:16:45.840 --> 0:16:47.840 to build an air cold system because every other system 0:16:48.280 --> 0:16:51.080 in the data center, you know, including all the GPU systems, 0:16:51.120 --> 0:16:53.400 were all air cooled, and we realized to make our 0:16:53.400 --> 0:16:55.960 system work, we would need to do we'd need to 0:16:56.000 --> 0:16:58.720 water cool it, and there was lots of questions around 0:16:59.040 --> 0:17:01.640 how that would work. APR kind of head of mechanical 0:17:01.640 --> 0:17:04.119 engineering and system design, had never done this. 0:17:04.200 --> 0:17:06.960 He kind of admended that to me later. And you know, keap. 0:17:06.800 --> 0:17:09.480 Dissipation at that scale is a really hard problem. And 0:17:09.520 --> 0:17:11.680 so we we built a lot of prototypes. We actually 0:17:11.720 --> 0:17:13.879 ended up having to bring in several consultants and I 0:17:13.920 --> 0:17:16.879 spent you know, probably you know it was not hundreds 0:17:16.880 --> 0:17:20.119 of hours, but you know up in that range working 0:17:20.119 --> 0:17:22.359 with the team on how do we actually solve this 0:17:22.640 --> 0:17:24.480 really really hard problems because you'd come out of the 0:17:24.520 --> 0:17:27.240 lab some days thinking, guys, we might just be trying 0:17:27.280 --> 0:17:29.400 to beat the laws of physics, which is never never 0:17:29.440 --> 0:17:30.040 a good idea. 0:17:31.200 --> 0:17:33.080 So you, in a sense you had all of these 0:17:33.560 --> 0:17:37.679 pod engineering problems and now engineering problems. Some of them 0:17:37.720 --> 0:17:39.400 are solved, but no doubt are new ones. But also 0:17:39.440 --> 0:17:42.800 of course you have you know, you have competition within Vidia. 0:17:42.960 --> 0:17:45.199 Is that is that essentially? I mean, how does the 0:17:45.240 --> 0:17:47.480 business look today? Is it? Is it a pitch battle 0:17:47.520 --> 0:17:48.040 of Nvidia? 0:17:48.080 --> 0:17:48.159 Like? 0:17:48.240 --> 0:17:51.080 Is that the central kind of business challenge? 0:17:51.400 --> 0:17:51.640 Yeah? 0:17:51.680 --> 0:17:54.159 I mean they are the eight hundred bound guerilla no question, 0:17:54.800 --> 0:17:57.480 and we are the insurgent and they've built you know, 0:17:57.520 --> 0:18:01.240 they've built an extraordinary category. When when Nvidia moved into 0:18:01.320 --> 0:18:03.760 the data center space, which was kind of in the 0:18:03.840 --> 0:18:06.480 mid twenty tens and it's now you know, it's more 0:18:06.480 --> 0:18:08.840 than half of their business, and so they are definitely 0:18:09.520 --> 0:18:12.400 you know there, they are the player that we most 0:18:12.440 --> 0:18:15.479 directly get compared with, which I'm fine with. And they 0:18:15.480 --> 0:18:18.560 have a different architecture and you know, we fundamentally believe 0:18:18.600 --> 0:18:22.000 we're doing we're doing things differently for again back to 0:18:22.040 --> 0:18:23.840 the sort of you know, initial premise here, because we 0:18:23.880 --> 0:18:26.320 think this workload deserves its own silicon. 0:18:27.320 --> 0:18:30.600 So so displacing them with clients, I guess is one 0:18:30.760 --> 0:18:33.640 challenge on the on the on the demand side, shall 0:18:33.680 --> 0:18:35.879 we say, what about on the supply side? I mean 0:18:35.920 --> 0:18:39.520 you mentioned that TSMC actually you know, fabricate these these 0:18:39.560 --> 0:18:42.560 chips for you guys, Are you like literally fighting of 0:18:42.640 --> 0:18:45.280 a factory flaw space in Taiwan with Nvidia to to 0:18:45.440 --> 0:18:48.560 to see which orders for chips they'll fulfill faster? I mean, 0:18:48.720 --> 0:18:50.800 how do you how do you navigate that relationship? 0:18:51.160 --> 0:18:54.800 Yeah, So TSMC has been an extraordinary partner to us, 0:18:54.840 --> 0:18:57.480 and they are to really everyone in the industry. And 0:18:57.520 --> 0:19:00.240 what I appreciate most about them, and I think what 0:19:00.320 --> 0:19:03.480 they are uniquely well suited to doing is working on 0:19:03.560 --> 0:19:07.399 new ideas even before the market is sort of clearly 0:19:07.440 --> 0:19:09.159 there for them. And they have been a great partner 0:19:09.200 --> 0:19:11.760 to us in that regard, because again, if you're looking 0:19:11.760 --> 0:19:15.679 at Cerebras back in twenty sixteen and your TSMC and 0:19:15.680 --> 0:19:18.399 you've got you know, customers like Video or Apple and 0:19:19.440 --> 0:19:21.840