Speaker 1: Bloomberg Audio Studios, Podcasts, Radio News.
Speaker 2: Hello and welcome to another episode of the Odd Lots podcast.
Speaker 3: I'm Joe Wisenthal and I'm Tracy Alloway.
Speaker 2: Tracy the Deep Seek sell off.
Speaker 3: That's right, it's pretty deep. Has anyone made that joke yet.
Speaker 1: We're in Deep Seek?
Speaker 2: Yeah, I don't think anyone who's made that joke.
Speaker 3: I will say, like, you know, it's bad in markets
Speaker 3: when all the headlines are about standard deviation, yes, right,
Speaker 3: And then you know it's really bad when you see
Speaker 3: people start to say it's not a crash, it's a
Speaker 3: healthy correction. Yes, that's the real cope.
Speaker 2: But just for like real scene setting. You know, We've
Speaker 2: done some very timely interviews about tech concentration in the
Speaker 2: market lately and how so much of the market is
Speaker 2: this big concentrated bed on AI et cetera. Anyway, on Monday,
Speaker 2: I think people will be listening to this. On Tuesday,
Speaker 2: markets got clobbered in video one of the big winners
Speaker 2: as of the time I'm talking about this three thirty
Speaker 2: pm on Monday, down seventeen percent. We're talking major laws
Speaker 2: is really across the tech complex. Basically, it seems to
Speaker 2: be catalyzed by the introduction of this high performance, open
Speaker 2: source Chinese AI model called deep Seek. I was born,
Speaker 2: from what we know, out of a hedge fund. Apparently
Speaker 2: it was very cheap to train, very cheap to build.
Speaker 2: You know, the tech constraints at this point didn't seem
Speaker 2: to be much of a problem. They may be a
Speaker 2: problem going forward, But yes, here is something the entire
Speaker 2: market betting on a lot of companies making AI and
Speaker 2: are now concerns about, of course, a cheap Chinese competitor.
Speaker 3: I just realized, Joe, this is actually your fault, isn't it.
Speaker 3: This last week you wrote that you were a deep
Speaker 3: Seek aibro and look what you've done. You've wiped five
Speaker 3: hundred and sixty billion dollars off of in videos market.
Speaker 2: Yeah, might be that's you anyway. One of the interesting
Speaker 2: questions though, is that this was sort of announced in
Speaker 2: a white paper in December. Why did it take for
Speaker 2: till January twenty seventh for related to freak people out?
Speaker 2: Big questions? Anyway, let's jump right into it. We really
Speaker 2: do have the perfect guest, someone who's was here for
Speaker 2: our election Eve Special, a guy who knows all about
Speaker 2: numbers and AI and quant stuff, and he writes a
Speaker 2: substack that has become for me a daily absolute must
Speaker 2: read where he writes an extraordinary amount. I don't even
Speaker 2: know how he writes so much on a given day.
Speaker 2: We're going to be speaking with Zvi Mashowitz. He is
Speaker 2: the author of the Don't Worry about the Vase blog
Speaker 2: or substack. ZV. You're also a deep seki brill. You've
Speaker 2: switched to using that.
Speaker 1: So I use a wide variety of different ais. So
Speaker 1: I will use quad paranthropic, I will use one from
Speaker 1: ta GPT, from open Ai. I'll use Gemini sometimes, and
Speaker 1: I'll use Perplexity for web searches. But yeah, I'll use
Speaker 1: R one, the new deep seat model for certain type
Speaker 1: queries where I want to see how it thinks and
Speaker 1: like see the logic laid out, and then I can judge,
Speaker 1: like did that make sense? Do I agree with that?
Speaker 3: So one of the things that seems to be freaking
Speaker 3: people out as well as the market is that purportedly
Speaker 3: this was trained on like a very low cost, something
Speaker 3: like five point five million dollars for deep Seek V three,
Speaker 3: although I've seen people erroneously say that the five point
Speaker 3: five million was for all of its R one models,
Speaker 3: and that's not what it says in the technical paper.
Speaker 3: It was just for V three. But anyway, oh I
Speaker 3: should mention it also seems like a big chunk of
Speaker 3: it was built on Mama, so they're sort of piggybacking
Speaker 3: off of others investment. But anyway, five point five million
Speaker 3: dollars to train, is that a realistic and then b
Speaker 3: do we have any sense of how they were able
Speaker 3: to do that.
Speaker 1: So we have a very good sense of exactly what
Speaker 1: they did because they're unusually open and they gave us
Speaker 1: technical papers, they tell us what they did. They still
Speaker 1: hid some parts of the process, especially with getting from
Speaker 1: V three, which was trained for the five point five
Speaker 1: million two R one, which is the reasoning model for
Speaker 1: additional millions of dollars, where they tried to make it
Speaker 1: a little bit harder for us to duplicate it by
Speaker 1: not sharing their reinforcement learning techniques. But we shouldn't get
Speaker 1: over anchored or carried away with the five point five
Speaker 1: million dollar number. It's not that it's not real, it's
Speaker 1: very real. But in order to get that ability to
Speaker 1: spend five point five million dollars and get the model
Speaker 1: to pop out. They had to acquire the data, they
Speaker 1: had to hire the engineers, they had to build their
Speaker 1: own cluster, they had to over optimize to the bone
Speaker 1: their cluster because they're having problems of chip access thanks
Speaker 1: to our export controls. And they were training on eight hundreds.
Speaker 1: And the way they did this was they did all
Speaker 1: these sorts of mini optimism, little optimizations, including like just
Speaker 1: exactly integrating the hardware, the software, everything they were doing
Speaker 1: in order to train as cheaply as possible on fifteen
Speaker 1: trillion tokens and get the same level of performance or
Speaker 1: you know, close to the same level performance as other
Speaker 1: companies have gotten with much much more compute. But it
Speaker 1: doesn't mean that you can get your own model for
Speaker 1: five point five million dollars, even though they told you
Speaker 1: a lot of the information. In total, they're spending hundreds
Speaker 1: of millions of dollars to get this result.
Speaker 2: Wait, explain that further. Why does it still take hundreds
Speaker 2: of millions And does this mean if it takes hundreds
Speaker 2: of millions of dollars that the gap between what they're
Speaker 2: able to do versus the say American labs is perhaps
Speaker 2: not as wide as maybe people think.
Speaker 1: Well, what deepseek is doing is they have less access
Speaker 1: to chips. They can't just buy Navidiot chips the same
Speaker 1: way that you know open ai or Microsoft or and
Speaker 1: throb it can buy Nvidiot chips. So instead they had
Speaker 1: to make good use, very very efficient, killer use of
Speaker 1: the chips that they did have. So they focused on
Speaker 1: all these optimizations and all of these ways that they
Speaker 1: could save on compute. But in order to get there,
Speaker 1: they had to spend a lot of money to figure
Speaker 1: out how to do that and to build the infrastructure
Speaker 1: to do that. And you know, once they knew what
Speaker 1: to do, it cost them five point five million dollars
Speaker 1: to do it. They've shared a lot of that information
Speaker 1: and this has dramatically reduced the cost of somebody who
Speaker 1: wants to follow in their footsteps and train a new
Speaker 1: model because they've shown the way of many of their
Speaker 1: optimizations that people didn't realize they could do or didn't
Speaker 1: realize how to do them. That can now very easily
Speaker 1: be copied. But it does not mean that you are
Speaker 1: five point five million dollars away from your own V three.
Speaker 3: So the other thing that is freaking people out is
Speaker 3: the fact that this is open source, right, we all
Speaker 3: remember the days when OpenAI was more open and now
Speaker 3: it's moved to closed source. Why do you think they
Speaker 3: did that? And like how big a deal is that?
Speaker 1: So this is one of those things where they have
Speaker 1: a story and you can believe their story. You're not
Speaker 1: with their story, but their story is that they are
Speaker 1: essentially ideologically in favor of the idea that everyone should
Speaker 1: have access to the same AI, that AI should be
Speaker 1: shared with the world, especially that China should help pump
Speaker 1: out its own ecosystem and they should help grow all
Speaker 1: of the AI for the betterment of humanity. And they're
Speaker 1: going to get artificial general intelligence and they are going
Speaker 1: to open source that as well, and this is their
Speaker 1: the main point of deep Sea. This is why deep
Speaker 1: Seak exists. They disclaiming even having a business model really
Speaker 1: and you know they're they're an outgrowth of a hedge fund,
Speaker 1: and hedge fund makes money and maybe they can just
Speaker 1: do this if they choose to do that, or maybe
Speaker 1: they will end up with a different business model. But
Speaker 1: it was obviously very concerning from a lot of angles
Speaker 1: if you open source increasingly capable models, because you know,
Speaker 1: artificial general intelligence means something that's you know, as smart
Speaker 1: and capable as you and I as a human, and
Speaker 1: perhaps more so. And if you just hand that over
Speaker 1: in open form to anybody in the world who wants
Speaker 1: to do anything with it, then we don't know how
Speaker 1: dangerous that is, but it's existentially risky at some limit
Speaker 1: to unleash things that are smarter and more capable, more
Speaker 1: competitive than us, that are then going to be free
Speaker 1: and loose to you know, engage in whatever any human
Speaker 1: directs them to do.
Speaker 3: I have a really dumb question, but I hear people
Speaker 3: say artificial general intelligence all the time. AGI, what does
Speaker 3: that actually mean?
Speaker 1: There is a lot of dispute over exactly what that means.
Speaker 1: The words are not used consistently, but it stands for
Speaker 1: artificial general intelligence. Generally, it is understood to mean you
Speaker 1: can do any task that can be done on a
Speaker 1: computer that can be done cognitively only as well as
Speaker 1: a human.
Speaker 2: I mean, it does most of these things do things
Speaker 2: much better than me. I don't know how to code,
Speaker 2: and so, but I get that there are still some things.
Speaker 2: Maybe they wouldn't be as good as proving some of
Speaker 2: the are you human tests? Everyone to talk about Jevins
Speaker 2: paradox and so we see in video and broadcom shares
Speaker 2: these chip companies, they're getting crumbled today. And one of
Speaker 2: the theories like, oh no, with all these optimizations and
Speaker 2: so forth, in researchers will just use those and they'll
Speaker 2: still have max demand for compute, and so it won't
Speaker 2: actually change the ultimate end for compute. How are you
Speaker 2: thinking about this question?
Speaker 1: So I'm definitely a Jevans pro right now from the
Speaker 1: perspective of this, you.
Speaker 2: Don't think it'll have a negative impact and just the
Speaker 2: amount of compute demanded.
Speaker 1: The tweet I sent this morning was Navidio down eleven
Speaker 1: percent pre market on news that his chips are highly useful.
Speaker 1: And I believe that what we've shown is that, yes,
Speaker 1: you can get a lot more in some sense out
Speaker 1: of each Navidia chip than you expected. You can get
Speaker 1: more AI. And if there was a limited amount of
Speaker 1: stuff to do with AI, and once you did that stuff,
Speaker 1: you were done, then that would be a different story.
Speaker 1: But that's very much not the case. As we get
Speaker 1: further along towards AGI, as these ais get more capable,
Speaker 1: we're going to want to use them for more and
Speaker 1: more things, more and more often, and most importantly, the
Speaker 1: entire revolution of R one and also Open Eyes O
Speaker 1: one is inference time compute. What that means is every
Speaker 1: time you ask the question, it's going to use more compute,
Speaker 1: more cycles of GPUs to think for longer, to basically
Speaker 1: use more tokens or words to figure out what the
Speaker 1: best possible answer is. And this scales not necessarily with
Speaker 1: out limit, but it scales very very far. So Opening
Speaker 1: Eyes new three is capable of thinking for you know,
Speaker 1: many minutes. It's capable of potentially spending you know, hundreds
Speaker 1: or even in theory thousands of dollars or more on
Speaker 1: individual query. And if you knock that down by an
Speaker 1: order of magnitude, that almost certainly gets you to use
Speaker 1: it more for a given result, not use it less,
Speaker 1: because that is effect starting to get prohibitive. And over time,
Speaker 1: you know, if you have the ability to spend or
Speaker 1: markly vittle of money and then get things like virtual
Speaker 1: employees and abilities to answer any question under the sun, yeah,
Speaker 1: there's basically unlimited demand to do that or to scale
Speaker 1: up the quality of the answers as the price drops.
Speaker 1: So I basically expect that as fast as the VIDIA
Speaker 1: can manufacture chips and we can put them into data
Speaker 1: centers and give them electrical power. People will be happy
Speaker 1: to pie those chips.
Speaker 3: At the risk of angering the Jeffons Paradox bros. Just
Speaker 3: to push on the point a little bit more so,
Speaker 3: my understanding of deepseek is that one of the reasons
Speaker 3: it's special is because it doesn't rely on like specialized components,
Speaker 3: custom operators, and so it can work on a variety
Speaker 3: of GPUs. Is there a scenario where, you know, AI
Speaker 3: becomes so free and plentiful, which could in theory be
Speaker 3: good for Nvidia, But at the same time, because it's
Speaker 3: easy to run on a bunch of other GPUs, people
Speaker 3: start using you know, more like ACIK chips, like customized
Speaker 3: chips for a specific purpose.
Speaker 1: I mean, in the long run, we will almost certainly
Speaker 1: see specialized inference chips, whether from the Video or they're
Speaker 1: from someone else, and we will almost certainly see various
Speaker 1: different advancements that today's chips are going to be obsolete
Speaker 1: in a few years. That's how AI works, right, There's
Speaker 1: all these rapid advancements. But you know, I think in
Speaker 1: Video is in a very very good position take advantage
Speaker 1: of all of this. I certainly don't think that like
Speaker 1: you'll just use your laptop to run the best agis
Speaker 1: and therefore we don't have to worry about buying TPUs
Speaker 1: is a porposition. It's certainly possible that rivals will come
Speaker 1: up with superior checks. That's always possible. The video does
Speaker 1: not have a monopoly, but the video certainly seems to
Speaker 1: be a dominantiation right now.
Speaker 2: It seems to me. I mean, I know there's others,
Speaker 2: but it seems to be in the US. There's like
Speaker 2: three main AI producers of models that people know about.
Speaker 2: There's Open Ai, there's Claude, and then there's Meta with Lama.
Speaker 2: And it's worth knowing that Meta is green today, that
Speaker 2: the stock is actually up as of the time I'm
Speaker 2: talking about this one point one percent. Just go through
Speaker 2: each one real quickly, how the sort of deep seek
Speaker 2: shock affects them and their viability and where they stand today.
Speaker 1: I think the most amazing thing about your question is
Speaker 1: that you forgot about Google.
Speaker 2: Oh yeah, right, yeah, that's very tilling.
Speaker 1: But everyone else has forgotten about Yeah, surprising Semini flash
Speaker 1: thinking their version of one and R one got updated
Speaker 1: a few days ago, and there are many reports that
Speaker 1: it's actually very good now and potentially competitive and effectively.
Speaker 1: It's free to use for a lot of people on
Speaker 1: AI studio, but nobody I know has taken the time
Speaker 1: to check and find out how good it is because
Speaker 1: we've all been too obsessed with being deep seep roads.
Speaker 1: Google's had its like rhetorical lunch eaten over and over
Speaker 1: and over again December. Like open a I would come
Speaker 1: up with advance after advance after Advance, then Google would
Speaker 1: love Advance after advanced after advance, and Googles would be
Speaker 1: seemingly actually, if anything, more impressive. And yet everyone will
Speaker 1: always just talk about open a eyes, so this is
Speaker 1: not even new. Something is going on there. So in
Speaker 1: terms of open Ai, Open Ai should be very nervous
Speaker 1: in some sense, of course, because they have the reasoning models,
Speaker 1: and now the reasoning model has been copied much more
Speaker 1: effectively than previously, and the competition is a hell of
Speaker 1: a lot cheaper Open Eye is charging, so it's a
Speaker 1: direct threat to their business model for obvious reasons, and
Speaker 1: it looks like their lead in reasoning models is smaller
Speaker 1: and faster to undo than you would expect. Because if
Speaker 1: deep Sea can do it, of course Anthropic and Google
Speaker 1: you know, can do it. And everyone else can do
Speaker 1: it as well, and Thropic, which produces Claude, has not
Speaker 1: yet produced their own reasoning model. They clearly are operating
Speaker 1: under a shortage of compute in some sense, so it's
Speaker 1: entirely possible that they have chosen not to launch a
Speaker 1: reasoning model even though they could, or not focused on
Speaker 1: training one as quickly as possible until they've addressed this problem.
Speaker 1: They're continuously taking investment. We should expect them to solve
Speaker 1: their problems over time, but they seem like they should
Speaker 1: be dressed directly concerned because they're less of a directly
Speaker 1: competitive product in some sense, but also they tend to
Speaker 1: market to effectively much more aware people, so their people
Speaker 1: will also know about deep Seak and they will have
Speaker 1: a choice to make. If I was Meta, I would
Speaker 1: be far more worried, especially if I was on their
Speaker 1: Genai team and wanted to keep my job, because Meta's
Speaker 1: lunch has been eaten massively here right, Meta with Lama
Speaker 1: had the best open models, and all the best open
Speaker 1: models were effectively fine tunes of Lama, and now deep
Speaker 1: Seat comes out, and this is absolutely not in any
Speaker 1: way a fine tune of Lama. This is their own product,
Speaker 1: and V three was already blowing everything that Meta had
Speaker 1: out of the water. Are one. There are reports that
Speaker 1: it's better than their new version that they're training now,
Speaker 1: it's better than Lava four, which I would expect to
Speaker 1: be true. And so there's no point in releasing an
Speaker 1: inferior open model if everyone on the open model community
Speaker 1: just be like, why don't I just use deep Sea Tracy.
Speaker 2: It's interesting that, as V said, the people who should
Speaker 2: be nervous are the employees of Meta, not Meta itself,
Speaker 2: because Meta is up, and so you gotta wonder. It's like, well,
Speaker 2: maybe they don't. I don't know, maybe they don't need
Speaker 2: to invest as much in their own open source AI
Speaker 2: if there's a better one out there now the stock
Speaker 2: is up.
Speaker 1: Anyway, The market has been very strange from my perspective
Speaker 1: on how it reacts to different things that Meta does.
Speaker 1: For a while, Meta would announce we're spending more in AI,
Speaker 1: we're investing in all these data centers, we're training all
Speaker 1: of these models, and the market would go, what are
Speaker 1: you doing? This is another metaverse or something, and we're
Speaker 1: gonna hammer your stock and we're gonna drag you down.
Speaker 1: And then with the most recent sixty five billion dollar
Speaker 1: announce spend. Then then Meta was up. Presumaly, they're gonna
Speaker 1: use it mostly for inference effectively in a lot of
Speaker 1: scenarios because they had these massive inference costs to want
Speaker 1: to put ail over Facebook and Instagram. So you know,
Speaker 1: if anything, like you know, I think the market might
Speaker 1: be speculating that this means that they will know how
Speaker 1: to train better lamas that are cheaper to operate, and
Speaker 1: their costs will go down, and then they'll be in
Speaker 1: a better position, and that theory isn't.
Speaker 3: Crazy since we all just collectively remembered Google. I have
Speaker 3: a question that's sort of been on the back in
Speaker 3: the back of my mind. I think Joe has brought
Speaker 3: this up before as well. But like when Google debuted,
Speaker 3: it took years and years and years for people to
Speaker 3: sort of catch up to the search function, and actually
Speaker 3: no one ever really caught up, right, So Google has
Speaker 3: like dominated for years. Why is it when it comes
Speaker 3: to these chatbots there aren't like higher wider moats around
Speaker 3: these businesses.
Speaker 1: So one reason is that everyone's training on roughly the
Speaker 1: same data, meeting the entire Internet and all of human knowledge,
Speaker 1: so it's very hard to get that much of a
Speaker 1: permanent data edge there unless you're creating synthetic data off
Speaker 1: of your own models, which is what Opening Eye is
Speaker 1: plausively doing. Now. Another reason is because everybody is scaling
Speaker 1: as fast as possible and adding zeros to everything on
Speaker 1: a periodic basis in calendar time. It doesn't take that
Speaker 1: long before your rival is going to have access to
Speaker 1: more compute than you had, and they're copying your techniques
Speaker 1: more aggressively. They's just a lot less secret sauce there's
Speaker 1: only so many algorithms. Fundamentally, everyone is relying on the
Speaker 1: scaling laws. It's called the bitter lesson is the idea
Speaker 1: that you know, you just scale more, you just use
Speaker 1: more compute, you just use more data, you just use
Speaker 1: more parameters and deep seek. You're saying, maybe you don't.
Speaker 1: You can do more optimizations, you can get around this
Speaker 1: problem and still get a superior model. But mostly, yeah,
Speaker 1: there's been a lot of just I can catch up
Speaker 1: to you by copying what you did. Also that I
Speaker 1: can see the outputs, right, I can query your model,
Speaker 1: and I can use your model's outputs to actively train
Speaker 1: my model. And you see this in things like most
Speaker 1: models that get trained. You ask them who trains you,
Speaker 1: and they will often say, oh, I'm from Open Ai and.
Speaker 2: The internet has gotten so weird. I just the internet
Speaker 2: is so weird to speak. Mashavitz, thank you so much
Speaker 2: for running over to the Odd Lots and helping us
Speaker 2: record this emergency pod on the Deep Seek selloff though.
Speaker 2: It was fantastic.
Speaker 1: All right, thank you, Tracy.
Speaker 2: I love talking to v We got just sort of
Speaker 2: make him our Ai or our Ai guy.
Speaker 3: I mean, to be honest, we could probably have him
Speaker 3: back on again because there's gonna be stuff happening.
Speaker 2: Maybe we will, and obviously it's we could go a
Speaker 2: lot longer. This is a really exciting story. This is
Speaker 2: a really exciting story, and things are just getting really
Speaker 2: weird these days.
Speaker 3: It is kind of crazy how fast all of this is. Yap,
Speaker 3: And then the other thing I would say is just
Speaker 3: the bitter lesson. Great name for a band.
Speaker 2: Oh, totally totally great. Maybe when we do our Ai
Speaker 2: themed proud rock band. True, Yes, that could be our name.
Speaker 3: Yes, let's do that. Okay, shall we leave it there?
Speaker 2: Let's leave it there.
Speaker 3: This has been another episode of the Odd Thoughts podcast.
Speaker 3: I'm Tracy Alloway. You can follow me at Tracy Alloway.
Speaker 2: And I'm Jill Wisenthal. You can follow me at the Stalwart.
Speaker 2: Follow our guests Vimashovitz, he's at this v Also definitely
Speaker 2: check out his free subs deck. It's a must read
Speaker 2: for me. Don't worry about the v OZ, really great stuff
Speaker 2: every single day. Follow our producers Carmen ra Rigaz at
Speaker 2: Kerman armand dash O Bennett at Dashbot and kill Brooks
Speaker 2: at Kilbrooks. For more oddlocks content, go to Bloomberg dot
Speaker 2: com slash odlocks. We have transcripts, a blog in a newsletter,
Speaker 2: and you can chat about all of these topics twenty
Speaker 2: four to seven in our discord Discord dot gg slash Odlots.
Speaker 3: Maybe we'll give zv to do a Q and A
Speaker 3: in there with oh yeah, that'd be great. And if
Speaker 3: you enjoy Oddlots, if you like it when we roll
Speaker 3: out these emergency episodes, then please leave us a positive
Speaker 3: review on your favorite platform. Thanks for listening.