Speaker 1: Also media.
Speaker 2: Hello on, welcome to Better Offline. I'm your host ed Zitron.
Speaker 2: Subscribe to the newsletter, joined the subway and buy a
Speaker 2: T shirt. All the links are in the notes. And
Speaker 2: here's a joke to kick us off. What do you
Speaker 2: call the CEO of a French RAM company? That's right,
Speaker 2: Immanuel Micron, And that's what we're here to talk to
Speaker 2: Steve Burke of Gamers. Next is about today, Steve, Welcome
Speaker 2: back to the octagon.
Speaker 1: Emmanuel Micron.
Speaker 2: Yeah, before we started, yeah, because you would have killed
Speaker 2: me before I read it on the But look, we're
Speaker 2: here to talk about RAM. So we'll start with a
Speaker 2: simple question. Why is RAM so expensive? Right now? What
Speaker 2: is doing this?
Speaker 1: I mean, the reason it's so expensive is not grounded
Speaker 1: and reac I think you know, it's the surface of
Speaker 1: it is that all the data center build out is
Speaker 1: consuming all the memory and therefore, you know, the price
Speaker 1: higher because everyone wants it. But I do think there's
Speaker 1: a little bit more to.
Speaker 2: It than that, right So from what so when you
Speaker 2: build chips We're going to really simplify this one for
Speaker 2: the audience because it's pain in the US to explain otherwise.
Speaker 2: But when you build chips. You have something called a waifer.
Speaker 2: You have the silicon out, and then you entch it
Speaker 2: using a lithograph machine. But you can make less high.
Speaker 2: So there's the high bandwidth memory that they use inside GPUs,
Speaker 2: and then there's the regular DRAM that they use in
Speaker 2: everything else. Right, is that it's a simplification.
Speaker 1: But is that failure? I think like the the most
Speaker 1: base level stuff to know is that memory, some form
Speaker 1: of RAM or memory is in almost everything. A common misconception,
Speaker 1: I'm not sure. I think your audience is like fairly technical,
Speaker 1: but a layperson misconception is that storage is the same
Speaker 1: as memory. They'll use it interchangeably. If storage is like
Speaker 1: your drive that you you know, you put data on,
Speaker 1: like photos or whatever, and then memory is is a
Speaker 1: temporary form of storage of volatile storage, like it's wiped
Speaker 1: when it's not actively being used when you restart and.
Speaker 2: It's and when they put stuff in memory, that's where
Speaker 2: that's like RAM and such where they like, that's where
Speaker 2: you put the stuff that you gonna need quickly.
Speaker 1: Right, Yeah, so yeah, memory is much faster than storage,
Speaker 1: permanent storage, and so it acts as the sort of
Speaker 1: transactional buffer to help get things to you quickly, So
Speaker 1: memories and phones, refrigerators these days, cars, there's all kinds
Speaker 1: of different grades of memory, and it's it's used in
Speaker 1: pretty much everything to do some kind of quick like
Speaker 1: in a car, Uh, maybe you need to retrieve something
Speaker 1: related to like if there's a built in map on
Speaker 1: the screen or something like that, or in a refrigerator.
Speaker 1: Who the fuck knows, honestly, but I think that's I
Speaker 1: think so that.
Speaker 2: You can have a giant, horrible screen on your Samsung
Speaker 2: fridge that displays you adds occasionally, yes, exactly, yeah, and
Speaker 2: so so yeah, to go to your wafer count. Yeah,
Speaker 2: there's different types of sort of production for memory, and
Speaker 2: you'll find DRAM is used in all of it in
Speaker 2: some capacity for the most part. So you'll find DRAM
Speaker 2: dies in HBM like inside of an HBM package. But
Speaker 2: data center high end GPUs will use high bandwidth memory,
Speaker 2: and basically every consumer device uses something called GDDR, which
Speaker 2: is just a lower bandwidth kind of more normal solution
Speaker 2: that's not as overboard on the performance for a consumer application.
Speaker 2: But they all kind of come from the same place,
Speaker 2: which is a wafer. They're all cut from right so
Speaker 2: right now, they're taking up more, they are using more
Speaker 2: of the how much of the compacity are they actually
Speaker 2: using to build high bandwidth memory? So the stuff that's
Speaker 2: going to the GP use.
Speaker 1: It appears to be an awful lot. I mean, so,
Speaker 1: I know that the recent class action lawsuit or complaint
Speaker 1: I should say that was filed notes that the profitability
Speaker 1: the plaintiffs attorneys claimed that the profitability of traditional conventional
Speaker 1: DRAM is greater than the profitability of HBM. That was
Speaker 1: one of their claims. And yet they sort of argue
Speaker 1: that it seems like their evidence is good, but they
Speaker 1: argue that why would you shift more and more capacity
Speaker 1: to producing this lower margin thing HBM? Right of like
Speaker 1: the easier thing to make is potentially higher margin, And
Speaker 1: so it does seem like more of it's shifting over there.
Speaker 2: You know.
Speaker 1: Open AI cause that sort of big kerfuffle previously when
Speaker 1: they claimed forty of just all DRAMs globally though yeah
Speaker 1: it didn't have it was all like basically.
Speaker 2: One of the most insane just I mean sane story.
Speaker 2: By the way, just last year, for those of you
Speaker 2: who don't remember, open A story came out the open
Speaker 2: A I was buying thirty five to forty percent of
Speaker 2: all the world's dram wafers. It never happened, Yeah, but
Speaker 2: we just talked about it like like we did, like
Speaker 2: it did, but.
Speaker 1: Now it did affect the price.
Speaker 2: It did. But here's the question, So why is everything
Speaker 2: so expensive that like is it? Is it the because
Speaker 2: are they just jacking the price up?
Speaker 1: I think so? I mean so, yeah, there's a couple
Speaker 1: of things here a layout like some of the highlights,
Speaker 1: and we can you can figure out how you want
Speaker 1: to go through them. But sure one of them is
Speaker 1: the memory industry has convictions actually in its past of
Speaker 1: price fixing, collusion and as the DOJ dubbed it, cartel
Speaker 1: like behavior. So this goes the early two thousands, there's
Speaker 1: a bus to twenty sixteen, twenty eighteen. There was an
Speaker 1: attempt at a similar lawsuit in the early nineties. The
Speaker 1: memory manufacturers were talking about how they thought their competitors
Speaker 1: were price fixing and colluding and so this is a
Speaker 1: long and story to history where they and now there's
Speaker 1: an active lawsuit, putative class action for price fixing. So
Speaker 1: there's that antitrust aspect, which that obviously you know, prices
Speaker 1: would be higher if you're just doing illegal things to
Speaker 1: make the prices higher. There's also the sort of physical
Speaker 1: manufacturing side, which is that to make memory, it's not
Speaker 1: quick to get another fab online, and the companies seem
Speaker 1: to sort of signal to each other through earnings reports
Speaker 1: whether they're going to expand capacity or maintain capacity of
Speaker 1: their fabs, which are factories for memory. So there's that aspect,
Speaker 1: which is if there's a real or not a sudden
Speaker 1: increase in demand, there's going to be a long latency
Speaker 1: to react to that with more supply, if they even
Speaker 1: want to. So the current projections are twenty twenty eight
Speaker 1: or twenty thirty for getting like more fabs online. The
Speaker 1: other reason it goes up, though, is really simple, which
Speaker 1: is that in a server that goes in a data center,
Speaker 1: some of Nvidia's newer solutions will use one and a
Speaker 1: half terabytes of system memory. So that's d RAM they'll use.
Speaker 2: Well they use this, they don't. They also use the
Speaker 2: same RAM as we find in smartphones as well.
Speaker 1: Yeah, LPDDR typically Yeah, yeah, so so Grace will use
Speaker 1: I think it's five X I think is what they're on.
Speaker 2: Yeah, I pulled the numbers and it's something like twenty
Speaker 2: terabytes for an NVL seventy two, which is seventy two
Speaker 2: GPU rack.
Speaker 1: Yeah, so they do. The single blade for the Big
Speaker 1: Boy is one and a half terabytes of system memory alone.
Speaker 1: And you know, for perspective, a consumer desktop computer might
Speaker 1: have sixteen gigabytes with Windows these days, you probably want
Speaker 1: like thirty two, and they're they're taking one and a
Speaker 1: half terabytes for one, you know, server solution. But that's
Speaker 1: just the system mary. They also have other processors on there,
Speaker 1: like the Blue Point or blue Field I think it's
Speaker 1: called DPU. It takes one hundred and twenty eight gigabytes
Speaker 1: if I remember correctly. The GPUs take hundreds of gigabytes
Speaker 1: to like close to half a terabyte per GPU of HBM,
Speaker 1: and the HBM has a there's kind of a yield
Speaker 1: factor at play where you know you can when you've
Speaker 1: talked about manufacturing, that means you're also talking yield how
Speaker 1: many good products do you get for every bad And
Speaker 1: HBM is more difficult to make. If you have a
Speaker 1: failure in manufacturing, then you're also potentially losing more silicon
Speaker 1: dyes that could have gone.
Speaker 2: And HBM is made by stacking the memory, right, so
Speaker 2: it requires just physically more memory. That's kind of an
Speaker 2: APES description.
Speaker 1: No, No, it's fine. Yeah, So HBM high bandwidth memory
Speaker 1: is literally just really high bandwidth. It's kind of like
Speaker 1: if you were to if you were a city and
Speaker 1: you wanted to solve a public transit problem and you
Speaker 1: just built a shitload of I don't know, let's.
Speaker 2: Take a massive tunnel.
Speaker 1: You build like a bunch of trains. You build a
Speaker 1: ton of trains for passengers, but you don't have enough
Speaker 1: tracks to put those trains on. They can't actually get
Speaker 1: on the track without blocking each other. You haven't really
Speaker 1: solved your problem. And so high bandwidth is trying to
Speaker 1: solve that by saying, Okay, we've got all this capacity
Speaker 1: for passengers, which is the memory itself. Now we need
Speaker 1: something like a bigger road system to actually feed that
Speaker 1: capacity through to access it. And so HBM tries to
Speaker 1: solve that, especially because you look at GPUs that might
Speaker 1: have like twelve plus times the amount of capacity that
Speaker 1: a consumer GPU would have, and so you don't want
Speaker 1: to starve the GPU core of the data that sits
Speaker 1: in the memory by not being able to get it
Speaker 1: fast enough. And so HBM tries to solve that problem. So, yeah,
Speaker 1: to go back to your point of stacking, high bandwidth
Speaker 1: memory is physically vertically stacked, as in, they take silicon
Speaker 1: dyes for memory and they stack them atop each other
Speaker 1: and then they connect them all together, as opposed to
Speaker 1: consumer memory, where it's closer to there's still layers, but
Speaker 1: it's closer to a flat plane.
Speaker 2: Right. But it just means that by virtue of that,
Speaker 2: it takes up more space in the place where they
Speaker 2: build the chips.
Speaker 1: Yeah. Yeah, if you're on a consumer device, then the
Speaker 1: consumer memory you'll need more physical area to put the
Speaker 1: same amount of capacity, whereas with the high end server
Speaker 1: stuff you just stack that capacity vertically. And then it's
Speaker 1: also hbm's really significant advantage is that it's very low
Speaker 1: power comparatively, right, Yeah.
Speaker 2: But just to be clear, it takes up more space
Speaker 2: on the fabricator itself.
Speaker 1: Oh yeah, yeah, if you're talking about the space, then yes,
Speaker 1: HBM requires a lot more space for a few reasons.
Speaker 1: In terms of manufacturing area. In terms of like footprint
Speaker 1: on a product, it's less, but in a fab it's more.
Speaker 1: And why is that in the fab One of the reasons.
Speaker 1: So there's a few, but one is that HBM requires
Speaker 1: additional silicon pieces to be able to function. So with
Speaker 1: GDDR consumer GPU memory, you pretty much you have the
Speaker 1: memory module. It looks like a black rectangle, and then
Speaker 1: if you were to use like acid or something to
Speaker 1: etch away the black plastic part, you'd be left with
Speaker 1: pretty much a singular silicon die underneath. That's all it
Speaker 1: is for consumer. For HBM, you also need a logic dye,
Speaker 1: So there's another piece of silicon that interfaces with the
Speaker 1: memory and the stack of memory, because you need something
Speaker 1: to kind of to command and control and understand, you know,
Speaker 1: the vertical tower you've got of memory. And then you
Speaker 1: also need an interposer die, which sits underneath everything. So
Speaker 1: this is a solution that will sit under your GPU core,
Speaker 1: which is a very large piece of silicon and is
Speaker 1: shared with the HBM stacks. So all of your your
Speaker 1: core and all your HBM is stacked on top of
Speaker 1: the same it's called an interposer. It looks like a
Speaker 1: PCB and they all communicate through that. So that's another
Speaker 1: piece of silicon. And so for this these reasons alone,
Speaker 1: there's also others, but you know these alone. Now you're
Speaker 1: involving more fabs. There's more logistics shipping things around, and
Speaker 1: there's more fab or fabrication plant area allocated. Even if
Speaker 1: you're just talking about making someone's got to make the
Speaker 1: logic dies TSMC makes the interposers, right. Yes, there's just.
Speaker 2: More moving parts and that kind of takes up more
Speaker 2: space exactly. Yeah, So why So to my early question,
Speaker 2: so this stuff is obviously lower margin, but they seem
Speaker 2: to be able to sell an absolute shit ton of it.
Speaker 2: So is that potentially why they're doing this that just
Speaker 2: they know there is insane demand on a level that
Speaker 2: for a high well, I guess they can charge what
Speaker 2: they want for it as well. I'm just having trouble
Speaker 2: trying to work out what it is they're doing other
Speaker 2: than just filling their boots.
Speaker 1: Have you try to asking the chat GPT.
Speaker 2: No, I could ask my cat and get an answer
Speaker 2: of respect, but it's genuine. But it's genuinely because it
Speaker 2: seems the unless I'm misunderstanding here, that these micro and
Speaker 2: eske High Inex and Samsung they're just kind of cranking
Speaker 2: up the price of regular RAM because they can.
Speaker 1: I mean to some extent. Yeah, So I think this
Speaker 1: is where the price fixing allegations come into I don't know,
Speaker 1: if you try it, if you try to take their side,
Speaker 1: which I really don't want it.
Speaker 2: I'm trying. I'm trying.
Speaker 1: Yeah, if you try to, I think their argument, whether
Speaker 1: it's for truthful or not, but would be We've got
Speaker 1: this crazy high demand for HBM. These are guaranteed customers.
Speaker 1: I can sell terabytes of it to Jensen, and it's
Speaker 1: it's one point of contact. It all moves quickly and easily.
Speaker 1: And this there's this data center demand right now, and
Speaker 1: we don't know if it's going to be here forever.
Speaker 1: But consumers, you know, they're they're always going to have
Speaker 1: to buy electronics or whatever. So I'm not exactly sure
Speaker 1: if that's to me, it seems like maybe you make
Speaker 1: some sort of you assign some value to the B
Speaker 1: two B relationship over B two C or something maybe,
Speaker 1: But also I guess if they yeah, I'm not sure,
Speaker 1: if you make.
Speaker 2: Why would that mean that they charge more for regular RAN.
Speaker 1: Oh As far as why more for regularly, that part's easy.
Speaker 1: That's just because the HBM itself takes the same a
Speaker 1: lot of the same machines as consumer memory. And so
Speaker 1: if you're made more HBM, you're naturally making less consumer,
Speaker 1: which I think is where you start seeing the supply
Speaker 1: squeeze and the price go up. Likewise, a single piece
Speaker 1: of HBM can use up to something I think it's
Speaker 1: like it might be like thirty two dies max or
Speaker 1: something of DRAM you'll commonly see like four referenced. But
Speaker 1: single piece of HBM will take more dram dies to build.
Speaker 1: And so one of the problems is if let's just
Speaker 1: say four, If you have a piece of HBM that's
Speaker 1: got four dram dies in it and you have one
Speaker 1: failure somewhere in that stack or in packaging an assembly,
Speaker 1: now you've destroyed you know, four individual pieces, rather than
Speaker 1: if you have a yield failure with a consumer device,
Speaker 1: you destroy one.
Speaker 2: Sorry to ask a stupid question, what is a die exactly?
Speaker 2: Is that just the what we would think of as
Speaker 2: a stick of ram.
Speaker 1: Uh No, a die is so a wafer we'll start
Speaker 1: the wavefer level. A wafer is just a disc, right,
Speaker 1: it looks like a CD that typically eight or twelve inches,
Speaker 1: and they they process it and they pattern it where
Speaker 1: they literally project like a pattern or they lithographically etch
Speaker 1: a pattern onto the surface of this disc. Once that's done,
Speaker 1: you know, these these patterns, they're called patterns because it's
Speaker 1: a repeating sort of set of shapes that ultimately form
Speaker 1: the logic that makes the silicon. And the word dye
Speaker 1: comes from the process dicing. So with a wafer, when
Speaker 1: they're done producing it and patterning it and it's basically
Speaker 1: ready to become a product, they need to cut that
Speaker 1: disc into a bunch of small squares.
Speaker 2: So it's like a big slab that they caught top.
Speaker 1: Yeah, it's I mean, it's like making a pizza with
Speaker 1: a pizza cutter. You know, they're they're doing a pattern
Speaker 1: that only psychopaths would do on a pizza, which is
Speaker 1: making a bunch of squares and uh, and so they
Speaker 1: cut it into the squares. That's called dicing. And then
Speaker 1: each of those pieces, the squares that yield from that,
Speaker 1: those are the dyes. And so if you were to
Speaker 1: pull up an image of any GPU, typically you'll see
Speaker 1: one large it's called the monolithic dye, which means one
Speaker 1: singular sort of piece of shiny reflective silicon mirror like finish.
Speaker 1: That's your die. And then for memory, same exact concept,
Speaker 1: just a lot smaller.
Speaker 2: Right, So right, okay, so there's less space to make
Speaker 2: regular rams, so they just crank up the price. That's
Speaker 2: really that really is it? I think that's their entire thing, right,
Speaker 2: But I saw Micron zernings and they had eighty four
Speaker 2: point nine percent margins. So it just kind of feels
Speaker 2: like they're setting the price to whatever they want it
Speaker 2: to be.
Speaker 1: You don't think there, you don't think that is? Is
Speaker 1: that not an I think they need more?
Speaker 2: Don't They should get ninety five percent? Well, it's just.
Speaker 1: Kind of why are you setting the setting it so low?
Speaker 2: They should get one hundred and ten percent margins? We
Speaker 2: should No, It's just it's very frustrating as well, because
Speaker 2: I've read now arguments in favor and against Micron where
Speaker 2: it's like, oh, Micron got screwed by Apple and consumer
Speaker 2: electronics companies during the last during the last supply chain
Speaker 2: crisis because they they kind of made a bunch more
Speaker 2: ram than they needed to from the river. You know, like, no,
Speaker 2: it really it really is. But what and it seems
Speaker 2: to be there's this boom and bus cycle in memory
Speaker 2: where they get a bunch of demand and then so
Speaker 2: do they build more capacity? Is it that they just
Speaker 2: ordered too much RAM and then people cancel their orders
Speaker 2: like what happened? What happened last time?
Speaker 1: Yeah, so it's cyclical, I will say right now, as
Speaker 1: nothing like I've ever seen. And we suppoke to a
Speaker 1: bunch of companies purchasers of memory like g Skill and
Speaker 1: course Air, who sell the sticks of RAM that you
Speaker 1: put in a computer, and those guys are telling us
Speaker 1: on record that likewise they've never this is not like
Speaker 1: part of the cycle. This is special. And as for
Speaker 1: how is it different, how the difference is, how much
Speaker 1: the difference is the percentage increase it is this time?
Speaker 1: So you know, you're looking at like seven hundred percent
Speaker 1: increase in price in some situations in a very short
Speaker 1: period of time. Normally, cyclically you'll see some ebbs and flows,
Speaker 1: but it might be like back in the day, a
Speaker 1: fifty dollars kit of DDR four became eighty five or
Speaker 1: ninety dollars, maybe even one hundred and twenty if it's
Speaker 1: a really bad year, you know, but you're still at
Speaker 1: like a little worse than double, Whereas now a two
Speaker 1: hundred dollars kit of memory is suddenly a thousand to
Speaker 1: twelve hundred dollars. Uh, And so it's just the the
Speaker 1: amount of the price in is what's different. And then
Speaker 1: also this time, the sort of the contest for the
Speaker 1: supply is it's coming from within the own industry, where
Speaker 1: the very companies that would be the cause for an
Speaker 1: increase in sales of consumer supply are now the cause
Speaker 1: of these like insane unprecedented volume of sales for AI compontents.
Speaker 2: But it isn't based like so the twenty twenty one
Speaker 2: supply chain crisis chip crisis was because there was a
Speaker 2: flood of consumer spending off the COVID, Yeah, and that
Speaker 2: led to a bunch of consumer demand for products and
Speaker 2: overall spending across the board. But like consumer demand drove,
Speaker 2: this one seems different because it's not demand outside of
Speaker 2: the outside of whatever like psychosis, the various hype sets.
Speaker 2: They're not spending a trullion dollars in CAPEX really feel like,
Speaker 2: it's not like consumers can't access whatever micros COVID.
Speaker 1: I think you know what's interesting is the big three
Speaker 1: memory makers decided not I think it was twenty twenty
Speaker 1: two or three, to bring on more capacity in some
Speaker 1: of these situations because they were talking about how they
Speaker 1: got burned by oversupply and sort of fellow in COVID.
Speaker 2: Right.
Speaker 1: But what's interesting is if you go and look at
Speaker 1: some of the small suppliers's not many of them, but
Speaker 1: there's a couple of small memory makers that make them
Speaker 1: like less than five percent of the market, Wind Bond,
Speaker 1: Nanya or two of them. Nanya's in uh Well, headquartered
Speaker 1: at least in Taiwana of where their fab is, but
Speaker 1: they like these two companies as far as I'm aware.
Speaker 1: I was just looking at this this morning. They increased
Speaker 1: their their production capabilities at the time when the big
Speaker 1: three we're saying that they weren't gonna do it because
Speaker 1: you know, they got burned, right. It's interesting to me
Speaker 1: because if you're this is the argument the lawsuit makes too.
Speaker 1: If you're a small company, I would think you're at
Speaker 1: the biggest disadvantage, the biggest risk to increase your production
Speaker 1: capabilities with this massive amount of spend. If your biggest
Speaker 1: competitors are saying like this, it looks like bad times ahead.
Speaker 1: You know, we don't want to spend into this. But
Speaker 1: if you're a small guy, like you're at the greatest
Speaker 1: risk of going under. If you fuck up your your.
Speaker 2: Cap back right? Why didn't that this is maybe this
Speaker 2: is a dumb question. Why didn't them increasing supply allow
Speaker 2: them to kind of on the cut the microns of
Speaker 2: the world.
Speaker 1: Yeah, that one is just because the there's a micron
Speaker 1: does have a technology advantage, so they can make memory
Speaker 1: that these other guys can't. HBM is one of them.
Speaker 1: Maybe this is I didn't think about this till now,
Speaker 1: but maybe there is a very a small part, I think,
Speaker 1: but possibly part of an answer to your question of
Speaker 1: why HBM that is something that's that other companies are
Speaker 1: struggling to make successfully in current generations.
Speaker 2: So it's well, I know, only like sk Heinex, Samsung
Speaker 2: and micro An not the ones who can make HBM.
Speaker 1: Yeah, and I the CXMT has also struggled with HBM,
Speaker 1: and they're a new Chinese memory maker that's actually done
Speaker 1: really well with DDR five. And so there may be
Speaker 1: a little bit of a pulling the ladder up, you know,
Speaker 1: after them factor.
Speaker 2: Well, here's the question as well, when it comes to
Speaker 2: high badmath memory, surely this is going to lead to
Speaker 2: an oversupply situation regardless because HBM is used in what
Speaker 2: video cards got that's such an anachrotistic term in the
Speaker 2: AI bubble, but like it's used in video cards and
Speaker 2: aigpus basically, right.
Speaker 1: Yeah, it's very limited. There's almost there's effectively zero consumer
Speaker 1: use case.
Speaker 2: So when the AI bubble bursts, where does the high
Speaker 2: bandwidth memory go? Oh? Yeah, I don't Oh okay, yeah,
Speaker 2: because I'm not.
Speaker 1: Talking about the bubble burst in all the.
Speaker 2: Bubble thing I saw, Samsung is talking about increasing their
Speaker 2: high bandwidth. They're spending like fifty something billion, five hundred
Speaker 2: billion dollars across an interterminate period, but like also building
Speaker 2: an entire new high bandwidth memory like factory, yeah and that.
Speaker 2: But I was looking I think I sent you the
Speaker 2: graph for coming on there was. I saw this chart
Speaker 2: where it's like they were making what a couple hundred
Speaker 2: million gigabytes of hbm is in twenty twenty two and
Speaker 2: they're now selling like seven or eight trillion a year.
Speaker 1: Yeah, So what's crazy is the hbm bit demand I
Speaker 1: think I I think I first saw this on semi
Speaker 1: analysis if I remember correctly when about the sort of site.
Speaker 1: But the hbm bit demand from envi Idea was recently
Speaker 1: projected to exceed all HBM bit demand of all other
Speaker 1: companies combined from twenty twenty five good, And that's just
Speaker 1: n VideA. And and again you have more wafer area
Speaker 1: per bit than with d RAM consumed. That counts yield losses, right,
Speaker 1: and so like. Increasingly, as they make more and more
Speaker 1: of this, presumably for Envideo's devices, you will continue to
Speaker 1: see consumers really get shafted if there's not new supply
Speaker 1: online soon. And likewise, if they're focusing all their efforts
Speaker 1: and resources on HBM for whatever reason, who's to say
Speaker 1: that the new fabs will even make consumer memory anyway? Right?
Speaker 1: Like there's they might just be more HBM.
Speaker 2: Well, the thing is, if they're all built to make
Speaker 2: maybe I'm missing something. They're building this fact, these these
Speaker 2: fabs specifically for hype band with memory behind bad with
Speaker 2: memory is only used really for the thing in the
Speaker 2: AI bubble, right, What they're gonna do with all this
Speaker 2: capacity when it's gone?
Speaker 1: They would switched it over it, they'd switched the line.
Speaker 2: I mean they would. But it's like, I guess they'll
Speaker 2: just have aren't they just creating a future nightmare? I mean,
Speaker 2: it's always good to have more capacity for them I guess.
Speaker 1: I yeah, I mean, I guess it's good that and
Speaker 1: more capacity. There's a lot of government money floating around
Speaker 1: to right, Like Micron got a five billion dollar tax
Speaker 1: break in New York as part of the the guess
Speaker 1: big beautiful bill, but the.
Speaker 2: Five billion Like Microns said they're going to spend two
Speaker 2: hundred and fifty billion, right, Samsung and s K Heinez
Speaker 2: have made two point one trillion dollars worth of commitments.
Speaker 2: It just feels like it's it's almost like they saw
Speaker 2: what happened in twenty twenty one and twenty twenty three,
Speaker 2: which ended with all of them doing at least a
Speaker 2: billion dollars of write offs, and when they were like,
Speaker 2: fuck that, we can go bigger.
Speaker 1: Yeah. Well, I mean I do think personally, I think
Speaker 1: that they are colluding again, I think they are price fixing.
Speaker 1: It might be through like a parallelism rather than a
Speaker 1: literal behind closed doors, meaning like we're just gonna announce
Speaker 1: our changes in earnings reports and then our friendly competitors
Speaker 1: will kind of mirror us, and so.
Speaker 2: Yeah, yeah, like it's kind of a prisoner's dilemma, like
Speaker 2: if they don't raise prices, the other guy will. If
Speaker 2: they and if they don't build capacity, the other guy will,
Speaker 2: they won't get the long term deals.
Speaker 1: And right now, yeah, and and also you know the
Speaker 1: the wafers themselves. I have some interesting numbers I wrote down,
Speaker 1: but I was looking up when you were asking about HBM.
Speaker 1: I went and pulled up some of the original coverage
Speaker 1: I did of HBM which started to come online or
Speaker 1: at least being actively developed somewhere around I don't know,
Speaker 1: twenty thirteen plus or minus a bit for AMD GPUs.
Speaker 1: So yeah, when AMD it was working on Fiji and
Speaker 1: its Fury GPUs, they debuted HBM. Actually for consumer devices.
Speaker 1: They had HBM on Vega and on a couple other
Speaker 1: architectures in like the twenty fifteen to maybe twenty eighteen
Speaker 1: Area plus or minus a year or two and massively expensive,
Speaker 1: just like it is today. The power efficiency was great,
Speaker 1: which is really why they used it, although they were
Speaker 1: banned with starved too. But I pulled up I'd done
Speaker 1: some reporting at the time where I spoke to a
Speaker 1: bunch of the people in the know in the industry,
Speaker 1: and at the time twenty seventeen or so twenty sixteen area,
Speaker 1: the HBM two memory for one of the devices should
Speaker 1: have cost around one hundred and fifty dollars cost for
Speaker 1: them to buy, and the interposer in the packaging should
Speaker 1: have been another twenty five dollars one seventy five. And
Speaker 1: this is these are you know, this is old technology. Now,
Speaker 1: I'm sure a lot different. But the point of bringing
Speaker 1: it up is that if you compare a sort of
Speaker 1: time for time and at the same capacity at the time,
Speaker 1: it would cost am D one hundred and seventy five
Speaker 1: bucks or so to put HBM hod its device, whereas
Speaker 1: with GDDR memory the non HBM stuff, they would have
Speaker 1: been at somewhere around like sixty eight dollars.
Speaker 2: So they would be just so, I guess the thing is,
Speaker 2: they absolutely would make more money selling d RAM or
Speaker 2: sorry GDR like regular people ram. It's just that right
Speaker 2: now they have the business to kind of. It's not
Speaker 2: just shitty. It's not these bullshit video cards will lose
Speaker 2: the gamers. It's selling aigpus to big strong boys.
Speaker 1: Yeah right, yes, like Sam Altman.
Speaker 2: And yes, the big strong honky men like Dario Amma
Speaker 2: day right right, yes, yeah, he I've never by the way,
Speaker 2: I don't think i've ever seen someone spread so much
Speaker 2: fear so anxiously as him. It's it's just pathetic. What's Yeah. Also,
Speaker 2: here's a random thing as well for these a question
Speaker 2: to the listeners and to you as well, Steve, you
Speaker 2: ever see anyone on You've seen this weird habit of
Speaker 2: Silicon Valley people of using AI now to add muscles
Speaker 2: to guys.
Speaker 1: This seen?
Speaker 2: What's that about all of the weirdest things I've seen
Speaker 2: in the tech industry's history, And there's a lot of
Speaker 2: weird ship.
Speaker 1: I don't know if it's just well, they're they're not
Speaker 1: used to working for intellectual property at this point. They
Speaker 1: just harvested, so why work for the muscles either.
Speaker 2: Well, it's just it's really strange. Like I like, I
Speaker 2: think the TVPN, the technology, the Blowjup Brothers network, like
Speaker 2: they they they do it with like Dylan Patel, who's
Speaker 2: just like a regular sized guy, and it's just it's sorry,
Speaker 2: I've just realized that's weird. But then I saw someone
Speaker 2: do it for Dario Amedey the other day, and this
Speaker 2: is like, your body's your body. I don't judge people
Speaker 2: for for like how they look, but at the same time,
Speaker 2: Dario Amma day is not like a like fifth course
Speaker 2: of steroids. Guy, Like, it's just the weird it's the
Speaker 2: weirdest thing I've ever seen.
Speaker 1: Did you see the story just recently of the copper
Speaker 1: thieves stealing it from data centers too?
Speaker 2: No, that was good. Have a rich better offline listener there, Yep.
Speaker 1: Yeah, they probably wearing one of your T shirts.
Speaker 2: Just don't god no disabout them.
Speaker 1: Yeah, the SAT there's a title of thieves stealing copper
Speaker 1: from data centers, and it's just like, oh, okay, I
Speaker 1: mean it's really I guess there's no honor among thieves,
Speaker 1: you know, it's just thieves stealing from thieves.
Speaker 2: It's also inevitable at this point because you make these
Speaker 2: giant things full of earth metals and then you put
Speaker 2: them in neighborhoods usually with like like disadvantaged farm neighborhoods
Speaker 2: or like like distressed towns in Texas. You're like, oh,
Speaker 2: I wonder what will happen with the other business.
Speaker 1: Yeah, I wonder what will happen when we screw over
Speaker 1: all these people and.
Speaker 2: And like mock them via the media constantly. Anyway, back
Speaker 2: to back to ram, So, is this something that's going
Speaker 2: to start affecting like storage so hard.
Speaker 1: Yeah, yeah, yeah, so you mean like.
Speaker 2: Actual yeah, like nand like solid state drives and the like.
Speaker 1: Yes, so it's already effecting it. Storage is made at
Speaker 1: the same fabs for the most part as memory, with
Speaker 1: basically the same tools and mostly by the same companies,
Speaker 1: although there's there's like one or two differences, but for
Speaker 1: the most part it's the same people and tools, and
Speaker 1: so definitely want affect the other anyway. But storage in
Speaker 1: general has hasn't gone up as violently as memory, but
Speaker 1: it has gone up. Some of the SSDs we had
Speaker 1: bought previously. I was just checking this actually for like
Speaker 1: ninety bucks maybe about a year ago, are now closer
Speaker 1: to like one hundred and eighty one hundred and ninety
Speaker 1: and Jesus Christ, yeah, I pull up a chart here.
Speaker 2: That's just this is genuinely like I think, I don't
Speaker 2: know if I'd said do anything that's activism, but like
Speaker 2: I think people just need to know this is AI's fault. Yes,
Speaker 2: And the worst part is it's not even as fault
Speaker 2: because they need these data centers because they're taking forever
Speaker 2: to build. It's like you can access claude and chat GPT.
Speaker 2: You don't. There's no one. No one's like unable to
Speaker 2: log on because there's not enough data sentences. Nothing's being stopped.
Speaker 2: They're just like, no, we need to do we need
Speaker 2: to buy buy the chips, need to build the data
Speaker 2: sense for some reason.
Speaker 1: Yeah, it's yeah, there's not like a I mean, I
Speaker 1: guess they're maybe they make the argument of like we're
Speaker 1: trying to make it better and the models will be
Speaker 1: more complicated in that.
Speaker 2: Require and what's not gonna happen.
Speaker 1: But yeah, the storage though, So, looking at an average
Speaker 1: price chart for four terabytes of solid state drive storage
Speaker 1: VME on PC part Picker, which averages it, they have
Speaker 1: the pricing at around it looks like maybe about two
Speaker 1: hundred and seventy dollars in January last year, and now
Speaker 1: it's somewhere around somewhere around eight hundred and nine hundred dollars.
Speaker 2: And that's for what kind of storage that is.
Speaker 1: For four terabytes. Uh so relatively large capacity, which is
Speaker 1: going to be affected the most.
Speaker 2: Jesus Christ.
Speaker 1: Yeah, And this is where this is interesting too, because
Speaker 1: one of the ways you'll see consumers getting shafted that's
Speaker 1: not as obvious is some of the modern devices that
Speaker 1: should be your next sort of you know, improvement upgrade
Speaker 1: on past generations. They're actually coming down in capacity, So
Speaker 1: I haven't looked at the exact devices names for a while.
Speaker 1: We ran a report a while ago talking about some
Speaker 1: of the phones and laptops that have shifted from say
Speaker 1: a prior model of twelve or sixteen gigaway it's down
Speaker 1: towards eight for memory, there's decreasing storage capacity as well.
Speaker 2: Yeah, that's fucking tech. So everyone just suffers here, and
Speaker 2: it's pretty like there's Micron's, Skehihex Samsung. They could just
Speaker 2: they could have not done this, just to be clear,
Speaker 2: just so they could have sat with like seventy percent margins.
Speaker 1: They're already colluding anyway, and my opinion, so yeah they could.
Speaker 1: They can clute it in a different way.
Speaker 2: It's just it's so fucking straight. Well, I mean it's
Speaker 2: actually no, that's not that's not fair. This isn't strange
Speaker 2: at all. This is exactly like, this is exactly what
Speaker 2: companies like this do. Yeah, like this, it's this kind
Speaker 2: of vulgar kind of chancerism. And their argument, from what
Speaker 2: I understand, is well, memory is cyclical, so we need
Speaker 2: to fill our boots when we can.
Speaker 1: And if you're calling something vulgar, then it's pretty vulgar.
Speaker 2: Yes, just to be clear, as the vulgarest man in tech,
Speaker 2: and that is a word now. But the thing is, though,
Speaker 2: just to be clear, that's their argument though, because things
Speaker 2: are good now, they can charge what they want so
Speaker 2: they can fill their coffice for the future.
Speaker 1: Yeah, I think so, and I mean you see it affecting.
Speaker 1: I think that's probably their argument. I think on the
Speaker 1: consumer side too, even a company like Lenovo, which is
Speaker 1: not a manufacturer of memory, but they buy it and
Speaker 1: they put it in stuff. I'm not sure if there
Speaker 1: may be this. I don't know for fact, but there
Speaker 1: may be a benefit for some of these companies as
Speaker 1: well for a few reasons. One is there's a lot fewer.
Speaker 1: It kind of reminds me a little bit of COVID
Speaker 1: when all the stores that were twenty four seven, all
Speaker 1: the grocery stores and whatever suddenly stopped being twenty four
Speaker 1: to seven, and at least where I am not one
Speaker 1: of them has gone back. And I think that the
Speaker 1: memory capacity kind of equivalent here is this offers a
Speaker 1: little bit of a reset, and so with.
Speaker 2: Lower those expectations exactly.
Speaker 1: Yeah, and now your next upgrade is more exciting. And
Speaker 1: what I noticed with Lenovo when I was buying a
Speaker 1: laptop was they had quietly snapped out of existence their
Speaker 1: sixty four gigabyte model memory system memory of the device
Speaker 1: I was looking at, and they only had thirty two.
Speaker 2: And I think, has the memory had sold it onto
Speaker 2: it as well?
Speaker 1: It is, Yeah, you can't.
Speaker 2: Oh so you just can't upgrade. You're just stuck. That's insane.
Speaker 1: Yeah, And so I think what happens there for them,
Speaker 1: you know, in their favor, I might argue, well, they
Speaker 1: can't they're having trouble getting supply, just like everyone else.
Speaker 1: That's probably true. But against them, if they've got sixty
Speaker 1: four gigabytes of memory, because that's just one part of
Speaker 1: the total bomb for a laptop they sell, they might
Speaker 1: rather sell two people two laptops at thirty two gigabytes
Speaker 1: right where they can get the markup on the whole thing, right,
Speaker 1: rather than one guy a sixty four gigabyte laptop where
Speaker 1: they're only marking it up. Whatever the hopefully whatever the
Speaker 1: difference in the memory is. So I think you see
Speaker 1: some of that here too, where there's.
Speaker 2: Well they're just going to use it as a chance
Speaker 2: to reset what people expect and just ohso prices are
Speaker 2: probably never coming down right itself.
Speaker 1: Funny enough that I was using them in this example,
Speaker 1: just like two weeks ago at Is said that prices
Speaker 1: will probably the word they used was never come back
Speaker 1: down to what they were last year. And so yeah,
Speaker 1: I think.
Speaker 2: They're truly evil era, just like a like a disgusting
Speaker 2: era full of horrible perverts. Because it's just it's so
Speaker 2: frustrating as well, because America has price fixing laws. Yeah,
Speaker 2: they we have price gouging laws, like we have means
Speaker 2: of doing this, we just don't do it because obviously
Speaker 2: of capitalism.
Speaker 1: And they've been enacted, you know, like the DJ itself
Speaker 1: busted these same companies in the early two thousands.
Speaker 2: Yeah, that was before. That's before we had social media
Speaker 2: for psychopaths to talk about stocks all day. Now we
Speaker 2: need to keep now, we need to We need just
Speaker 2: borderline corruption or real corruption in everything. That's the only
Speaker 2: way to get things done these days.
Speaker 1: And I think also there's probably incentive to you know,
Speaker 1: if you're no administration of any party wants the economy
Speaker 1: to fail on their watch and I think if you're
Speaker 1: worried about and a bubble pop, and especially if maybe
Speaker 1: spooking investors with the DJ investigation or something, might you know,
Speaker 1: trigger part of that. It just seems like you wouldn't
Speaker 1: be incentivized to not probe too much.
Speaker 2: But isn't this going to cause just already is causing
Speaker 2: hyper inflation within all electronics related products, because yeah, it
Speaker 2: seems that these companies are fully dedicate to the graveyard
Speaker 2: smash of making everything more expensive forever and at some
Speaker 2: point that will start crushing all conceimer electronics.
Speaker 1: Yeah, I think it will, you know, for some things
Speaker 1: that can migrate more to software as a service and
Speaker 1: hardware as a service. I think there's an end game
Speaker 1: there for some of these companies where tell me more,
Speaker 1: I think if it's okay we crushed demand for the
Speaker 1: gaming GPUs market because filthy, poor casual gamers can't afford
Speaker 1: our big strong GPUs that we need for data centers,
Speaker 1: you know, I think if you crush that market, there
Speaker 1: is potentially an out in the form of cloud gaming,
Speaker 1: or maybe a better example where you don't have latency
Speaker 1: as an issue, would just be if you as a company,
Speaker 1: if you're worried about consumers getting systems that are powerful
Speaker 1: in their hands and doing using LLLMS at home or whatever.
Speaker 1: If that is suddenly out of reach because the memory
Speaker 1: is too expensive and the GPS are too expensive, all right,
Speaker 1: I guess they come crawling back to claude or chat GPT.
Speaker 1: So I do think not everyone's going to be able
Speaker 1: to grab that parachute. But I think that moving services
Speaker 1: to cloud oriented services that you pay for every month
Speaker 1: is a way that they can capture the people, which
Speaker 1: is going to be most of them at some point,
Speaker 1: who can't afford to actually own the full device to
Speaker 1: do stuff cloud gaming. Cloud gaming sucks. But I also
Speaker 1: I don't think that's the only place where you know.
Speaker 1: I think one of the real conception if anyone here
Speaker 1: is genuine about LM's and AIB in the future, I
Speaker 1: would think at least companies like open aye would be
Speaker 1: worried about that about lms getting too good if they're
Speaker 1: locally hosted for like, let's just say, the vast majority
Speaker 1: of basic tasks someone might want, and by by all
Speaker 1: the prices of hardware remaining elevated, it does kind of
Speaker 1: keep that at bay a little longer.
Speaker 2: I just because I understand that, and you've said this before.
Speaker 2: The idea that they want to push people towards not
Speaker 2: owning anything. You'll own nothing and you'll love it, I
Speaker 2: think is the phrase. And but at the same time,
Speaker 2: do you think that that's actually their plan or do
Speaker 2: you think my version of this is much simpler, which is,
Speaker 2: do you think they have one? Or is this just
Speaker 2: incredible short terminus thinking.
Speaker 1: Yeah, I don't think it's the plan. I think it's
Speaker 1: like the fallback for the future.
Speaker 2: Google Stadia we all know is a huge success.
Speaker 1: I contributed to the failure of that one. Yeah, that one.
Speaker 2: There are ten people who must be so angry.
Speaker 1: Oh, I'm sure there are.
Speaker 2: Dozens of us G Force.
Speaker 1: Now I had numbers at one point. I don't have
Speaker 1: them currently, but I had a lot of Actually, I
Speaker 1: think we talked about it on your show because I
Speaker 1: pulled up their website. Yeah, however many millions of users,
Speaker 1: and both of us were kind of surprised at it.
Speaker 1: But yeah, I think, you know, I think gaming is
Speaker 1: sort of the easy one to point to. I think
Speaker 1: for a lot of casual users they might be okay
Speaker 1: with cloud connected gaming, but I do think there's more
Speaker 1: services than just gaming, you know that could be sucked
Speaker 1: into the cloud. Even just stuff like QuickBooks has been
Speaker 1: the one I've been ranting about recently.
Speaker 2: I'm surprised that that isn't already sassed.
Speaker 1: Well, basically is because the desktop version, it used to
Speaker 1: be a buy at once for three hundred dollars, then
Speaker 1: I went to seven hundred, and then they went to
Speaker 1: fifteen hundred dollars annually for the desktop version, and you
Speaker 1: have to log in when you launch it, just like
Speaker 1: Adobe software. Oh good, And now they're killing desktop entirely
Speaker 1: and you have to go to cloud, which I actually
Speaker 1: I had just abandoned it and went to a free
Speaker 1: tool instead. But like the point being, yeah, I think
Speaker 1: a lot of these you know, these services they do
Speaker 1: just increasingly go cloud connected.
Speaker 2: But the problem is is cloud connected services still require RAM.
Speaker 2: I mean, just anyone who's ever used an AI tool
Speaker 2: that's still in your browser, well, I mean, Chrome itself
Speaker 2: is a hog, as we all know. And so it's
Speaker 2: just kind of like it feels that there's just going
Speaker 2: to be a point that this escalates to the point
Speaker 2: that it kills large parts of the consumer electronics industry.
Speaker 2: I mean, so Neaster laying the next PlayStation. Microsoft is
Speaker 2: increasing the cost of the next Xbox while firing all
Speaker 2: the people who know how it works. Just an evil company.
Speaker 1: I mean, it's and a steam machine is a great
Speaker 1: example too.
Speaker 2: I love that I just got when. I fucking love it?
Speaker 1: Oh did you really? Wow?
Speaker 2: I really love it. It's just what I But the thing is,
Speaker 2: it's also insanely expensive. Yeah, it's unbelievable.
Speaker 1: Yeah, it's because the Steamer Shione. We had talked to them,
Speaker 1: and the quote they gave us was amazing when I'll
Speaker 1: paraphrase it for the best of my memory, but they
Speaker 1: were talking about how I asked them how did it
Speaker 1: work buying from the memory manufacturers, like did you get
Speaker 1: a contract price or how did you know? How did
Speaker 1: you guarantee your memory supply? And the answer was there
Speaker 1: are no contracts, And the valve guy told us, uh,
Speaker 1: they're offered a price by the memory supplier. In this case,
Speaker 1: the one we got was Heinex and if they say
Speaker 1: no to that price, then the quote was they never
Speaker 1: talked to us again, which is why I feel like
Speaker 1: l like behavior.
Speaker 2: I got some I got some thoughts. I don't think
Speaker 2: I want to share on the record on that one.
Speaker 2: That is just that you know what I'm right back
Speaker 2: round to just I hope these companies supper, Like I
Speaker 2: think the memory companies are evil, Like that's just an
Speaker 2: evil way of doing business.
Speaker 1: Yeah, I mean if it's coercion, right, Because it's like
Speaker 1: if if the player is I offer you a price,
Speaker 1: you say no, fine, I'll take my supply and self
Speaker 1: to someone. I'll never even talk to you again. Like
Speaker 1: you're now down to two companies you can buy from, right.
Speaker 2: And it's not like and it's not and if they'll
Speaker 2: never talk to you again, it doesn't matter if they
Speaker 2: build more supply, right yeah, if you're on there Valve
Speaker 2: as well, who are like the like royalty within the PC.
Speaker 2: It's just this era is just about the companies hitting
Speaker 2: on the people that made them rich.
Speaker 1: And it's you know, the crazy part to me is
Speaker 1: it's like the Valve is not a poor company. You know,
Speaker 1: they're a private so we don't know how much they have,
Speaker 1: but they are definitely a multi billion dollar company. And
Speaker 1: Valve is getting bullied by the trillion dollar companies.
Speaker 2: Which are only the trillion dollar companies because of the
Speaker 2: stock speculation.
Speaker 1: Right, Yeah, not like actually but yeah, it's just it's
Speaker 1: insane to me because it's of course they're saying thing
Speaker 1: Porster has no real power in the purchasing and they're
Speaker 1: almost a billion dollar market cap, not quite. I don't
Speaker 1: know if they've how far they've fallen, but they were
Speaker 1: at one point a billion dollar market cap, and at
Speaker 1: point being like, these aren't nobodies who want the memory,
Speaker 1: and they're struggling to get the memory suppliers to talk
Speaker 1: to them. They're like, they're serious players and they need
Speaker 1: a lot of it, you know, and the response is
Speaker 1: take it at this price or go fuck yourself, which
Speaker 1: is crazy with me.
Speaker 2: I was reading about the long term agreements the ske
Speaker 2: Heinex is presenting and apparently they're considering removing price caps
Speaker 2: because apparently they do the long term agreements. Yeah, and
Speaker 2: then they say, okay, we like they won't go above
Speaker 2: this bound so that you don't just get Roger. They're
Speaker 2: just not doing that anymore.
Speaker 1: I guess rodri it is. I don't think i've seen
Speaker 1: the rider.
Speaker 2: Classic British class, British, beautiful, British phrase.
Speaker 1: I was gonna say, like, that's when I took British
Speaker 1: literature and before I dropped out of college, is the
Speaker 1: last time I read that? Yeah, I think the the
Speaker 1: in the memory supply typically contracts, they do set a
Speaker 1: ceiling and then my understanding is the way the contract
Speaker 1: works if you're lucky enough to get one, is that
Speaker 1: you're offered the supply and you have to buy it
Speaker 1: or you have to pay a penalty.
Speaker 2: Basically, yeah, take or pay right.
Speaker 1: Yeah, Yeah, that's.
Speaker 2: It just feels like the natural end of this is
Speaker 2: going to be horrifying. It's just going to be. It's
Speaker 2: going to be because they're going to be sitting on
Speaker 2: all this hype bad memory. And I was having a
Speaker 2: conversation friend of the Showcase Could Go yesterday and he
Speaker 2: was saying that, like, well, they'll sell it to someone.
Speaker 2: And while I agree on that, in theory, we're talking
Speaker 2: about going from a few hundred million gigabytes of HBM
Speaker 2: to trillions of it. So someone is going to be
Speaker 2: left holding the bag. Maybe it won't be the memory companies,
Speaker 2: but there's a situation where Nvidia or someone else is
Speaker 2: just sitting on the stack of this crap.
Speaker 1: Yeah, if they can't find devices to put it in
Speaker 1: and they oversupply, then it'll be the same type of
Speaker 1: apocalypse we've seen in the past, I mean twenty sixteen.
Speaker 1: Like typically though, this benefits consumers on the crash inside
Speaker 1: of the wave because you it becomes commoditizing and get
Speaker 1: it super cheap. I'm not sure what happens here because
Speaker 1: when we've already reset the expectations, like we were talking about,
Speaker 1: so there's always already been a regression in the technology
Speaker 1: that'll take a few years to kind of get back
Speaker 1: to where we were. But then also whatever crash happens
Speaker 1: next seems like it's not going to be isolated to
Speaker 1: the memory industry, right, it seems like it'll be uh
Speaker 1: like I sent you a photo during Computax of n
Speaker 1: Videos advertisement in Taipei that said it all starts here.
Speaker 1: Do you remember what? You replied with, No, I do not.
Speaker 1: You replied with it parentheses next global financial crisis.
Speaker 2: Yeah, it really is though, because it's like, think about it,
Speaker 2: if we have just this glut of high bandwidth memory,
Speaker 2: how many graphics cards do they realistically make? Because when
Speaker 2: I was saying hundreds of millions of gigabytes, that was
Speaker 2: back in twenty twenty two. So this was the demand
Speaker 2: for these things for every games console manufacturer, every PC manufacturer, everyone.
Speaker 2: As of twenty twenty two, when the economy was doing
Speaker 2: a lot better, right and they only needed a few
Speaker 2: hundred million, we could be sitting on arguably the largest
Speaker 2: supply in memory history. Oh for like an for like, like,
Speaker 2: you can't unbuild that memory, right, you can't pass it out.
Speaker 1: You can't really there's no way to. You would need
Speaker 1: an HBM device to put it on. There's no real
Speaker 1: consumer HBM devices these days, and it's not something you'd
Speaker 1: spin up fast. So yeah, if there's like too much HBM,
Speaker 1: you don't really have a lot you could do with
Speaker 1: it other than I guess sell it for cheaper or something.
Speaker 1: But like, at the end of the day, you need
Speaker 1: something to attach it to. And I mean one thing
Speaker 1: to think about is part of those numbers, although I
Speaker 1: don't think all of it, but part of it is
Speaker 1: that the capacity per device has increased a lot for
Speaker 1: non consumer devices since twenty twenty two, So some of
Speaker 1: it is accounted for there where you just put more
Speaker 1: on a device, But then you look at consumer and like,
Speaker 1: somewhat notoriously, it has remained stuck at like eight gigabytes
Speaker 1: to maybe sixteen gigabytes for a consumer GPU, which eight
Speaker 1: is barely even enough to play modern games with any
Speaker 1: kind of reasonably you know, high texture settings, and so
Speaker 1: you don't have that multiply and effect in consumer. It's
Speaker 1: just in data center. Which all that to say, if
Speaker 1: more supply capacity goes online, if they're able to produce more,
Speaker 1: but the data center demand slows down or the data
Speaker 1: center GPU demand slows then and they switch back to
Speaker 1: just consumer DRAM, they're still going to have too much
Speaker 1: because the consumer devices are they don't use that much memory.
Speaker 1: So I don't know, you finally spec them up maybe.
Speaker 2: But but you'll just have all of this insane amounts
Speaker 2: of high bandwidth memory to go. We're just going to
Speaker 2: get graphics cards with one hundred and twenty eight gigabytes
Speaker 2: of high bandwidth RAM just.
Speaker 1: Just because they need to put it somewhere, right, Yeah, it.
Speaker 2: Could, we got to do something with it. We're just
Speaker 2: gonna have a laptops and you'll be able to open
Speaker 2: seventeen or eighteen Chrome tabs. All right, it's gonna be incredible. No,
Speaker 2: I think I wonder if there's also going to be
Speaker 2: a knock on effect of all of this on the
Speaker 2: used gaming market like they used PC market.
Speaker 1: Well, that's a really common I see people asking about like, oh,
Speaker 1: surely this is gonna be good for secondhand in a
Speaker 1: few years when they retire all these devices or the
Speaker 1: bubble pops.
Speaker 2: Or I just I mean just like right now, I
Speaker 2: mean just people won't be buying new devices.
Speaker 1: Yeah, well, I mean on both of those right now.
Speaker 1: It means that if you go look up ArtX thirty
Speaker 1: series cards from I don't know, like shit, I think
Speaker 1: it might be four or five years ago now on eBay,
Speaker 1: some of them are like within twenty percent of what
Speaker 1: they sold for new, you know. So it's yeah, it's
Speaker 1: wrecking the used market because it's just like it's not
Speaker 1: you're paying close to the new price for something that's
Speaker 1: forty years old. So that's not good. And then on
Speaker 1: the other side of it, none of the data center
Speaker 1: hardware is useful in consumer there is no secondhand use case.
Speaker 1: Yeah yeah, lpdd R five x is either on some
Speaker 1: form of server specific cam card or is soldered to
Speaker 1: the board, so you can't use that. And maybe in
Speaker 1: China they'll deslder it, you know, and solder it onto
Speaker 1: new sticks and sell Frankenstein thing but and then the
Speaker 1: GPUs it's HBM, so it's it's attached to the GPU.
Speaker 1: You can't just you can't just like take it off.
Speaker 2: This is the thing as well, when it comes to
Speaker 2: like this post dot com bubble recovery story, this is
Speaker 2: I'm glad you brought that up, because it's like people like, oh, yeah,
Speaker 2: they'll just use it for something else, useful infrastructure. It's like,
Speaker 2: what useful infrastructure? What are we gonna do with all
Speaker 2: these gps? It's not you can't run a black well
Speaker 2: like a GB three hundred is what's a grace and
Speaker 2: too blackwell? You've got over a terabyte of different kinds
Speaker 2: of fucking rams on that of saying rams and what
Speaker 2: you're gonna do with that? It's not got it's not
Speaker 2: got DisplayPort. Yeah, it's not going to do anything. It's
Speaker 2: still it's built for one thing.
Speaker 1: Right, maybe a local AI model or whatever, but anything reason.
Speaker 2: You couldn't do that unless it was in a consumer
Speaker 2: hardware shell. Yeah, you assume you need like a dishwasher plug.
Speaker 1: For it, or a washing machine plug even like yeah,
Speaker 1: probably need to forty vault Jesus Christ.
Speaker 2: People really, people need to do some do some actually
Speaker 2: like reading about this because it's like they need to
Speaker 2: realize that this can't go anywhere where else like this
Speaker 2: just we're just going to have racks of these things
Speaker 2: and it's not like in the future they'll be cheaper
Speaker 2: to use.
Speaker 1: I think that the But you know, if the dot
Speaker 1: Com era argument of infrastructure was dark fiber or something
Speaker 1: where the fibers in the ground, it's just it's not
Speaker 1: used because all the companies went belly up. Eventually they
Speaker 1: used it. Maybe that's your argument there. I really I
Speaker 1: agree with you. I don't think it works for this one,
Speaker 1: because the closest thing we have to an infrastructure gain
Speaker 1: maybe is power, except the problem of their not really
Speaker 1: bringing more online fast enough, and it's it's at least
Speaker 1: currently it's not helping the prices for the rest of us,
Speaker 1: Like almost everyone's had their power costs go up lately.
Speaker 1: So maybe that would be the advantage if you're like,
Speaker 1: because of data centers, we brought back nuclear and we
Speaker 1: have shitloads of solar and wind and whatever. But that's
Speaker 1: not what we're doing. It's not like our minds on
Speaker 1: the roof, you know, And I don't know.
Speaker 2: It's also what if the power doesn't what if the
Speaker 2: power gets built and the demand isn't there. It's just
Speaker 2: there's so many things that could go wrong here, and
Speaker 2: I think people just kind of I think the reason
Speaker 2: that this is all coming to a head is because
Speaker 2: we're finally facing the physical problems. Yeah, they were like,
Speaker 2: there's no so much capacity, there's only so much money,
Speaker 2: there's only so much power, there's only so much silicon.
Speaker 2: At some point this all has to break, and it
Speaker 2: might break everything.
Speaker 1: Yeah, I think I don't know what the I don't
Speaker 1: know what the Enron moment will be. You know, I'm
Speaker 1: not sure where it'll tip. But it does seem inevitable.
Speaker 1: But I don't know. It could be a year, it
Speaker 1: could be several years, right, Like, I have no idea.
Speaker 2: It could just it feels like the moment the vibe shift,
Speaker 2: because the trigger I'm looking for is that kapek s pullback.
Speaker 2: But what scares me somewhat is the in video is
Speaker 2: the I think, what fifty five sixty five percent of
Speaker 2: all high bandwidth memory demand all for AIGPUS. That just
Speaker 2: feels like a single point of failure.
Speaker 1: Yeah, I think and video, I mean and video itself.
Speaker 1: Jensen Juan himself said of core weave, that without and
Speaker 1: Video there is no core weave, you know, like, if
Speaker 1: I remember correctly that was, I'll go dig that up.
Speaker 1: But they, you know, they create the circle, and video
Speaker 1: is part of the circle, and it's Yeah, if anything
Speaker 1: happens to them, then all of it falters, and then
Speaker 1: they are a kinmaker and yeah.
Speaker 2: But they're also going to be the king loser. I
Speaker 2: think by the end of this just it kind of
Speaker 2: it's just so fucking faster, I guess. Yeah, it's so
Speaker 2: fucking frustrating as well, because I feel like if we'd
Speaker 2: have slowed this down in twenty twenty four, we could
Speaker 2: have avoided this. But everyone was like, no, we need
Speaker 2: to go further and harder and faster and spend even
Speaker 2: bigger more amounts. But one final question, surely with the
Speaker 2: increase in high band with me, like, actually, maybe it's
Speaker 2: two questions in one. The memory companies are kind of
Speaker 2: have Nvidia over a barrel too, don't they.
Speaker 1: Uh, in the sense that Nvidia doesn't really have alternatives.
Speaker 1: But and Video is also heavily involved in the R
Speaker 1: and D for all this new memory, so like they
Speaker 1: were involved in the R and D for GDR seven
Speaker 1: g five X and for the new HPM options and
Speaker 1: so like over a barrel, I feel like I think
Speaker 1: you could make that argument, but I also think that
Speaker 1: Nvidia is more likely pulling a lot of those strains
Speaker 1: despite being the customer to very few suppliers.
Speaker 2: Right. But at the same time, doesn't this just mean
Speaker 2: that in Vidio's prices are going to get linearly more
Speaker 2: expensive the longer this goes on.
Speaker 1: Yeah, I do think, I do think the prices go up.
Speaker 1: I mean, we're you know, with Nvidio on the consumer
Speaker 1: side too. You can look at that as a gauge.
Speaker 1: They were supposed to launched their refresh of consumer GPUs
Speaker 1: like a year ago or something almost and not quite
Speaker 1: a year ago at close and like rumors r end
Speaker 1: of twenty twenty five. Then it was first quarter twenty
Speaker 1: twenty six, and then the rumors where they were all canceled.
Speaker 1: And the reason for that is because this particular type
Speaker 1: of refresh they call the Super series. Normally you see
Speaker 1: an increase in memory capacity at roughly the same price,
Speaker 1: where you see a decrease in price at the same capacity,
Speaker 1: and right now, neither of those things as possible, and
Speaker 1: so Nvidia hasn't shipped the card. It may come out
Speaker 1: at some point, but for now they've delayed that indefinitely.
Speaker 1: And so to your point, it does start to self cannibalize,
Speaker 1: right where like they are the reason that their own
Speaker 1: refresh for consumer can't come out.
Speaker 2: And also but regular the cost of the things that
Speaker 2: were already making the current generation will go up.
Speaker 1: Yeah. Oh, the current generation has already gone up in
Speaker 1: terms of like real world pricing, and officially they haven't
Speaker 1: changed their MSRP from launch, you know, but it's I
Speaker 1: mean a lot of this stuff, the fifty ninety is
Speaker 1: I guess technically sometimes you can get them at MSRP,
Speaker 1: but more often they're like two to.
Speaker 2: I mean, like the aigpus Oh yeah, I mean that
Speaker 2: I would just like, yeah, all of them will go.
Speaker 2: It just feels at some point the money's gonna run out,
Speaker 2: and it's the longer it goes, the worst it will get.
Speaker 1: Well, and then you get the weird lease back stuff,
Speaker 1: right like with Meta and the GP.
Speaker 2: Oh yeah, and all the SPV share. What a horrible era,
Speaker 2: truly awful, And I think that's a that's a great
Speaker 2: place to end it for us. Steve where can people
Speaker 2: find you what you're working on at the moment. I
Speaker 2: know you have some sort of campaign campaign, a dystopian
Speaker 2: campaign you want.
Speaker 1: To Yeah, we do have our AI Dystopia coverage. We're
Speaker 1: doing it. We have a we're trying something new. We
Speaker 1: do these documentaries that are like three hours long, and
Speaker 1: we're trying something new and doing a short one. So
Speaker 1: we're doing two three minute ones coming up. And one
Speaker 1: of them is on the hard Drive cartel, which is
Speaker 1: pretty interesting, nice, and the other one is on this
Speaker 1: memory class action lawsuit. We read through all one hundred
Speaker 1: and eighteen pages of the filing and did a bunch
Speaker 1: of history research. So those are coming up within the
Speaker 1: next week on gamers Nexus, and.
Speaker 2: We'll get the links in there. I don't know what
Speaker 2: I'm going to be doing next week on the show
Speaker 2: because I'm recording this a little early. I'll have him
Speaker 2: on log for you on Friday, though. I have been
Speaker 2: at Zutra on this is Better Offline. You know where
Speaker 2: to find all my stuff will be in the links.
Speaker 2: Thank you so much for listening. Thank you for listening
Speaker 2: to Better Offline.
Speaker 3: The editor and composer of the Better Offline theme song
Speaker 3: is Matasowski. You can check out more of his music
Speaker 3: and audio projects at Mattasowski dot com m.
Speaker 1: A T T O.
Speaker 2: S O w Ski dot com.
Speaker 3: You can email me at easy at betrofline dot com
Speaker 3: or visit Better Offline dot com to find more podcast
Speaker 3: links and of course, my newsletter. I also really recommend
Speaker 3: you go to chat dot Where's youread dot at to
Speaker 3: visit the discord, and go to our slash.
Speaker 2: Better Offline to check out I'll Reddit. Thank you so
Speaker 2: much for listening. Better Offline is a production of cool
Speaker 2: Zone Media. For more from cool Zone Media, visit our
Speaker 2: website cool Zonemedia dot com, or check us out on
Speaker 2: the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.