WEBVTT - Ray Wang on How AI Is Causing DRAM Prices to Surge 0:00:02.720 --> 0:00:16.520 Bloomberg Audio Studios, Podcasts, Radio News. 0:00:18.280 --> 0:00:21.799 Hello and welcome to another episode of the Odd Lots Podcast. 0:00:21.880 --> 0:00:24.200 I'm Joll Wisenthal and I'm Tracy Alloway. 0:00:24.400 --> 0:00:26.119 Tercy, you know what I'm worried about. I don't know 0:00:26.160 --> 0:00:28.200 if I'm worried about it, but it kind of feels 0:00:28.280 --> 0:00:30.280 like the direction things are going, which is that like 0:00:30.880 --> 0:00:36.720 all energy, industrial commodities, anything that we use for any 0:00:36.720 --> 0:00:40.720 purpose is just going to like feed the AI beast 0:00:41.080 --> 0:00:43.760 and us humans We're just gonna get left out in 0:00:43.800 --> 0:00:46.040 the cold. They're like nothing for you. We gotta like feed, 0:00:46.120 --> 0:00:48.760 We gotta feed it all to the AI. And and 0:00:48.840 --> 0:00:51.880 maybe in ten or twenty or fifty years, the AI 0:00:52.040 --> 0:00:53.960 is it's so powerful. They just decide, why why do 0:00:54.080 --> 0:00:56.440 humans get any of this? We just thought we should 0:00:56.520 --> 0:00:57.480 keep it all to ourselves. 0:00:57.760 --> 0:01:00.880 A legitimate concern, I would say, I mean, to some extent, 0:01:00.920 --> 0:01:04.120 we are already seeing this crowding out effect. Right, So 0:01:04.360 --> 0:01:07.640 energy prices in certain areas have been going up, and 0:01:07.920 --> 0:01:12.040 most recently memory chip prices. So this has been all 0:01:12.080 --> 0:01:16.000 over earnings. Recently. You had companies like Apple saying that 0:01:16.160 --> 0:01:19.520 because of a crunch in memory chip supply, they might 0:01:19.560 --> 0:01:21.839 have to either raise their prices or like cut down 0:01:21.840 --> 0:01:25.440 on the amount of phones they use. Nintendo, Like, if 0:01:25.440 --> 0:01:27.480 you pull up a chart of Nintendo shares right now, 0:01:27.560 --> 0:01:30.920 they are just getting hammered, and supposedly it is all 0:01:30.959 --> 0:01:35.080 because of this memory chip shortage. By the way, do 0:01:35.160 --> 0:01:38.959 you remember do you remember like buying your first PC 0:01:39.840 --> 0:01:42.200 in like you bought one before I did, I think, 0:01:42.240 --> 0:01:44.360 But mine was in the mid nineties, ye, I think, 0:01:44.880 --> 0:01:48.000 I go. Do you remember how much memory those things had? 0:01:48.120 --> 0:01:48.280 Oh? 0:01:48.320 --> 0:01:48.840 Like nothing? 0:01:49.080 --> 0:01:52.200 Yeah, Like I looked it up, something like less than 0:01:52.200 --> 0:01:54.440 ten megabytes for a bunch of them. You know how 0:01:54.520 --> 0:01:55.639 much PCs have nowadays? 0:01:55.680 --> 0:01:57.120 Actually, I have no idea. 0:01:57.160 --> 0:02:01.360 I think it's let's see sixteen gig by Yeah, isn't 0:02:01.400 --> 0:02:02.040 that crazy? 0:02:02.240 --> 0:02:02.520 It is? 0:02:02.680 --> 0:02:04.800 It is really crazy. I remember, like, do you remember 0:02:04.880 --> 0:02:08.320 zip drives? Yeah, oh my god, or like, and there 0:02:08.320 --> 0:02:12.359 were all these special you know, obviously various memory peripherals 0:02:12.400 --> 0:02:14.200 that you could buy and they came in sort of 0:02:14.200 --> 0:02:17.280 dedicated cartridges and stuff like that. I think they're called 0:02:17.360 --> 0:02:19.680 zip drive. Yeah, I think right, there was zip drive. 0:02:19.760 --> 0:02:22.160 I don't remember. Yeah, yeah it was. Yeah, that's wrying, 0:02:22.200 --> 0:02:24.960 additional memory storage, etcetera. It used to be this big 0:02:25.000 --> 0:02:27.880 part of the consumer process. I'm actually glad. You know, 0:02:27.960 --> 0:02:30.480 my son really wants a Nintendo. But if they can't 0:02:30.480 --> 0:02:33.200 produce them anymore because they can't get the memory, then 0:02:33.240 --> 0:02:34.880 I say, sorry, no video games. 0:02:34.960 --> 0:02:36.560 That's so mean. You're gonna have to sit them down 0:02:36.600 --> 0:02:39.119 and be like, I'm so sorry, son. There's this thing 0:02:39.200 --> 0:02:42.000 called the supply shortage. I'm going to use this as 0:02:42.000 --> 0:02:45.760 a learning opportunity to explain how global supply chains actually work. 0:02:45.960 --> 0:02:48.119 Totally, you know what. The other the one interesting thing 0:02:48.320 --> 0:02:51.120 is if you look at spot dram prices, which we 0:02:51.160 --> 0:02:53.480 have on the terminal, this is I think maybe the 0:02:53.520 --> 0:02:56.800 interesting dynamic. There's a lot of AI related things on 0:02:56.800 --> 0:02:59.520 the terminal that have been surging for several years, most 0:02:59.560 --> 0:03:02.160 notably some of the big chip companies and video et cetera. 0:03:02.360 --> 0:03:04.799 If you look at a chart of Spot dram prices, 0:03:05.240 --> 0:03:07.720 the big surge wasn't until late last year, and then 0:03:07.840 --> 0:03:10.240 since then it's gone nuts. So for a long time, 0:03:10.280 --> 0:03:13.640 even as this AI story, the AI meme, the AI 0:03:13.680 --> 0:03:16.720 industry like propagated, people were really paying attention to that 0:03:17.280 --> 0:03:19.960 area and then suddenly it's just gone like you know, 0:03:20.000 --> 0:03:24.320 the haywire. Yeah, and so obviously huge supply demand and balances. 0:03:24.440 --> 0:03:26.440 We need to understand what's going on because I think 0:03:26.480 --> 0:03:29.120 the stakes are you know, it's getting so much. As 0:03:29.120 --> 0:03:31.520 you mentioned, a lot of companies are losing out pretty big. 0:03:31.560 --> 0:03:33.520 Others are making a lot of money. Yeah, we started 0:03:33.600 --> 0:03:35.480 need to get a handle on what's going on with 0:03:35.480 --> 0:03:35.920 the memory. 0:03:36.320 --> 0:03:36.840 Let's do it. 0:03:36.960 --> 0:03:39.400 I'm really excited to say we really do have the 0:03:39.400 --> 0:03:41.600 perfect guest. We're going to be speaking with Ray Wong. 0:03:41.720 --> 0:03:45.320 He's an analyst at Semi Analysis, and he's recently published 0:03:45.360 --> 0:03:49.600 a brand new report on exactly this topic, talk about 0:03:49.600 --> 0:03:52.880 a memory super cycle and so forth. So truly the 0:03:52.880 --> 0:03:56.839 perfect guest. As I mentioned, he's comes to us from Korea. Ray, 0:03:57.000 --> 0:03:59.360 thank you so much for coming on odlots. 0:03:59.680 --> 0:04:02.320 Nice to me in Bove and thanksful having me so 0:04:02.360 --> 0:04:03.080 many questions. 0:04:03.160 --> 0:04:06.520 Really appreciate you joining us. Let's just start with the basics, 0:04:06.600 --> 0:04:09.960 like what is it about AI or what is going on? 0:04:10.040 --> 0:04:13.560 What is the core idea such that demand for memory 0:04:13.840 --> 0:04:17.440 in various forms is absolutely seem to be booming right 0:04:17.480 --> 0:04:20.000 now in a way that must have caught the supply 0:04:20.080 --> 0:04:21.440 side of the equation very off guard. 0:04:21.839 --> 0:04:23.880 Yeah, I think you know, to really do get kind 0:04:23.880 --> 0:04:26.680 of the some of imbalanced supply TMO dynamics. Really we 0:04:26.760 --> 0:04:28.400 wanted to get like you know what happened a few 0:04:28.480 --> 0:04:31.120 years ago, right, because you know your capacity to check, 0:04:31.240 --> 0:04:34.480 like your investment few years ago, and what's happening you know, 0:04:34.680 --> 0:04:37.280 like in twenty twenty twenty four, it's it's sort of 0:04:37.320 --> 0:04:40.480 a down cycle, right because remember doing the COVID, all 0:04:40.560 --> 0:04:43.000 the companies so try to flug in to buy tons 0:04:43.040 --> 0:04:46.000 of DRAM, right, and you know there's a lot of 0:04:46.320 --> 0:04:48.880 purchases there, right, and there's sort of like upcycle, right, 0:04:48.880 --> 0:04:51.080 but it's like a very very short up cycle because 0:04:51.120 --> 0:04:54.600 like when the COVID gradually getting better, right, you actually 0:04:54.600 --> 0:04:57.839 don't need that much DRAM And back then the company 0:04:57.839 --> 0:05:00.520 will be like, hey, you actually don't have a sustainable right, 0:05:00.560 --> 0:05:04.200 So they don't want to over invest to expand their capacity. 0:05:04.680 --> 0:05:07.279 And when you go into the down cycle, your capet 0:05:07.360 --> 0:05:10.440 typically be Americans trained Meri like, you know company from 0:05:10.440 --> 0:05:13.400 their perspective, they try to be conservative. So that leads 0:05:13.440 --> 0:05:17.000 to twenty twenty four and twenty five that you're incremental 0:05:17.000 --> 0:05:20.599 waver for capacity that goes So DRAM is actually quite limited. 0:05:21.200 --> 0:05:23.719 But what's happening on the demand and it's absolutely the 0:05:23.720 --> 0:05:26.159 demand was asserrating so fast, right, you know, it's you know, 0:05:26.200 --> 0:05:27.599 one way to look at it is to look at 0:05:27.600 --> 0:05:30.240 a media's earnings, right, and if you when you when 0:05:30.279 --> 0:05:33.320 you're on one side, your demand asserting so fast. On 0:05:33.360 --> 0:05:36.520 the other hand, on the supply side, your supply just 0:05:36.560 --> 0:05:40.279 couldn't catch up. And there's more nuances behind the supply, 0:05:40.440 --> 0:05:43.720 right because before they are just supplying all the commodity 0:05:43.760 --> 0:05:47.920 DRM like DDR LPDDR, right, the all the memory that 0:05:47.960 --> 0:05:51.640 goes to PCs, goes to laptops, goes to your mobiles, 0:05:51.720 --> 0:05:55.200 goes to Nintendo switches. Right. But you know what's happening, 0:05:55.560 --> 0:05:57.560 you know, in this AI, it's like there's a new 0:05:57.600 --> 0:06:00.000 thing called SBN. The think about. It's like a special 0:06:00.520 --> 0:06:04.920 memory for directly for airs arter and that memory is 0:06:04.920 --> 0:06:08.680 extremely drent wayfar intensive to give you a sense, right, 0:06:08.839 --> 0:06:11.280 you know, on the same way, for a basis, you 0:06:11.320 --> 0:06:15.200 can produce three more bits if you do commodity DRM, 0:06:15.279 --> 0:06:18.360 but you can only produce one bit of HBM, right, 0:06:18.400 --> 0:06:21.120 and that ratio will actually going higher when you go 0:06:21.200 --> 0:06:24.040 to HPM four HBM five in the next few years. Right. 0:06:24.360 --> 0:06:26.560 So in that way, on the same way, for basis 0:06:26.600 --> 0:06:29.159 you can you can only produce that much HBM, and 0:06:29.240 --> 0:06:31.599 right now what's happening is like, hey, this HPN is 0:06:31.640 --> 0:06:34.760 super super profitable, and why not we that? Okay, more 0:06:34.800 --> 0:06:38.320 wafare for HBM. But so here's the problem, right, you 0:06:38.440 --> 0:06:40.400 only have that much wafer and you need to try 0:06:40.480 --> 0:06:43.919 to do everything. So that's funding out a lot of 0:06:43.960 --> 0:06:46.680 the supply going to the commodity dram. Right, so we 0:06:46.760 --> 0:06:48.919 have we have sort of both things. That's happening and 0:06:49.080 --> 0:06:51.080 essentially comes down to like I think, really, you know, 0:06:51.120 --> 0:06:53.359 things start really kind of people are wearer. It's like 0:06:53.400 --> 0:06:56.120 probably Q two T twenty five that people realize, hey, 0:06:56.200 --> 0:06:59.440 like you know, we actually couldn't find enough like enough DURAN, 0:07:00.360 --> 0:07:04.040 enough commodity d REM and also HPN for their demand. 0:07:04.560 --> 0:07:07.440 So you mentioned the sea word just then oh wait, 0:07:07.680 --> 0:07:09.640 not that seaword commodities. 0:07:10.240 --> 0:07:11.280 You mentioned commodities. 0:07:11.320 --> 0:07:14.720 And when I'm reading these articles about a memory chip shortage, 0:07:14.800 --> 0:07:19.040 people use commodity type language. Right, So you hear super 0:07:19.120 --> 0:07:23.600 cycle a lot, you hear commoditized chips versus I don't 0:07:23.640 --> 0:07:25.960 know other types of chips. Are they customized? I don't 0:07:26.000 --> 0:07:29.920 think so. But anyway, how much does this particular industry 0:07:29.920 --> 0:07:33.560 actually resemble a commodities like industry, and then maybe explain 0:07:33.600 --> 0:07:37.160 a little bit more why historically it has been really 0:07:37.200 --> 0:07:38.040 really cyclical. 0:07:38.800 --> 0:07:40.240 Yeah, so I think a couple of reasons. 0:07:40.320 --> 0:07:40.520 Right. 0:07:40.880 --> 0:07:44.080 You know, Fundamentally, the sort of the cost of bits 0:07:44.120 --> 0:07:48.520 for DRM was actually coming down every single year significantly, right, 0:07:48.560 --> 0:07:50.720 even you know in most recent years was also coming down. 0:07:50.760 --> 0:07:53.280 But like the extent that they are coming down for 0:07:53.600 --> 0:07:56.000 every single year is actually slowing down, like you know, 0:07:56.080 --> 0:07:58.640 over the past like ten twenty years. Right, So the 0:07:58.680 --> 0:08:02.000 cause was key going down, right, So because it's hard 0:08:02.040 --> 0:08:05.160 to become of like you know, the most competitive part. Right. 0:08:05.320 --> 0:08:08.720 Another thing is there's the nest of the committee that 0:08:08.800 --> 0:08:11.680 let's set up the standard for d REM right. So, well, 0:08:11.720 --> 0:08:14.360 you're doing the similar products, right, you're just having you know, 0:08:14.360 --> 0:08:16.520 oh this is kind of a little better power efficiency, 0:08:17.160 --> 0:08:19.760 the performance was a little better right here and there. 0:08:20.080 --> 0:08:22.320 So like it's hard for you know, a memory supplier 0:08:22.400 --> 0:08:27.520 to significantly differentiate their products compared to their competitor. Right, 0:08:27.560 --> 0:08:30.280 So essentially, what's the what's the things that will compete 0:08:30.440 --> 0:08:32.640 in M will be in the market prices? Right, And 0:08:32.679 --> 0:08:35.440 then if you are selling the similar products and you 0:08:35.480 --> 0:08:38.760 are talking in similar audience. Right, that's sort of like 0:08:39.160 --> 0:08:42.160 other commodity you know, sort of semic adopt the commodity 0:08:42.200 --> 0:08:44.400 products to me, and I think those are the two 0:08:44.520 --> 0:08:47.839 man characteristics I'm saying for DRAM, and I think that's 0:08:47.840 --> 0:08:50.160 a little a little different for HBM. 0:08:50.520 --> 0:08:54.560 Tell me explain this. So there's commodity d RAM and 0:08:54.600 --> 0:08:58.320 then there's HBM. What HBM is a type of d RAM? 0:08:58.400 --> 0:09:01.480 Is that what you're saying? Yes, can you explain this 0:09:01.640 --> 0:09:04.120 further and like sort of yeah, walk us through what 0:09:04.200 --> 0:09:06.720 the commodity version and why there is this thing called 0:09:06.880 --> 0:09:09.079 HBM High bandwidth memory what that's all about. 0:09:09.720 --> 0:09:13.439 Yeah, Yeah, So essentially the emergence of HPN is really 0:09:13.520 --> 0:09:16.280 go down to the kind of model layer, because you know, 0:09:16.440 --> 0:09:18.600 this is when you like try to scale and stary 0:09:18.679 --> 0:09:23.920 scaling your AM model. Right, your thoughts, your followsing increasing 0:09:23.920 --> 0:09:26.600 significantly every single year, right, because your compute was increased 0:09:27.160 --> 0:09:30.280 lat faster, but your memory bandwidth was actually sort of 0:09:30.320 --> 0:09:33.160 it's increasing, but it's very limited, especially if you are 0:09:33.240 --> 0:09:36.640 using the memory from kind of traditional commodity dan that 0:09:36.760 --> 0:09:39.760 the bandwidth is actually mare limited. So from a memory supply, 0:09:39.800 --> 0:09:42.199 they're thinking, hey, like how can we expand the bandwidth 0:09:42.480 --> 0:09:47.160 to supporting the memory. They're going to scale the AI, right, 0:09:47.640 --> 0:09:50.320 so let's consoles HBM, so the way they do hBN. 0:09:50.400 --> 0:09:55.040 It's so they are stacking multiply eight were told for 0:09:55.160 --> 0:09:58.720 even in the future sixteen DR and dies together, right, 0:09:59.040 --> 0:10:02.920 So that making you you have like essentially having higher 0:10:03.040 --> 0:10:07.400 bandwidth and have enough capacity to support AI developments over 0:10:07.440 --> 0:10:09.439 the past few years and in the next few years. 0:10:09.559 --> 0:10:11.480 And let's very different, right, you are stacking all this 0:10:11.640 --> 0:10:13.320 dies together compared to a Hey, I would just we 0:10:13.440 --> 0:10:15.720 just have like one DRM products as a chip. 0:10:15.800 --> 0:10:16.000 Right. 0:10:16.240 --> 0:10:18.360 And also that also comes down to the point that 0:10:18.880 --> 0:10:22.480 HP and manufacturing also the packaging, both front end and 0:10:22.559 --> 0:10:25.679 back in there's a lot more complex compared to commodity 0:10:25.720 --> 0:10:28.360 d REM. And that's why I say commodity DRM is 0:10:28.400 --> 0:10:30.680 more like commoditize, but I think HP is less so 0:10:30.840 --> 0:10:34.000 because it's lumbo complex, right. And for supplier you can 0:10:34.040 --> 0:10:37.480 actually differentiate either from the front end or back end technology. 0:10:37.800 --> 0:10:39.920 And when you you know it's better, right, it gives 0:10:39.920 --> 0:10:43.080 you a better margin, better ability to compare compete with 0:10:43.080 --> 0:10:46.240 your competitor. Right, you know, stuff like thermal things like 0:10:46.280 --> 0:10:48.440 that is the thing that will really stand out right, 0:10:48.440 --> 0:10:50.320 and let's marry different commodity. 0:10:50.000 --> 0:10:52.760 D rem getting more and more layers on HBM. It 0:10:52.840 --> 0:10:55.120 kind of reminds me of the Razor Wars, where like, 0:10:55.400 --> 0:10:58.360 you know, Gillett would unveil a three blade thing and 0:10:58.400 --> 0:11:01.120 then the next day someone has like twelve or whatever. 0:11:16.840 --> 0:11:19.880 So if I'm a company like Nintendo, how am I 0:11:20.040 --> 0:11:23.920 actually purchasing chips? Because I think this is going to 0:11:23.960 --> 0:11:26.560 be important when we talk about what's going on right now. 0:11:26.600 --> 0:11:30.160 But do I have like a forward contract with a 0:11:30.200 --> 0:11:32.400 major supplier, or do I have I don't know, a 0:11:32.440 --> 0:11:35.280 warehouse full of chips that I bought previously. How do 0:11:35.360 --> 0:11:36.320 they actually manage that? 0:11:37.280 --> 0:11:39.080 So there's a couple of ways they do it, right. 0:11:39.200 --> 0:11:41.000 They will they will do it in in terms of 0:11:41.080 --> 0:11:42.439 like kind of they will do a different way like 0:11:42.600 --> 0:11:46.240 different environments and also different vendors, right, So it's hard 0:11:46.240 --> 0:11:48.800 to say like, hey, this is exactly exactly the way 0:11:48.840 --> 0:11:50.600 they do it. But typically I think you can think 0:11:50.640 --> 0:11:53.199 about like, hey, this is amount of the chips we're 0:11:53.200 --> 0:11:55.760 gonna purchase in the next four months, and this is 0:11:55.760 --> 0:11:57.600 a month we're going to commit, right, and we're gonna 0:11:57.600 --> 0:12:00.000 sign a counter partly saying whether this term will be 0:12:00.200 --> 0:12:02.800 four months or for the LTA, you can go to 0:12:02.840 --> 0:12:04.400 like as long as like to a year or even 0:12:04.440 --> 0:12:06.600 two years. Right, So let's see the people the way 0:12:06.640 --> 0:12:08.480 you think about it. But people you will be like, hey, 0:12:08.480 --> 0:12:11.400 we signed this country for four months, right and by 0:12:11.400 --> 0:12:13.360 the end, sorry for three months for a quarter. So 0:12:13.400 --> 0:12:15.640 by the end of the quarter, you negotiate a new 0:12:15.760 --> 0:12:18.640 country pricing for the next quarter. So that's typically the 0:12:18.679 --> 0:12:21.640 way it works. But you know the problem for right 0:12:21.679 --> 0:12:24.600 now is for consumer and toronic, it's like the pricing 0:12:24.600 --> 0:12:27.559 prob is sort of I would say secondary, because if 0:12:27.559 --> 0:12:30.079 you couldn't secure value, you couldn't even make any money, right, 0:12:30.240 --> 0:12:33.400 So number one is secure in value, whether that you know, 0:12:33.520 --> 0:12:36.199 make sure that fits to the demand all look, you 0:12:36.240 --> 0:12:38.920 are same two. It's whether you can negotia the best 0:12:38.920 --> 0:12:43.679 pricing you can. But given the overall pricing environment was slow, volatile, 0:12:43.920 --> 0:12:46.680 was so ascanting so much, it's hard to like not 0:12:46.840 --> 0:12:51.000 taking some kinds of marching impacts because essentially, you know, 0:12:51.760 --> 0:12:53.800 you try to get the chips right and you cannot 0:12:53.840 --> 0:12:56.679 just you know, get too much away from your vendors. 0:12:57.240 --> 0:12:59.720 Yeah, I want to get more into the demand destruction 0:13:00.200 --> 0:13:02.199 because I think This is very important to think about 0:13:02.200 --> 0:13:04.920 where the e globrium is going, just to clarify what 0:13:05.120 --> 0:13:09.400 is it about AI specifically? Is it in the training, 0:13:09.640 --> 0:13:12.920 is it in the inference? Where specifically in the process 0:13:12.960 --> 0:13:16.840 of building out AI services does this voracious demand for 0:13:17.200 --> 0:13:18.200 DRAM come from? 0:13:18.559 --> 0:13:21.120 So honestly, is to be up, very honest, I think 0:13:21.160 --> 0:13:24.439 is everywhere, Like, right, you know AI training you need 0:13:24.440 --> 0:13:27.000 to council HPM. That's just everyone, no, right, you also 0:13:27.080 --> 0:13:29.480 need a lot of the CPU d RM and that's 0:13:29.600 --> 0:13:31.520 LP d d R and a DDR that going to 0:13:31.600 --> 0:13:34.079 the server. You also need HPN, right, So let's on 0:13:34.120 --> 0:13:38.000 a training part. In an inferencing you also need HPM, right, 0:13:38.080 --> 0:13:40.840 and you also need the CPU DR and the ghost 0:13:41.000 --> 0:13:43.520 loss vocal right, But you know the main one will 0:13:43.559 --> 0:13:46.240 be the HPM and even the inferencing right right now 0:13:46.240 --> 0:13:49.240 they are separating to pre fill and dec and you know, 0:13:49.320 --> 0:13:51.680 I think eighty percent I think right now, like the 0:13:51.679 --> 0:13:53.720 most important thing will be to code, right, and the 0:13:53.800 --> 0:13:57.119 CO is super super like memory bound and memory intensive 0:13:57.679 --> 0:14:01.120 and HPN and memory importance of their will just will 0:14:01.120 --> 0:14:04.160 only increase to my understanding, especially given the long contest 0:14:04.160 --> 0:14:07.599 window continue to expand and the inference usage. You know, 0:14:07.679 --> 0:14:10.679 douctions continue to going up, right, So I think those 0:14:10.720 --> 0:14:12.840 are typical way I will think about those, right. And 0:14:12.880 --> 0:14:14.120 the last thing is I think a lot of people 0:14:14.200 --> 0:14:17.520 talk about these dates is about argentic AI, right, agentic AI. 0:14:17.920 --> 0:14:20.360 You want to power those? You need a lot of CPU. 0:14:20.520 --> 0:14:22.360 You're not a lot of CPU based server, right, So 0:14:22.400 --> 0:14:25.280 what's what's including in the CPU server? There's a lot 0:14:25.280 --> 0:14:27.680 of d R in there, right, and a lot of 0:14:27.720 --> 0:14:29.680 urgentic world law also mean you need a lot of 0:14:29.680 --> 0:14:32.160 inference as well, right, And then go back to my 0:14:32.200 --> 0:14:35.080 previous point, you need a lot of XP and d REN. Right, 0:14:35.120 --> 0:14:36.680 So I would say like kind of d RN and 0:14:36.720 --> 0:14:38.680 I get HP and d R and kind of everywhere, 0:14:38.720 --> 0:14:40.720 and we have to talk about storage bout storage or 0:14:40.720 --> 0:14:43.960 so everywhere. You know, in doing this process. 0:14:43.760 --> 0:14:47.360 I have a basic question, but why does AI need 0:14:47.680 --> 0:14:50.920 a lot of memory at all? So you just explained, 0:14:50.960 --> 0:14:54.400 you know, it's important for inference, it's important for training whatever, 0:14:54.520 --> 0:14:57.880 but like why Yeah, I think. 0:14:57.680 --> 0:14:59.680 I think this this needs to like goes down to 0:14:59.720 --> 0:15:02.400 the to the model later, right, because essentially when you 0:15:02.560 --> 0:15:05.160 try to train, uh, let's say you want to train 0:15:05.240 --> 0:15:07.600 the model, right, and then you need. You just need 0:15:07.640 --> 0:15:09.760 the pure tons of data. You need to write tons 0:15:09.760 --> 0:15:11.680 of data there to train your model. And then for 0:15:11.800 --> 0:15:14.760 the influencing partline, you need one data after another to 0:15:14.840 --> 0:15:16.760 kind of compute one after another, sort of the chain 0:15:16.840 --> 0:15:19.360 of salts. Right, yeah, So to to do that process, 0:15:19.440 --> 0:15:21.600 how can you keep the previous data on your process 0:15:21.680 --> 0:15:23.400 to keep it in there and the process to another 0:15:23.480 --> 0:15:25.880 data that's coming out after right, So that's why you 0:15:25.920 --> 0:15:28.280 need not of memory and go back to my previous point. Right. 0:15:28.440 --> 0:15:32.160 The colon context window basically means like when you're when 0:15:32.280 --> 0:15:34.960 the things that your process is longer and longer, right, like, 0:15:35.000 --> 0:15:36.840 how can you how can you have enough memory to 0:15:36.920 --> 0:15:39.280 process those? Right? And you know, one for a good example, 0:15:39.360 --> 0:15:42.320 if you use TGBT, you guys definitely remember right, you 0:15:42.320 --> 0:15:46.000 know back togbts, which starts coming out people just like hey, 0:15:46.160 --> 0:15:46.960 like what's the weather? 0:15:47.080 --> 0:15:47.240 Right? 0:15:47.280 --> 0:15:51.480 Like marriage poems people right now? Is or nastier people 0:15:51.480 --> 0:15:54.720 are doing like hey, can you write a twenty page 0:15:54.760 --> 0:15:57.320 of poor? How canser it can happen on people? 0:15:57.360 --> 0:15:57.440 Right? 0:15:57.520 --> 0:16:00.000 For example? Then will give twenty page of poor and 0:16:00.000 --> 0:16:02.120 they were wait like five minutes, right, And so if 0:16:02.120 --> 0:16:04.520 you think about the process like they do all the calculation, 0:16:04.680 --> 0:16:08.040 all the researches, and also give you the output tokens 0:16:08.040 --> 0:16:12.720 that the answer you get from your prompts, that's significant, 0:16:12.800 --> 0:16:15.400 way longer than the proms that you are given, right, 0:16:15.400 --> 0:16:17.920 And that's also a lot longer response that you are 0:16:17.960 --> 0:16:20.880 getting when you call back to the GPD three. And 0:16:21.040 --> 0:16:24.320 remember the usage we know back to GPD three was 0:16:24.440 --> 0:16:27.920 it was the U kind of the monthly daily users, 0:16:28.200 --> 0:16:29.800 but right now it was I think the user for 0:16:29.840 --> 0:16:32.440 GGPT was eight hundred million, right, and we are not 0:16:32.600 --> 0:16:37.240 included the users for the Germini, for clud for XAI. Right, 0:16:37.280 --> 0:16:39.720 So you think you multiply those effects. I think the 0:16:39.760 --> 0:16:43.680 memory they management significant and it's very clear to me. 0:16:43.920 --> 0:16:46.480 Yeah, it's really interesting. I saw someone people were talking 0:16:46.520 --> 0:16:48.520 about this, but it really strikes me in my own 0:16:48.680 --> 0:16:51.920 usage of AI just basically how much more use it 0:16:52.040 --> 0:16:55.200 as capabilities have grown. Like it was sort of cute 0:16:55.240 --> 0:16:57.480 in late twenty twenty two. It's like, oh, get it 0:16:57.520 --> 0:16:59.720 to write a poem about this, and it's like okay. 0:16:59.800 --> 0:17:02.200 Then I would go, you know, several weeks sometimes without 0:17:02.440 --> 0:17:05.840 having anything to query, and then you know, these days 0:17:05.880 --> 0:17:08.159 like I'm using it all the time, like trying to 0:17:08.240 --> 0:17:11.000 do things with code or scrape data, et cetera. So 0:17:11.800 --> 0:17:17.800 overall token consumption has grown massively with increased capabilities. All right, 0:17:17.880 --> 0:17:24.199 let's talk about the demand destruction element. Any commodity is 0:17:24.280 --> 0:17:25.639 going to be the price is going to be a 0:17:25.680 --> 0:17:29.000 function of supply and demand, and eventually prices get high 0:17:29.119 --> 0:17:31.320 enough such that certain forms of demand just are no 0:17:31.359 --> 0:17:33.639 longer economical. And it's like, you know what, We're just 0:17:33.680 --> 0:17:36.440 not going to make as many video games or cameras 0:17:36.560 --> 0:17:40.240 or whatever else uses a memory. Are we starting to 0:17:40.280 --> 0:17:44.320 see any of that yet such that certain uses and 0:17:44.520 --> 0:17:45.679 consumption of memories. 0:17:45.920 --> 0:17:48.320 Joe doesn't have to buy switch for his son, Yeah, 0:17:48.320 --> 0:17:48.720 that's right. 0:17:50.440 --> 0:17:53.240 Yeah, I actually both have switched RAC So the price 0:17:53.359 --> 0:17:54.159 you should resell it? 0:17:54.480 --> 0:17:55.399 Are you going to resell it? 0:17:57.160 --> 0:17:59.240 I don't know. We'll see it up hol the opts 0:17:59.280 --> 0:18:02.080 rocks there works. But yeah, I think I think we're 0:18:02.119 --> 0:18:04.240 already seeing a lot of impacts. Actually, but kind of 0:18:04.480 --> 0:18:06.720 you kind of vary in different types of products, right 0:18:06.800 --> 0:18:10.320 right in p you are showing seeing that thell non 0:18:10.320 --> 0:18:13.360 noble answer sous. They are all having a price tip 0:18:13.440 --> 0:18:16.840 for kind of different their problems, right, and you know, 0:18:16.960 --> 0:18:19.240 on the kind of demand side. You're also seeing that. 0:18:19.640 --> 0:18:22.560 I think for the Chinese small phone market. Right, a 0:18:22.600 --> 0:18:24.720 lot of the analysts, a lot of the research firm, right, 0:18:24.800 --> 0:18:26.360 a lot of the company was saying, like they are 0:18:26.359 --> 0:18:29.600 cutting off their small phone outook a full example. I 0:18:29.600 --> 0:18:31.880 think media attacked recently left for Arniest. Right, they're saying 0:18:31.880 --> 0:18:34.919 they're cutting off the mobile to I think it was 0:18:34.960 --> 0:18:37.880 twelve for ten to fifteen percent of the twenty twenty 0:18:37.920 --> 0:18:39.960 six outlook. That's mere significant. 0:18:40.040 --> 0:18:40.159 Right. 0:18:40.200 --> 0:18:43.200 You're saying this year is supposed to be a hundred million, 0:18:43.640 --> 0:18:46.720 but you will essentially go to a nineteen million. Right, 0:18:46.840 --> 0:18:49.520 that's mere significant. So I think we are already seeing that, 0:18:49.560 --> 0:18:52.000 but you kind of vary different places. Right. Apple will 0:18:52.040 --> 0:18:54.920 be a great example here because they're saying, hey, like, hey, 0:18:54.960 --> 0:18:56.359 we have all the price TI like, right, but we 0:18:56.440 --> 0:18:58.960 actually managed to put it well. Uh, they are saying 0:18:59.000 --> 0:19:01.800 the real sort of for marketing impact will actually show 0:19:01.920 --> 0:19:04.240 up in the second half twenty twenty six instead of 0:19:04.280 --> 0:19:07.120 like the first quarter or second quarter. I think those 0:19:07.119 --> 0:19:09.119 of at of the things I'm seeing, So it's definitely 0:19:09.200 --> 0:19:11.320 already showing out in the market. Right. I think what 0:19:11.440 --> 0:19:14.640 you can expect this sphere will be two things one 0:19:14.760 --> 0:19:17.879 is sort of demand destruction in PCs right, the border 0:19:17.880 --> 0:19:22.280 consumer and astronics or the dispect whether it's the memory 0:19:22.280 --> 0:19:25.560