AI Made Everything Expensive (ft. GamersNexus)

Also media.

Hello on, welcome to Better Offline. I'm your host ed Zitron. Subscribe to the newsletter, joined the subway and buy a T shirt. All the links are in the notes. And here's a joke to kick us off. What do you call the CEO of a French RAM company? That's right, Immanuel Micron, And that's what we're here to talk to Steve Burke of Gamers. Next is about today, Steve, Welcome back to the octagon.

Emmanuel Micron.

Yeah, before we started, yeah, because you would have killed me before I read it on the But look, we're here to talk about RAM. So we'll start with a simple question. Why is RAM so expensive? Right now? What is doing this?

I mean, the reason it's so expensive is not grounded and reac I think you know, it's the surface of it is that all the data center build out is consuming all the memory and therefore, you know, the price higher because everyone wants it. But I do think there's a little bit more to.

It than that, right So from what so when you build chips We're going to really simplify this one for the audience because it's pain in the US to explain otherwise. But when you build chips. You have something called a waifer. You have the silicon out, and then you entch it using a lithograph machine. But you can make less high.

So there's the high bandwidth memory that they use inside GPUs, and then there's the regular DRAM that they use in everything else. Right, is that it's a simplification.

But is that failure? I think like the the most base level stuff to know is that memory, some form of RAM or memory is in almost everything. A common misconception, I'm not sure. I think your audience is like fairly technical, but a layperson misconception is that storage is the same as memory. They'll use it interchangeably. If storage is like your drive that you you know, you put data on, like photos or whatever, and then memory is is a temporary form of storage of volatile storage, like it's wiped when it's not actively being used when you restart and.

It's and when they put stuff in memory, that's where that's like RAM and such where they like, that's where you put the stuff that you gonna need quickly.

Right, Yeah, so yeah, memory is much faster than storage, permanent storage, and so it acts as the sort of transactional buffer to help get things to you quickly, So memories and phones, refrigerators these days, cars, there's all kinds of different grades of memory, and it's it's used in pretty much everything to do some kind of quick like in a car, Uh, maybe you need to retrieve something related to like if there's a built in map on the screen or something like that, or in a refrigerator.

Who the fuck knows, honestly, but I think that's I think so that.

You can have a giant, horrible screen on your Samsung fridge that displays you adds occasionally, yes, exactly, yeah, and so so yeah, to go to your wafer count. Yeah, there's different types of sort of production for memory, and you'll find DRAM is used in all of it in some capacity for the most part. So you'll find DRAM dies in HBM like inside of an HBM package. But data center high end GPUs will use high bandwidth memory, and basically every consumer device uses something called GDDR, which is just a lower bandwidth kind of more normal solution that's not as overboard on the performance for a consumer application.

But they all kind of come from the same place, which is a wafer. They're all cut from right so right now, they're taking up more, they are using more of the how much of the compacity are they actually using to build high bandwidth memory? So the stuff that's going to the GP use.

It appears to be an awful lot. I mean, so, I know that the recent class action lawsuit or complaint I should say that was filed notes that the profitability the plaintiffs attorneys claimed that the profitability of traditional conventional DRAM is greater than the profitability of HBM. That was one of their claims. And yet they sort of argue that it seems like their evidence is good, but they argue that why would you shift more and more capacity to producing this lower margin thing HBM? Right of like the easier thing to make is potentially higher margin, And so it does seem like more of it's shifting over there.

You know.

Open AI cause that sort of big kerfuffle previously when they claimed forty of just all DRAMs globally though yeah it didn't have it was all like basically.

One of the most insane just I mean sane story. By the way, just last year, for those of you who don't remember, open A story came out the open A I was buying thirty five to forty percent of all the world's dram wafers. It never happened, Yeah, but we just talked about it like like we did, like it did, but.

Now it did affect the price.

It did. But here's the question, So why is everything so expensive that like is it? Is it the because are they just jacking the price up?

I think so? I mean so, yeah, there's a couple of things here a layout like some of the highlights, and we can you can figure out how you want to go through them. But sure one of them is the memory industry has convictions actually in its past of price fixing, collusion and as the DOJ dubbed it, cartel like behavior. So this goes the early two thousands, there's a bus to twenty sixteen, twenty eighteen. There was an attempt at a similar lawsuit in the early nineties. The memory manufacturers were talking about how they thought their competitors were price fixing and colluding and so this is a long and story to history where they and now there's an active lawsuit, putative class action for price fixing. So

there's that antitrust aspect, which that obviously you know, prices would be higher if you're just doing illegal things to make the prices higher. There's also the sort of physical manufacturing side, which is that to make memory, it's not quick to get another fab online, and the companies seem to sort of signal to each other through earnings reports whether they're going to expand capacity or maintain capacity of their fabs, which are factories for memory. So there's that aspect, which is if there's a real or not a sudden increase in demand, there's going to be a long latency to react to that with more supply, if they even want to. So the current projections are twenty twenty eight or twenty thirty for getting like more fabs online. The

other reason it goes up, though, is really simple, which is that in a server that goes in a data center, some of Nvidia's newer solutions will use one and a half terabytes of system memory. So that's d RAM they'll use.

Well they use this, they don't. They also use the same RAM as we find in smartphones as well.

Yeah, LPDDR typically Yeah, yeah, so so Grace will use I think it's five X I think is what they're on.

Yeah, I pulled the numbers and it's something like twenty terabytes for an NVL seventy two, which is seventy two GPU rack.

Yeah, so they do. The single blade for the Big Boy is one and a half terabytes of system memory alone. And you know, for perspective, a consumer desktop computer might have sixteen gigabytes with Windows these days, you probably want like thirty two, and they're they're taking one and a half terabytes for one, you know, server solution. But that's just the system mary. They also have other processors on there, like the Blue Point or blue Field I think it's called DPU. It takes one hundred and twenty eight gigabytes if I remember correctly. The GPUs take hundreds of gigabytes to like close to half a terabyte per GPU of HBM, and the HBM has a there's kind of a yield factor at play where you know you can when you've

talked about manufacturing, that means you're also talking yield how many good products do you get for every bad And HBM is more difficult to make. If you have a failure in manufacturing, then you're also potentially losing more silicon dyes that could have gone.

And HBM is made by stacking the memory, right, so it requires just physically more memory. That's kind of an APES description.

No, No, it's fine. Yeah, So HBM high bandwidth memory is literally just really high bandwidth. It's kind of like if you were to if you were a city and you wanted to solve a public transit problem and you just built a shitload of I don't know, let's.

Take a massive tunnel.

You build like a bunch of trains. You build a ton of trains for passengers, but you don't have enough tracks to put those trains on. They can't actually get on the track without blocking each other. You haven't really solved your problem. And so high bandwidth is trying to solve that by saying, Okay, we've got all this capacity for passengers, which is the memory itself. Now we need something like a bigger road system to actually feed that capacity through to access it. And so HBM tries to solve that, especially because you look at GPUs that might have like twelve plus times the amount of capacity that a consumer GPU would have, and so you don't want to starve the GPU core of the data that sits

in the memory by not being able to get it fast enough. And so HBM tries to solve that problem. So, yeah, to go back to your point of stacking, high bandwidth memory is physically vertically stacked, as in, they take silicon dyes for memory and they stack them atop each other and then they connect them all together, as opposed to consumer memory, where it's closer to there's still layers, but it's closer to a flat plane.

Right. But it just means that by virtue of that, it takes up more space in the place where they build the chips.

Yeah. Yeah, if you're on a consumer device, then the consumer memory you'll need more physical area to put the same amount of capacity, whereas with the high end server stuff you just stack that capacity vertically. And then it's also hbm's really significant advantage is that it's very low power comparatively, right, Yeah.

But just to be clear, it takes up more space on the fabricator itself.

Oh yeah, yeah, if you're talking about the space, then yes, HBM requires a lot more space for a few reasons. In terms of manufacturing area. In terms of like footprint on a product, it's less, but in a fab it's more. And why is that in the fab One of the reasons. So there's a few, but one is that HBM requires additional silicon pieces to be able to function. So with GDDR consumer GPU memory, you pretty much you have the memory module. It looks like a black rectangle, and then if you were to use like acid or something to etch away the black plastic part, you'd be left with pretty much a singular silicon die underneath. That's all it is for consumer. For HBM, you also need a logic dye, So there's another piece of silicon that interfaces with the

memory and the stack of memory, because you need something to kind of to command and control and understand, you know, the vertical tower you've got of memory. And then you also need an interposer die, which sits underneath everything. So this is a solution that will sit under your GPU core, which is a very large piece of silicon and is shared with the HBM stacks. So all of your your core and all your HBM is stacked on top of the same it's called an interposer. It looks like a PCB and they all communicate through that. So that's another piece of silicon. And so for this these reasons alone, there's also others, but you know these alone. Now you're involving more fabs. There's more logistics shipping things around, and

there's more fab or fabrication plant area allocated. Even if you're just talking about making someone's got to make the logic dies TSMC makes the interposers, right. Yes, there's just.

More moving parts and that kind of takes up more space exactly. Yeah, So why So to my early question, so this stuff is obviously lower margin, but they seem to be able to sell an absolute shit ton of it. So is that potentially why they're doing this that just they know there is insane demand on a level that for a high well, I guess they can charge what they want for it as well. I'm just having trouble trying to work out what it is they're doing other than just filling their boots.

Have you try to asking the chat GPT.

No, I could ask my cat and get an answer of respect, but it's genuine. But it's genuinely because it seems the unless I'm misunderstanding here, that these micro and eske High Inex and Samsung they're just kind of cranking up the price of regular RAM because they can.

I mean to some extent. Yeah, So I think this is where the price fixing allegations come into I don't know, if you try it, if you try to take their side, which I really don't want it.

I'm trying. I'm trying.

Yeah, if you try to, I think their argument, whether it's for truthful or not, but would be We've got this crazy high demand for HBM. These are guaranteed customers. I can sell terabytes of it to Jensen, and it's it's one point of contact. It all moves quickly and easily. And this there's this data center demand right now, and we don't know if it's going to be here forever.

But consumers, you know, they're they're always going to have to buy electronics or whatever. So I'm not exactly sure if that's to me, it seems like maybe you make some sort of you assign some value to the B two B relationship over B two C or something maybe, But also I guess if they yeah, I'm not sure, if you make.

Why would that mean that they charge more for regular RAN.

Oh As far as why more for regularly, that part's easy. That's just because the HBM itself takes the same a lot of the same machines as consumer memory. And so if you're made more HBM, you're naturally making less consumer, which I think is where you start seeing the supply squeeze and the price go up. Likewise, a single piece of HBM can use up to something I think it's like it might be like thirty two dies max or something of DRAM you'll commonly see like four referenced. But single piece of HBM will take more dram dies to build.

And so one of the problems is if let's just say four, If you have a piece of HBM that's got four dram dies in it and you have one failure somewhere in that stack or in packaging an assembly, now you've destroyed you know, four individual pieces, rather than if you have a yield failure with a consumer device, you destroy one.

Sorry to ask a stupid question, what is a die exactly? Is that just the what we would think of as a stick of ram.

Uh No, a die is so a wafer we'll start the wavefer level. A wafer is just a disc, right, it looks like a CD that typically eight or twelve inches, and they they process it and they pattern it where they literally project like a pattern or they lithographically etch a pattern onto the surface of this disc. Once that's done, you know, these these patterns, they're called patterns because it's a repeating sort of set of shapes that ultimately form the logic that makes the silicon. And the word dye comes from the process dicing. So with a wafer, when they're done producing it and patterning it and it's basically ready to become a product, they need to cut that disc into a bunch of small squares.

So it's like a big slab that they caught top.

Yeah, it's I mean, it's like making a pizza with a pizza cutter. You know, they're they're doing a pattern that only psychopaths would do on a pizza, which is making a bunch of squares and uh, and so they cut it into the squares. That's called dicing. And then each of those pieces, the squares that yield from that, those are the dyes. And so if you were to pull up an image of any GPU, typically you'll see one large it's called the monolithic dye, which means one singular sort of piece of shiny reflective silicon mirror like finish.

That's your die. And then for memory, same exact concept, just a lot smaller.

Right, So right, okay, so there's less space to make regular rams, so they just crank up the price. That's really that really is it? I think that's their entire thing, right, But I saw Micron zernings and they had eighty four point nine percent margins. So it just kind of feels like they're setting the price to whatever they want it to be.

You don't think there, you don't think that is? Is that not an I think they need more?

Don't They should get ninety five percent? Well, it's just.

Kind of why are you setting the setting it so low?

They should get one hundred and ten percent margins? We should No, It's just it's very frustrating as well, because I've read now arguments in favor and against Micron where it's like, oh, Micron got screwed by Apple and consumer electronics companies during the last during the last supply chain crisis because they they kind of made a bunch more ram than they needed to from the river. You know, like, no, it really it really is. But what and it seems to be there's this boom and bus cycle in memory where they get a bunch of demand and then so do they build more capacity? Is it that they just ordered too much RAM and then people cancel their orders like what happened? What happened last time?

Yeah, so it's cyclical, I will say right now, as nothing like I've ever seen. And we suppoke to a bunch of companies purchasers of memory like g Skill and course Air, who sell the sticks of RAM that you put in a computer, and those guys are telling us on record that likewise they've never this is not like part of the cycle. This is special. And as for how is it different, how the difference is, how much the difference is the percentage increase it is this time?

So you know, you're looking at like seven hundred percent increase in price in some situations in a very short period of time. Normally, cyclically you'll see some ebbs and flows, but it might be like back in the day, a fifty dollars kit of DDR four became eighty five or ninety dollars, maybe even one hundred and twenty if it's a really bad year, you know, but you're still at like a little worse than double, Whereas now a two hundred dollars kit of memory is suddenly a thousand to twelve hundred dollars. Uh, And so it's just the the amount of the price in is what's different. And then also this time, the sort of the contest for the supply is it's coming from within the own industry, where the very companies that would be the cause for an

increase in sales of consumer supply are now the cause of these like insane unprecedented volume of sales for AI compontents.

But it isn't based like so the twenty twenty one supply chain crisis chip crisis was because there was a flood of consumer spending off the COVID, Yeah, and that led to a bunch of consumer demand for products and overall spending across the board. But like consumer demand drove, this one seems different because it's not demand outside of the outside of whatever like psychosis, the various hype sets.

They're not spending a trullion dollars in CAPEX really feel like, it's not like consumers can't access whatever micros COVID.

I think you know what's interesting is the big three memory makers decided not I think it was twenty twenty two or three, to bring on more capacity in some of these situations because they were talking about how they got burned by oversupply and sort of fellow in COVID.

Right.

But what's interesting is if you go and look at some of the small suppliers's not many of them, but there's a couple of small memory makers that make them like less than five percent of the market, Wind Bond, Nanya or two of them. Nanya's in uh Well, headquartered at least in Taiwana of where their fab is, but they like these two companies as far as I'm aware.

I was just looking at this this morning. They increased their their production capabilities at the time when the big three we're saying that they weren't gonna do it because you know, they got burned, right. It's interesting to me because if you're this is the argument the lawsuit makes too. If you're a small company, I would think you're at the biggest disadvantage, the biggest risk to increase your production capabilities with this massive amount of spend. If your biggest competitors are saying like this, it looks like bad times ahead.

You know, we don't want to spend into this. But if you're a small guy, like you're at the greatest risk of going under. If you fuck up your your.

Cap back right? Why didn't that this is maybe this is a dumb question. Why didn't them increasing supply allow them to kind of on the cut the microns of the world.

Yeah, that one is just because the there's a micron does have a technology advantage, so they can make memory that these other guys can't. HBM is one of them. Maybe this is I didn't think about this till now, but maybe there is a very a small part, I think, but possibly part of an answer to your question of why HBM that is something that's that other companies are struggling to make successfully in current generations.

So it's well, I know, only like sk Heinex, Samsung and micro An not the ones who can make HBM.

Yeah, and I the CXMT has also struggled with HBM, and they're a new Chinese memory maker that's actually done really well with DDR five. And so there may be a little bit of a pulling the ladder up, you know, after them factor.

Well, here's the question as well, when it comes to high badmath memory, surely this is going to lead to an oversupply situation regardless because HBM is used in what video cards got that's such an anachrotistic term in the AI bubble, but like it's used in video cards and aigpus basically, right.

Yeah, it's very limited. There's almost there's effectively zero consumer use case.

So when the AI bubble bursts, where does the high bandwidth memory go? Oh? Yeah, I don't Oh okay, yeah, because I'm not.

Talking about the bubble burst in all the.

Bubble thing I saw, Samsung is talking about increasing their high bandwidth. They're spending like fifty something billion, five hundred billion dollars across an interterminate period, but like also building an entire new high bandwidth memory like factory, yeah and that. But I was looking I think I sent you the graph for coming on there was. I saw this chart where it's like they were making what a couple hundred million gigabytes of hbm is in twenty twenty two and they're now selling like seven or eight trillion a year.

Yeah, So what's crazy is the hbm bit demand I think I I think I first saw this on semi analysis if I remember correctly when about the sort of site. But the hbm bit demand from envi Idea was recently projected to exceed all HBM bit demand of all other companies combined from twenty twenty five good, And that's just n VideA. And and again you have more wafer area per bit than with d RAM consumed. That counts yield losses, right, and so like. Increasingly, as they make more and more of this, presumably for Envideo's devices, you will continue to see consumers really get shafted if there's not new supply online soon. And likewise, if they're focusing all their efforts and resources on HBM for whatever reason, who's to say

that the new fabs will even make consumer memory anyway? Right? Like there's they might just be more HBM.

Well, the thing is, if they're all built to make maybe I'm missing something. They're building this fact, these these fabs specifically for hype band with memory behind bad with memory is only used really for the thing in the AI bubble, right, What they're gonna do with all this capacity when it's gone?

They would switched it over it, they'd switched the line.

I mean they would. But it's like, I guess they'll just have aren't they just creating a future nightmare? I mean, it's always good to have more capacity for them I guess.

I yeah, I mean, I guess it's good that and more capacity. There's a lot of government money floating around to right, Like Micron got a five billion dollar tax break in New York as part of the the guess big beautiful bill, but the.

Five billion Like Microns said they're going to spend two hundred and fifty billion, right, Samsung and s K Heinez have made two point one trillion dollars worth of commitments. It just feels like it's it's almost like they saw what happened in twenty twenty one and twenty twenty three, which ended with all of them doing at least a billion dollars of write offs, and when they were like, fuck that, we can go bigger.

Yeah. Well, I mean I do think personally, I think that they are colluding again, I think they are price fixing. It might be through like a parallelism rather than a literal behind closed doors, meaning like we're just gonna announce our changes in earnings reports and then our friendly competitors will kind of mirror us, and so.

Yeah, yeah, like it's kind of a prisoner's dilemma, like if they don't raise prices, the other guy will. If they and if they don't build capacity, the other guy will, they won't get the long term deals.

And right now, yeah, and and also you know the the wafers themselves. I have some interesting numbers I wrote down, but I was looking up when you were asking about HBM. I went and pulled up some of the original coverage I did of HBM which started to come online or at least being actively developed somewhere around I don't know, twenty thirteen plus or minus a bit for AMD GPUs.

So yeah, when AMD it was working on Fiji and its Fury GPUs, they debuted HBM. Actually for consumer devices. They had HBM on Vega and on a couple other architectures in like the twenty fifteen to maybe twenty eighteen Area plus or minus a year or two and massively expensive, just like it is today. The power efficiency was great, which is really why they used it, although they were banned with starved too. But I pulled up I'd done some reporting at the time where I spoke to a bunch of the people in the know in the industry, and at the time twenty seventeen or so twenty sixteen area, the HBM two memory for one of the devices should have cost around one hundred and fifty dollars cost for them to buy, and the interposer in the packaging should

have been another twenty five dollars one seventy five. And this is these are you know, this is old technology. Now, I'm sure a lot different. But the point of bringing it up is that if you compare a sort of time for time and at the same capacity at the time, it would cost am D one hundred and seventy five bucks or so to put HBM hod its device, whereas with GDDR memory the non HBM stuff, they would have been at somewhere around like sixty eight dollars.

So they would be just so, I guess the thing is, they absolutely would make more money selling d RAM or sorry GDR like regular people ram. It's just that right now they have the business to kind of. It's not just shitty. It's not these bullshit video cards will lose the gamers. It's selling aigpus to big strong boys.

Yeah right, yes, like Sam Altman.

And yes, the big strong honky men like Dario Amma day right right, yes, yeah, he I've never by the way, I don't think i've ever seen someone spread so much fear so anxiously as him. It's it's just pathetic. What's Yeah. Also, here's a random thing as well for these a question to the listeners and to you as well, Steve, you ever see anyone on You've seen this weird habit of Silicon Valley people of using AI now to add muscles to guys.

This seen?

What's that about all of the weirdest things I've seen in the tech industry's history, And there's a lot of weird ship.

I don't know if it's just well, they're they're not used to working for intellectual property at this point. They just harvested, so why work for the muscles either.

Well, it's just it's really strange. Like I like, I think the TVPN, the technology, the Blowjup Brothers network, like they they they do it with like Dylan Patel, who's just like a regular sized guy, and it's just it's sorry, I've just realized that's weird. But then I saw someone do it for Dario Amedey the other day, and this is like, your body's your body. I don't judge people for for like how they look, but at the same time, Dario Amma day is not like a like fifth course of steroids. Guy, Like, it's just the weird it's the weirdest thing I've ever seen.

Did you see the story just recently of the copper thieves stealing it from data centers too?

No, that was good. Have a rich better offline listener there, Yep.

Yeah, they probably wearing one of your T shirts.

Just don't god no disabout them.

Yeah, the SAT there's a title of thieves stealing copper from data centers, and it's just like, oh, okay, I mean it's really I guess there's no honor among thieves, you know, it's just thieves stealing from thieves.

It's also inevitable at this point because you make these giant things full of earth metals and then you put them in neighborhoods usually with like like disadvantaged farm neighborhoods or like like distressed towns in Texas. You're like, oh, I wonder what will happen with the other business.

Yeah, I wonder what will happen when we screw over all these people and.

And like mock them via the media constantly. Anyway, back to back to ram, So, is this something that's going to start affecting like storage so hard.

Yeah, yeah, yeah, so you mean like.

Actual yeah, like nand like solid state drives and the like.

Yes, so it's already effecting it. Storage is made at the same fabs for the most part as memory, with basically the same tools and mostly by the same companies, although there's there's like one or two differences, but for the most part it's the same people and tools, and so definitely want affect the other anyway. But storage in general has hasn't gone up as violently as memory, but it has gone up. Some of the SSDs we had bought previously. I was just checking this actually for like ninety bucks maybe about a year ago, are now closer to like one hundred and eighty one hundred and ninety and Jesus Christ, yeah, I pull up a chart here.

That's just this is genuinely like I think, I don't know if I'd said do anything that's activism, but like I think people just need to know this is AI's fault. Yes, And the worst part is it's not even as fault because they need these data centers because they're taking forever to build. It's like you can access claude and chat GPT.

You don't. There's no one. No one's like unable to log on because there's not enough data sentences. Nothing's being stopped. They're just like, no, we need to do we need to buy buy the chips, need to build the data sense for some reason.

Yeah, it's yeah, there's not like a I mean, I guess they're maybe they make the argument of like we're trying to make it better and the models will be more complicated in that.

Require and what's not gonna happen.

But yeah, the storage though, So, looking at an average price chart for four terabytes of solid state drive storage VME on PC part Picker, which averages it, they have the pricing at around it looks like maybe about two hundred and seventy dollars in January last year, and now it's somewhere around somewhere around eight hundred and nine hundred dollars.

And that's for what kind of storage that is.

For four terabytes. Uh so relatively large capacity, which is going to be affected the most.

Jesus Christ.

Yeah, And this is where this is interesting too, because one of the ways you'll see consumers getting shafted that's not as obvious is some of the modern devices that should be your next sort of you know, improvement upgrade on past generations. They're actually coming down in capacity, So I haven't looked at the exact devices names for a while.

We ran a report a while ago talking about some of the phones and laptops that have shifted from say a prior model of twelve or sixteen gigaway it's down towards eight for memory, there's decreasing storage capacity as well.

Yeah, that's fucking tech. So everyone just suffers here, and it's pretty like there's Micron's, Skehihex Samsung. They could just they could have not done this, just to be clear, just so they could have sat with like seventy percent margins.

They're already colluding anyway, and my opinion, so yeah they could. They can clute it in a different way.

It's just it's so fucking straight. Well, I mean it's actually no, that's not that's not fair. This isn't strange at all. This is exactly like, this is exactly what companies like this do. Yeah, like this, it's this kind of vulgar kind of chancerism. And their argument, from what I understand, is well, memory is cyclical, so we need to fill our boots when we can.

And if you're calling something vulgar, then it's pretty vulgar.

Yes, just to be clear, as the vulgarest man in tech, and that is a word now. But the thing is, though, just to be clear, that's their argument though, because things are good now, they can charge what they want so they can fill their coffice for the future.

Yeah, I think so, and I mean you see it affecting. I think that's probably their argument. I think on the consumer side too, even a company like Lenovo, which is not a manufacturer of memory, but they buy it and they put it in stuff. I'm not sure if there may be this. I don't know for fact, but there may be a benefit for some of these companies as well for a few reasons. One is there's a lot fewer.

It kind of reminds me a little bit of COVID when all the stores that were twenty four seven, all the grocery stores and whatever suddenly stopped being twenty four to seven, and at least where I am not one of them has gone back. And I think that the memory capacity kind of equivalent here is this offers a little bit of a reset, and so with.

Lower those expectations exactly.

Yeah, and now your next upgrade is more exciting. And what I noticed with Lenovo when I was buying a laptop was they had quietly snapped out of existence their sixty four gigabyte model memory system memory of the device I was looking at, and they only had thirty two.

And I think, has the memory had sold it onto it as well?

It is, Yeah, you can't.

Oh so you just can't upgrade. You're just stuck. That's insane.

Yeah, And so I think what happens there for them, you know, in their favor, I might argue, well, they can't they're having trouble getting supply, just like everyone else. That's probably true. But against them, if they've got sixty four gigabytes of memory, because that's just one part of the total bomb for a laptop they sell, they might rather sell two people two laptops at thirty two gigabytes right where they can get the markup on the whole thing, right, rather than one guy a sixty four gigabyte laptop where they're only marking it up. Whatever the hopefully whatever the difference in the memory is. So I think you see some of that here too, where there's.

Well they're just going to use it as a chance to reset what people expect and just ohso prices are probably never coming down right itself.

Funny enough that I was using them in this example, just like two weeks ago at Is said that prices will probably the word they used was never come back down to what they were last year. And so yeah, I think.

They're truly evil era, just like a like a disgusting era full of horrible perverts. Because it's just it's so frustrating as well, because America has price fixing laws. Yeah, they we have price gouging laws, like we have means of doing this, we just don't do it because obviously of capitalism.

And they've been enacted, you know, like the DJ itself busted these same companies in the early two thousands.

Yeah, that was before. That's before we had social media for psychopaths to talk about stocks all day. Now we need to keep now, we need to We need just borderline corruption or real corruption in everything. That's the only way to get things done these days.

And I think also there's probably incentive to you know, if you're no administration of any party wants the economy to fail on their watch and I think if you're worried about and a bubble pop, and especially if maybe spooking investors with the DJ investigation or something, might you know, trigger part of that. It just seems like you wouldn't be incentivized to not probe too much.

But isn't this going to cause just already is causing hyper inflation within all electronics related products, because yeah, it seems that these companies are fully dedicate to the graveyard smash of making everything more expensive forever and at some point that will start crushing all conceimer electronics.

Yeah, I think it will, you know, for some things that can migrate more to software as a service and hardware as a service. I think there's an end game there for some of these companies where tell me more, I think if it's okay we crushed demand for the gaming GPUs market because filthy, poor casual gamers can't afford our big strong GPUs that we need for data centers, you know, I think if you crush that market, there is potentially an out in the form of cloud gaming, or maybe a better example where you don't have latency as an issue, would just be if you as a company, if you're worried about consumers getting systems that are powerful in their hands and doing using LLLMS at home or whatever.

If that is suddenly out of reach because the memory is too expensive and the GPS are too expensive, all right, I guess they come crawling back to claude or chat GPT. So I do think not everyone's going to be able to grab that parachute. But I think that moving services to cloud oriented services that you pay for every month is a way that they can capture the people, which is going to be most of them at some point, who can't afford to actually own the full device to do stuff cloud gaming. Cloud gaming sucks. But I also I don't think that's the only place where you know.

I think one of the real conception if anyone here is genuine about LM's and AIB in the future, I would think at least companies like open aye would be worried about that about lms getting too good if they're locally hosted for like, let's just say, the vast majority of basic tasks someone might want, and by by all the prices of hardware remaining elevated, it does kind of keep that at bay a little longer.

I just because I understand that, and you've said this before. The idea that they want to push people towards not owning anything. You'll own nothing and you'll love it, I think is the phrase. And but at the same time, do you think that that's actually their plan or do you think my version of this is much simpler, which is, do you think they have one? Or is this just incredible short terminus thinking.

Yeah, I don't think it's the plan. I think it's like the fallback for the future.

Google Stadia we all know is a huge success.

I contributed to the failure of that one. Yeah, that one.

There are ten people who must be so angry.

Oh, I'm sure there are.

Dozens of us G Force.

Now I had numbers at one point. I don't have them currently, but I had a lot of Actually, I think we talked about it on your show because I pulled up their website. Yeah, however many millions of users, and both of us were kind of surprised at it. But yeah, I think, you know, I think gaming is sort of the easy one to point to. I think for a lot of casual users they might be okay with cloud connected gaming, but I do think there's more services than just gaming, you know that could be sucked into the cloud. Even just stuff like QuickBooks has been the one I've been ranting about recently.

I'm surprised that that isn't already sassed.

Well, basically is because the desktop version, it used to be a buy at once for three hundred dollars, then I went to seven hundred, and then they went to fifteen hundred dollars annually for the desktop version, and you have to log in when you launch it, just like Adobe software. Oh good, And now they're killing desktop entirely and you have to go to cloud, which I actually I had just abandoned it and went to a free tool instead. But like the point being, yeah, I think a lot of these you know, these services they do just increasingly go cloud connected.

But the problem is is cloud connected services still require RAM. I mean, just anyone who's ever used an AI tool that's still in your browser, well, I mean, Chrome itself is a hog, as we all know. And so it's just kind of like it feels that there's just going to be a point that this escalates to the point that it kills large parts of the consumer electronics industry.

I mean, so Neaster laying the next PlayStation. Microsoft is increasing the cost of the next Xbox while firing all the people who know how it works. Just an evil company.

I mean, it's and a steam machine is a great example too.

I love that I just got when. I fucking love it?

Oh did you really? Wow?

I really love it. It's just what I But the thing is, it's also insanely expensive. Yeah, it's unbelievable.

Yeah, it's because the Steamer Shione. We had talked to them, and the quote they gave us was amazing when I'll paraphrase it for the best of my memory, but they were talking about how I asked them how did it work buying from the memory manufacturers, like did you get a contract price or how did you know? How did you guarantee your memory supply? And the answer was there are no contracts, And the valve guy told us, uh, they're offered a price by the memory supplier. In this case, the one we got was Heinex and if they say no to that price, then the quote was they never talked to us again, which is why I feel like l like behavior.

I got some I got some thoughts. I don't think I want to share on the record on that one. That is just that you know what I'm right back round to just I hope these companies supper, Like I think the memory companies are evil, Like that's just an evil way of doing business.

Yeah, I mean if it's coercion, right, Because it's like if if the player is I offer you a price, you say no, fine, I'll take my supply and self to someone. I'll never even talk to you again. Like you're now down to two companies you can buy from, right.

And it's not like and it's not and if they'll never talk to you again, it doesn't matter if they build more supply, right yeah, if you're on there Valve as well, who are like the like royalty within the PC. It's just this era is just about the companies hitting on the people that made them rich.

And it's you know, the crazy part to me is it's like the Valve is not a poor company. You know, they're a private so we don't know how much they have, but they are definitely a multi billion dollar company. And Valve is getting bullied by the trillion dollar companies.

Which are only the trillion dollar companies because of the stock speculation.

Right, Yeah, not like actually but yeah, it's just it's insane to me because it's of course they're saying thing Porster has no real power in the purchasing and they're almost a billion dollar market cap, not quite. I don't know if they've how far they've fallen, but they were at one point a billion dollar market cap, and at point being like, these aren't nobodies who want the memory, and they're struggling to get the memory suppliers to talk to them. They're like, they're serious players and they need a lot of it, you know, and the response is take it at this price or go fuck yourself, which is crazy with me.

I was reading about the long term agreements the ske Heinex is presenting and apparently they're considering removing price caps because apparently they do the long term agreements. Yeah, and then they say, okay, we like they won't go above this bound so that you don't just get Roger. They're just not doing that anymore.

I guess rodri it is. I don't think i've seen the rider.

Classic British class, British, beautiful, British phrase.

I was gonna say, like, that's when I took British literature and before I dropped out of college, is the last time I read that? Yeah, I think the the in the memory supply typically contracts, they do set a ceiling and then my understanding is the way the contract works if you're lucky enough to get one, is that you're offered the supply and you have to buy it or you have to pay a penalty.

Basically, yeah, take or pay right.

Yeah, Yeah, that's.

It just feels like the natural end of this is going to be horrifying. It's just going to be. It's going to be because they're going to be sitting on all this hype bad memory. And I was having a conversation friend of the Showcase Could Go yesterday and he was saying that, like, well, they'll sell it to someone.

And while I agree on that, in theory, we're talking about going from a few hundred million gigabytes of HBM to trillions of it. So someone is going to be left holding the bag. Maybe it won't be the memory companies, but there's a situation where Nvidia or someone else is just sitting on the stack of this crap.

Yeah, if they can't find devices to put it in and they oversupply, then it'll be the same type of apocalypse we've seen in the past, I mean twenty sixteen. Like typically though, this benefits consumers on the crash inside of the wave because you it becomes commoditizing and get it super cheap. I'm not sure what happens here because when we've already reset the expectations, like we were talking about, so there's always already been a regression in the technology that'll take a few years to kind of get back to where we were. But then also whatever crash happens next seems like it's not going to be isolated to the memory industry, right, it seems like it'll be uh like I sent you a photo during Computax of n

Videos advertisement in Taipei that said it all starts here. Do you remember what? You replied with, No, I do not. You replied with it parentheses next global financial crisis.

Yeah, it really is though, because it's like, think about it, if we have just this glut of high bandwidth memory, how many graphics cards do they realistically make? Because when I was saying hundreds of millions of gigabytes, that was back in twenty twenty two. So this was the demand for these things for every games console manufacturer, every PC manufacturer, everyone.

As of twenty twenty two, when the economy was doing a lot better, right and they only needed a few hundred million, we could be sitting on arguably the largest supply in memory history. Oh for like an for like, like, you can't unbuild that memory, right, you can't pass it out.

You can't really there's no way to. You would need an HBM device to put it on. There's no real consumer HBM devices these days, and it's not something you'd spin up fast. So yeah, if there's like too much HBM, you don't really have a lot you could do with it other than I guess sell it for cheaper or something.

But like, at the end of the day, you need something to attach it to. And I mean one thing to think about is part of those numbers, although I don't think all of it, but part of it is that the capacity per device has increased a lot for non consumer devices since twenty twenty two, So some of it is accounted for there where you just put more on a device, But then you look at consumer and like, somewhat notoriously, it has remained stuck at like eight gigabytes to maybe sixteen gigabytes for a consumer GPU, which eight is barely even enough to play modern games with any kind of reasonably you know, high texture settings, and so you don't have that multiply and effect in consumer. It's just in data center. Which all that to say, if

more supply capacity goes online, if they're able to produce more, but the data center demand slows down or the data center GPU demand slows then and they switch back to just consumer DRAM, they're still going to have too much because the consumer devices are they don't use that much memory. So I don't know, you finally spec them up maybe.

But but you'll just have all of this insane amounts of high bandwidth memory to go. We're just going to get graphics cards with one hundred and twenty eight gigabytes of high bandwidth RAM just.

Just because they need to put it somewhere, right, Yeah, it.

Could, we got to do something with it. We're just gonna have a laptops and you'll be able to open seventeen or eighteen Chrome tabs. All right, it's gonna be incredible. No, I think I wonder if there's also going to be a knock on effect of all of this on the used gaming market like they used PC market.

Well, that's a really common I see people asking about like, oh, surely this is gonna be good for secondhand in a few years when they retire all these devices or the bubble pops.

Or I just I mean just like right now, I mean just people won't be buying new devices.

Yeah, well, I mean on both of those right now. It means that if you go look up ArtX thirty series cards from I don't know, like shit, I think it might be four or five years ago now on eBay, some of them are like within twenty percent of what they sold for new, you know. So it's yeah, it's wrecking the used market because it's just like it's not you're paying close to the new price for something that's forty years old. So that's not good. And then on the other side of it, none of the data center hardware is useful in consumer there is no secondhand use case.

Yeah yeah, lpdd R five x is either on some form of server specific cam card or is soldered to the board, so you can't use that. And maybe in China they'll deslder it, you know, and solder it onto new sticks and sell Frankenstein thing but and then the GPUs it's HBM, so it's it's attached to the GPU. You can't just you can't just like take it off.

This is the thing as well, when it comes to like this post dot com bubble recovery story, this is I'm glad you brought that up, because it's like people like, oh, yeah, they'll just use it for something else, useful infrastructure. It's like, what useful infrastructure? What are we gonna do with all these gps? It's not you can't run a black well like a GB three hundred is what's a grace and too blackwell? You've got over a terabyte of different kinds of fucking rams on that of saying rams and what you're gonna do with that? It's not got it's not got DisplayPort. Yeah, it's not going to do anything. It's still it's built for one thing.

Right, maybe a local AI model or whatever, but anything reason.

You couldn't do that unless it was in a consumer hardware shell. Yeah, you assume you need like a dishwasher plug.

For it, or a washing machine plug even like yeah, probably need to forty vault Jesus Christ.

People really, people need to do some do some actually like reading about this because it's like they need to realize that this can't go anywhere where else like this just we're just going to have racks of these things and it's not like in the future they'll be cheaper to use.

I think that the But you know, if the dot Com era argument of infrastructure was dark fiber or something where the fibers in the ground, it's just it's not used because all the companies went belly up. Eventually they used it. Maybe that's your argument there. I really I agree with you. I don't think it works for this one, because the closest thing we have to an infrastructure gain maybe is power, except the problem of their not really bringing more online fast enough, and it's it's at least currently it's not helping the prices for the rest of us, Like almost everyone's had their power costs go up lately.

So maybe that would be the advantage if you're like, because of data centers, we brought back nuclear and we have shitloads of solar and wind and whatever. But that's not what we're doing. It's not like our minds on the roof, you know, And I don't know.

It's also what if the power doesn't what if the power gets built and the demand isn't there. It's just there's so many things that could go wrong here, and I think people just kind of I think the reason that this is all coming to a head is because we're finally facing the physical problems. Yeah, they were like, there's no so much capacity, there's only so much money, there's only so much power, there's only so much silicon.

At some point this all has to break, and it might break everything.

Yeah, I think I don't know what the I don't know what the Enron moment will be. You know, I'm not sure where it'll tip. But it does seem inevitable. But I don't know. It could be a year, it could be several years, right, Like, I have no idea.

It could just it feels like the moment the vibe shift, because the trigger I'm looking for is that kapek s pullback. But what scares me somewhat is the in video is the I think, what fifty five sixty five percent of all high bandwidth memory demand all for AIGPUS. That just feels like a single point of failure.

Yeah, I think and video, I mean and video itself. Jensen Juan himself said of core weave, that without and Video there is no core weave, you know, like, if I remember correctly that was, I'll go dig that up. But they, you know, they create the circle, and video is part of the circle, and it's Yeah, if anything happens to them, then all of it falters, and then they are a kinmaker and yeah.

But they're also going to be the king loser. I think by the end of this just it kind of it's just so fucking faster, I guess. Yeah, it's so fucking frustrating as well, because I feel like if we'd have slowed this down in twenty twenty four, we could have avoided this. But everyone was like, no, we need to go further and harder and faster and spend even bigger more amounts. But one final question, surely with the increase in high band with me, like, actually, maybe it's two questions in one. The memory companies are kind of have Nvidia over a barrel too, don't they.

Uh, in the sense that Nvidia doesn't really have alternatives. But and Video is also heavily involved in the R and D for all this new memory, so like they were involved in the R and D for GDR seven g five X and for the new HPM options and so like over a barrel, I feel like I think you could make that argument, but I also think that Nvidia is more likely pulling a lot of those strains despite being the customer to very few suppliers.

Right. But at the same time, doesn't this just mean that in Vidio's prices are going to get linearly more expensive the longer this goes on.

Yeah, I do think, I do think the prices go up. I mean, we're you know, with Nvidio on the consumer side too. You can look at that as a gauge. They were supposed to launched their refresh of consumer GPUs like a year ago or something almost and not quite a year ago at close and like rumors r end of twenty twenty five. Then it was first quarter twenty twenty six, and then the rumors where they were all canceled.

And the reason for that is because this particular type of refresh they call the Super series. Normally you see an increase in memory capacity at roughly the same price, where you see a decrease in price at the same capacity, and right now, neither of those things as possible, and so Nvidia hasn't shipped the card. It may come out at some point, but for now they've delayed that indefinitely.

And so to your point, it does start to self cannibalize, right where like they are the reason that their own refresh for consumer can't come out.

And also but regular the cost of the things that were already making the current generation will go up.

Yeah. Oh, the current generation has already gone up in terms of like real world pricing, and officially they haven't changed their MSRP from launch, you know, but it's I mean a lot of this stuff, the fifty ninety is I guess technically sometimes you can get them at MSRP, but more often they're like two to.

I mean, like the aigpus Oh yeah, I mean that I would just like, yeah, all of them will go. It just feels at some point the money's gonna run out, and it's the longer it goes, the worst it will get.

Well, and then you get the weird lease back stuff, right like with Meta and the GP.

Oh yeah, and all the SPV share. What a horrible era, truly awful, And I think that's a that's a great place to end it for us. Steve where can people find you what you're working on at the moment. I know you have some sort of campaign campaign, a dystopian campaign you want.

To Yeah, we do have our AI Dystopia coverage. We're doing it. We have a we're trying something new. We do these documentaries that are like three hours long, and we're trying something new and doing a short one. So we're doing two three minute ones coming up. And one of them is on the hard Drive cartel, which is pretty interesting, nice, and the other one is on this memory class action lawsuit. We read through all one hundred and eighteen pages of the filing and did a bunch of history research. So those are coming up within the next week on gamers Nexus, and.

We'll get the links in there. I don't know what I'm going to be doing next week on the show because I'm recording this a little early. I'll have him on log for you on Friday, though. I have been at Zutra on this is Better Offline. You know where to find all my stuff will be in the links. Thank you so much for listening. Thank you for listening to Better Offline.

The editor and composer of the Better Offline theme song is Matasowski. You can check out more of his music and audio projects at Mattasowski dot com m.

A T T O.

S O w Ski dot com.

You can email me at easy at betrofline dot com or visit Better Offline dot com to find more podcast links and of course, my newsletter. I also really recommend you go to chat dot Where's youread dot at to visit the discord, and go to our slash.

Better Offline to check out I'll Reddit. Thank you so much for listening. Better Offline is a production of cool Zone Media. For more from cool Zone Media, visit our website cool Zonemedia dot com, or check us out on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.

AI Made Everything Expensive (ft. GamersNexus)

Kapitelklicken springt direkt hin

Shownotes

Transkriptvom Podcast bereitgestellt · Zeitmarken springen hin