Casey Destroys Optimization Myths

0:00 If you guys do not know, chat, chat, everybody in here, Casey, trash, the startup is going through a bit of a change, all right, there's been some investment interest and we are now raising a series A like any proper multi-billion dollar unicorn startup. And so that's a deck of corn, deck of corn, really, we're a deck of corn.

0:20 And so I don't know what that means yet, but someone told me it's like that's the best kind of unicorn. I think it means it has multiple horns. I don't know what it means. 10 horns. 10 horns. 10 horns. Okay. Yeah. Okay. So it's kind of like the beast in the Bible, the devil, right? Very good. Doesn't have 10.

0:37 That's it. Yeah, yeah, yeah, yeah, yeah. Yeah. Yeah. Yeah. Anyway, sorry. Back checkers. Back checkers. And so what the, I actually thought TJ was going to fact check me, but he fact checker me. Correct. It didn't seem 10, but it could be. It could be. I know. I know. I'm doing on here, guys. Hold on.

0:59 Well, that is kind of what they would, what it answers, you know. Anyway. All right. So he has 10 horns. Called it. Okay. Primary reference. I got it. Okay. Okay. That was correct on that one. Okay. Okay. Good. Referenced. All right. So with that in mind, we're trusting AI for theology. All right. Trust.

1:20 Let it happen. Everybody. Okay. My gosh. All right. All of that said that means as part of our amazing content. I have now organized us an amazing linear board, which will help us walk through a more organized standup. Because if there's one thing I know about raising a series A is they want to see agile.

1:42 Okay. I'm pretty sure that's correct still in today's day and age. So we're getting really agilized. No, it's actually going to be, it's going to make us hopefully have a more organized and amazing standup so that the world's most attended standup, which is our current standup right now, right now, just in case you're wondering, you know, you may not know this casey.

1:59 There's, there is a, there's a 2,500 people on the standup right now. This is the world's most attended standup. It's like you couldn't even, like it, we previously, no one knew if that many people could even stand at the same time. We've just shattered that. We shattered that. We shattered that. Exactly.

2:15 Okay. Definitely. What was the record for like 100 people? Maybe. Yeah. It was like 100. I don't know. I don't know. It's a game changer. Yeah. I want to add though, we didn't get to invite or the topics until this morning, like, no, yeah. So this, this, this standup is going to be amazing because, because Prime's like, uh, we're going to talk about this new, well, you know, I won't spoiler it, but let's point it, put it this way.

2:38 It probably was a topic that would have been nice to research before and if we're going to talk about it. That's all I'm going to say. I don't want to spoil the gym. Let's think of it. It would have been nice. It was nice. Okay. That's all. Okay. You can react to trashes workout gym knowledge. Okay. Rather than the trash, you get trash.

2:58 I am not the expert. Trash facts. Trash facts. Dude, if there was a trash fact every episode that would be, that would be fire. It's literally going on the linear board. Yeah. I literally have endless lore. I'm sure that I would like to ask at this point is, if we are serious about this series A, which I think it should be.

3:21 Is it time for us to start putting together a pitch deck? Oh, wow. I still don't know what one of those are, but I think we need to do that. I think you're just so excited to do it. We need some slides. My startup dailybjj.ai is going nuclear. Congratulations on product market fit. Sorry, I called you the differ.

3:40 I knew you weren't the differ. I always thought it was the intern with the weird glasses. I should have seen this code. But anyways, if you want to get in the ground floor of this thing, you take an angel of estimates. So you finally cracked the case merge cop. You put the white space in the diff. You made the synthetic traffic.

4:24 You made me approve that PR. You're the diff learned, I always knew it. You've merged your last PR diff learned. I already clicked to merge on my PR 35 seconds ago. I merged this block. Curse you, gold rabbits. You saved the city from a code review related crime. Thanks, commas. Well, technically it was code rabbit that saved the day.

4:55 Not only do they have advanced AI features that can detect security vulnerabilities, like what the Dithler was trying to merge. They also have ways to enforce styling, linting, and a variety of other tools. And so you can stop wasting time reviewing code that humans don't need to review. You can try it yourself at coderabbit.ai.

5:15 So merge cop, I would say it's a little misleading to say you did it all by yourself. Huh. Would you like some cake? Oh, sure. Thanks. Come on. Merge cop. We need slides that say like, that say like, here's our total addressable market, which is every human on earth. That the market is like everybody.

5:37 People in like uncontacted tribes in like indigenous places will be using the standup somehow. They'll be standing up. Because cases, you know who our market is, right? People who are affected by AI. That's everybody. Which is everybody. That's earth. They're done. Is the tam. There you go. Tam infinite.

5:58 Is the tam for next. Aliens on another planet, like they're going to be getting the signals of this standup beamed into space. They're going to, they're part of the market. They're being advertised. When they show up to contact earth, they will buy your product if it is advertised on the standup, ladies and gentlemen.

6:14 True. What else do you need to know for whatever's on the standup? That's right. Trash. I'd like to see the answer. The aliens will be like, ooh, I wonder what that tastes like. Great point. Great point. I'd love to see next week maybe we each bring a pitch deck for what we believe the standup pitch deck would work.

6:31 Oh, let's do that. I don't even know what to do. It's going to be so good. Let's each have a pitch deck. Each present ours next week then. Yep. Well, what would your pitch for the standup getting a sponsor or like for your own personal sponsor? No, this is for the standup. The standup is probably a trillion dollars long term.

6:52 A trillion dollars long term, I'd say, just to be conservative. I don't want to, I don't want to sound too aggressive, but the pitch deck for why it's worth a trillion dollars long term. Yes, I like it. We are a deck of corn. All right. So with that in mind, I figured we'd do our first task here, which is trash fact.

7:08 Trash, can you hit us with the trash fact really quickly? Yeah, I got to think about this now. What you said you got an endless supply of corn. Yeah, it's true. I tried to become a pro bowler and my average was like 203. What? What? How is it every time something I never could have imagined? Trash. That is a good fact.

7:29 I bowed for like 10 years and I went super hard. And I had to stop because I rock climbed a lot. So I was like, you know, rock climbing and bowling. Too much. Competing. Yeah. But I used to do like cash leagues after work and would go play for money. It was fun. You were a bowling hustler. I wasn't a hustler like I competed like in leagues or whatever, but I had a really good average.

7:53 So I thought I could really like make it, make it pretty far. So I would be just like a professional average. That's good. I think that's a professional average. I think like 190 plus maybe is pretty good. Really? I think my highest score is like 279. So I was like one strike away from a 300. Oh. You know?

8:13 True story. I grew up in Massachusetts. And so I like the kind of bowling that everyone talks about when they say bowling is like giant, giant ass ball with like the three holes in it, right? Yes. You stick your fingers in, OK? Yeah. Never seen that. We didn't have that. We had candle pin bowling because we're near Rhode Island.

8:35 And so it's this little tiny ass ball. Oh, that's like that big. Duck bowling. It's not duck pin. Duck pin's a different kind of bowling. It's basically 10 pin. Well, it's not even 10 pin builds. It's long, thin, fairly heavy pins. And a tiny ass ball. There's no way you have a 200 average unless like the best bowl in the world.

8:56 It's like really hard because the ball is super tiny. And it's pretty hard to get a strike. Like a good game is, you know, I don't know, three strikes or something like that. Whereas if in real bowling, you know, or a 10 pin bowling, you'd be like, oh, three strikes, OK, like you're probably not, that was your an amateur or something, right?

9:14 OK, I never heard of this game. I've literally never heard of some alternative bowling game. Then duck pin bowling is even more narrow. I think that's like Vancouver and Rhode Island are like, and like that's it. I don't even know who does duck pin. Not many people. Oh, were it a bowling or a Vigenade then?

9:30 Did it not make its way up there? I don't know. I don't know. It's got, but like in my town, there was only one bowling alley there. And there was, and it was candle pin. And there weren't, I don't even know where the next bowling alley would have been. Oh, are you like a small town? Are you from like a small town?

9:46 Very small. Yeah. I think at the time there were 2,400 pieces in the town. That's why the pins are smaller. Oh, that's small. OK, that's actually small. You're from a real small place. OK, I didn't realize that. Yeah, yeah, very, I mean, it's not, there are tinier towns, but yeah, 2,400 houses, I think, in it, when I was growing up.

10:02 That was a population. Well, when I was, oh, and when I lived there, we, I lived to see the first actual stoplight put in. Like a stop, like a thing that could control. Before that, we had only, we just had this hanging light that would just blink at the, at the one main intersection. Dude, that's right.

10:19 Are you from Radiator Springs? Yeah. Oh. T.J.? Dude, Radiator Springs. Dude, you just have the bridge in your light. That's not for you, buddy. I think it's longer. OK. Wow. But anyway, moving on. So that was an amazing trash fact. And now I'm just like, what don't you do, trash? Like, is there a thing you don't do?

10:42 You're like, you do, you do that for next week. We have to do that for next week. We have to do that for next week. I will, I will share more as the episode goes on. OK, good. But I actually like, I have like a weird person out, like as soon as I get interested in something, I take it as far as I can go, get bored, and then I find the next thing that I take as far.

10:57 So I just have like accumulated many random skills. This is great. You know, the thing is, Trash, is I've also done that. Except for I've done it on much more useless talents, such as yo-yoing. I used to be a professional grade yo-yoer at one point. And so one time I went to a conference where it was called the Server Challenge, and I forget IBM, I believe.

11:17 We had the plug-in 18 ethernet cables and plug-in 12 hard drives. And I was the fastest person at a 3,000-person conference and got my wife a MacBook Air for winning it. And it turns out like, they're like, hey, there's a useless talent award. And I'm like, oh, that's me. I'm that guy. You give me, oh, I was plugging in six ethernet cables at a time, and they're just like, this is a revolutionary status going on, so it was pretty great.

11:42 All right, but hey, we can just continue on. Because Trash, I don't know if you know this, but it's also your snack picks. OK, you dominate the intros around here. Well, one on eating right now is the Pipcorn Spicy Nachos. And I got my favorite red yummerth bag with me. This is my current duo. I've already went through four of these bags this week, which is terrible.

12:03 I have no excuse, so I've been working out. So I work out in the morning and the afternoon. So just to try and make up for the yummers. Just to like, to make up for my sins that I'm doing. I feel so bad. And I make sure I risk my mouth out with water and all that stuff. And I show which one I told you to just clean the breath.

12:24 Well, that's the way I tend to say. He said, don't listen to your teeth. It's just rinse your water and that's a good thing. You talk to your dentist about it, you're saying? Well, fun fact, almost all the teeth in my mouth are like, basically fake. I could never have guessed. We actually talked about that already.

12:41 That was a different fun fact. I'm all right, dude. All right. We should honestly trash that's fantastic. TJ, do you have any UUG jump in with a little bit of snack? I don't really have anything to do. I'm drinking a little bit of cinnamon spice tea because my nose hurts really bad because yesterday I was playing basketball at our friend's kid's birthday party.

13:05 And then one of the kids just ran directly into my nose. And I had a enormous nose bleed for like the rest of the evening. The kids did not attempt to steal the ball. They just lowered their head and ran straight into my nostrils. Like a sheep. Yeah. What was the plan? What was like, were they trying to do some weird form of a box out that we haven't seen?

13:31 Some new, some new revenue theory. Well, it was mean versus like probably seven or eight children. And then I was like, I would pretty little like between five or eight. TJ, if you're one of them. Yeah, really sure to guess. Well, you get low to do good basketball moves. OK, OK, yeah, that's a good point.

13:52 So I'm spinning around and I'm dribbling like a maniac. And then one of the kids just like wasn't, it just was excited. I don't know. And then ran right into my face and hurt pretty bad. Well, isn't this, I mean, this could be a valid tactic in basketball, right? Like you just sacrifice, you know, you take the technical.

14:09 Number one. And you just, yeah, they tried it in karate, kid. Yeah. Yeah, yeah, there you go. And then you just, you know, since you're not only the star player of the other team, you're the only player of the other team taking you out would be very valuable for their win chances. Yeah, win by default.

14:28 That's a good move, honestly. Yeah. So I don't have any snacks right now. Sorry. I'll turn to this point. That's OK. All right, well, let's move on. By the way, I've been using this also for my stream. It's shocking how much more organized you are if you list everything out. It's like honest, it's, it's, it's, it's, it's crazy.

14:47 So Casey, what'd you do as a software engineer? Talk about Casey. I kept it all my head because Giro was the worst experience in my universe. And so I was just like, I would rather forget what I am doing than ever use that prop. Like I'd rather have to come to a meeting and explain to my boss why I didn't do something, then use that.

15:06 OK. That's a valid excuse. It's a valid excuse. And you know, you're at Netflix. You know exactly the experience. All right, so Casey, I saw this tweet and I figured you could maybe give me like a little, I think I understand the purpose here and what the problem is of it. But I thought maybe you could help me a little bit on this one.

15:25 This is my blocker, by the way, for the day. OK. I saw this where someone said, hey, don't use division. Use reciprocal math instead. OK. Can you, can you, can you, can you maybe give us like us dumb people why this is either good or bad idea? Oh boy. I have no idea what that means. So right here, he's doing array index i divided by pi versus array index i multiplied by the reciprocal.

15:54 So one divided by pi. So he does the division thing once and then hits it with a bunch of multiplications. Yeah. So there's a lot of things that you would want to talk about here. And that's, I actually saw that tweet that you're talking about. Yes. I don't want to delete your account. No, I mean, see, here's the, here's the problem, right?

16:16 So delete your account. I love delete your account. And I love what Ryan's doing there. But what mostly what he's doing there is he's just like, and he even described to someone of his own streams, he's just saying like, look, basically like you guys are all the people who like come up and beg for change on the street or something.

16:36 Like you're just totally making the world difficult to live in by like trying to get all it like just, just hounding everyone who goes into a particular area to try and get them to give you something, right? Which is your attention, whatever. They're like these AI shell accounts or whatever. People trying to get engagement.

16:53 They're trying to get follows, whatever it is, right? And it just really degrades the experience of being in that place. That's basically what he said on like one of his streams. I'm probably phrasing it poorly. I think he's, I think he said the phrase. If you go to a third world country like France, was literally I think the phrase he used.

17:15 I love Ryan. In that stream. So I don't know, don't let me, don't come after me for this. I'm just repeating what I believe it was. But anyway, the point is just that that was sort of the point of delete your account. And I'm, he's right. Like there's just all of this kind of really bad behavior that's going on there.

17:35 And this tweet, I don't really like being harsh on people who are trying to talk about performance because I'm glad that we somehow went over the past 10 years or so to people actually starting to think that it matters that you're thinking about how the CPU is going to do things. So I don't want to be mean to anyone who's at least trying to say like, hey, let's think about what this is doing.

18:00 The problem is that it requires some actual knowledge to actually get these things right. And a lot of times if you just ask the AI or if you just read something on a blog and you then repeat that, it's not accurate, right? Because performance is a very specific thing. It's kind of like repairing a car or something.

18:18 You can't just know like one thing about like, oh, yeah, you know, the carburetor or something, right? It's like, you have to actually know how the whole thing you want to know, right? Carburetor. That's what car is short. No, me exactly, like I'm using that as an example. Like I just be like, oh, the spark plug or something, right?

18:35 You know, it's like, I don't even really know much beyond like there's a thing that ignites the gas, right? Though, okay, that's it. I couldn't tell you like, is it broken, is it working properly, does it need to be tuned? What's going on, right? Park plugs are for electric cars, right? Because you get it from all the batteries.

18:48 I mean, that's there you go. That's what's to mean. Must be, I mean, clearly. So yeah, this particular tweet is not good though because it misleads people, I think, on a lot of points. I'm not sure if the person actually tested this, they've got like seconds listed there, which is a little weird because I don't, those numbers, I mean, there's a lot of things that could have happened there, I don't know.

19:14 So I don't know the circumstances of it, but I just don't want people to be misled. When you're looking at a loop like this, both of those loops that are in that tweet, because I remember this. Of course, I can't read what's on prime screen because Riverside won't let me, but I remember it fairly well.

19:30 You're basically going, you're going through arrays, and it's basically a read, modify, write, right? It's loading a value out of array. The value is a floating point number, loading a floating point value out of an array. It's modifying that floating point number and writing it back to the array, right?

19:48 And the operation that's being done on the array, the only difference between the loops is one of them is doing a divide. So it's loading, dividing, and rewriting. And the other one is loading, multiplying and rewriting. Correct. Now, obviously in math, if you do a divide of a number, if you say I'm going to divide by x, there's no difference between that and saying, I'm going to multiply by one over x, right?

20:20 True. That's true in math, meaning in grade school, you write it down. If someone said, oh, you know, they divided by, what's wrong? Okay. All right. T.T. was a math major. So, okay, he said he didn't know of these things. You're a math major, get out of here. Yeah, T.E.G. This is like in junior high, right?

20:42 You really are a professor, right? Yeah, yeah, totally junior high, yep. It's just totally. Come on, guys. Seriously, like I am not making the math. I am not making the math if I know this, you know this. Come on. That's true. This is true and straightforward, yes. This is just something people did in grade school, right?

21:01 And so, the idea this was certainly true in the old days for computing, right, is look, if I have floating point, right, if I have the ability to use floating point numbers, integers, obviously if we're using integers, then you have to do a lot of much more crazy stuff if you wanted to divide instead, I mean, multiply instead of divide.

21:22 Go into that, because that's a whole other ball game. But if assuming you're in floating point, then that means that you could literally just store the number one over x, whatever you're gonna divide by, and then multiply by it instead. So if you're going to do a bunch of multiplies, this is potentially attractive option, if dividing is slow, right?

21:41 So if we look at divide, we go the dividing is too slow, we could just do it once, and then multiply if the multiply is faster. Now, it's worth noting that these are not necessarily the same operation, because in floating point, right, an actual divide, this divided by that, right, may produce a different answer, because you don't know like when you do an inversion like that, you're only capturing part of the answer, like the answer might have gone on for quite some time, and you're only getting a smaller part of it.

22:11 So you're not doing infinite precision math, and so you also have to be aware of the fact that this will change the answer. So you have to know, right, if that mattered to you or not, when you're doing this. So if it's just some random thing you're doing for some UI element or something like that, then it probably will never matter.

22:29 If this is scientific computing, it could be a very big difference. And so this is the first thing that I want to point out is that this is not a thing you can just tell somebody, always invert your numbers, and then multiply, because that may be very bad advice in certain circumstances. That's thing one that wasn't mentioned in the tweet that you have to understand.

22:48 The butter and live act. Yeah, well, it's basically like the water going down is hand effect. It might even just be the 1,000 mile butterfly, like it's this huge butterfly, because it could literally, it could just be just literally the answer you get right here could be wrong enough that you care, right?

23:08 Depending on the circumstances, and depending on what those input numbers were, and what their scales were, and all this stuff, right? Who does that? Just, there is a decent chance of few people in the audience. Don't understand why that happens on a computer. Well, so to be fair, I'm the wrong one to explain it because I don't work on scientific computing, right?

23:30 And people who work on scientific computing are much better at analyzing things like, you know, what's called ULP, or basically like the last, what happens to the final bits in floating point numbers and things like that. But in general, when you're working with floating point on a computer, what's actually happening there is that you're taking the size of the storage that you made for those numbers.

23:54 So let's say that you're doing 32-bit floating point numbers in scientific computing, it might be 64-bit numbers, whatever it is. If you just have 32 bits, you only have a certain amount of space to store what you would prefer was an infinite series. Like, let's say we want to compute with pi, right?

24:12 Everyone knows from grade school, pi is this infinitely long series. Like, there is no storage that can store pi. It just keeps going. So we have to approximate that, right? By putting it into some format and all the numbers that we work with in floating point computing, they have this kind of format.

24:29 Just something simple. Let's interrupt you for a quick second. Please. Trash, when he says pi, he's not talking about a snack. Just in case you're wondering. I don't even like pi actually. I actually like pi. Me neither. Trash, yes. There's another reason Trash is the star of the standup. It's because it's no pi zone.

24:43 Yeah, that's right. Yeah, what's your favorite pi, a Jason kind of thing? Like the cake or brownies or like what is it then? Loads brownies. But pi is not a Jason to cake and you'll take that back. We know we just got to move on because that was very offensive. OK, that's it. We'll put that on next episode.

24:58 We're going to get in the backlog. How it makes it, how a Jason is take the pi. OK, so anyway, if you look at what happens to these things, during all your computations, every computation that you do is going to have some infinitely precise answer that's the answer you actually wanted and that you would have wanted to carry through all of your subsequent operations.

25:20 But at every step of the way, the computer has to like round it basically to whatever will fit back into that 32-bit encoding. And that 32-bit encoding is basically, you know, it's a sign bit that says whether things positive or negative. It's some number of bits for the exponent, right? Which says. I think is the sign.

25:42 The mantis is the next thing. Oh, the next thing, that's right, that's right. The exponent basically says what the scale is, like how big roughly is this number? And then the mantis are the bits that actually say like what the number is. So the reason it's called floating point is because it's broken up into these pieces.

25:59 Floating point, the point is the decimal point, if you will. The floating part is that we store an exponent. So we say where the decimal point is, and then the mantissa is just the actual digits. So we're basically floating the point to where it needs to go using the exponent, right? And the reason we do this is because we're trying to compute with extremely large and extremely small and everything in between.

26:24 And so we can't just take 32 bits, which has a very small range if we just use an integer, right? If we fixed point, if we just said the decimal point is here, it would kind of suck. That's what floating points for it's why we use it for everything that isn't just nice, you know, typical whole number kinds of stuff that we're doing, right?

26:41 You know, usually when we use just insert things like that, just integer numbers. The common one that people laugh at, right, is what 0.2 plus 0.1 in JavaScript and you get the 0.2, 999, right? So it's like, this is why that happens though. So just, I feel like there's probably a decent amount of people who are like just ha ha JavaScript stupid.

27:03 I mean, you're right, but like that's not, that's like a thing of the wrong reason JavaScript problem. Yeah, that works in Java, it works in all of them. Yeah. Yeah, JavaScript is weird in that it kind of only has double precision arithmetic period. Like it's just like, that's just what we do everywhere, I think, just kind of.

27:22 Unless he's bitwise operations, then it converts to a signed 32 bit number. No, just magically at that time for that one moment, you can actually do a overflow at 32 bits. So only to use a bitwise stuff. Again, this is always why JavaScript compilers are so hard, because they have to go like, oh, I don't, you don't want to do double precision arithmetic for like your loops and like counter and things like that.

27:44 So it has to analyze and go, OK, I can turn this into an integer for this code, right? It's got to do that work. Normal language is, it's already knows, right? It's just been told like this integers is not the think about that, right? But anyway, so that's the floating point math thing. And that's sort of what this snippet was trying to say is just invert the floating point number so you only pay for the divide once, then multiply by it, because that'll give you the same answer.

28:10 Again, the tweet didn't say, no, it's not quite the same answer. And it might matter depending on what you're doing and depending on what the inputs are. But anyway, the other problem I have with this is it misrepresents the performance as well for a number of reasons. So the first reason it misrepresents the performance is because it has some weird statements about how long a division is.

28:31 So it says something like 20 to 40 cycles for a divide or something like that, I think. Yes, 20 to 40. And I just have no idea where that's coming from, right? I have no idea where they got that number from. I really wish it said where it got that number from, right? Came from his heart, all right? It came from his heart.

28:49 Yeah. You could feel it. You could feel those cycles. CPU whisperer. So a floating point divide, you know, if you look, so if you look at a modern chip, like let's say a Zen4 or Zen5 chip, right? Something that you might be running today that might be in a computer you'd buy today. Typically, their floating point divide units are extraordinary, very fast.

29:16 I want to say that they can complete these things in, I don't know, five, six cycles, something like five cycles. Maybe I'm not sure what it is. We should probably look it up, but 40 is nowhere to be found. I mean, that's, it's not quite off by 10, but it's something like 10, like it's very, very wrong.

29:35 I think it's from a chat shippity, because if I'm not mistaken, I actually asked this question not too long ago, and I got the answer of like eight to 30 or something like that, cycles for a divide depending on the CPU and architecture. I mean, the part, the part that that chat GPT got right is the defending on the architecture part, because it does, okay, okay.

29:55 But like, let's just, here, I'll just look it up on my machine right now. Like, we'll just see what roughly it is. So if I go in and I say, I want to do a, you know, a 32 bit, we're going to do a div PS here or a div SS, let's say, and we look at what's going to happen on his N5 chip, the total latency, right, is less than 10 cycles for that operation, latency, right?

30:20 So that means that if you were to apply then, sorry, what, what about for multiply? Multiply will be four. Okay. I want to say, right? So three, three on the FPS though. 10.5. We just need to know how many FPS is 10 cycles. Right. If it were bouncing ball, which I'll pass that ball. So three on that, and I think four is on my CPU, which is Zen4, I think it should be four, right?

30:47 Is that correct, Zen4? Also three. So, all right, this is better than I would have expected. I'm used to four, these get three, right? Now, so in general, that is the number that you would use if you were trying to analyze how long it takes you to get the answer back, right? These loops don't care how long it takes to get the answer back because nothing is waiting for that, right?

31:16 On the iteration of the loop, as you go through, right? You're going to do a load, the floating point up, either divide or multiply, and then the store, right? When you do that, the next iteration of the loop doesn't have to wait for that to complete. So those are just in the scheduling queue going. They can be dispatched.

31:42 The next iteration of the loop is already decoded and probably also in the scheduler, given this is a hot loop, presumably, we're just running without, you know. It's good level of that. It's good looking, for sure. It's just cooking. So these are all going to pile up in the scheduler, it's going to dispatch them every cycle as it can.

31:59 And so what we care about is not how long it takes to get the answer because no one cares. What we care about is how long will it take us to issue the next one? So if we issue them on this cycle, how long does it take us to issue the next one, right? And so that's what we typically call, that's a throughput number, so we typically call that, right?

32:19 And if you look at the throughput numbers for multiply, right, typically you can issue two floating point multiplies per cycle, which means it takes half a cycle effectively to multiply, half a cycle, okay? That is the number you should be listing or thinking about when you're looking at this loop. For divides, the throughput again, is remarkably good on modern CPUs, so it's like in the two to three range.

32:50 So is multiply faster than divide, sure, right? Half a cycle versus three cycles when we're analyzing this loop, let's say, or something along those lines. But then you have to ask the question, do you care? This is a load, a op, and a store. Unless you're entirely out of the L1 cache, which is going to service, you know, at about, you know, like let's say, one, you know, it's probably like two per cycle, three per cycle, it could get you answers back from the L1 cache, right?

33:27 If you're talking about the L2 cache, then I mean, I don't know, the number of clocks it's probably gonna take 14 cycle latency, something like this. If we're looking at how long it takes you to actually get the memory back, it's not clear to me that unless the, unless the CPU was predicting extremely well what you were going to do and always had things, like you'd have to go look at the actual bandwidth to see if you were actually gonna get enough back such that you could operate them on them at that speed.

33:56 Maybe you can, it's not in my brain right now as to whether you would actually be able to do that at a speed that would matter at the like, let's say three cycles, throughput number that you're gonna get on the divide. So it really just, it was very misleading the whole thing and as I, you know, you heard me say all that stuff.

34:18 I would wanna go look at this loop and check it first and I'd wanna think about it a little bit before I would post anything like that and it was just that clearly wasn't done because none of those numbers make sense that are placed there. Is it true that multiply is faster than divide? Yes, doesn't even matter in this case, no, I don't think it probably does, but maybe.

34:39 And furthermore, usually in most cases, right? When you're looking at actual loops, more is going on. And so it's also gonna mislead people. So point number three, it's also gonna mislead people in this final way which is to think that they always have to do this. It's that this is the problem with these kind of like out of context performance things.

35:00 People read it that think, oh, I just need to invert all my numbers and then my code goes fast and then they go around inverting all of their numbers and turning them into multiplies. And the problem with that is in a lot of cases, unless you had serial dependency chains that really could have been unblocked by this change, you're just wasting your time.

35:19 The divide would have been just as fast. It would have overlap with other things you're doing in the loop. You were, the thing was waiting on an uncast read or something so the whole loop was waiting 80 cycles anyway. So none of this mattered, right? So the problem again is that like performance is a thing, performance is a process.

35:38 It's an understanding. You have to understand all those things that I just said. They're not actually that hard to understand. You just, you can't, you just need to learn them. You need to spend a few weeks learning this stuff. And then you can think about, do I need to invert this number? Do I not need to invert it at that point?

35:52 What you don't wanna do is try to package it into a tweet like that when you haven't really done the work of understanding why it matters or anything like that because other people will take away the wrong lessons. So that was a very long winded way of trying to explain everything I didn't like about that tweet.

36:05 Again, no offense, sorry to the person who posted it. It's just, I don't think that kind of thing is helpful. It's sort of like saying always call memset or never call memset or these other weird performances that people pass around, which out of context don't help anybody because you're just gonna make incorrect decisions if you think that the key to performance is always do X or never do X.

36:25 That's not how it works. Spark plugs. Great job, Casey. Thank you. I did my best. I figured you'd be able to help me on that one because I had the rough idea that I would introduce air but I had no idea to the extent that divide and multiply may not make any sort of actual real difference in your program.

36:44 Plus all the crappy things I do anyways, probably destroy my program's performance long before we get to this loop. Yeah, but I'm, we've got other issues before we get to whether we're multiplying or dividing, I think yeah. I just really wanna take my Lua program seriously, TJ, okay? It's usually, the one thing I can put in someone's head that's generally not wrong is don't confuse integer multiply with floating point multiply.

37:08 Floating point multiplies are much faster than integer multiplies typically. And so some of the time when you see people saying, oh my god, like divide is so bad, sometimes they're thinking about integer divide and there's a lot of reasons, there are a lot of reasons why integer divide is bad, a lot of reasons.

37:26 I'm gonna put that on the block list for next week. You can, we can talk about that because that's actually a, that seems very interesting. Okay. Is it on the block? It's on the block list. The pregnant pause. Yeah, I was like, sorry, I was, I was just putting it on. I thought in case you had finished anything.

37:42 Now, track, hold on, can I just jump in for a second? Can I jump in here? I'm gonna say that this instruction thing is done. Trash, I had a, TJ said you had a blocker that you need to discuss. I don't know what it is. We're a blocker trash that we need to get out to the world. What is this blocker that you had to talk about?

37:57 Oh, dude, I'm locked out of my GoDaddy account that I set up like a decade ago. And I got hit with like a renewal, whatever. And I try to get it. And apparently it's using like my college email which got the activated God knows when. So then I hit up, I hit up like their chat bot, not chat bot, but there's like apparently it's supposedly a real person.

38:19 Yeah, for real sure. And he just kept asking me, show me a screenshot of the air. And I was like, dude, I'm locked out of my account. I just need to change my email. He's like, oh, show me his screenshot of the account. And I was just, I got to the point where I just got so fed up so I was trying to do it for like an hour.

38:36 I went on Twitter and then I just like tagged him. Now I'm like making some progress here because I want to get a refund from my like domain renewal because before I was like a program, I did photography. So it was a photography domain. Of course you did. Of course he did. Anyway, next week's fun fact, bro.

38:52 You just spoiled it, dude. Wow, whatever. But anyways, I'm just trying to get rid of that domain and get my money back. I was like 200 bucks or something, but just insane. It's spoken by cards. I can't believe you could have something renew a decade later successfully. Like how did you keep hold of us some sort of long running credit card?

39:10 Like what do you got on there? I don't know. I just got a random email. I was like, this domain's gonna renew and I was like, huh, I haven't seen that. I was expecting like domain trust to be like, well, before I was a photographer, I was a infintech. And so I have this, I started a credit card company.

39:27 Before I was an archaeologist, I used to do deep sea diving. How are you holding breath? Yeah. I got some other things. I got some other things that must leave. But yeah, that's my block. Hopefully go daddy if you're watching. I want my refund and I need my email changed. ASAP, please. Nice, I like it.

39:47 All right, well, we've now finally hit the part where I can introduce the main topic everybody. Today's stand up. Welcome to the stand up today. We're gonna be talking about Nvidia and Microsoft talking about what they're releasing. I got some hot takes on Microsoft. I didn't tell Casey long enough beforehand.

40:06 So he probably has zero hot takes on what's happening. But guess what? We're still gonna talk about it and today stand up. One thing I'm truly disappointed by with today's stand-up introduction is that TJ has an in progress task called interrupt Prime's intro. And he did not in fact interrupt my intro.

40:21 I am now done. TJ, what have you done? See, in the business, that's what we like to call power play. And I just made you do the whole intro waiting for me to interrupt the intro. I didn't have to do anything. Living in progress and I did nothing. Anyways, that actually sounds like your daily life doing nothing.

40:49 Okay, so here we go, now we got him. All right, so I think the first thing we have to talk about is this Microsoft surface. Has everybody seen the Microsoft surface? Yeah. Oh, you mean in the 15 minutes between when you sent out the invitation and we're up recording this? Did we see the mic? Is that what you're talking about?

41:08 Yes, yeah, yeah, yeah, that's a specific one. That's the one that has the DJX in there, or the RTX in there, right? You have the RTX spark. And so they're saying it has one petaflock, which by the way, tossing out one petaflock, I don't know what that actually means. Is that a real word? Yes, petaflock, yes, I petaflock too, if it looks good enough.

41:28 But here's the deal. I think it's for how bad the product's gonna fail. It's going to be for a flop. It's a petaflock. So it's it's FB4 tensor tensor apps. I'm pretty sure. Okay, you just had a bunch of words at me. Like I should understand that. I just think it just means it's what is that? That's one above terror, right?

41:53 So that's like 12 e to the 12 amount of floating port operations per second. Yeah, so, okay. What'd you say? I said FB4, so four bit floating point, right? FB4 tensor apps. So meaning these have to go through the tensor core like largest, largest dimension mat mall. Okay. I'm probably just speaking gibberish at this point.

42:23 So we had this big long discussion coincidentally about floating point just moments ago and we were talking about that whole like exponent, mantissa, whatever thing. So in the quest to quote higher and higher numbers, GPU vendors have been making progressively smaller and smaller floating point types.

42:45 So we have 32 bit floating point. They made 16 bit floating point. That's been around for a while actually. Different kinds of stuff. What do they call a 16 bit floating point? Because I know they call them doubles. Floats for 32 back in the day and doubles for 64. What's the 16? They have various names because there's different encoding.

43:04 So there's like B float 16 and I don't know. Like I don't use this stuff. So I have no idea what the, there were 16 bit floating point numbers for graphics, which were used for things like, oh, we can, this frame buffer that we're storing results in. We'd like to be able to store lighting results in something better than eight bit color, right?

43:26 So we're gonna go up to 16 bit color where we store that as a 16 bit floating point number because we don't wanna go all the way up to 32 bit floats because that's a lot more memory bandwidth and blah, blah, blah, blah, blah, right? You can imagine all the things that go downstream of this stuff like that.

43:39 So 16 bit floating point is nothing new. Although the particular type of floating point you use can change, like how many bits you dedicate to each thing, right? Cause remember we talked about exponent mantissa, there's like how many bits you dedicate to scale and how many bits you dedicate to the actual digits is up to you in something like this cause you can move that trade off around depending on what you were, whether you wanted more range or whether you wanted more digits in your number, right?

44:05 So yeah, so we went down to 16. Then there was eight, so eight bit floating point numbers. Again, very, very small amount of information but still doing it. And now they're actually at four. So they actually do four bit floating point which I don't even, since I don't do AI, I don't fit. Isn't that like the AI super quantitized models one and a half bit numbers representing?

44:30 I don't know, I don't, I don't look at AI stuff so I'm not sure. All I can tell you is I believe this is an FP4 number. That's all I know. And that could be wrong because like I said, I only had like a very brief amount of time to even see what the heck this announced it is we're supposed to be talking about.

44:43 So everyone out there take literally everything I'm about to say with a pretty big grain of salt. I mean, always a good idea anyway. Anytime anyone's talking these days, take it with a big grain of salt but even bigger grain than you ordinarily would have shaken out of the salt shaker into your mouth.

44:58 So the way that they quote these numbers now, I believe cause their marketing numbers is they take the smallest possible thing they can do on the chip. So if your chip does FP4, you're quoting FP4 numbers, right? And you may ask why. The reason is because all GPUs, they are wide ops. They're like SIMD ops, single instruction multiple data.

45:23 The lane width is typically depending on how you want to consider it either 32 or 128, depending on how you look at it. In terms of the sort of the size, the bytes that you're gonna operate on at a particular time. So when you talk about how many operations you can do, they're not talking about how many like instruction operations like I told this thing to multiply.

45:51 They're saying when I tell it to multiply, it's gonna take this giant byte pattern with a ton of stuff packed into it and do this big multiply, right? So they take that and they say, okay, how many of these can we pack in there? Take the smallest thing which is FP4, right? Take the most efficient thing we have, which is when we multiply these two things together, what instruction can do the most of these multiplies in the shortest amount of time, right?

46:23 And then that is the number that they quote you. And just for good measure, because typically what you're doing is F-mads, floating point, multiply, add, right? It's a multiplying an add in one step. They just for good measure multiply the number by two because they're like, well, hey, you're doing two operations.

46:42 So technically it's twice as much, right? So you have to do all of this to figure out the actual instruction rate, right? But yeah, so that's what's happening. Sometimes they quote dense, sometimes they quote sparse. I don't know whether this is a dense or a sparse number, I didn't have time to go look at the math to see you can tell pretty easily by just looking at the coup d'-core count of the thing in the clock rate.

47:05 So that's not hard to figure it out. But and the way that that works is these kinds of cores have this ability to effectively pack data sort of, pack and unpack data as part of the operation so that if you have something where you know there's a lot of zeros in it, like it's a sparse, like it doesn't have, not all the numbers are always filled in.

47:29 They can exploit that sparseness and get higher throughput by just the chip actually seeing the sparseness and only issuing the multiplies it actually needs. So it can basically fit like more, you know, it can pack the zeros out. So one of those big 128 byte lanes can hold more than 128 values because the zeros have been eliminated or something, right?

47:54 There's all these kinds of things they do in there and I don't study this. So I don't study the specifics of that. And I don't know which number this would be, whether it's a dense or a sparse. But that's everything and you think what they quote these numbers all those shenanigans are on the table. Okay, this makes a lot more sense because that number, you know, in my head, I'm thinking, okay, this is like, you know, what's called architecture wide floating point operation so they're doing like double, like a flop of doubles.

48:19 I was like, this seems like that's really fat. Like in my head, that's like super computer fast. So I must be misunderstanding what we're talking about at this point because then I went on the internet, you know, as I come and do from time to time and I started asking like, I went to the AIs, of course, to ask about the AIs, like with the new spark thing, like how fast are these models actually being able to be ran because it looks like it can be ran at like super fast, right?

48:42 You could have like local models running at infinity speed and a lot of the forums slash what the AI had to say was saying you're gonna be running at somewhere between like eight to 16 tokens a second, which may on a 32, on a 32 B model. So they're not the speed and all that. I don't, I don't know actually what these speeds mean because that just seems like the ultra fast number I've ever seen in my lifetime, but really it may not be nearly as fast as, as at least my brain would assume that means.

49:10 Well, again, I mean, so if you look at the actual chip, like what's in this thing, right? It's basically like the same number of coup d'acours as like a 5070, right? That's what I'm saying. And so I'm fortnight really good. So I assume that was a pretty good, you know, graphics unit. Well, I mean, maybe the other problem though is, it's so the memory is gonna be different, right?

49:37 Cause this thing, this thing is LPDDR. So low power, the stuff that's in laptops. And so if you compare it to something like a 5070, a 5070 is using GDDR, which is the like the high clock rate, high power consumption, high clock gains, right? And so the trade off, the actual thing you have to think about here is a 5070 has like three times the memory bandwidth or something of this thing.

50:08 So this thing is actually a third the speed of getting you the actual memory. So I don't know how good it would be at fortnight, maybe fine, I'm not sure, but it's definitely worse than a 5070 at fortnight. So you're kind of doing a trade. You're trading the fact that this thing will have much worse memory bandwidth.

50:25 I mean, a third of the memory bandwidth is not good, right? Which I think is roughly what it would get. Again, take it with a grain of salt because we had all of zero seconds for me to look at this stuff. But the trade is that because this is using LPDDR, right? It can have a ton of memory, can have 128 gigabytes of memory.

50:47 Whereas a RTX 5070 is like, I don't know, it's like 12 gigabytes. It's like a 10th of the memory. So you're making a trade and I assume that this is a trade that AI people are willing to make because although it would be slower, so evaluating any particular part of your model in theory would probably be slower on this chip because it's got one third of the memory bandwidth for pulling it in.

51:14 So it can only go at one third the speed of pulling it in if it's memory bound, which again, I don't study this stuff. My understanding is that AI workloads often can get memory bound depending on what you're doing. But on the flip side, you can fit a much larger model in there, right? You can fit 10 times larger model.

51:33 So I'm assuming that's a trade that AI people would be happy with. Otherwise, why did they make this thing, right? Can I ask a dumb question or look quickly? I probably can't answer if it has to do with AI. Oh, no, it has to do with memory and specifically why you said like one tenth of memory. Why does the low power enable more memory versus the high power counterpart?

51:53 Is it purely just a power draw problem or is there something I don't understand? I really couldn't tell you so I would assume that it is probably more to do with cost than anything else and or market segmentation. Meaning, I don't think there's anything in particular that's, well, you know there isn't because you could go by a 5090, right?

52:17 And what does that have? 24 gigabytes, 32 gigabytes, I don't remember what they have, but they've got way more. And that's on a consumer level card. You're not talking about buying like a black well for the data center of it. It's a 32, 32, right? So obviously you could have stuck more GDDR on there.

52:33 It's just the cost is higher and the, you know, so I think they're trying to get 128 gigabytes into something that doesn't cost you 10 grand. If you wanted to go pay data center prices, you can get, I mean, you can get high bandwidth memory for that matter. HBM modules on their like actual data center parts, they can do it, right?

52:53 So it's really I think it's just a cost issue in terms of the manufacturing of these things. I imagine GDDR is just a lot more expensive than LPDDR, that'd be my guess, but I don't know. Probably a little bit for laptop form factor too. I mean, it's just like you'd rather not have necessarily like that.

53:12 I don't know how much power draw, but I imagine that going faster means more power, more heat, bad battery life, et cetera. Well, he doesn't really consider himself a battery life. I don't know if you've seen his pictures, but he has a battery life that he could run for days on a laptop. Well, I didn't show, I didn't post that picture, but I can grab it if you want from it.

53:33 It's a pretty good picture. Casey, I did verify that the RTX Spark has 300 gigabytes a second for the unified memory system, whereas the 5070 has 670 or something like that. Yeah, it has an over double of whatever it is. And then the 5090 has like 1.7 and high 300 sounds high for the spark, where are you getting that from?

53:53 This is this is the summary from an Instagram verge post. And then Tom's hardware as well as reporting apparently that exact number. Right here. So Tom's hardware is saying it's 300 gigabytes a second right here, because it looks like 273 on the old OK, so they may have done a little bit of rounding.

54:14 It's like 273 versus 672. So yeah, I mean, yeah, I maybe, maybe saying a third is, is being a little bit harsh, but I wouldn't say double. So it's more than double, let's take a look at it. Let's take a look at what it actually is if people want to get that specific about it. 2.5. So split in the middle, like split the beta.

54:45 So 2.5. And again, that's the trade, right? You're just trading like, OK, we got, you know, we're going to have less speed at pulling this stuff in for memory, but you're going to get a lot more memory. And presumably that's what you want. Otherwise, again, I don't know why they would build this, right?

54:59 Presumably, that's what people want. Because if it wasn't, I then what's, I don't know what the pitch for this thing is, right? So I think I think I understand more of what this pitch is going to be and kind of the general direction Microsoft's taking is that they kind of, he said a very interesting thing, which I just wanted to bring up in here.

55:16 Who's he? Who's he? Well, I'm about to show you his name is Satya. And he said the following. He said, our goal is to deliver unmetered intelligence to every home and every desk with windows. And then they do this RTX Spark and part of RTX Spark is that you can run a bunch of local models yourself. So you don't have to necessarily rely on the cloud models.

55:33 You can run models yourself as the general kind of tenor or goal it seems like for where Windows is going. And there's a very specific phrase in there, which is unmetered intelligence. Sam Alvin for the last two years has been saying that he envisions open AI to be a metered intelligence. It should be no different than your electricity all this.

55:53 So this kind of seems like, hey, they're kind of going against this general idea. And instead, they're moving a lot of it down into the individual's laptop and then building harnesses. So my personal guess is that what Windows is wanting to do with these larger memories but a bit slower is that you can run a bunch of these smaller models to do mundane tasks, simple tasks, and then have a cloud subscription to run a model up in the cloud that doesn't need nearly as much power because they're just telling little models what to do.

56:20 And you're running like four models in parallel to go and do these basic operations. And so thus, that's why they're fine with these, say, lower bandwidth memory, these four to tens tokens per second kind of production because it's not really, they don't really have to care about that because they have multiple going and they have the big powerful one up in the cloud.

56:40 And they're just trying to win the basic consumer by reducing costs and reducing data center. So that's what I think is generally happening here and that's why they've chosen this general architecture. I mean, I got two things that would be fun to chat about for it. I think the first one is isn't not a little bit weird like that it feels like a direct shot at OpenAI that Microsoft has a really big stake in and has spent a lot of money on it.

57:10 It feels like Sam's thing's been the, it's like it's too cheap to meter or like it should be treated like utility or we should have UBI and tokens. They kind of gave up on that train but that was I recall talking about it on the puzzle. I'm interested to hear sort of what people are thinking for like, is there some trouble in paradise over there?

57:31 What's going on, you know? Well, we do know from the court leaks that Satya was a major reason why Sam held onto a CEO ship of OpenAI. And so there is something, it is very interesting that he's using very, because we all know that Satya does not tweet a tweet that has not been looked at by human resources by legal by, yeah, especially not this one.

57:55 Like they've really, this is a very intentional tweet which makes it seem very intentionally against whatever that vision is of the world, which to me, I think part of this is also in response to like the data center hate is that generally speaking, people seem to be very anti-data center in our current world, whether it's founded for good reasons or bad reasons people generally are so having the ability to have agents run on your computer as opposed to open the cloud means less data centers.

58:24 Thus, it feels more tenable, I guess. And so my two guesses is that there is trouble in paradise when it comes to this, because Windows has also created their own frontier model. And so there's like 900 things going on to where it seems like they've went from competing against the other models using OpenAI to saying now we're competing against OpenAI with our own models and we're like directly going against it.

58:48 And there are contracts I think run out of 2032, so maybe there's something about that. Trash thoughts? No, I was gonna say, I'm actually, I've been thinking about doing local AI stuff, we have like a local AI group that I'm part of and they all have like these like DGS sparks are just way too expensive.

59:06 So I feel like this was very good time to wait, wait, wait, wait, wait, Trash, turn around. What? What? Can't you expensive? It's, he said it's too expensive. DGS sparks is like $5,000 or like almost $5,000. It's crazy. What do you know how many Pokemon Picards I could buy? It's a fraction of this, but.

59:27 That's right. That's right. No, but like to be serious here for a second, I think it's pretty neat. The one thing I don't like is like, there's no Linux support, now I'm a Linux knob now. Thanks a lot. Go ahead and have a good one. I was like, I was like, no Linux support. I'm not even take a part of this, show right now, but apparently it might be coming.

59:48 But I think the interesting thing is, I'm curious how, because I already know a lot of these companies are underwater with as far as revenue goes. So if I have this computer, I don't need to subscribe to ChatGPT anymore. I can do my mundane tasks. I feel like most consumers aren't doing anything crazy besides actual enthusiasts and developers.

1:00:08 So your average consumer doesn't need that like crazy power. So they can run a Q132 model on their machine and get away with it. I don't know much about what Microsoft releasing with their seven frontier models. I'm assuming they're shoving that down their employees' thoughts right now, and it probably sucks.

1:00:22 They did cancel my account. So yeah, exactly. I'm assuming that's definitely like leading up to this moment. But anyways, I'm just curious how it's going to work out for Anthropic and ChatGPT, because I know like the big thing was like, we need more compute, we don't have enough compute, but now everyone has it on their machine now.

1:00:40 That's no longer going to be an issue. And also, you're going to lose all their subscribers. And that's the next issue. So I'm just curious how ChatGPT is going to like respond to this. And if they're going to think like they're potentially in trouble here, I don't know. Yeah, the other trash which you mentioned, which I was kind of interested in, is I'm wondering how much of this is like a, you know how when someone says no to you, so you just reframe it like it was always your idea and that you're really happy that you can't do option A and now you have to do option B.

1:01:11 Like I do wonder if some of it is like, hey, we thought we'd be able to like build all these data centers and everyone would be like stoked about it and we'd get to like do all that stuff and then that doesn't seem like it's going to be possible. They can't get the electricity, they can't get anything else.

1:01:25 And so is, are they doing like a pivot to try and be the first like, you know, first pass the finish line on these sort of like home AI set up things because they can't get the data center stuff approved, right? You know what I'm saying? Like in terms of like, well, we need to get AI to everybody, obviously, but we can't do the data center thing.

1:01:45 You know, as I'm sure that's a part of it, but I feel like they're probably working on this well before like the hate on data centers potentially. Well, some of these have been work, but like the packaging is very different. Like packaging it inside of a surface laptop. Like they've been building all these components but putting together in one is interesting.

1:02:04 Yeah, I'm curious how it's going to shake out. I don't know the, I don't think they mentioned prices. Did they prime? I don't, I didn't see the prices. I haven't watched that. I watched the keynote and the keynote didn't seem to mention prices or at least that I remember. I don't think they really, I don't, I couldn't have found prices anywhere.

1:02:21 I'm assuming it's just going to be kind of insane. Hard 28 gigs of RAM, even if it was low memory, low power along with a 5070. Come on, come on. Well, I mean, I can't is like, what would even be the explanation for how it would be cheaper than a DGX? Like the thing that trash doesn't want to buy because it costs too many Pokemon cards.

1:02:45 Like you take some of the networking out of that thing, but now you got to pay for a screen and a keyboard and chassis and all this stuff that you didn't have to pay for before, right? And a battery. So how does it get cheaper? Like it seems like it will just cost the same as the thing you can already buy, right?

1:03:03 I mean, I don't know, I don't know why this would be cheaper. I'm not sure about that. Is that there's a huge amount of people that are just Windows and East, shall we say? They're not really into the computer side of the computing. They don't really want to, because if you've ever played with local models at least last time I did it last year, there's a lot more work to it than simply plug it in.

1:03:23 Hey, I'm up and running. And I think that's what Microsoft is selling is this really kind of like, hey, oh, you're just in Excel, you're just in these things. Well, here's our integrated. It runs locally, so it's fast. You don't even have to pay money. You just have to, hey, just buy Microsoft 365 and pay the once a year fee for that and you're good to go.

1:03:42 It runs all locally, it's all yours, right? And so to me, they're making a play to purchase people in by saying it's free effectively, but do you have to buy all their services anyway? So they're like still winning in the end. It's just giving people the access. Well, I'm just pointing out for this particular product, I'm not really seeing how this comes in at a price point below even $3,000 possibly higher.

1:04:07 So it doesn't seem like this is not something very many people are going to buy, I wouldn't think. It's geared towards programmers predominantly. Like that's what at least their presentation really made it seem like a programmer thing. The person that did the examples, talked about how they got all the effectively unix tools, like GREP is now in Windows, all these things are now in Windows, they show they have like 53 tools or something like that.

1:04:27 And so it did seem like it's geared towards the working professional as a working professional computer. Right, and that's why I was saying like, so this isn't really something that you're going to sell to somebody who just runs Windows and doesn't know anything, right? Like it's not really targeted at them, it's only targeted at people who already know what they're doing and are using the command line and stuff, right?

1:04:49 I think. It's a you're not thinking about this, you're not thinking about this in the right way. Okay, so here, we were talking about TAM earlier, obviously the TAM for this podcast and the company that it represents everything that's ever lived. That coincidentally, everything that's ever lived, they can be programmers now, Casey, because of AI.

1:05:10 You haven't even thought of that probably. True, no. So everybody at the company is a programmer now in 2026. So you just have to convince the CEO of the company that since everyone's a programmer, they need a programmer computer and therefore they need to get this computer for every single person at the company so that they can run it.

1:05:31 Hey, Casey, do you want a $500 million token bill? Or would you rather just pay, let's just say a million dollars for a few laptops and then it's all run locally. What do you think, Casey? Well, I'm just saying, I'm just saying, which would be one case. Laptops or Prime, a thing that looks like a griddle that you can get.

1:05:50 Did you see that? I saw people, it looks like a hot plate that you could cook on, which I think is kind of fun. Can you make an egg on this thing by putting a piece of teflon on top of it? I want to know. I did, it actually did remind me very, I bet you'll even have one in here, on Xbox One X, the top to the Xbox One X, they just lifted the design of that and just put it as the gritting or whatever, the grit on top of the griddling.

1:06:20 On top of this one. Here's what I want, and I'm dead serious. I want someone to take one of these things, the griddle one. I want them to put a solid metal piece in place of the waffle pattern, and then I want them to create a thing called bacon mark. And bacon mark is basically an AI throughput benchmark where you put a piece of bacon onto it, and then the benchmark result is just a picture of the bacon after the workload has run.

1:06:54 Okay. Okay. And how do you determine if the bacon result? At the bacon still raw? The raw, the bacon, the better the result? Yes, yes, yes. I mean, maybe, or, but yes, like if the thing computes the same, if it's computing the same answer, I guess, then the more, like, intact, the more original the bacon, the more efficient this AI is.

1:07:18 It should be a Friday. Unfortunately, Casey, this, we did miss the bow on this one, this would have been perfect for Apple to release, because then we could have called it Let Tim Cook. Okay. That would have been the perfect. Too late. Too late. He's gone. It's passed. It's over. Yeah. Yeah. Unfortunately, they did let Tim cook and he got kicked out of the kitchen.

1:07:40 He couldn't handle it. He was cooking to zero over. He couldn't handle the heat. He couldn't handle the heat. Huh. Hey, I do think this is really interesting, though. I am very, I'm very curious where this is going, because there is something that's very, like, unique about this moment, that Windows did have a three and a half hour keynote.

1:07:59 And this was obviously like the kind of the center fold of this. I'm not sure if that's the appropriate term. Now that I think about it, because I do remember this. Yeah, let's touch it and be a center fold. Do you really want that? Yeah. Are you really one of my full music? In my head. I don't think that's not good.

1:08:13 I can't, I literally can't put the center fold back in the magazine at this point. So we're just going to keep on going. Okay. And someone mentioned that they never mentioned any of their pro, any other programming apparatus, other than AI, except for VS Code once. And so like they're fully going towards the programmer, but they're going towards them exclusively through model usage.

1:08:38 Like there they don't talk about TypeScript.net. Like.net, they've always talked about.net. They're not talking about.net anymore. They're not talking about C-sharp. They're talking about models. AI. I'm curious how those models are going to be. But I said their flagship one is like equivalent to 4.8 or something.

1:08:55 I want to see what he's going to do a lot and gets distracted. It goes on X and tells me about things that are happening. Okay, I do have to stay. So Casey 4.8 Opus 4.8 came out. So I gave it a test run and it was hilarious because the very first message it sent back to me, I haven't been using cloud models in quite a while.

1:09:18 The very first message goes back and goes. Excellent insight. That's straight to the point. And I'm like, ah, bro, bro, put the code in the bag. Bro, stop, I'm telling me how smart I am. I already know that, okay? Please. Gotta get a little glaze in there. It's so bad. Gotta get a little glaze on that code, don't I?

1:09:40 Guys, it was crazy. Immediately, it was like so obvious. And I was like, this is why people are falling in love with anthropic. It tells them how smart they are every time. They send something to it. Oh, man. Yeah, as far as models go, as like which one's better, honestly? At this point in my life, I can't really like, besides for like some kind of weird implementation details, like some will create many tests, some will create one test.

1:10:06 Like there's like kind of these weird, like kind of variations in that. As far as performance goes, I'm having a harder and harder time disambiguating. Which one is better? Right, like I can't, if you were just show me a function, a snippet of code, it'd be hard to understand, which one's better. So I'm sure the Microsoft one does fine.

1:10:23 Right, like I'm sure you wouldn't even, you would probably not be able to understand at a more micro level, the difference. I could also imagine if I used my powers of imagination to really imagine in an imagining way. Yeah. Yeah. That if your goal is to program on Windows, which at this point, I don't know who.

1:10:46 That's the imagination. Bar Casey. OK, that's the madness. Microsoft obviously has a huge history of source code control check-ins to Windows code bases that nobody else has, because they're all internal. They weren't done publicly. And so there AI is presumably trained on all of the entire development of the NT code base and all the development of the Windows 95 code base and all of the development of Microsoft Office and all that stuff, which nobody has, as a training data set, except people who maybe like hacksworded out of Microsoft or repaid to smuggle it or whatever happens in AI circles now.

1:11:22 I'm sure it's all very illegal and that they're all fine with it because they've convinced themselves of something. But I would imagine that Microsoft is probably the only people who can train on that. In less open AI, I got the right to train on that as part of one of their sleazy backroom deals that they do or whatever.

1:11:38 However, I would just like to bring this back to another point, which is, again, I think it's worth underscoring that this is a non-announcement. Like the only thing that's actually, as far as hardware is concerned, this is just an announcement that Microsoft is like getting Windows to run on this thing, right?

1:12:01 Like that's it, because there's no, it's just a spark, like the surface thing is kind of like that. I guess you can say that the laptop part is a form factor announcement. It's like someone's gonna put this thing you could have bought from Nvidia since last year, right? We're putting it into a laptop for you if you prefer a laptop.

1:12:20 Like that's the hardware announcement. There isn't another thing right there. It's not a new chip. It doesn't have new capabilities, that's it. So really, the announcement is more of a software announcement. It's like Microsoft got Windows on ARM running on this thing. That's really the announcement, right?

1:12:36 I mean, and laptop form factor. So just want to make sure that that's kind of like, there is no hardware. There's no chip thing here, right? I mean, are we all... Yeah, good idea. I think the chip thing is just like, how small and integrated it is, right? With unified memory as opposed to having its own dedicated V RAM, right?

1:12:54 Isn't that like the... No, that's the thing Trash was talking about that he didn't want to buy, because it was too expensive. It's the same chip, same memory. I'm an idiot, sorry. So we took this thing that you could have bought since last year and we're putting it into a different thing now. That's it for the hardware side of things as far as I can tell.

1:13:15 I didn't see any announcement of like a new actual capability or a more modernized chip, right? So it's really more of a software announcement, like the actual new thing here in terms of what's gonna change for you if you, instead of buying one of these NVIDIA little desktop boxes, we'll call it a Mac Mini Compete or whatever.

1:13:37 Instead of buying one of those, the only difference really is that now you'll be able to run Windows, right? You'll be like, before you couldn't run Windows on those, I don't think Windows on ARM worked on that little NVIDIA box. But it's the same box, just in a different form factor now. It's a nice little skinny laptop.

1:13:56 And KZ, it's a great improvement, wrong. This was, there was some sort of like Jonathan Blow was tweeting like, that what if it was like a completely different style of computer, right? That was like in terms of the things of, like. Yeah, and it's definitely isn't that. Like whatever the mysterious tweets at all of these secret little signs and stuff.

1:14:18 And you're like, oh wow, they're gonna release like a GPU computer or something insane. No, and then it was like, yeah. Whatever John Blow wanted, he definitely didn't get it because literally the chip, because literally the chip is the same chip that you could have bought it last October, right? So, so there was no, like they didn't announce any new like architectural interesting things where John would have been like, oh, that's a step towards the thing I want it.

1:14:44 Like, no, there was no step at all. Well, because they were doing the mysterious like, oh, NVIDIA sent the date and Microsoft sent the date. And you're like, whoa, no. Is this gonna be like the first time I was alone? I didn't tie one if I'm not mistaken. It was the location of the, it was the GPS coordinates of where they were gonna give the keynote, right?

1:15:04 Right. Yeah, yeah. So you thought it was gonna be like, whoa, I bet their stock's gonna double after this. Like it's gonna change the game. Oh, look at the stock actually. Oh yeah, trash, let us know. Yeah, so that's the longest short is if you want this thing early, just go buy an NVIDIA, one of those little NVIDIA spark boxes, whatever they called it, the RTX spark or something.

1:15:25 I don't remember what it was called. Yeah. Just go, you can literally buy this chip right now if you want it. By the way, you can see it in the stock price. It actually did it directly affect the stock price. And then after everybody saw it directly unaffected the stock price. Yeah. People are like, literally, look, this is, you're talking about somewhere in the teens, four, you know, 15 will just call it four, 15 per average right there.

1:15:51 All the way up to four, 60, so a 10, 15% boost, all the way back down to now four, 27. So, oh well. Oh well. I mean, so to be fair, though, I am gonna at least try to glaze it a little bit here. Yeah. I think that trying to prevent, like, first off, 128 gigabytes of unified memory and then saying this 120b plus, this is very misleading as far as what you can run locally.

1:16:15 But excluding all of that, the idea that you should be able to run your stuff locally and then that, like, that's a good idea. I think is a really good message to be said, because, you know, the whole Dario's of the world talking about how everything's dangerous and they should own everything you're doing.

1:16:31 There are so many companies that have really, like, competitive data about a activity, right? They have users who pay them money, who send them specific data. They have, like, all this good stuff and they have to effectively ship everything off to open AI. If they want to use any sort of these models to be able to do any sort of kind of cool things that they say can't afford a team of data scientists to figure out how to do.

1:16:53 And they don't want to share this data with anybody. I'm thinking of, like, mining factories, I'm thinking about just a lot of just more kind of brick and mortar tech kind of combo places. And the ability to run things locally, I think becomes really appealing to them. And so this idea of moving towards it, I think it's really good.

1:17:09 I mean, I personally think this is a good direction overall for people, because you stop having to be tied to large companies. And I think that that is overall a good thing. I would agree with that. And like, it would be nice if Microsoft decided to go that direction for some reason. They're not. They don't triple it.

1:17:28 They don't seem like it because, you know, Sachin Adela is famously the person who came in and basically, like, I mean, I'm not going to say killed Windows, but he was definitely the person who came in and said, look, Cloud is what we do. That's where we're going to make all our money. Is subscription, software, the service.

1:17:46 We don't really want Windows to be good, because the better things are locally, the less we can charge you remotely. That's Sachin Adela. So if he is, well, yeah, but that's kind of separate from this thing. But so if he's actually now changing tech on that and he's going to turn Microsoft back to an actual personal computer company, that would be fantastic.

1:18:09 I don't believe it, but if that actually happened, that would be a great outcome. Yeah. So that's, that's, so I'm trying to like be at least Microsoft positive, because I'm always pretty negative on Microsoft. I think they, they somehow just fumbled the bag so hard for the last decade. I cannot believe that I think Steve Balmer is a better C or CEO than Saudi.

1:18:30 I'm saying it right now. That's my personal opinion. Halo three and he was ruling during that time. Absolutely love the man. And so it's just, this has to be a good, and that's what I'm hoping for is that people realize the need for local and perhaps I will push Microsoft, but Microsoft, they did it with VS code, which is they built a bunch of cool open source stuff and then they just immediately started EEEing people.

1:18:55 I assume this thing is going to be the exact same thing, like all these cool local models, but if you really want to actually use it with Windows, you have to pay for all these services to be able to build your soul or your memory thing and your management and all this kind of stuff, right? And so it's just going to be, they're going to re-e-e-e-e-e-u again.

1:19:12 They're just in the embrace season of local AI right now. And someone's asking what EEE is, embrace, extend, extinguish, right? You become the good guys. Everybody comes to you. You extend it and make it really, really functional, but then it becomes also like privatized to where it's very hard for anybody to compete with you, but you're also now extracting value.

1:19:34 And then you extinguish by diving your prices and effectively killing everybody else out while you can then reap all the benefits being a single monopoly winner. Classic Microsoft, by the way. So I'm trying to give them a little bonus. Okay, hey, I hope they're good. All right, now that the second donut has been glazed.

1:19:55 We were going to talk about Nvidia as well, like the Spark. Is there anything interesting in that or is it really just purely a, this is just a, it's just the same thing because I was reading through it and there's nothing that really feels interesting about it. I mean, they really do love DLSS and I know that that just ruffles some serious feathers out there.

1:20:16 If you mentioned DLSS, SSS, SSS, SSS. That's all I know. Yeah, I mean, this thing is not, like I said, it's not a new chip. If you want to know how it performs with GDDR, no. Still, no, in fact, it's still LPDDR in the other box. So you can literally just go see how the thing performs. And I guess we don't know what kinds of concessions might have to be made for like the laptop version.

1:20:39 So maybe the laptop version performs worse for some reason because of some thing that has to happen there because of the power. That part I don't know. But in general, like if you want to know how this thing works, well, you can just go look at the benchmarks for the thing that we already have. And I believe the benchmarks for that thing put it somewhere around a 3080.

1:21:00 So like if you want to know how your graphics would run on this thing, it's like a 3080, roughly, which is fine performance. I think, like again, take all this to the grain of salt, but I think that's roughly where the spark is. And again, that's just because, as we said before, games care a lot about the memory bandwidth on a GPU.

1:21:19 And so having not that much memory on a GPU is actually fine if the bandwidth is really high. That's why we don't typically gamers don't have 128 gigabyte GPUs, right? That's not a common kind of skew. And that's because games are used to fitting in eight gigabytes, 12 gigabytes, 16 gigabytes now. So they just don't need it as much.

1:21:42 I can't wait for everybody to have 128 gigs. So that games can start taking up that much space. That's all that will happen. That's exactly right. All that will happen is now the games will just take 128 gigabytes. They're all going to shit down. That's all that was going to happen. It's going to be so good.

1:21:57 All right, sorry for ruining everyone's day. You guys thought it was bad downloading 100 gigs for Call of Duty. They're going to have it be several terabytes and it's going to be 128 gigs of RAM required. It looks so good though. And I believe, I mean, I think the, again, didn't have time to actually look at this myself, but vague understanding is, you know, so it's media tech, right?

1:22:22 I mean, presumably you guys have heard of media tech, right? They make cell phone CPUs. TV CPUs too. They make CPUs that are in a lot of, one point eight gigahertz quad cores, right? Something like that. That's like it depends entirely on which one there. I thought that was one of their newer ones, but anyways, yeah, sorry, keep going.

1:22:44 They're the, like, they make this CPU, Nvidia doesn't make the CPU part of this thing. It's two, it's actually two, like, kind of things on a package where the Nvidia side is the part with, you know, the GPU stuff and then the CPU side was made by media tech. And it's got just the standard, the CPUs that they make, right?

1:23:06 So it's a, I think it's 10 cores of the X, of the X line and 10 cores of the A line. So they have like, there's like 10 performance cores and 10, like, more efficiency kind of cores. That's, that's what it is. So it's, it's not a mystery. Like you can just go look like, like I said, they made it sound like this was this big grand unveiling and stuff.

1:23:29 It's all stuff we already know what it is. You can already look up the benchmarks. It's not as good, for example, as a Qualcomm, like the Qualcomm Windows on ARM CPUs. Those CPUs are like better for performance, I believe, than these are. So on the CPU side of things, if you think of one of those Qualcomm notebooks, that's probably like best case CPU performance for at least for like single threaded, like the things that you would normally be looking at for CPU there, maybe, maybe if you're, if you're doing like a big multi-threaded or a little change.

1:23:59 But so it's, again, it's a really kind of a nothing burger from the standpoint of like hardware announcement. There just wasn't much to look at here, which is good because Prime only gave us about 15 minutes to look it up. KC. What was, what was, what was, why the announcement? They said something about a thousand games.

1:24:15 Like what, what does that mean? You know what I'm talking about? I'm talking about a thousand RTX enhanced and accelerated games and apps are available. They probably just mean that they tested them and they run because remember they just, they all have to run through a Compat layer because nobody ships ARM compiled binaries.

1:24:32 So they have to run through a compatibility layer that makes the X64 code run on ARM, right? So they probably just mean that that's, or like it's got the LSS support or something or it's got some of the other like AI capability. Maybe, I don't think there's anything special about that because again, like, you know, that's just what you would expect from a Windows, like what you expect for a Windows 3C is all the games.

1:25:03 All the games run exactly. So it's not much of, again, not much of announcement, but I don't know what they mean there. I didn't see that part of the announcement, so. Okay, well, I'm watching a little video of it going and this one looks like you're playing on a PlayStation 1 and this one looks fantastic because I don't really know what's exactly happening.

1:25:19 Is that that new thing that's going on here? I don't know what they're going to do for DLSS 5 or whatever, where they just replace the graphics with different graphics that are like better. Yeah, it looks like a nice graph to the left. Yes. Yeah, so this thing is just like slowly zooming in. Yeah, I'm not really sure what this comparison is because I don't think I've played a game that looks like this.

1:25:40 I'm actually replaying Dark Souls 1 right now and it doesn't look this bad. So I don't really know what's going on. I don't really get this example, but whatever, right? I guess I'm not a, I'm not in the no on whatever that is. You're a boomer. I'm a boomer. Yeah. Yeah. Yeah. All right. Well, I think that's about about it.

1:25:59 You know, appreciate you guys. Great work, everyone. I think we really knocked it out of the park this time. I think we did. I want to say the floating point operation thing was super interesting talking about the local agents. I think is actually not only interesting, it's actually important. I think it's really good.

1:26:15 I'm hoping that more people realize that they don't need to be beholden to these big companies. That would be great if the industry go in that way because that's much better than the current thing. Yeah. I'm against current thing. I am against the current thing and for the new current thing. I support current thing.

1:26:32 Oh, dang. Yeah, I actually hate you is what that means, TJ. What is the current thing? I don't even know. Trust me, that's actually smashes red button. That's a smag. Yeah, red button. Red button. Team red button. Oh, are we talking about the police, the police thing again? No, what? I don't know what the police thing is, but we're not talking about the red first blue button.

1:26:55 I thought that's what we were talking about. Yeah, that has nothing to do with police, but I really appreciate it. Really, you can talk about it as the mini-fig situation. Dude, the mini-fig situation is insane. Should we talk about that next week? It's a mini-fig. Fender guitars. Oh, my gosh. Can we do a slideshow as the blocker for next week and show Casey the Lego situation?

1:27:17 Brick, brick, brick, brick, brick, brick, what is this? How about this one? Trash and Casey. If you see anything about Lego news, just navigate away. I will prepare a roughly 10 slide presentation about it, by the way, TJ, there's a lot of pictures required. I'm going to have to figure out how to make it so that the blocker isn't the entire episode.

1:27:41 I think it should be the entire episode. It might actually have to be, honestly. Just do it. You will like it. It will be. Just do it, Prime. Just do it. I know. I'm very interested. Hey, dude, I got a lot of Lego over here, but it's a very interesting story, but TJ, there's a lot of lawyers and other lawyer tubes coming out saying that the inverse party is completely incorrect, and that many figs is the correct one.

1:28:07 And so there's a lot of drama going on, and I've been trying to keep up with all of it, so I'll be glad to do a whole lot of no idea what you're doing. Dude, you just can't, what's the many fig? Oh my gosh. Again, Trash, do not even Google it. If you Google many fig, you'll be led into the entire situation, so you've got to stay away from it.

1:28:28 Is that your wife's name? I've been up. No, it's my name is Pidge Deco. I'll be like, a lax on. Your wife comes in. Please, could you play? Yes, sir. Oh, like the play desk, Bacito, and she starts playing it on the end. It's all just walks into my frame, it just does whatever. That'd be insane. That would be good.

1:28:49 All right. So the next week, Pidge Deco is working Pidge Deco. I don't think, honestly, TJ, I don't think we can Pidge Deco and do many figs. Many figs next time. Many figs this next time. That's what we're doing. That's great. Dude, I've been holding my pee for like 45 minutes. All right. All right, that's it.

1:29:08 Bye, everybody. That's the end of the episode. Bye, everybody. You