Whats really going on with AI, Expert weighs in

0:00 Welcome everybody to the stand-up. We have an extremely special episode of the stand-up if you have been living under a rock You did not realize that Casey has started his own podcast a competing podcast It really is I know Casey. How could you sit down and how could you He and Dimitri are talking about and having a real conversation about what is the actual effects of AI and all that and you're probably asking Well, why would they do that? Well, Dimitri turns out to be a legend when it comes to AI He knows a lot about it's been working in it for a very very long time and just like Casey is significantly more competent than TJ and I combined And so therefore their conversations are actually useful and good to listen to and we thought we'd bring them on and ask some of our own

0:48 Very own questions kind of a podcast melding There's a lot less cars References and jokes in their podcasts. So if that's what you're looking for just make sure you're still coming back here But if you're you know just wanted to make that clear where are sort of competitive edges is the cars references Specifically the cars references and that is it exactly Oh, they also do like technical scale and excellence and all these other things and have deep reason Well, we weren't listening what they did better. I was listening what we did better. Oh, Hearts references focus on the positive. Yes, right. I'm more of a glass half full kind of guy or like glass 110 Fulls kind of situation here glass has water kind of guy. That's that's the first night

1:31 All right, you know all seriousness though. This was the podcast was actually Dimitri's idea It wasn't even my idea I should have been my idea because I agree that I should ditch you guys and go with some like some higher quality Co-hosts here. I mean, that's obviously clear But but it wasn't because Dimitri you know You were actually the one who was kind of like I want to talk about AI because I'm just basically you weren't happy You were like I I don't like that. I don't like what's being said It feels like you go on Twitter by accident. Is that what happened? You went on Twitter Wait a second. We need a podcast immediately Yeah, so let me add a little bit So actually I did not set out to to have a podcast

2:18 I set out so I get so I've been working this a long time like 20 plus years I have lots of friends who are not programmers or any kind of like you know lawyers, accountants, doctors, whatever And they're frequently asking me you know first of all can I use this in my job and other things like how does this go to effect like I have kids in college Should I stop sending them to college right? You know all of those kinds of questions so at some point I thought I should just try to put this stuff together or somehow and first I started writing stuff But that felt awkward then I thought maybe I'll record something but I'm not really a recording personality And then Casey and I have been talking about AI on and off for four years and my suggestion was what if we record like one session

2:59 And I had a list of things that we could talk about and Casey said well There's so many things here that we could just turn this to a podcast. So that's how we ended up Ended up here, but let me have one more thing to that which is So you know, I know that there are a lot of people out there who are you know, I see Casey. I click personalities. I am that kind of personality as well. So part of Part of what I want to do here is to Facilitate having more Casey out there On stuff where I can be the wingman right that Casey Understand so and one of the things that I appreciate about appreciate about him quite a bit is that He wants to know what he's talking about before he says something which you know puts him at the top top 20% of podcasters at least

3:46 At least at least at least 50% of this podcast Hey, is that HTTP get that out of here? That's not how we order coffee We order coffee by a SSH terminal dot shop. Yeah, you want a real experience. You want real coffee You want awesome subscriptions, so you never have to remember again Oh, you want exclusive blends with exclusive coffee and exclusive content then check out cron you don't know what SSH is Well, maybe the coffee is not for you The What I hope comes comes out of this A total is that we get kind of Casey culture aligned commentary on On AI and I mean Casey can I don't want to put words in his mouth, but Like we see software in a very similar way right? I'm similarly skeptical about

4:50 The quality of anything coming out of big tech these days about questionable ethical business practices and so on so On on general software not AI software Casey and I see I do I on on most things And so what I'm hoping is that I can be the the wingman and Like give a platform for more Casey culture commentary specifically on this crazy eye I think that's taking over everyone's life including my own right like as I know in our inaugural episode I mentioned that my life used to be much more quiet and nobody believed they I would work and we were just quietly doing fun stuff and And in a way things were better. I mean, it's good that That things are working now or sort of working And the sort of working actually is a big part of

5:33 What I care about trying to make it better than sort of work Um, so I don't I've gone on long enough. Maybe Casey you should put a bow on it I think you're I think you're being way too generous to me Yeah, I'm I'm the wingman on our podcast because I don't really like I just don't work with AI stuff So to me it was just like a good opportunity like have somebody on who like I trusted to give Solid perspective on AI because like it's really hard to you know The only real way I thought otherwise that I would be able to give any counters I'd have to go spend a ton of time with it, right?

6:04 And I just didn't really want to do that So so it's been great Uh, and I I've really enjoyed it obviously to each one I've recorded a couple things that you know we haven't posted yet So but uh, yeah, it's it's been really great having you on so And I I should kick it back to to prime and teach so like We don't want to just talk about our podcast here So like you guys had some AI questions you wanted to talk about uh or I don't know take take it wherever you guys want to take it We are here at your disposal. I did want to say part of it is to have both of you talk about the podcast Because I said Casey you should come on and then talk about the podcast with Demetri so that people can hear about it

6:43 I really enjoyed episode one So we at least wanted to make sure everybody knows that this thing is happening So don't feel bad about talking about it. That's why part of the reason that we're doing this today So that's that's number one. I really liked it um prime. I don't know anything you want to start Otherwise, I've got a few questions. I do want to start with some things which is Demetri can you please uh We didn't give you really like an actual qualifying intro other than your legendary Uh, yes, I should downplay that a little bit. Okay They say this for all the time, too, Demetri It's very embarrassing. They're Casey Meuritori So they're legendary for what?

7:17 It's like I talk like romp as just a warning everybody's legendary everybody You've been hearing some very good things about me prime. It's like it's a beautiful thing Best game developer Beautiful code everybody agrees You are the best every amazing code totally corrupt but but beautiful code It'd be a beautiful code All right, thank you guys No, I was just gonna say I appreciate the kind of words like I consider myself relatively competent But I mean there are like big names in the industry who are either friends or friends of friends And it's useful for me to keep keep myself calibrated relative You know Relative to people with much more at least public Public renowned So yeah, I don't I guess

8:01 In terms of introduction. We I've been doing Broadly AI-related research for 20 plus years like shipped shipped to many people my very first Custom design AI in 2025 at Google it was one of their first 2025 sorry 20 2005 Yeah, that's like that's like That's like banana and nano banana if you're shipping that one Okay Uh, sorry I've been doing this for a long time and seen lots of the ups and downs in the business and also I study the history of the business as well So I know about You know the ups and downs for like the preceding 50 years Like going back to you know going back to even the the 60s And so one of the things that really interests me is how this is going to affect other people Like people that I worked with for example who are now

8:55 I mean in the first episode we just talked about the impact on junior engineers and separate Probably should talk about the impact on senior engineers right because there's they face a different problem Which is if you're like let's say you're whatever 45 years old now You've been doing this a long time you have a good stable career making good money And now you're wondering what happens if They like crack the nut and in five years I'm useless, right?

9:24 And I know I know people who are wondering this myself um And that's an especially bad time to be useless right if you're like late mid career What do I do if I'm 50 years old and I don't know like what do I do next right And there there it's not anything like I'm the the problems that the junior engineers face where it's Again big problem for the junior engineers, but they have their whole lives ahead of them And they can't they have time to try to do something else But if you're 50 years old and don't have You know 15 years of Building up a new career ahead of you. What do you do?

9:57 So that's not something that I see people worrying about a lot as well Yeah, I don't I guess maybe I Talk too much about myself. You can't talk too much. Were you staying out with Casey? All right. It's fine We like it when people talk that's why we bring smart guests on Yeah, so I guess another thing I will add and this connect back to Kind of having a Casey style perspective on the industry is that I've seen a lot of how the sausage gets made both in terms of what the technical side but also Also the business side and this is something that We we talk about in later episodes about How how much can you take certain claims at face value and how to evaluate which claims can be taken at face value and so on

10:47 so what I'm In the same way that Like you can't take whatever Microsoft's claims at face value about improvements in windows performance Right you can see well, this is going to be the best windows yet. There's going to be perfect and flawless I was promised will be safe. It was going to be the last windows ever exactly as well I was told we lost windows well I did I they are still working on making it the last windows ever That's a good point I hear the Linux desktop is what tj's trying to say they they put that into the AI and it gave them a plan back of how to make it the last windows ever Executing on that flawlessly. Yeah, it is they're like we got this thing called windows 11 and it will ensure that windows 10 is the very last

11:38 All right, and if that doesn't work we have windows 12 Yep I um, so I've just got a few topics that I thought would be fun We can just see where uh, like they lead us and what happens I think there's similarly to on on the episode we said I've listened to You start on a topic and there's like 95 different branches that we can go off of So I'm not if we don't get to all of them. It's whatever. I think it it'll be fun But um one of the things I'm interested in and like it's just hard to get anybody's like unbiased I everyone's got to bias whatever just like a more rational take on like this current state of token cost and what that's looking like going forward I don't know if you have any thoughts to meet you about like is it going to be that

12:28 Same Altman was right and we're gonna get 10x cheaper every year or whatever I don't know that's what twitch chat always tell us what sam sam has it on the record. He said it's okay now He said it. He said it twice and made projections into the future saying it's a hundred x cheaper GP2 5.2 high will be a hundred x cheaper in two years Okay So a lot of that is Infrastructure development and algorithm development that is trade secrets that Like I can't evaluate and even like even an open AI insider. It would be illegal for them to to evaluate in public right I Don't know what they're cooking up there. I mean they have an extremely talented team a hundred x cheaper seems hard to believe but I guess wait and see so I mean one of the thing one of the problems with these

13:18 With evaluating claims in this business is that The timing matters as much as the content of the claim so like Like Musk is a good example just to take sam out of this for a moment where What was it? Musk was saying, you know, we'll have I mean we'll have You know reusable rockets in I think you first made the claim in 2005 or something like that And it took many years before that actually worked later. You know, when was the first promise of Tesla full self-driving that was like on e60 that he was saying and then every six months after that So I would I found it useful to try to separate out the Content of the claim from the timing of the claim So I would not be surprised Um, I would not be surprised if eventually we can get

14:07 A hundred x cheaper token cost whether or not they can do it now. I That's beyond my knowledge because that's uh Partly that's infrastructure partly that's uh like custom hardware partly that's can you get cheaper electricity part like there so many things that go into the into the business that it's hard Uh Hard for me to claim one way or another What I can tell you is that it has been as I'm sure you've seen it has been a Landrush mentality, right? So everyone's building as quickly as they can whatever they can. Yeah, put it out. Does it work Does it barely work? Okay move on to the next thing and some of that is Some of that is maybe Recklessness, but some of that is just market forces right because the

14:54 um Right now there are multiple really big players and probably there won't be as many big players in ten years And so they were given how much money they've all put into this already They really don't want to be the ones caught out, right? Yeah, and so the like the the arms race is Rational from a business perspective whether or not it's it's producing You know completely optional engineering artifacts. So uh all of that is to say I'm sure that there is very substantial Opportunity for For improving efficiency in the current stock and I don't know if they so my biggest question would be Is the is the internal stock stable enough that you can optimize it now, right? Because let's say you spend a year optimizing it to get the token costs down

15:44 and then there's Whatever the next architecture architectural innovation is does that invalidate your optimization or not? So I I don't know right and so you will hear me say I don't know perhaps more than most People talking about AI just because it's There's a lot that's unknown even to people who are deep like deep inside these companies about what's what's going to happen in like two or three years, right?

16:10 Yeah, I would add one thing to that that you know It's not not adding something as in information adding something as in sort of like a thing to think about And that is that It's worth noting that I If that costs were really going to get that low that fast I think I would have expected Google's costs to be very low already and I know it's a weird thing to say but If you assume that there's any hardware component like if you think that that all hundred X is going to come from software Then that seems okay, maybe that could happen But if we're counting on a substantial portion of that hundred X coming from infrastructure builds It feels to me like Google has kind of been building AI centric Like stuff for quite some time and has gotten

17:01 They're thinking that like their TPUs are very much just like we built machines whose only purpose is to do this job As opposed to like say for example in video who's not doing that like the And video cards are kind of like still in a weird like hybrid state where Only the most lay only the very latest and videos could be said to be focusing on AI really and even those are still a little bit hybrid feeling to me so I could imagine if someone said well, you know If all we had was Nvidia to look at Maybe you could believe that like you know, there's a very long lead time on hardware so You know, we don't know what they you know when they shifted probably four or five years ago to going like oh my god

17:42 AI is going to be the biggest thing we need to completely redesign everything to just focus on that That stuff will only be you know will only be seeing the end of that pipeline You know now or something like that right So I could see that but I feel like Google's kind of I mean am I wrong about the speaker?

17:58 I feel like Google's been doing this for a long time if there was a huge if there was huge gains to be had by doing just AI hardware I would have thought they'd have them already is that my way I'm not sure if they don't right so okay first the first time I heard about TPU I think the first time I heard that they were starting to work on it was like 2014 or something so that's just when I heard about it like okay I'm not a Google insider so I'm sure like Knowing how Google works. I'm sure that they were talking about that like three years before I heard about it in public right But when I heard when I first heard about TPU development was somewhere around 2014 Yeah, so I guess I have not been studying the

18:41 Whatever internal economics reports Google puts out it's possible that they are already substantially cheaper than I would think on operating cost substantially cheaper than opening I don't know and but I mean this is a something that's a common Folk belief in the industry which is The Google is sort of the quiet monster in the business because like they If you compare them for a moment to say open AI They have had they have been infrastructure leaders forever. They have been AI leaders forever Unlike open AI they have They have um trying not to say the word surveillance, but I'm just gonna say it surveillance So On the entire internet right so they have just a constant flow of potentially trainable information and you know xai has

19:33 Has that flow that that is their own proprietary flow of data so Many people have have this feeling of Google was kind of slowed on the on the product side I emphasize on the product side with like chat Chat LLM products But there is many people have this feeling of Possibly Google is just going to quietly leverage all those advantages that Nobody else has and like Xai is the only one that's sort of sort of close in And having that capability. I mean it's interesting to note that As far as I've seen in public xai They're merging with start link. They're talking about sorry with the space x Space x And they're talking now about orbital data centers and something that I thought was interesting there

20:25 was That the designs that they've talked about are GPU based not TPU based and I don't know if that's because They don't think they can Build it up as well as Google did or they don't think they can license it so I don't I don't know right But the thing I'm going to point out the designs I've seen I mean I haven't looked looked in like three months, but the designs I've seen were Estimates based on you know Nvidia style GPUs and not TPUs so they must know something because this is This is the kind of thing that they do right like the the big The biggest competence advantage that I would put in the kind of musk category of companies is The relentless execution on on building and optimizing and streamlining physical stuff

21:13 That's something that they've been doing really well for a very long time. So if Anyway, that caught my eye that the designs I saw were based on GPUs and not TPUs And I do know that they're working on their own Their own AI chips. I don't I don't know that much about how far they are there But it's interesting that that when they were talking about The orbital data centers designs they were talking about Get conventional GPUs as we understand them I Got to say orbital data centers like the coolest sounding thing to build Like it it seems really not practical. I get like I get there saying oh, we're gonna have robots do it And it'll be outer space and it's really cool But I got I mean you I just feel like everyone's got to admit orbital data centers

21:58 Probably the coolest thing to say you could work at and software development. So Yeah, I think like yeah, based on what I've seen for orbital data centers. It sounds like it's It is either much cooler or much less cool than what you're thinking about pending depending on your perspective, right? So If you're imagining like a giant building in space like yeah Like it it sounds like the kind of more leading designs that are like more plausible They're more like starlink They're more like satellite clusters where you just have lots of little things that talk to each other over Light links basically for high-speed communication Well and giant giant radiators right and huge. Yeah, it's it's almost all radiator

22:53 And then there's a little like nub in the center contains that contains the GPUs, right? Yeah That doesn't sound as cool as what I had in my head of a big skyscraper in outer space We've got racks of GPUs and you can walk around in it and it's like really cool But that's okay because when you tell your parents that you work at an orbital data center They won't know so it's fine. That's what they imagine. Yeah I say it's exactly like Star Wars mom and dad trust me Trust me Unfortunately like heat is just very I guess the as the kids say say problematic in space like you don't have anywhere We should kids like it's very hard to get rid of heat in space So this is a problem as Demetri was saying you you end up having to have a lot of surface area

23:42 Per chip like per per heat generating and videos like gp 200 or whatever I mean it won't be a gp 200 by the time they get these things launched but the uh, you know That's something I didn't I didn't get I don't What is the advantage then because in in like I get that It feels like if you talk to like a regular person about putting it in outer space They're like well, it's cold in outer space. So that's gonna be really good for GPUs But it's the opposite I think right. I mean I so I don't get it's not that it's not cold in outer space cold Is the lack of available, you know like yeah having energy. Right. There's just there's no density. So you have nothing you can Yeah, I mean like more like they said it cooling would be super easy

24:25 I feel like that's like the normy thing if you say put something out It'll be cold sick. It's like a huge refrigerator, bro But that's not how heat works. I like so I get that far so then what is the advantage? This is we also don't have to go way down this route, but I have been wondering this myself So what like why do they want to put it in outer space?

24:44 Incredible investment tj like the amount of investments that are coming in will be the tame the tame for outer space is big Think about how many data centers we can put in outer space Well, it's so it's free power and difficult, but free cooling, right? Okay So if you if you can make the physics work and I don't I don't know what they have really really good like thermal people I mean you can imagine they have very good thermal people at yeah at SpaceX. So I I assume that they have done simulations that that suggest of them that this can be made viable But yeah, so it's free power and also like more free power than you would get for the same panel on earth because there's no Right atmosphere eating up your atmosphere or weather for that matter. Yeah, no clouds right so free power and

25:30 tricky, but free cooling And so it's like when you build a really complicated thing, but then it's passive in a video game I understand this is this mechanic So this is entirely Business strategy speculation So whatever we take like poor big it's financial vice got it. It's exactly right Just investing right now If it works it works and you are Uh Musk's AI team and you just get to launch your AI into space and everyone everyone else is like fighting local governments and can I build a hydro plant here? Oh how much you have to pay you to right And you'll just be like we're going to space right like you losers can hang out here We're going to space and by the way if you want to go to space you have to come through me, right? Yeah

26:20 Yeah, he pretty much owns the the passage to space. So it's yeah, it's pretty synergically And ask Katy Berry. She's the novice place Well, and also like it's pretty trivial for like when you look at the projections for these sorts of things It's all about launch cost like whether or not it makes sense to put a data center in space It quote unquote data center in space. It's like here is the launch cost like at this launch cost We think we could do it because that and that's based on like the failure rate of the thing how you know long It's expected to be able to be up there, you know all those sorts of things And so at the end of the day if you know if You look at somewhere like SpaceX and they're like well for us

27:00 We could get our launch cost down to $50 a kilogram or whatever it you know the magic number is We just charge every one else 75 dollars a kilogram and then it's it's it's not worth it for them to put it in space Or there will always be worse than are right. It's a very good position to be in and at a strangely profitable bet for SpaceX If it turns out that these are actually useful in the future, so So I think we should probably attempt to get a little bit more on On a more practical approach to AI. I do think orbital data centers does sound amazing You don't want the title this video to be orbital data centers. There's gonna be like AI in space prime. That's good. How many clicks is that gonna generate?

27:42 That's my rock op rhyme writing that's crazy. How did you know? Okay? Because I am also writing a rock opera copy in space That's not gonna happen. He's just he's just gonna take it off for me. Okay, so We should probably talk. I think the number I get a lot of concerns from people and one of the most frequent questions I've been getting lately is is my future just reviewing code Yeah, so quite possible Let me let me try to be measured in what I like they think that is So look one one thing that I've been I have discovered in my own work and that I've been saying publicly for a while now is So I try to work on like repeatable and reproducible AI results So the bucket term I use for that has has been reliably I but it turns out that there's a company called reliable AI

28:42 So So anyway, the thing that I I think you can do right now like completely hands off and not have to review anything Right, so this is what I mean by reliable right that like if I ask you teach to whatever go implement Like download this thing from this HTTP endpoint and like save it into the database and give me a dashboard that Whatever calculates the averages of some things, right? I can just tell you that and expect the result and it will work And I don't have to talk to you about it again, right? So that's what I mean Inly is the completely hands-off style Right now I think the limit of that is somewhere around a couple thousand lines of Like relatively standard junior level code And I want to emphasize because some people some people have

29:22 Justifiably pushed back on this a bit. I'm not saying you can't do more by Having oversight of doing multiple tries and having algorithm test weeks. I'm not saying you can't do more right now What I'm talking about is what's the limit of what you can do hands off and you can mostly just trust that it happens Mm-hmm I think I think at at a minimum we are going to go through a Review heavy phase Because the the businesses are as I'm sure you know The businesses are trying to shove this into everyone's workflow Whether they like it or not, right? And I again lots of people lots of people even in the AI business You tell me like I hate how much they're trying to push us to use these things, right?

30:02 Yeah, I just want to do my job But like they're monitoring my token usage Right And it totally too not just like literally like I have friends who are like we have to use this tool Part of my KPIs like we use we have to use clawed code only That's and My manager has a dashboard with my token count for the month And it is the thing we talk about and review not even like oh you can use whatever you want We just want to see you guys experiment. They're like you have to use clawed code after this. Yeah, it's crazy Yeah, think about Amazon Kuro That's a lot of people are talking about that right now with the whole Amazon accidentally having several seven ones Yeah, they're like you have to use our stuff to take ourselves down if anyone's gonna bring it down

30:45 It's gonna be okay. That's you want to make sure we are But we want to make sure our site goes down and our AI looks bad at the same time We don't want like someone else's AI to look bad. No, that would be mean Okay, if it's gonna if it's gonna go down to the AI. It's ours should be our in the eye in AI. Okay. There you go. Very good. Yes So yeah prime on on that point. I think it's very likely that we will go through a Review heavy phase I think I think that at some point Like the workers will object to this enough that we're going to have to find something something more acceptable Right because I don't know if you've tried it, but if you're just like Reviewing stuff like you know claw dumps and other thing on you every you know five minutes

31:31 You're like I cannot have to review this and approve and merge whatever or or I mean you can just yolo right and but but we've seen what happens if you just yolo So I do think we will at a minimum go through a Like reviewing mostly reviewing phase and I think people are not going to like it That's my expectation. I don't know I don't know how I don't see what the alternatives is right now because yeah, so the either The two possibilities I see right now are like they're going to be monitoring your token count And so you need to be consuming and generating tokens and then that has to go that has to get linked which you know in some places that actually link the like tokens to PRs right And so You need to be generating lots of PRs. You need to be consuming lots of tokens

32:22 How can you do that other than Just generating many many PRs with AI and reviewing and like hoping that you're not taking down you know Taking down by by missing something in your review right so I don't know how we get out of that other than At some point the business side says okay, like we can cool it right we're going to We're going to stop monitoring your token use which I mean I personally I feel like That's not that different from it. That's top monitor like a keyboard monitor or something It feels very invasive to me like hey, you know, I understand you're asking me I think it's fair to ask me hey can you use this tool can you try to do better work?

33:00 I think that that's fair right like I'm ultimately you're like you're being paid to do Something for the business. I think it's fair for them to ask hey, we'd we'd like to be using these tools And we think you can do whatever like 20% more work if you can if you use this I think that's fair I think yeah the micro micro tracking feels worse than the past and It's really easy to game too, right?

33:25 You just like sequentially look at every single file and it's just going to be like dang dang dang I know people who are like at big text who are working on stuff and They have to do exactly what you're saying and they're saying I like I do I do this just to keep up appearances for my a i use and You know by the end of like my output changes maybe 10% right?

33:51 I do like how it's literally just setting money on fire and they're like this makes me look fast Because so that actually before we got distracted on the orbital data centers Which I'll repeat cool this name ever Um, we're cool. We're my company. I I Was I was interested in you know some of your thoughts about sort of the tokenomics side of things because right now It feels like people are burning a bunch of money trying stuff out and exploring and willing to just be like Oh, yeah, you just spent $5,000 this month whereas before like you couldn't get like a $45 like online course approved And they're like oh, but yeah, yeah $5,000 of tokens for nothing. Yes, we're totally down for that do that every month

34:31 Because I was interested to if you know Uh, like if we're going to see a big you know or your thoughts on if we're going to see a big push on like generating less or like more focus on like we're going to use more tools to check what's being gender You know because at some point, you know Quad-clogious released their thing about code review and they're like yeah, it usually is like 15 or 25 bucks or review And you're like well that adds up kind of fast. Yes, especially at the rate people are saying I'm shipping 10,000 lines of code a day Or I made a 30 300,000 line Ruby on Rails application for my blog. That's Gary Tann by the way Casey Casey that tweet Okay, um, hey he's boiling a lake and then he's going to boil the ocean. All right, that's what you have all this

35:17 Yeah, I side note I cannot tell if he's the most genius rage Vader of all time But we can circle back to that on a different episode. Yeah Um Yeah, so because I'm interested in in sort of that angle because I do find like some tasks I can get done Like a lot faster with letting agents work on it or like agents spin on something like forever It has a really clear outcome. There's some stuff. I already have a bunch of patterns in my code base spin But I'm like is everyone just going to be chill forever with me spending $500 of tokens overnight on this that doesn't seem like what A business would like. I don't think we're going to get $500 of value back From this feature. So I don't know if you have any thoughts like in that vertical or yeah, so this is speculation

36:03 Yeah, of course speculation a lot of this is Like principal agent problems for the business. Yeah, it's like The so there are for sure like CEOs and like one step down from CEOs who one way or another Um their Their personal benefit is tied to claiming that they did a lot with AI Yes true. Yeah, um, and as you know Uh, like in any big organization you never just evaluate something for whether it's good or not it has to be evaluated through some KPI right So yes, there are people. I'm sure like I haven't seen this personally, but I'm sure that something close to this exists that um There is someone with many millions of dollars worth of bonuses tied up to Can I get our token usage to double this year and can I get our PRs to increase by 20% as attributed to

36:58 That token increase right right like extremely bureaucratic thinking about Using this new technology, right? So I'm sure that there are many people like that um, and so Uh, those to those people They're not they're not spending the money right they're spending the company's money right They're telling you to spend the company's money to do something that makes them have to go up that gets them a bonus, right? Yeah So I don't know how long that's going to last there, but there's certainly right now Um, a very large premium at multiple levels of the kind of the business stock where people are like, you know Why why isn't your token usage double right and yes Yeah, so some people that sounds like a joke. I have friends who literally they say my boss came and asked me

37:37 Why has my token use usage double right but you only used three hundred dollars of cloud last Last month honey you barely touched your cloud last month Is something wrong sweetheart? Okay, so I do actually want to follow up on that because this is the you know This was one of the big things that has been going around is this Amazon Kiro thing taking down everything And now they're saying hey, we're gonna make it so that juniors and mid-level people who use generative AI must have a senior sign off On this can I do a quick question on that as well prime, which is was there policy before seniors didn't have to review juniors code Just a general I still I assume it's like a lot of companies, which is that a junior could have a mid-level person review the code

38:17 Say hey, this is not really good like every change needs to have you know the same because you you effectively will exhaust and lose your senior population if they have to review every single PR and so kind of that makes sense So excluding the personnel problem that will likely develop from this with Amazon Is this like the first sign of people realizing that token measurement isn't Necessarily the best metric because I assume that's what's hurting Amazon is that they really push super hard as token metric is the The greatest thing that has ever existed and you must only push on this one thing And now they're feeling some of the effects of maybe moving too fast Is there a world that's going to exist where people are going? Okay, instead of

38:55 Trying to double metrics. Maybe we should double some other Thing say I don't know uptime Maybe like uptime could be the thing that we're like you know that we're valuing and then that can be That could be it like is is this an actual Like path forward or are we From your perspective are we just going to see continued push on double your token usage double your token usage Because they both sound super appealing I guess from my purely theoretical point of view of like oh more features We could know we could get all the features or no, let's be stable Again, I think the so as we as we discussed with Casey and Casey you should you should jump in on on this part um Like we've seen these cycles of like everything has to be you know like

39:39 25 years ago everything had to be online and then everything had to be uh Web 2.0 and then everything had to be like social local mobile app and then everything had to be crypto right and that right so Um, we are in that phase. Absolutely right now. We're separate from Any utility that the technology is offering. There's just this social media of everything has to be this right now Uh, I don't know when that fever will break Um, I think certainly events like the Amazon um Amazon event will Will help with that and I do think that at some point people someone is going to say Okay, look like um We just can't be burning this kind of money all the time and also like setting our infrastructure on fire and driving our engineers crazy

40:25 At some point at some point we should actually get a benefit from this instead of Getting ourselves you could only pick two points on that triangle driving Crazy burn a lot of money uptime you get a pick two of those points. Yeah, that's true um But I think I think this could last for years and the reason I say it could last for years I know people will not like that Uh, I think the reason you this can last for years is uh, it's easy to underestimate um like on like you know, you guys are like people who watch this uh, you know watch the stream How far how far up the the like power lock herb of early adopter you are and so true We just meet lots of people all the time who Still conceptualize AI as like it's a better search engine, right and I mean it's uh, it's easy to

41:17 forget that Actually the vast majority of people don't know much about this and aren't really trying to use it and um So all those people somehow will have to be carried through the uh, the fever swamp Um, and I don't know like I think that could take years I would agree with that uh for and add the sort of the chaser Which I think we did talk about on that first episode of the podcast, which was I don't see it as mattering whether it takes down proud all the time either is the problem because like At least my experience for the past 20 years has been it doesn't matter how bad the practice is If it's just something that is in the zeitgeist then people do it and you can show them Clearly like you can literally demonstrate look if you had done it this way

42:04 This is how much better it would be they don't care because the best practices is that you do it this other way The best practice is you use python then we're using python or whatever it is and you're like look If you just written it this way It's a hundred times faster and you're like we don't care That's How this industry has operated and so it would be very weird to think That in the special case of AI that anyone would care one way or another whether it even was better, right?

42:29 So you can almost take your take your guesses about how Good AI will be in practice about generating code in the first place You could even put that aside I think the fact that it has this much momentum is enough to know that this will be here for a long time The current way we're doing things. Even if it never gets any better than it is right now Even say it exactly as good as it is right now. No improvements literally at all I don't think that would change the outcome honestly. I really don't Well, there's like a trillion dollars invested in a two right so they're like we exactly even if even and it's I mean It's not really even sunk cost fallacy. It's just like literally sunk cost truth to see which is like we really

43:13 Yes, we really need this to work guys or we're really underwater, right? It's not like yeah So it will become the workflow. I think that's like inevitable and Like I wish that I wish that we could say like well if the AI's don't improve Substantially then people would like rethink the I don't I don't think that's true Like I think whether or not they're able to push the quality up I mean, hopefully they are because you know Everyone is putting a lot of money into pushing the quality of the as so hopefully they do get better But even if they don't I don't think it's going to matter personally So I actually have a bit of a fall-up to that one Casey and I'm No offense to meet you. I'm actually very curious on Casey's not on this one

43:56 in 2024 GDC started doing this survey asking about The perceptions of AI inside of the game development space and in 2024 18% of the developers held a negative view on Gen AI in 2025 30% of the developers at GDC had a negative view on AI in 2026. It's now at 52% And only 7% have a positive view is there Is is this like a special unique thing to games because games does contain a bit more art than the average programming Shall we say is this like a special games thing or is this actually a A momentum change that people are feeling due to hey, you must use this is the greatest thing ever You have to use it to therefore and then you start using you're like, okay, well, it's great here. It's bad You know, you you feel this giant disconnect from Twitter because I have my own opinions that are largely formed

44:45 Because of how much I hate what Twitter has to say I mean obviously I can't speak for people who are taking that survey I mean they know what they were thinking when they checked the boxes or were Pushed whatever they pushed But I you know, I think as I've said before I think it's very difficult to separate out people's opinions of Gen AI The technology from Gen AI the people and the companies and I think When we look at things like negative surveys, I will say I'm right there with them I have nothing but disdain for this sort of A current moment But that's not based on an objective look at generative AI and where I think it's going It's based more on the behavior, the legal behavior Um the attitude it's based on you know like many things when you know we're humans we make decisions largely based on

45:45 What we think the intentions are of the people who are in charge of things because at the end of the day Like that's you know if we have a really good feeling about someone who's running something We are less concerned about that thing that if we have a really bad feeling about it right um And so I would suspect that a lot of it comes from there and that you know Especially like you said there's a lot of more there's more artistry in games There's a lot more people making our assets a lot of people more people making music sound effects making narratives And you know the behavior with respect to that uh of the AI companies has been utterly deplorable Right i mean i've said this many many times on twitter

46:24 The you know it's it's a Pure disdain for the law no concern about what they were doing They just they basically just made a decision We're just going to do what we're going to do and we'll deal with the lawsuits later was clearly what they decided I mean i don't i don't have evidence that they literally decided that in a meeting But their external behavior makes that clear They were if they had been at all concerned about the you know Whether they're breaking the law they would have done a lot of things first that they didn't do Uh by the way Casey uh what's it called uh the Facebook actually was caught with email saying this is all legal right yes This is don't do it on a company computer idiot

47:00 We don't multiple of them like now if you look in their in court documents Multiple of these AI companies are on the record as having known they stole things Not not about fair use of that like just literally Pirate like this is transformative no no no there's none of that there's like yeah there's like we We are literally taking an illicit step to acquire data and we know There's multiple companies that are now in court records like They have that evidence right so yeah So i think also know like like get hub could have just filtered out Non permissively license stuff to train co-pilot for example like that you know So that's the easiest one in, that's one of the most easiest, like you own the search thing that you're doing to find the code.

47:45 You could have clicked your own checks, it's not like, oh, we just called the internet, and we got back 10 million things. I mean, like one of the primary things that these LLMs are good at is understanding language. They all now are at the point where they could analyze their own training data set and assign a probability of legality to each document.

48:06 That could be done today. For all I know, they have this, right? 100% legal for my company deals. So you don't get it. Mark, as you don't get it from us, it's legal. OK, so that's from most. So I think that's from an artist's perspective, you can see why this is a big deal, because it's not just looking at something and saying, I'm afraid that this thing is going to take my job, or I don't like using this thing, because it doesn't feel like art to me when I just, it feels like management, because I think that's another thing that we haven't talked about that much.

48:40 But an artist wants to directly manipulate the thing that they're doing sometimes. You tell them, oh, well, now you don't do that. You just ask it to make this thing for you. That doesn't feel like art to a lot of artists. Some artists are fine with that. There's technical artists and there's non-technical artists.

48:57 And a technical artist might be more used to that, because they're used to kind of going through this technology intermediated process to produce what they're doing. But a non-technical artist, they're in that business because they like the direct tactile work of making art. And so not only is it bad in that sense, where they don't like it, because they're like, I don't really want to use this, and I feel like it's threatening my job.

49:19 That's obviously bad. But then you add on top of that, oh, and by the way, it was your own artwork that they used to take your job from you without paying you anything extra and without asking you first. I mean, how do you not check the negative, like it's obvious that that's just bad. Nice about it.

49:36 It's also the same. No, they're not there in there. Hey guys, guess what? Guess what? We're putting all of white color workers out of work. Am I right, boy? Yeah. Yeah. TJ, yeah. TJ, you are, to be fair, TJ, that was a bit of a misinterpretation, okay? What actually happens is Dario first makes a really painful face and goes, I'm so sorry, but we're going to have to have some economic struggle, okay?

50:03 We're just going to have to lose some jobs, okay? Some are going to have to go now. And when I say we, I mean you, because I'm not interested in struggling at all. Okay, also cookies, you're a broad boy. I mean, what I would, I have proposed before I was like, they always say like, we're going to have universal basic income.

50:22 My thing is like, look, if everyone at AI companies demonstrate that, all of you go on UBI first. Like all of you go on a fixed low salary first because if that's what you believe the future is, let's see it. Let's see you guys all be happy with that first and then sell it to the public, right? But they're like, no, actually we would like all the really high salaries for us and everyone else can have the UBI.

50:47 Like it's just, it keeps getting worse every time they step into these things. And so, I don't know, my opinion on that is like, it's obvious why someone at GDC would check that box. I think that's probably going to be true for a lot of workers, not just game developers. It's just that game developers are more exposed to this technology currently because they tend to be more technologically savvy than say someone working at an accounting firm randomly right now, who maybe hasn't really seen this stuff all that much, their firm hasn't started adopting it, whatever, the average worker may not have been encountering AI to the degree that a game developer has.

51:23 They're further ahead on that curve. That would be my answer prime. Dimitri might have a different take on this, but that's what I would say. I completely agree. So the two points I would pull out starting from the most recent one is the accounting world has not been dragged through this yet. And they will, the HR world, so I happen to know someone who works on apps for HR surveys and stuff like that.

51:49 We've all seen these very TDS HR surveys right there. But all those were made by someone who had a job which was like a HR survey designer, that's probably not surviving, at least not in the current form. And they don't, like that world doesn't even know that this is happening yet, right? So all those other industries, like the reason they're not yelling yet is because they just haven't been under the gun yet, right?

52:16 And then the other thing was, I think Casey made this really good point about if you are, if you're the kind of, if your position in the art production pipeline is more editorial and composition, so you're not making the assets yourself. You are, you're up artists under you who are making the assets and then you're doing editorial work or composition or something like so someone's, I don't know if art director is exactly the right word, but this is parallel to, you know, the people in tech right now that I think see the biggest buff boost from what's happening right now is a technically trained project manager or product manager.

52:56 And I know a couple of those guys who are talented enough to review their own code and understand when something is obviously stupid and they seem to be doing plausibly good work. And so whereas they used to be managing three or four junior engineers, they just manage 20 cloths or something, right? So anyway, the, this idea of if you're, if your position in the pipeline is editing, assembling, composing like the big picture structure of somebody else's work, I think those are the people right now who are least disrupted and most, most benefited.

53:39 Prime will go back here about to say no. I'm reading for Prime to, no, no, no, no, no, no, no, no, no, no, no. There was a stroke. I'm considering, I'm considering his position, which is that I'm just playing through my head, like the biggest thing that I really enjoyed doing was not in fact managing juniors, but in fact writing the code.

53:54 And I've also had some of the hardest transitions in an AI world because I still just like writing the code and I find it less, you know? So I'm like, oh, yeah, I see myself in what you're saying is that there's a bunch of people. And now this is where things, this is where the, I call the, I call it the classical bullshit thing, which is that people say they're good at this, but I don't think you can review code and actually be good at it, beyond a certain amount of lines per day.

54:21 So it's like, if you're reviewing more than like 500 lines of code a day, I just don't, I just don't believe you. Like at some point, there's like some line that exists somewhere where you're just like, oh no, I really have to think about this. It's higher than 500. Okay, a thousand, right? Oh, I mean, but people like, I'm for, I can, I think what he just trying to say is like, he can do, you know, a lot more lines of code than average.

54:44 What I think he just trying to say, there's a, there's a, maybe it's 500 for you prime. For me, I have, for teenagers like 10, 20,000. Yeah, no problem. 10, 20,000, there we go. That's closer. No problem. That's why we call, that's why we called, you know, JT and TJ, they're just mirrors of each other.

55:04 Okay? You use code, by the way. Yes, I agree. That's why JT is Gary Tan. If you're wondering, even though the JT, but, oh, that was a stretch. Yeah, he managed to get a power cable from the Best Buy. How's he doing? Did he, did he get in line there? I mean, that is just these things. You're able to get that power, that MacBook power it up.

55:21 Because the whole future of civilization is riding on Gary. And if he doesn't get that power cable, I think we're on a lot of trouble. If Gary's list, don't keep listing. Yeah, I don't know if you guys ever saw, there was a fantastic movie starring Ice Cube called War of the Worlds that came out recently.

55:35 I know if any of you saw this movie. I watched it in five separate small clips. All right. As you know, you think I'm joking, but as you know, the fate of humanity itself can very well hinge on one delivery of an Amazon thumb drive. So I don't think what I just laid out is all that far fetch. Just go watch the movies, you don't believe me.

55:57 It's a document. I heard it was, that's what I heard, too. Yeah. I just am super proud that Ice Cube was like, give me that bag. I don't care what's about to come out of the other end. Just give me the bag. I, you know, I, all more power to that guy. But what I'm gonna say, do you think they asked before Ice Cube who they like read the script?

56:12 And they're like, nah, I can't do this. Bro, I cannot do this. What are you talking about? No, in the meeting, they were like, do you think we can get Ice Cube for this? Nah, he's never gonna do it. Well, just ask. What's the worst gonna happen? Just ask. And he was like, yes, I'll do it. They were like, oh my God, thank God, we didn't have to go to Daniel Day Lewis.

56:29 Yeah. All right. I believe I have, generally forgot what I was going to say. Sorry. I will just at least. You were saying there's a limit on code review. You were like, you're saying you don't believe, you're calling bullshit is what we're doing. Okay, okay. There was, I forget where the professor was.

56:45 I forget her name. I think it's Susan Simpson or another. I just read it just two days ago. She's coming up with this for call the cop. Yeah, I know, cognitive debt. It is, and so instead of just being like tech debt, which is here's part of our code in which we know is really, really bad, but we've kind of covered it up.

57:01 And someday we're gonna be able to come back to this and actually make it better. We can't add more features on this one set of our product. Cognitive debt is when you have like a senior leave your team, and then there's just a portion of your code base that nobody knows how it works. And so her whole argument is that these people effectively claiming that they're reviewing, as TJ said, 20,000 lines of code a day, really all you're stacking up is.

57:22 I want to make it clear, I'm not saying I can review 20,000 lines of code. Yeah, just fine. All right, 50,000, 60,000, whatever. Yeah, that's too low. That's way too low for me. But you're getting this idea of cognitive debt, which is that we're stacking up something in which nobody knows how it works.

57:40 And you now are only like, your fate is directly tied to the performance of say, clawed or chat jippity or any of these programs. Because you can't like me as the human eventually, like I can't even go in there and make a meaningful impact in the day anymore. Because it's just like me starting a brand new job at, you know, Google.

57:58 It's like, okay, figure out the code base. You're like, I don't know, there's millions of lines at this point. And so I'm just curious what the future is gonna look like here. In six months, because I was a lot of companies that are currently stacking cognitive debt. And we don't really know the outcome.

58:12 I had a really good insightful question here, but I don't remember what it was. So that was my best swing at all. I'll just say to really quick prime that I was reading somebody talking about this problem. And one of the things that I thought was really insightful about it was they were talking about how like, maybe even if 99% of the time, the LLM can understand all of it.

58:33 And it works really good. Like, there's so many catastrophic outcomes that can happen. Like, for example, let's say you're running a SaaS and someone depends on your site and something goes wrong. And then the LLM gets stuck in a loop and cannot solve, right? Which like, everyone who's done this, it happens.

58:52 It happens even on sometimes stuff where you're like, bro, it's literally, it's right here. Like, I'll just make the change. I'll make the change, that's fine. Put the token in the background. These hands were not meant for anything besides prompting, but I'll still do it. These hands were built for Excel, OK?

59:06 And so I was tippy-type in the editor still occasionally. And it's like, but so, OK, so then you imagine that. And then now you're like, OK, I need to run my agents in a loop. They have to do all this crazy stuff. No one can understand the code. And then it's like, three weeks of downtime. Like, you just lose all your customers.

59:26 You'll all leave. So there's so many things where I feel like people are not applying like a proper level of like even just simple common sense game theory. You can call it whatever you want to make it sound cooler orbital data center. And orbital data knowledge, if you will, to this problem where it's like, yeah, you can get to that point, but then what happens when somebody goes wrong?

59:51 You actually need to think about that if it's your business and not your hobby blog for Ruby on Rails boiling the lake. I don't know, sorry. The feature you were going to say something. It's been really one quick second. Sorry. There was a paper just released. Just release the classic guest experience.

1:00:08 Sorry, me and teacher are talking everybody. Well, that's it for the stand at this. Sorry, we didn't get to talk to the guest at all, but maybe they can come on again next week. That's actually how we're doing it. Yeah, we did clip a section out to put on YouTube and bash happened to not talk a lot in that section.

1:00:23 Like, guys, that was so rude. Bash never spoke that episode. And I'm like, no, she spoke a lot later. We had a whole segment for her, guys, relax, relax. But anyway, sorry, Brian, continue on the thing. I was just going to say they did release this paper. There are some holes with which like models they tested, but effectively the floor rate, the bottom rate that if you give it a document and cannot retrieve answers out of it, its hallucination rate was, or its incorrect response rate was at 1.19% at the most bottom rate.

1:00:51 And so this is out of these like, and it gets worse as the context window gets open. So like, you know, just imagine what that means compounding in a code base at some point. You could find yourself in a situation where you'd like TJ describes where you're just, you're beholden to use somehow conjuring the correct context and the correct prompt until you can master this or else you're just spinning tokens.

1:01:11 I personally had this happen on a problem, so. It's just a very exciting thing. And it was a very simple solution. All right, sorry. Go on, Dimitri. You're very, very smart. Yeah, so I'm just going to say that on a talk to that and connecting to what TJ was saying about the 300,000 line Ruby on Rails blog.

1:01:29 I think people are easily kind of duped by confusing is this thing like a software tool or is this thing like a personality? So sometimes I like to suggest this mental exercise. Whenever you get an AI output, I want you to ask yourself, if someone told me that they hired someone on Upwork to do this and they delivered it, how would I feel about it, right?

1:01:54 So for example, if you hired someone to make a very simple like 2010 era blog and they showed up with 300,000 lines of Rails, something's wrong here, right? I probably... There's 150,000 lines of tests, would you say there's something wrong there? Quite likely. Yes. Yes, I would. There's something wrong up here.

1:02:13 There's something wrong up here, right? Right, so this thing is an inhuman artifact, right? So, like, in the discussion with Casey, I was making this joke about like a formerly online app to order pet food, right? And it was an exaggeration, but it's only like a factor of 10 exaggeration, right? Like a 300,000 line Rails blog is like, at least 10 times the size it needs to be.

1:02:36 It's made to do this. Right, it's made to be. Exactly. Yes, it's... Rails new blog, or Rails... Yes, yes, yes, I was gonna say, there's probably just a built-in thing in Rails to like have this happen, yeah. This is how people were like, holy cow Rails is amazing. Is DHH built a Rails blog on stage? I'm pretty sure.

1:02:54 I'm pretty sure that's the initial thing, yeah. So you have this inhuman artifact, right? A 300,000 line blog. And you also now have to rely on an inhuman tool to do anything to it, right? So you're like, first of all, you could be screwed because the tool can't do it, right? As you were saying, but like it just spins and doesn't solve a very simple thing.

1:03:15 You could just be screwed and you're locked in, right? So you may not be locked into exactly the buyer. Not the good way. Right, right, right, right, right. You are locked in somehow to using an AI tool to deal with it because like, what engineer, let's say that the blog goes down tomorrow and it's like, it's a big tragedy.

1:03:33 Lives are on the line somehow. And you go and try to get an engineer and say, like, we're gonna pay you anything you want. We need you to dive into this 300,000 line Rails blog and solve this subtle problem. Who's gonna do that? Or at what price? Quick question, Casey, what would it cost to get you to go solve the 300,000 line Rails problem?

1:03:55 So the answer is very simple. I'm gonna message DHH really quick and ask him how much he would charge and I'm gonna add about 50 bucks to it. Not at all. And I'm gonna do it as a pass through. Smart, I love that. This is a really interesting strategy for monetizing and contract. Get someone who knows how to do it and then charge slightly more.

1:04:18 That's my motto. I love the strategy. That's my motto. But I'm gonna go ahead and guess that he will charge a lot of money. I'm guessing he won't be like $30. Like I'm guessing it's gonna cost you a lot of money. Sorry, I interrupted you to ask Casey that. Go ahead Demetri. So I'm just riffing on that.

1:04:36 I think one of the things that, so there's something I partly, I do with partial success in my own work and I think many people are going to be doing this and the coming years is working on driving AI outputs to something that might be human scale. So getting it to optimize for personness or optimize for legibility in the human sense.

1:04:59 I don't know of any well-established practices for this, like I basically just mess around with it until I get it into a state of like, okay fine, maybe I'll accept 15,000 lines of rails for a blog. I'm definitely not accepting 300,000 lines, right? So I think that that's definitely, that's something that is coming because it has to, right?

1:05:20 Like we're not going to be shipping 300,000 lines of blog. Like every single blog is going to be 300,000 lines of, like not, so not just 300,000 lines of code but 300,000 lines of completely custom code, right? So it's like, one thing if it's, I can't remember how big WordPress was, I saw someone do like a side-by-side of this thing versus WordPress.

1:05:42 And WordPress was like, there are lots of like side modules and stuff but they said that the core was like 150,000 lines, which, okay fine, that's bloated as well but whatever, but it's WordPress and the whole world knows that code is, like metaphorically, the whole world knows that code is. So it's not, it's not 300,000 lines that, not only does the whole world not know it, not even one person knows it, right?

1:06:05 Not even Gary Tan knows it, right? So I just don't think that that can work long term and you know, to play the other side for a moment here, like everything here is new, there are growing pains, people try weird stuff, right? Like early in any technology or even like in games, like the early 3D games all looked weird and played weird until people figured out roughly how to get like stick controls and cameras working reasonably.

1:06:33 So some of this is just, you know, people are working through the initial weirdness and yeah, it's weird. So the part that I find a little, frustrating is we could be a lot more honest about that, right? Yeah, we could just say, hey, look, I made this thing, it happens to work for me, this 300,000 line thing, but also obviously this can't be, this can't be the path going forward, right?

1:07:04 Or at least can't be the best thing possible, right? Like probably the blog would still work at 200,000 lines of code. Like probably, I don't know about that, TJ. Okay, this is the blog. I guess I would add something to that, which is, this is like, if I had to summarize my primary frustration with AI currently, it is sort of just the condensed version of the things that Dimitri was talking about, which is basically like, it's being adopted too soon.

1:07:36 That's like, because it's a gold rush and everyone fears being left behind and like you've got to get on the AI train and all that stuff, the result of that is people are using it prematurely. So they're tackling things with it that should not be tackled, like, they should be tackled in an experimental sense, right?

1:07:55 So to be fair to Gary Tan, just something I'm never gonna do again, but if I had to get Gary Tan this one time, thank you. No one gives a crap what he's doing, right? So like the, you know, he makes a blog, it doesn't matter, that's not gonna take down AWS, right? It's just some, how much rail code there is.

1:08:14 Also, we don't know, maybe Kiro depends on the blog now. So. So it's just like fine, like that's actually, Gary Tan making a 300,000 lines of code blog is actually very harmless to the industry, I would argue. So it's actually not that bad, you know, it's fun to kind of poke at him on this podcast or whatever, but like actually I don't have a problem with that.

1:08:37 I really don't. Great, you know, he's playing around with this thing and we're learning something from that and hey, the results weren't that great, but and maybe he's not ready to admit that yet, but like, you know, that's, we learned something from that, right? And that's all fine. Yeah. The worst thing is when, yeah, like when Amazon starts using this at scale and that was a bad idea and now people suffer the consequences because the infrastructure gets more brittle and all this other stuff.

1:09:04 And so really that's most of it. It's like I would much prefer if we could have done this in a way where we were validating that the AI could really do this better than humans for whatever the task was and then we roll it out, that would have felt a lot more comfortable. And instead we're very much riding the curve where like, oh, can an AI sort of maybe do this at all then we're doing that?

1:09:31 Like in production as the actual model and I understand why people are doing that because of the gold rush sort of, or the FOMO sort of problem, they don't want to be the last company to adopt. I get it, but the problem is like, I think we're going to pay a lot of, I think we're going to suffer a bunch over the next two or three years for no reason other than people, if they had just waited like each task, if they had just waited another six months or another year to like vet these things out and let the AI's get better at the same time and all that, we could have prevented.

1:10:06 They're like, they're riding just below where they should be instead of just above where they should be, right? I don't know how you feel about that to me, Tree, but that's kind of like, yeah. The thing that I see being the bad part currently. So it is perhaps worse even than what you are imagining right now.

1:10:23 So multiple times. So you're talking about within the tech industry, I very frequently work with industry that are not tech grade. I have worked with people who are in charge of various kinds of regulatory committees. And for example, they have a dispute about how a particular category of product, I can't say what it is, but a particular category of product should be regulated.

1:10:46 And they'll say, well, maybe we should get the AI to write the regulation. And like I have to talk them down and say, look, wait, stop, you know, stop right where you are. You obviously have an extremely dangerous idea in your head, which is that this thing can, that this thing is smart in the way that you understand a human to be smart, and that when you ask it to do, write me, you know, 10 pages of regulations on the safe use of this device, that it's executing like human value judgments the way you might, right?

1:11:16 And they don't know that, right? These are like smart people, like lots of degrees, 30 years of industry, these are not stupid people, but they just have very dangerous mental models of what these things are and what they can do, right? So like how are we getting told all the time? Yeah, this thing is, this thing is literally God.

1:11:34 This thing is literally God. It's impossible for it to make a mistake. It knows every book that's ever been read, every scientific fact of all time, and it can't be wrong. That's because me, Dario, me Sam, or me Elana, or whoever, we're saying yes, and I created God. Yeah, and they say that exact thing to me that well, all the, you know, like all of the federal register, like all the federal rules, right?

1:11:56 All the federal register is in the training data, right? So it should know all the rules, and I can just ask you to generate, and I spent a lot of my time talking people down from these kinds of ideas, and the AI companies absolutely do not make that any easier, because they're obviously highly incentivized to say the opposite, right?

1:12:14 Like it's a lot harder, like, and partly there's a, like a, not a person whose dilemma is a Mexican standoff, I guess, where the, like if, like, if whatever Dario were to come out and say, you know, actually, let's talk about having reasonable expectations. This isn't going to solve all your problems in life.

1:12:35 It's not going to get you a girlfriend, it's not whatever, right? That person would almost certainly be then at a disadvantage, at least in the financial markets, you know, compared to what, you know, Musk or Altman, or, or, as obviously, the deep mind might be saying, right? I didn't mean to single out, like I think the problem is pretty equally, for each of them, they have this exact same problem, and they also need boundless amounts of capital to execute on what their current vision is.

1:13:06 No, no malice required, just they think the way to get really awesome AI's to build a trillion dollars of data centers, well, they don't have a trillion dollars on hand, they have to raise the capital, so they can't, like it, like it, they'd actually make sense, it follows, yeah. Well, everyone is motivated by the same thing.

1:13:24 Everyone is worried about being cars references. In second, third place, fourth place, whatever, right? So if you're terrified of that, and you think that it will justify, you know, that that justifies it, then, you know, the AI companies are behaving in that way, for the same reason that the other ones are.

1:13:44 What is, it's not that funny the cars references. That's just being motivated only by cars preferences. But we know that about him. We know that he's only meant, like, that Twitch chat just, Twitch chat noticing my joke always makes me laugh, sorry, Casey. Okay, so I was done anyway. I want to throw one thing out there that, I think one thing that is kind of, I would argue as being massively understated in this is that it's not that I think all these AI companies and all these people are thinking that they're going to be losing their job and they have to be keeping up with the times or else they're going to be left behind and in the permanent underclass.

1:14:21 I do think a large amount of people just don't perform that hard on their jobs. I've worked with plenty of people who, they just want to really arrive there and do three hours of work, right? We've seen the TikToks, we know this just simply exists. Sure. And if you have something that's going to take the 30 minutes you would normally spend on writing an email and trim that down to three minutes, you can just compress your four hour workday into 20 minutes and you're just like, why would I not press the easy button?

1:14:46 They're going to press the button. And so for me, it just seems like a very simple, like a large percentage of this is just like, oh, this just makes all the little mundane things I do not in software world super, super easy and for those that just don't care. And I mean, we've all worked with plenty of people that can produce amazing code, including myself.

1:15:05 And then you produce this amazing and you're just like, oh, dude, I can just easy button this one thing out, easy button in. But I don't, that's on an individual base. That's like somebody individually who decides to use AI and they're really not going to be the problem, quote unquote, because the problem comes from institutional adoption, right?

1:15:25 The product, that person is not really an issue for anyone other than maybe, you know, if you're getting slightly lower quality work because the AI isn't doing as good a job writing the email, but it probably is. So like, I don't know that I'm that worried about even that because if someone was that checked out to begin with, like you say, I mean, how much worse is the AI really going to be?

1:15:46 So the problem is more when you have this institutional adoption because then what you're doing is you're forcing people who, the people who were taking their job seriously to take it less seriously effectively, right? Because you're forcing them to use this tool and they may not be able to get the results that they want from the tool and you're forcing them to use it and all of that stuff.

1:16:08 And so if that's premature, that's to me the problem. I don't see it as being a huge problem with a lazy person just wants to use AI to make themselves even lazier like because they were already kind of a problem to begin with and like, you know, how much more of a problem are they really becoming? I don't know.

1:16:24 To me anyway, that would be my gut reaction to that. But probably maybe you disagree there though. It's not that, like I mean, obviously the institution is the catalyst, but when, you know, I don't know what percentage of people at these, like super large companies tend to have a very large amount of checked out people.

1:16:39 It's just like a reality. But they were already non-contributors sort of. It's more sort of non-contributors, but they're also non, they also didn't make a big impact. Now they can move at a rate that is, like, unknown to them previously. And so now they can't, because they're lazy. They can make bigger impact than they've ever made before, right?

1:16:59 But they can make bigger. But maybe I don't want you to be 10x more productive. Like right now, I can tell you that I've had more problems on YouTube studio than I've ever had in the last five years. I've had in the last year. And it's almost like every single time I use that product, a new thing is just completely broken constantly.

1:17:14 And it's just like, I know Google is, like, historically a place where you go not to work. That's just like one of the big thing of the general software. And I know there's some places in Google that are very rigorous and very difficult to work in. I'm positive that exists. But I've met enough people to know that there's a lot of coasting going on.

1:17:31 And now there's a lot of changes happening, but everything feels broken. And I don't know what to attribute this to. Other than like, hey, we've given a bunch of people easy buttons. Now everyone's moving at a rate faster than they normally would. But they still have the same decision making level that they did before they went into it.

1:17:46 Which is like, to me, the dangerous part is like, I don't want you moving too X faster. I want you moving more. I want you to move the same speed. You can be right. That would be terrible. If a bunch of people who don't really care are not suddenly doing a lot of, quote unquote, work, that would be very bad, obviously.

1:17:59 That is, I don't know. What appears to be happening, okay. When I agree with you, that's pretty bad. If that's actually true. It doesn't have to be they don't care. It's just they may not have even the skills to recognize. Like to meet your point of saying, like, hey, I thought the AI could solve this problem well, but it actually can't.

1:18:15 It actually can't write a good rigorous spec for this thing for 10 pages. Or like, I'm a junior dev. I don't know that these five things are actually hidden foot guns that I shouldn't merge that are going to turn into gigantic foot cannons in PR2345. But now I could just merge those in and they're like a cascade of effects because nobody is putting the stop to those things.

1:18:39 That part does strike me as kind of dangerous. Okay. Demetri, I do want to circle back to something you said. You said you're working on trying to minimize these outputs with like, okay, hey, we have 300,000 line Ruby blog, but really that should be, you should be able to get that down to 50,000 lines, 100,000 lines, right?

1:18:56 You're trying to work on how to reduce the entropy not grow it. How like, is there actual meaningful progress being made in that region to the point where a year over a year you're saying, hey, this would have taken 300,000 line Ruby blog posts, but now with our work right, 275. And is there any sort of like direction that you're saying, hey, we're actually improving this kind of stuff?

1:19:18 Because that's, to me, that's very appealing. Like, I'd much rather see it play code golf than play whatever it's playing right now. So that's a, this is a small slice of the industry where I'm working and I'm not aware of any, I know very few people who are even talking about trying to solve this problem, right?

1:19:39 On my side, I do this all through sort of like meta practices of, I mean, just the most basic, starting from the most basic thing, which is I have some project, let's just say hypothetically a blog, right? I know roughly what the components of the blog should be, I'll make like an architecture diagram metaphorically speaking, and then I'll say, hey, like this thing should be ballpark 2,000 lines of code and they'll have it generate that and separately evaluate that, right?

1:20:05 And then write out an interface file and then separately generate another module of, okay, we've got whatever, like the back end and the RRM, that's one thing, it's standalone, it's done, right? And then I'll go and do whatever, like here's the load balancing layer, right? If you have like a super high performance blog or something.

1:20:22 I found that's generally my, the thing that I would like to encourage people to do is, like instead of trying to solve the biggest, most hypothetical game you can right away, just like try to maintain quality and increase your productivity by 10% and see if you can do that, right? And that's, so like people are trying to go from, you know, like whatever my current pace is, I'm going to try to go 10 times as fast.

1:20:49 And you know, I mean, that would be great, but actually in the history of computer programming, like a 10 or 20% productivity improvement is already enormous, right? Like there are very few things that actually be great, right? So what I encourage people to try to do is, can you just get 10% better using, using AI?

1:21:08 And if you frame it that way, I think I approach things very differently, which is I sort of spec out, this is how I would do the project myself, here are the components, here's how they talked to each other, write out a spec for the interface, and then generate each thing separately. And so I have like 20 AI generated modules that are a couple thousand lines of code each, and then I just ditch them together at the end with like 500 lines of code that I wrote using the interfaces that I spec'd, and then it filled out.

1:21:36 And there I'm able to generate things that are ballpark the same size of code that I would generate, like AI is more of a boost than that I would be, but I come from a mathematical computing background. So my style is very, very tertiary to begin with, right? But I think you can generate quite reasonable artifacts that way.

1:21:54 I don't know of any standard organized push to try to do what you suggested prime, which is, okay, I got the inhuman artifact to the 300,000 lines, can we now shave it down? I haven't seen anything, I haven't seen anything like that that works. Now I've tried here, take these 50,000 lines and can you reduce it down to 40,000 lines and it's extremely unreliable.

1:22:19 I haven't done that test in a few months, so you've probably heard the meme look like everything changed December of 2025, and there's some truth to that. It's not as true as some people are suggesting. Oh, can I follow up with that after you're done with what you're about to say? I actually do want to ask you about that.

1:22:35 Yeah, I'm done with this. Oh, yes! Okay, so I have a hypothesis that the reason my people are seeing the big change that they're seeing in December 2025 is not because the new model was just like 20x better, but because we've been investing non-stop effort into all these harnesses and they just have gotten better and better and better over time.

1:22:57 So this nominal improvement in the model actually made something that perceivably was a larger improvement, but it actually is just the stuff around the model that has so vastly improved over the course of the last year and not the model itself. Yeah, I would agree with that without like 70%. I mean, there is substantial improvement in...

1:23:15 I'll think so. Like in 4.6, but also you're absolutely correct that I would attribute most of the excitement of the post-Christmas 2025 meme is sufficient maturation of these meta-practices like agent loops and agent teams and stuff like that. So I think broadly I agree, although there really wasn't any full improvement with 4.6.

1:23:43 I've got one last bigger question that I wanted or like topic that I wanted to talk about, that I wanted to make sure we covered from the podcast. Prime, did you have anything else you wanted to add? Or Dmitry, say anything else? Dmitry, in case you're still okay going because we were a half an hour over our projected amount of time.

1:23:58 And I don't want to take your guys' time if it's too valuable. I should probably go in like five minutes. Okay. Just have one quick big topic. He wants to talk about it. Really quick big topic, okay. I gotta get off the way for two. I thought that was a take one. Oh no, you gotta have one, that's a green screen and then one that's a real one.

1:24:15 That's the trick, that's the trick. Okay, it's not see-through though, so I have to look away when I'm ready to get. So I'm just gonna draw this really quick. Just kidding, that's I'd instead. There we go. Supply and demand, I have a quick question about this. There you go, that's all I need. Okay, simple topic.

1:24:31 That was a really, really good diagram. Thanks. It's one that a lot of people aren't familiar with. So I wanted to make sure that we drew it out for those that aren't out there. There's two parts, two part question just so you know where I wanna go with this. The first one is sort of like, I don't feel people meaningfully talking about, like let's say we a hundred or a thousand X or like a million X demand for tokens, I which like would have to be the case if everybody's using tokens for all their jobs, aren't we gonna like a hundred billion X or 300 million X?

1:25:03 It's like a big number. Yeah, I'm using way more tokens than like my dad. My dad like occasionally asks Chatchy Pt something. He doesn't even know about deep research mode, right? Like, so I can burn a million tokens before even noticing, right? So it's like, so there's that part of the equation where if the demand is gonna go crazy on one side, is it not gonna push the price of tokens up higher even if we have technical things.

1:25:26 But on the flip side of that, of this like we're getting tokens cheaper and all this other stuff. And you guys talked about Javan's paradox. So I just want to at least give a spot for you to at least bring up this topic of some things. Like hey, if we're gonna get tokens a bunch cheaper, but then we're gonna keep pushing up demand for tokens is that gonna make it more expensive and what does that mean for like everything that people wanna do with tokens.

1:25:51 They wanna do agent loops, they wanna, everyone can't possibly just have an agent loop running 100 million tokens a second. So it's a big broad question, just interested in your thoughts sort of on like those market pressures where you see that and like how that affects people's usage of AI if it actually does get broadly adopted.

1:26:12 I guess I'll say two things, one connecting to the previous conversation of the 300,000 line blog. Yeah, obviously that's generating, that's consuming and generating a lot more tokens than you might think would be required to write a blog in Rails, right? Right. So there's that. There's this question of how long will we be in a period of subsidized growth with multiple players so that no one can really afford to try to squeeze the market for margin.

1:26:41 And as I said with Casey, the big thing everyone is worried about is somehow someone gets a position in Gen AI that is as dominant as Google had or has in search, right? So Google for a long time had like 90 plus percent of the market and the next player was like eight percent or something. Right, so it was like 90 percent Google eight percent Yahoo, or that later being and then like two percent random other stuff, right?

1:27:08 At least in the US market. Sorry, I should say that. But almost all my experiences in the US market, right? So I mean, notably once Google got that dominant position, they were able to eat tons of the margin in advertising, right? And so without someone else, without some competing search engine to force Google to reduce margin on ad clicks, well, they just sat and ate almost all of the margin in the online advertising business.

1:27:41 If that happens, it could be very scary economically for anyone who's lying on AI, right? Because right now we have not only this VC honeymoon period of they don't really have to make the money back right now. Right, and that phase could last a long time. Like Uber was in that phase for like 10 years, roughly.

1:28:04 So we have this, like at some point the honeymoon will be over. And possibly also at some point there will be a Google of Gen AI and it might even be Google, right? So like the Google of Gen AI might be Google. We've seen what they've done with a Google style advantage in the past in terms of how much of the value in the market they were able to capture.

1:28:26 Right. And so I think, like we won't know anything about that for years. But I think that's the biggest factor. Right. I do think that the open stuff is getting better and better. And I think it will be good enough, I mean, it's already good enough soon actually that you can use it as a substitute in many cases.

1:28:48 The main thing that is missing in the open stuff is the relatively polished product wrapped around it and like all the extra features that go into the chat. Now I understand like you and I are using it through the API, but like the, the, the, the mass of the market that still has not moved through the phase of, hey, what the heck is this thing?

1:29:13 And can I, can I use it? They're going to go through the chat interfaces first, right? Even if they eventually end up using, using it through the, through the API. So yeah, I guess the open models are really quite good and getting better. So at some point, that will have to come to a head where people will say, okay, look like, yeah, Cloud is great, but you know, between DeepSeek and Quen and Mistral and Lama, I can stitch together a pipeline that does 90% of what, what Cloud does and all it costs me is electricity, right?

1:29:49 Right. And that's, that's so far on the other end of the economic structure that I don't have good predictions. Yeah, okay. Frequently set up such systems for, you know, for customers where we just do like, everything is open because I, like, what are regulatory reasons that has to be on premises? Nothing can leave, there can't be some third party involved in the deal or whatever, right?

1:30:10 So, that's so far on the other side that it's hard to compare directly because it's like setting it up is also more complicated. But I think probably, it was not probably for certain, there are people right now who are trying to build kind of like bring your own open model and they'll just post it and like pipe stuff together for you something like that has to succeed in the next few years.

1:30:35 I haven't seen anyone who has a strong lead there but something like that I think has to, has to exist in the market if only as a, as a check on, on how greedy the commercial players can be. I guess I don't know if that answers your question, but sorry, Frank, go ahead. Just a quick question. If, if, let's just say all things are given the same, I don't know much about kind of the infrastructure overhead that OpenAI or Quad currently has, like how much room do they have to be able to grow?

1:31:07 But if everybody were to turn on a Cloudbot tomorrow and they all just ran 100,000 tokens or which is not a very hard amount to run through or maybe 250,000 tokens and just in America, I mean, that would be hundreds of trillions of tokens that just weren't there beforehand. Can that, in this theoretical situation, would we see, I would assume the infrastructure would be so overwhelmed that there would, like we would have to skyrocket price.

1:31:33 It would that be the only path forward. So if something doesn't come out of this, then price does have to go up if usage goes up. Or throttle speed or throttle speed, which effectively will reduce the usage. Yeah. But that's why everyone is fighting for rights to build data centers and get electricity and hydropower.

1:31:51 You're thinking of it the same way, they're thinking about it at least in public, which is we are maybe 10% to 20% into market penetration of AI use. So they're hoping to get to 100% penetration, right? So that's like five to 10 times where we are now. In order to do that, they need vastly more physical resources just to run the thing, never mind making better models and just freeze right now everything you have and just serve it 10 times as many people doing, not just 10 times as many people, but like 10 times as many people doing, maybe 10 times as many tasks, right?

1:32:26 Because maybe right now we're only using it for coding, but later we'll use it for, you know, I don't know. Well, like Kerser's got remote agents, it takes a video and sends you a video of the output to see if the thing actually still runs so that you can know like did your front end change actually work?

1:32:46 Like that burns a lot of tokens. It's a super cool feature, but like that's a ton of tokens. To me, if you said you had to go, yeah, Dimitri, I'm sorry, I have to let him go. Dimitri, where can everybody find you out? What's your most like favorite place that we could link you at? It's just a touch to do cases regular thing and computer ads, there's waiting through AI.

1:33:06 That's case in me, roughly once a week and then if you want to find me specifically, or whatever, it's DimitriSpanos.com. Sweet. DimitriSpanos.com. We'll link that in the description for people later as well. Yeah, I just want to make sure I get it. PhD. PhD level intelligent. I am doctor. I am generally anti-credentialist, but yes, I do have a PhD.

1:33:29 OK, well, I just want to say that's like, yeah, me too, if I had a doctor and I be against credentials too, but then I would say it like I'm cool as well. I'm not against credentials. I'm against credentialism, yeah. All right, well, thank you very much, Dr.Spanos. Thanks, Dimitri. We do need to get you back on here.

1:33:48 Hopefully, our style, which is very different than Casey's, which involves a little bit more joking in orbital data centers. Hopefully, you enjoyed cars. We have certain cars too. You chat was wild. They want you back. A lot of people say, bring back Dimitri. They really, really appreciate you, so. Yes.

1:34:05 Thank you very much for joining us. I appreciate it. This is a great conversation. Yeah, all right. I'm out. Thank you, Mr. Mayor. Bye. Bye.