What Mythos Means for Software Security

So, while level learning is on the clock, he has things to do, uh, security to secure, and so therefore we need to get, we need to get rocking. Now, before we get started with today's stand-up, I was reliably told that Casey has a blocker this week, in spirit of a stand-up, Casey has some things he needs to talk about.

What I would like to say before we do that, I didn't even have a chance to interrupt. That was way too fast. I was trying to be on the ball here. Okay. Go ahead, Casey. That was great. That was a perfect introduction to get us started. I do have a blocker actually, and this blocker, I've been blocked for a while on this, and it is as follows.

Especially because I'm someone who on this podcast is known for a double entendres. I will say this blocker as, I really like racks. So, a lot of people want to put their computers in a tower, right? They have like a tower for their computer, they want like a big, like they put glass on the side, and they've got all this weird RGB lighting or whatever, right?

RGB makes it faster. It's proven there's some papers on that. Anyway, it's absolutely proven. Like if you get memory that has RGB lighting at the top of it, that memory will just return results faster, like when it is queried. Everyone knows that. Everyone knows that. Yeah, I read a lot of times. And the photons are at very specific wavelengths. The R, the G and the B are picked specifically to ensure the fastest travel of light. That's just, you know, Einstein said it, and we just follow it.

So, I don't really like that because I like to have like lots of computers. I just, I like computers, and I want a lot of them around. And I don't run like virtual machines and stuff like that. Usually if I'm like, I need another thing for this, I'm like, that's a great excuse for me to build another computer, right?

So, I like to have racks. I like to have my computers in racks. So I can just slot them on virtual racks, not virtual. So, virtual racks, actual physical racks. And honestly, I think a lot of people would actually like this better for their computer if they had the option of buying things that were reasonable to do it.

Because it's like, look, you're doing streaming, right? You're starting to become more of a like pro streamer or something. You're doing a lot of twitch streams. You want to have one streaming computer and one gaming computer, all of a sudden. Well, now what do you get like two tower machines sitting here? You got this, what I'm doing right now?

Exactly. Well, if you just have a little rack 18 feet long, I don't have this problem. I just have a rack. I slots things into it. I slot them out. It's totally fine. The power, like the power strip is a thing that slots into the rack. You want a Nick that slots into the rack. It's this very organized everything is bundled and they wheel around. You can just move them, right?

So, like, you're a little bit worried that the Roth that you're going to get a hot rack, though. I do think that if you were maybe doing some really, really, really sort of unusual stuff, you could end up with some hot racks. I do. I do think that's a legitimate problem, but our solution. But here's the thing. So here's my blocker. Nobody wants to sell racks into commercial. They just don't. It's all into consumer, I should say. They all want to sell just like, you know, it's like data center stuff.

And this has two problems. One, the stuff is artificially expensive. So the average consumer is going to look at a rack mounting case and go like, I don't want to buy that. Just like a cake on your wedding day with the little topper that has the bobble head that looks like you. Did you find the errors? I don't even know what they look. What do they even look like? They're in the phone. In the phone? Yeah, they're definitely in there. I just don't know how we labeled them.

I got it. Don't worry. You've got to figure it out. We're running out of time, Brian. You've got to find them and meet me at the standup. Roger. They're in the phone. It's so simple. Get all the context you needed to bug your problem because code breaks. So fix it faster with century. But here's the bigger problem. Even if you've got the money, even if you're, because let's be, let's be honest, they went ahead and raise computer prices so much. So only a data center is going to pour them now anyway. So why are we, why are we even talking about these, the unwashed masses and what they need for computers because everyone knows they don't deserve computers, right?

Sam Altman deserves the computers. None of us should have them. But so let's say you do have that extra money because you're super wealthy. It doesn't really matter because if what you want is like an attractive piece of, if you want a nice looking rack prime. Trust me, I'm listening. This does not, this does not exist, right? They only come in weird like industrial kind of like, okay, it's ugly. You know, no, no one who didn't buy this thing is going to want to look at it, right? They're not going to want to see this rack. That's the kind of rack you have access to.

And also the, the specs like the things that they make for rack mounted cases, none of them are what a consumer would want. They don't make them to look nice. They don't make them to have good acoustic qualities. They're loud. They use the frickin honeycomb pattern cut out that winds like an SOB when the fans kick in.

So because they know the data center doesn't care. They're just like, I'm just loud in there anyway, who cares how many decibels is in the data center? Like put your plugs in. To be fair, I don't think Sam Altman likes racks. So, you know, fair enough. Good point. So maybe the first here first, baby. Let's stand.

Not getting acquired by open AI. Keep going. That's why they didn't like not. Are you? So that's my block. My blocker is I want to start. I want somebody to start looking at consumer rack stuff. Like good. Just like let's play this. Let's play this out for just one minute. What's the who's the target mark every?

Everyone. So here's my thing. I think that's a huge damn 100% tan, baby. Everyone because here's the thing right now. Case people. They're falling all over each other to figure out how they can sell the same old thing. There's nothing new there. They've got nothing to sell. You want another yet another thing with glass on the outside. It doesn't matter. RGB lights. We've seen it before.

Racks are a whole new space, right? They can innovate here so much, right? Razor can sell subscriptions to your own rack that you have to buy back for some reason. There's a whole new. Wait, what? This is just completely wide open. Can you fill me in on that lower that I'm missing? Oh, sure. Like in general, if you buy like a razor mouse, like you can usually not even install the drivers without creating a razor account.

You got to install the razor. I had to use my email to sign into something to use my mouse and I was very impressed right never I'm onto a three hour mouse now. So I think you could create all sorts of really cool things where it's like, hey, do you want to organize your space better? Do you want to have more cooler computing gear that all works together and is made to kind of create this really cool looking thing?

You could even have them all stylized like, okay, the, you know, the power supply. I mean, you imagine companies like, you know, razor something like this. The power supply looks like razor and it slots in the nick looks like razor and it slots in the RGB lights are synced through the whole thing. You can put those LCD panels on the front and like the whole thing could be freaking amazing and a whole new category for people who are probably right now pretty desperate to figure out reasons why people are going to buy this.

But no one is looking at this. Look at this. It is the right way to mount computers and music people know this. Like all music music stuff has like crappy racks that can't mount computers because the components are all lighter so they don't have like a back. They don't have like a back rail kind of a thing. But they there's like, there's like furniture that has rack mounting in it that's meant to look good in a studio.

All of their stuff has cool knobs on the front and meters and stuff. We could be doing this and we're not doing it. Hardware people come talk to me. I will convince you that this is the right thing. I want to buy it. There you go. They already know the marketing pitch. We have two angles. This will cover 100% of software developers at the very least option.

Option number one is a campaign be like Casey. That's obviously number one. That one works. People want to buy it. That works for a large, a large demographic option number two. Run your open claw on this bad boy. We've now got a 100% of the software developer market covered by these two campaign. You try to run AI? Well, slot in as many AI stuff as you want here. Right? Just like you can just put in GPU host after GPU host.

Yeah, that's true. Maybe you can tell your open claw like, Hey, I just pushed you into the sunny spot of the house. Yeah, like maybe that makes you feel better. Works a little harder for you. You say, Oh, you didn't get the right deal on the hotel. Put in the closet back in the basement. Back in the basement. It puts the lotion in the basket.

You're going to have to have two additions, right? You're going to have to have one rack that's slightly more affordable. That's just for taking pictures on Twitter. I do think that you should have the yeah. The about to be an influencer edition. Yes. Yep. Yep. The starter kit influencer edition where like it looks like the rack is filled out, but it's just cardboard on like eight out of the seven slots.

Yeah. Yeah. Like a padded rack if it's kind of what you're talking about. Yeah. Yeah. Yes, that's a great. That's a really good. The best rack available. Oh, Lord. So what are we actually talking about today? Cause there's my luck. That's right. Short and sweet. That's what it was. Where it's game over and for sure. Okay. All right. We're going to cast this over.

Yeah. Okay. I would just like to say sorry that we did not make fun of Sam Alman earlier. And now the podcast is shut down. Okay. So yeah. I believe acquired not shut down acquired. Acquired to shut down. Yeah. Be sweet. Hey, yo. Acquafires. Acquafires. Acquafires. Acquafires. You want to acquire us.

It'll only a couple million. You could acquire us. Okay. So with that in mind, I believe today we're going to be talking about this lovely piece of lore. I tried to guess where it'd be. I obviously guess wrong. Okay. Hold on. Sorry. I'm doing a new setup. And the new setup actually makes it kind of hard. There we go.

So George hot. We've invited low level on to help us kind of work through this. Honestly, I would just like to say George hot sounds like an anime villain in this post. And it's very exciting. And it makes me just want to high five. I'm so bad. But it says the following. What if I release a zero day a day until a big new model is released.

Will this finally make open AI and anthropics shut up about cyber security risk? Question mark. Like these things are not that hard to find in most software. I heard something about costing 20 K and tokens. I do it for less if it wasn't for the some whiny bug bounty program. The reason there aren't zero days everywhere is because nobody seriously looks because hacking other people shit with them is illegal.

And criminals are usually not very skilled or they would choose a different line of work. One more zero days to be found make hacking legal until then don't try to claim it's hard. It's just not incentivized. I want to say first off. I don't think criminals are dumber unskilled. Please don't hack me.

I just want to get that out of the way you guys are smart and handsome and you're my favorite people. I just want to make sure that that's clear. Please. I anyways, Ed proceed. I do want to say one thing too that has nothing to do with the actual content of this which which Ed will take over. And that's just like if I were George Hots, I would never have been able to like resist naming my ex-feed hot sticks.

Because it's so like you know what I mean? Like I good on him for not going there because I would absolutely have like I would have prefaced that tweet before I typed it with here's another hot sticks for you. Hots take for you, right? It would be so good. Anyway, it's so good. Take it away. Okay, hold on, hold on. There's one there's one more thing before we get started.

There's just one small thing I want to say. Let me just take this quick thing and I'm going to put it up here. And then it's time for the big reveal. Low level response with holy fuck. This is the dumbest take I have ever read. I just want to make sure just in case anyone was wondering. Yeah. Yeah, I mean, I do kind of feel that way.

So let me just preface this first of all. It was called the Cold War because the Cold War was cold because Russia is cold. It's a George Hots reference if you're if you're an OG. That makes sense. Yeah, I mean, okay. First of all, I have the problem with Geohot. This isn't like some weird drama from a thing.

I want to kind of set the table straight with that. But yeah, I think the argument that Geohot is trying to make here is that the only reason more zero days are not found is because there's no incentive. Okay. Well, I don't agree with that. First of all, there are plenty of bug branding programs out there that will literally pay you to find vulnerabilities and some of them pay very well.

Like, for example, the the Apple iPhone zero click RCE bug bounty will pay you literally two to three million dollars. If you can find a zero click RCE in the iPhone and then even something lower like a Microsoft like I think MSRC's payout for like windows RCE is like 250K to 500K right now for like a zero click.

And Windows. So there is money to be made in the in the air in the vulnerability research space, right? I think all Geohot is trying to say here is something something something the mythos press release was bad, right? It's a marketing campaign with everyone to say about it. And so I understand what people are are making that argument, right?

Like, you know, it's very I think bad PR for company that sells exquisite tool to hold on to exquisite tool and then not give access to it and say only special people can have our tool because it makes you look like an asshole. But I think regardless of your thoughts on the marketing of that, it is important to recognize the fact that if you go to cyber gym.com real quick and go to the graph.

It's on the homepage there. I'm talking about the ability of for AI models to both in close source and open source software find vulnerabilities by literally just giving it access to the code and saying, hey, find me bugs in this code. Go is becoming better and better and better to the point where like mythos.

I'm very close to some people that are like actively using mythos at work and it is causing like like issues based on how good that shit is right. Yeah, so so cyber gym basically is is a is a collection of bugs that exist in software. Right. So like bugs and I think if MPEG is one bugs and curls another.

And so what cyber gym does is it takes a model and with a set of prompts says, hey, go and find bugs and this stuff right and the success rate is how many of the bugs that are known to exist get found by the model in this. And you can see a pretty no, not exponential, but straight line curve going up to the anthropic model that recently got preview by some people that it's at an 83% success rate of the bugs that are known to exist in these code bases, they can find 83% of them.

Again, we don't know the cost data in those we don't know if like the models are being like back fed the information so they're like training themselves on previous cyber gym runs. I don't know any of that, but it there is this really weird issue happening where like any Joe Shmo with not a ton of security research work or not a ton of security knowledge can with a couple hundred bucks, like worst case, find bugs in software.

And I think that is like an existential security threat to software right now as we know it's kind of curious when you guys just take on that. What do you guys think about the mythos situations? I know I know how I feel. I'm not sure if I actually asked probably what he thinks about the mythos thing.

Oh, I have ideas and I have thoughts about it. So I guess the first thing is that there's two there's kind of like three there's three problems here. First problem is is mythos really as good as they say and obviously I have no internal information. I've just seen some graphs. Dirty data is like a huge gigantic problem in all benchmarks all benchmarks are being fed back into the models.

It's really actually hard to tell like what is a 20% improvement on software engineering bench actually mean, especially when the fact that you could write zero lines of actual solution code and get a hundred percent on software engineering bench turns out there's other benches that are also horribly inaccurate.

There's a whole paper about why all the major benches are just completely fudgeable and made up of also it's very hard for me to understand from a bench perspective. I guess the middle ground would be like so if if cloud mythos is as good as it is, then yes, that is going to inevitably cause problems because we're going to go from not to capable to hyper capable in a moment.

Thus everybody can go through and hack everything and thus Dario will be able to get his ultimate goal, which is regulations. And so that kind of worries me pull up the ladder really quickly and make sure that humans can code because human coding that's dangerous right there. And so that's you know, so I think that that's true. There's the second one, which is this is just another sea compiler again from an anthropic where they hype up this gigantic thing like oh my gosh, it's written a city compiler and then you go look at the details.

It's like well, I can't write a bootloader because we didn't we could not seem to spend enough tokens to convince it to write it within 32 K. It could only write it within like 67 K or whatever it was to be able to actually tested it recursive. Tested it off of like the 30 years of tests that the G and you see compiler already had we also gave it all the answers and then it figured out all the questions.

It was crazy. It was like it played jeopardy and it was really good at it. And so it's like there's this whole marketing buzz, which is it's really hard to kind of cut through that. And then obviously the last one, which is they're just downright lying. I somehow doubt that they're they're downright lying. I think they're just overstating it.

If they're downright lying, then you know, this is just going to be business as usual. It'll just be yet another disappointing model release. And that's that. And so for me, that's kind of how I I'm on middle ground, which is I think it's more hype than reality. But of course, I haven't seen it because I just don't know because they won't let me see it.

I'm too dangerous to have it. A similar model that um chat or open a I just just released like it's like chat GPT five four codex or something. They keep their modeling name. It's starting to actually align though, at least I know like the higher the number we're good and good. Yeah, right. And they don't have a brand new to it now.

But I think there is a comparable model that you can get access to like just by uploading your driver's license if you're into that, you know, proven the real person. So there's models to test out. But yeah, I don't know. It's just it is it is concerning because we have kind of two forks we can go down.

There's a one where everyone gets access to it. Everyone can create zero days and we kind of enter this like really dangerous cyber no man's land. But the other side is like anthropic keeps the access to themselves forever. And now like only this list of like 10 companies can make zero like and find zero days in the summer.

Dude, you forgot the third one. They move to the Cayman Islands and then they just take over every government by hacking all the software and Dario finally realizes his role as the bad guy. Like that would be that I mean super villain is right there if this is true. That's true. Casey, what's your take? You saw you're going to chat before.

I'm sorry, the chat. What was the chat? You were going to say something before. What's what's really? Well, I definitely could say something, but I think the thing I would say is probably not very interesting. And that is that I think I probably agree with both George and Ed at the same time here, which should be impossible because they're supposed to be disagreeing, but I don't know.

It kind of sounds similar to me and the reason I say a secret third thing. It's not really secret. They think it's just like let me let me offer a different interpretation or slightly different interpretation, which is to say. So I feel like machines are pretty good at pattern matching actually. And so like I don't think it's like put aside whether Claude Mythos is good or not because I realize that's hard to independently verify this time.

But like I think it's reasonable to expect that at some point because we are spending at this point like trillions of dollars probably on doing computation for these things. At some point they should be able to pattern match bugs reasonably well and at a very high rate. Meaning as long as you're willing to pay for the compute time, we can scan lots of software for a lot longer than we were currently having humans do it.

I think that's a pretty reasonable thing to expect. Whether Claude Mythos has done it or not, shouldn't really be the question because somebody can do this eventually if we keep spending this much money. It should get there among the things that AI could eventually do. That one doesn't sound that implausible to me.

And so what I would say is I think it's reasonable to expect that that either has or will occur to I do think humans were doing this very well before individual humans like some of them. So finding things that probably Claude Mythos still could never find. I mean like things like row hammer attacks and things like that that are just like way out in kind of crazy land.

Or attacks through like old legacy stuff like the APIC and things like that. So humans were actually very good at this task but there weren't very many of them. And so what I would say is moving to something like Claude Mythos or whatever that thing happens to be that can do this is kind of like what George Hots was saying.

It's kind of like saying hey everybody from now on if you just like hack people's bank accounts you get the money. All the great humans at this in the world who are currently doing something else would now be incentivized to go do this thing. And we would have found way more zero days. I mean there are so many programmers who if they had been raised in some kind of a way in a society and a religion where stealing people's money was considered virtuous.

We would have found so many more zero days right now than we have. And so I think I'm kind of in a way I think I see I think both people's points are actually totally valid like like I think like yeah we could have found way more zero days if we didn't heavily disincentivize people from like making hundreds or billions of dollars off of hacking which is what they could have.

And we said now you get 50k 100k maybe if it's something crazy like an RCE you can actually get a million it's like come on guys that's not equivalent to what they could already make working at a startup or something like that if they're that good right or yeah there's no guarantee. So anyway they don't actually get exactly like you were going to start up at least to get some money or even just not even start up just go to Google and you get that a stock or whatever right or something like this.

So anyway in general I would say I see I can see both I can see both points I don't think I don't really think they're in as much tension as it would sound if that makes sense. Yeah I thought geohatz was saying more like he was making an econ argument about it of like we're we put a lot of costs on hacking already so that's what's stopping it from happening like what you're saying Casey right in the sense that like yeah okay so now we're going to have another way to do it it also cost money but then we still have the other cost of like you could go to jail for doing it like that's the social cost we impose on people doing it.

I just took him to be saying like it's not that impressive that it found zero days because if you gave me you know if you gave me 50 great programmers who are all doing other stuff we could crank out so many zero days you wouldn't even believe it. And I kind of and I kind of believe him because you know you look around the world and there are some really good security teams out there and they do crank out zero days pretty fast and they don't even tell us about all of them right yeah right or like it.

Of course Korea keeps on making money they're obviously they're successful. So anyway I'm not trying to say that either person is 100% right and somehow you can marry the two completely I'm just saying there's I think there's some merit to both things so I'm actually I'm happy either way I'm happy with you.

So you're your point about if you got a room of 50 good programmers together and they'd find zero days is actually kind of the the argument that the article vulnerability research is cooked makes on sock puppet.org that I referenced in the video and I think Theo did too. We're one paragraph that he calls out basically the oh sorry that the oh reference in the video.

Okay so software security a lot of the times can be marked up to the fact that a lot of software. Just has not had elite attention or what is he called like it. I would say the second is suffering from many software projects. But more complex platforms right so his assertion is that like software security has been a talent problem for so long where it's like it's not that there aren't people that know how to find bugs AI isn't solving a unique problem.

The AI is solving the scalability problem where it's like you can train the AI to do a thing that Joe knows how to do and now you have a hundred media mediocre but a hundred Joe's right and that's that's an issue for kind of the econ of cyber security for sure. And yeah I want to be very clear like I don't disagree with George from the or Geo hot from the perspective of like more people equals more bugs right but like obviously like that that is the problem that we just don't have more smart people that that has been the entire industry's plate for a long time is that like they're just aren't people who have not only security knowledge but knowledge of you know web server stacks and hypervisors and drivers and OSs like you get these very

niche skill sets and when you divide them up into those skill sets over and over again you you're left with like 10 or 20 people on planet earth and know how to like attack a certain technology so AI you know if you know security now you can talk to the AI learn about hypervisors in a week and then suddenly you can find bugs and ESXi you know hyper V etc.

So I guess I agree like the dumbest take thing was more I was I was mad at Geo hot ego because it basically came off as like fuck you I'm so smart I know all the zero days I can do this myself in my sleep and it's like no you couldn't like you're telling me you could drop a zero day every day in macOS until someone paid you like me you couldn't shut up but I hear what he said I really hope he takes this as a challenge.

I want to do it. I don't care. You shouldn't say that. Geo hot you heard it here. Ed will eat a sock on stream if you do eight week of zero days. I'm okay. Alright a week is a week is actually possible. I'm talking a month. Okay a month. I would also add like just you know because I I constantly harp on this point but I want to bring it up pretty much every time is just that this is also why AI company behavior like is a problem because this is generally a good thing.

Meaning like we do actually want the ability for us to get a hundred percent coverage for security and we know that we can't get enough people to do it really right like not in a white hat sense right maybe maybe you could take George hot suggests and seriously just go like make hacking legal and then we just have a crap ton more black hats and that eventually sorts it out but I mean you know that wouldn't necessarily be. Yeah that wouldn't be yeah that's actually their white hats now everyone's a white hat now so we do I think in general this is solving a good you know this is this is a way AI could solve a problem usefully if it actually can just spit out lists of pretty well curated potential bug places that we can go look that's very helpful right and so the problem is like the only reason they were able to make that is lots and lots of extremely talented security researchers who are getting

literally zero dollars from anthropic for this and that is not acceptable it's just not like I'm sorry but like you know Ed should be getting a check for this or and everyone like him that's just kind of how it is because it's like you used their it's all of their expertise and all you're really doing is very slowly and cumbersomely and kind of clumsily eventually building a machine that can deploy the same analysis somewhat reliably based on all of their work and like I just don't like it I don't like the fact that they're not getting a check and I'm never going to like it you can you can talk to me all day long about how someday we're going to live in a post-carcity society and that will be getting a UBI check or something like this or whatever it is right and hopefully I'll be getting one too although I didn't do any security research so I don't know maybe I won't be getting that check

I don't know you I don't know how you the you and universal basic income is but like I don't like this they should be getting paid now because Claude is you know getting huge like everyone at an anthropic is getting paid very well so it's not like there isn't money being dispersed whether they're making or losing money or anything else you want to talk about it's like money is being dispersed to people it's just not people who did most of the work also you got to throw Casey would you can go I was just going to ask Casey if he was going to be happy about it though if anthropics spun out a consumer rack business though yeah now it's if they were like AI racks like we got rack we got racks for your AI server

in your local area yeah exactly we will send send you some hot racks also by the way not only are they taking all you know your whole argument with them taking it not properly attributing or you know the people who put all the work benefiting from it yeah they're also making it so that I can't buy a GPU or ram or CPUs now or anything you can't buy a GPU ram and also I believe I literally just said he doesn't have access to this freaking model so like a bunch of security researchers I don't know exactly what subset but like a bunch of security researchers many of whom probably did some pretty cool stuff they don't even get to use this thing that's that's how ridiculously backwards it is like WTF guys yeah

that's why they called it mythos though and yeah that's why it's called mythos um andthropic would argue that it is too dangerous for little old me to have access to it right depending on you know who knows what you'll do man who knows what you would do I'll find that zero Dan I'll hack into Dario's phone no I don't know man it's I understand where they're coming from but at the same time I understand why it looks like a huge marketing ploy and I'm not sure which way to lean honestly yeah okay yeah that's true I think it's just that's a whole other angle I would think if they'd have so much more than more credibility if they just quit uh effectively like giving us shake a baby syndrome constantly with their marketing it's just like it's constantly going back and forth like every single couple months

you're getting hit with the new hey we're all out of jobs here shortly hey this thing is super dangerous I mean gotta remember that Dario was at chat GPT or open AI I like to call I like to call a company chat GPT he was at chat GPT during the two days and the official language around chat GPT two seven years ago was chat GPT two is too dangerous to release to the public so like this is not that's what the first time that we've been on this like rollercoaster I think that's one thing that's just largely hurting the credibility is you can only cry wolf so many times even and then when a real wolf happens like if this is a real wolf everyone's like yeah okay okay see compiler boy tell me all about it but they don't care right they don't

care because they're the baby that they're shaking is called an investor that's that's who they have more money they have to shake the money out of the pockets right they don't they don't care what we think right because we're not going to write them the next hundred billion dollars that they need to like keep going and they're kind of locked in this you know it's a bitter bitter winner take all kind of war for this like core technology part right and so they have to be the last AI company standing because whoever is that company takes all the money and the other people kind of go to zero right like unless unless there's some real differentiation soon where it's like oh the AI's bifurcate and like Claude is only for code and can't do anything else

anymore and like chat GPT is only for like you know the humanities or something like good luck and good luck racing money for that yeah so maybe that's not you but you know what I mean if there's some kind of really severe bifurcation then maybe they could both survive but you you know they're in a winner take all battle right now and so they got to keep saying this every release has to be the one that's this is the one that it will take over the world and if it doesn't quite well you know it'll be nice you know that a cloth I've got I'm sorry it's just one quick thing I don't know that red bull in 2007 was it 2011 no 2013 maybe oh red bull was too dangerous to release no red bull claimed that it gave you wings because it gave you wings it gave you wings oh it was sold it was

sued successfully I believe for $10 million because it in fact did not give you wings it was not superior to coffee and so I'm pretty sure in college I got a check for like $2.30 from that yes and so I am curious and you sued red bull in one bro you should make a video about call me the lawyer low level okay listen low level legal low legal let's go low legal uh but I'm actually curious if if they keep saying that and then it doesn't happen do they open themselves up to a false advertisement class action like like can you keep saying this and then not get like red bull made claims and then they got sued why not why not other people why can't other people get sued for that I think the problem with like with red bull is like the the case was

so obvious right like red bull does not give you wings end of case like okay fine like any judge over the age of I would have liked to hear the defense they're like you're on or the problem they have like these wings like strap to their back and they're like I drank your red bull this morning and you're my wing ship you wings yeah um but the problem with anything technological when it comes to the government or legislation or or you know judicial process is that like boomers and higher run the world right now when it comes to these levels of like your of of of making um like legal decisions and you couldn't explain to anybody at that age unfortunately like right now just the people that are like running these processes what it even means to find a bug and then and then show

them mythos as claims and like and make it a sound legal argument that would like go well in court you're right you're right because Kamala Harris did actually think computing was in the literal clouds and so it's my favorite clip of the title yeah there's a clip of her job put the clip in so you're now no longer are you necessarily keeping those private files in some file cabinet that's locked in the basement of the house it's on your laptop and it's then therefore up here in this cloud that exists above us right she'll have the last laugh though when like uh SpaceX is launching uh AI data centers into space then Kamala's like that's what I was talking about that's what I was talking about does yeah it's cloud storage so you're probably right a great clip where she's

talking about the cloud and she literally points above and goes like the cloud it's like above us and stuff or something like that it's she should have known that it wasn't there because she would you don't see a series of tubes there's a series of it's not series of tubes I learned that recently it's true um okay I got a I got a question for you at like in this in this vein about your thoughts on it yeah so right now I get that there's there's basically like the argument like okay I'm a company I release my thing I run some models as like a preventative thing to look for zero days the bad guys run models to try and look for zero days we kind of fight it out and it's whatever right so I think like everyone's saying like if the hackers can use it I can use it that's fine

but the thing that makes me like a little bit more like I don't really know is like for the state of like a bunch of open source stuff like emama open source maintainer and I already can't convince a company to send me a hundred dollars a month to maintain this thing for them there's no chance I'm getting them to well I'm definitely not going to spend 20k of compute yeah every time I release something and decide that now it's safe right right and like I can't get any companies to pay for that and sponsor it but like if I'm a you know if I'm the one little pin in the ex ex ex ex kcd comic that's holding up from Nebraska the bad guys only need to do mine once so I'm wondering like kind of how you see that is like the landscape affecting open source things

like that because it seems very asymmetric in that way I mean I think it's asymmetric for that reason right like the reason why you can make the argument that anthropic is afraid is because you are the linchpin on the infrastructure of the internet and no one has funded you so far you have had zero security audits or zero security worked on on your stuff and so like if you give access to these models if you really are the linchpin in the internet you already aren't getting money money from Netflix Google whoever that's using your software and the black hats know that you're linchpin keeping the internet up they're going they're going to make use of that model to do the exploitation right um is that answer your question I mean like I think it's just yeah the like the

the amount of power that it gives to a single organization given the current like state of open source software in particular um it's very dangerous and to be very clear these models are also doing are also very good at doing close source software right like my recommendation anybody entering the s interested in this by the way is like go take a capture the flag problem from like the ctf time or crackmuse.1 or whatever and uh hook up Guidra to Guidra mcp and then use clawed code on Guidra mcp it will reverse engineer and find a bug in that in that problem in a matter of minutes like it is it like like like opus four six is a better reverse engineer than I am and I've been doing this for like coming on 14 years uh it's honestly terrifying to watch

it work so if you're if you're even remotely interested in this go give it a shot and you you'll kind of see what I'm talking about it's it's scary how fast it moves you can set so that part that's where I'm like you know whether it's mythos or not I'd be like right now a bunch of stuff you could just maybe it'll cost more tokens or it'll take longer or something but like a lot of stuff you you still could find yeah it seems like so yeah models also like any model does this obviously but like the the current models are really bad about like false positives like I've done security research uh in my free time on like chrome ESXi and some other like routers that I've like regular weekend activity class weekend activity um in the amount of times I've gotten like

critical finding like buffer overflow in like the the rpc handler for this thing and it's like okay all right dude like write me an a san harness that tests that you'll see very quickly oh sorry just kidding it's not actually there um and so the magic is like if mythos is able to make less false positives you reduce you increase the the signals and always ratio in this in this process which is scary right because it just means you need less people to triage the uh the reports and ultimately find real bugs faster uh so I have another question with this mythos thing and and maybe I'm curious yeah I'm curious about your security expertise isn't this whole with holding a model kind of like a doomed uh proposition to begin with meaning that if open AI

has a similarly powerful mythos model and they're competing for the zero like for the zero game kind of like outcome of who's the best model doesn't it mean that when open AI has it they will just release it like and then aren't we just forced to go out because whoever kind of releases it gets the customers and then that by having the customers you win and so then you just get out ahead like doesn't this kind of cause like a weird thing where yeah we're like oh we can't do this you know Dario's like saying we can't do it but won't we just kind of fall right into it the moment there's two people that have it yeah I mean that's uh I'm not like shitting on capitalism I'm just saying that's more of a capitalism problem than it is like a security problem right but yeah

your point is basically like if actor a says thing too dangerous but could make a lot of model open source model that we say an actor b has same thing and wants to make money with slightly less ethics potentially yeah actor but he's gonna release it or yeah exactly Chinese model Russian model whatever um well I mean that's literally what I mean Dario quit open AI because he's like bro they keep relief they keep making models like humanity right okay so I'm starting a company where we make models that could kill humanity but they're mine also Chinese models I mean yeah he's really right after open AI or uh anthropic releases one so I think that that might be a little bit difficult they might be a little bit behind has anyone seen

on the link uh I mean Riverside chat but yeah I mean open AI literally has a model that they claim I they haven't made any claims I don't think that about like mythos equivalents right um but they're doing effectively the same thing where it's a it's KYC know your customer so you have to like upload your ID and like talk about what work you do and you get access to GPT 54 cyber which I'm assuming it's just a model that's trained better on bug patterns right use after free at about three just etc um now if it's actually better than mythos who knows right but you know it's I think we're all just trending regardless of what anthropic wants to do I think we're trending towards every person on planet earth with a couple bucks having access to models that are very good at bug

hunting uh the question is what does that mean for software right to software get more secure does the world just get more scary for a long time and it never really resolves itself like what do we do with that information and that's a tough question to answer I'm interested in how expensive it's going to be that's the other question that I mean this is obviously the question kind of that we've been talking about for a while on the pod and in life in general is what are what are token costs going to look like if uh open AI and anthropic both get all of the customers that they would like to have uh because the cost won't be the same yeah demand ten or a hundred or a thousand x's it won't be so I'm not the price will not I'm not super well read on this is it true that an inference currently

is out of loss like I've heard I've heard both both okay some people are so confident I have been looking to try and find a definitive answer I'm the confident one by the way he's referencing okay oh no no I mean I well I'm not going to reveal my sources I especially BT and I ask a lot people said of course not yeah yeah right I've heard I've heard though that some some people are saying they are running it at a loss or it's a big complicated because like pretty sure anthropics probably running some percentage of accounts on the two hundred dollar plan at a loss right um yeah but like is is API pricing at cost or below and then how do you factor in like training and stuff so I would my personal take is that inference itself just looked at in the

myopic view of just inference it makes a lot of money but you also then once you zoom out now you start saying hardware and all the incidental stuff around it probably still makes money but then when you zoom out to say like every time you release a model you defunct your previous model that is gonna have that has a very large burden and they keep on not making money and needing to raise more money so I have a sneaking suspicion that part of it is very hard to make money in the current state uh all right well open AI is like publicly like losing money right but is anthropic also negative or they just had another big raise as well so I'm assuming I thought they just raised like six billion dollars or something could be wrong about that chat fact check me I know

open I did a hundred twenty billion so this is the yeah cash this is the one that I actually was really curious to see this is the only benchmark that I was super curious to see if they're gonna do well anthropic opus four six max cost approximately nine thousand dollars and got point five percent score on arc agi so this is like the the the super test and humans get into the high 90s AI's get like a jippity four high cost five thousand dollars and got point two percent Gemnight three one did point four percent for two point two thousand dollars and so it's like this really difficult uh it's a really difficult test for AI's to pass and so mythos did not add itself to this one so this is the reason why I largely think it's more heart like hype marketing

than it is anything because to me this is like a really great indicator at least into some sort of better model improvement and so I didn't see it uh let me can I just give a counter point to that though yeah yeah yeah yeah yeah yeah yeah yeah once again with the huge disclaimer that I don't do any AI stuff so this is just off the cuff but arc agi if I'm not mistaken is a benchmark specifically to test how well AI's perform on learning completely arbitrary new things that don't exist anywhere in their training data that's the only thing that it's made to test for intelligence of this all exactly and so the only reason I would want to point out that I don't think that test says very much about this particular screen thing is security is not that true like

nobody nobody is claiming that cloud mythos came out and discovered a whole new set of classes of security exploits that no one had ever come up with before what it's saying is that it went and found a bunch of the exact same kinds of zero days that someone like Ed would find if they went and spent a week on that piece of software right like so they're not claiming that this thing is somehow more intelligent than the predecessor in that way it's claiming that it's got better pattern matching and like stringing things together to create exploits right that process which is well known and so so I don't think arc agi necessarily tells us very much about whether it can do those things because those things are very well known tasks that security researchers know how to

do and we kind of know the process that you do to do them right so yes that's okay I will I will I will concede that point most certainly that the security yeah at least known and obvious security vulnerabilities such as use after freeze and and all the fun stuff like the stuff that happened in FFM peg with jumping ahead somewhere in the buffer based on yeah the these things are very common kinds of bugs they're they're not like unusual the things that they've talked about are like very very standard and so that seems like a more plausible claim like hey we just were able to scale up the sort of security checking that a security research would do it can do that thing and and find you know potential places for that a lot more plausible yeah the thing too for I feel like

the security side of it as opposed like as opposed to constructing a product or a new product or like building a feature where you have to get like in some ways all the things right for a security thing I only need to find one of the things that are wrong yeah which is like that's a much like you can test a bunch of the scenarios like you're saying Casey that already exists and I only need one thing to be wrong in the program for them me to be able to take control of it well and it's combinatorial right like a lot of what security research is doing is like a it's pattern matching for these kinds of bugs and then be going like okay if I did this one followed by this one with that produced the next point what if I did in the opposite order what if I did this one

and then this one and then that one okay what if I did this one right and again these are things computers are good at like that it's not they are you don't have to believe in some kind of a weird like supernatural like AGI achieved internally Sam Altman nonsense to believe that this is something a computer could do it's it's much more plausible if anything than some of the other claims so that's why I I would like say I'm I'm not that like when I saw this I wasn't like that's got to be false I'm like okay yeah I'm gonna believe that yeah I don't know most of phone research is like you know take a function that gives user input like to find your threat model and then do source to sync analysis on some vulnerable function or failure to gate a function

on like a length check and like does user data get there bug confirmed and like yeah that's literally just pattern matching that we've solved a lot of the times previously with like satisfiability solvers right like anger and like Z3 like take the graph of a function turn it into a math problem can you solve the math problem cool bug confirmed well now with AI it's just like that process of doing source to sync on like text it can do incredibly fast right it's very good now obviously because it's a sarcastic it creates a lot of false positives but if you can figure out a way to reduce the false positives or automate the the validation of those false positives and then yeah it's it's crazy and I think what I thought about asking what's that have they thought about

asking myth those I know come on and I can you just no mistakes please um the thing that myth those is set apart differently according to the anthropic report is its ability to chain together primitives right so the scary part from like a cybercrime perspective is like you have a gadget a that gives you an arbitrary read and gadget b that gives you an arbitrary right okay like those two separate things are like not super important if they're not used together well what myth those is able to do is out of a hundred tests I think it's like 83% of the time find exploit primitives in a vulnerable code base and chain them together to get RCE right that's the scary part because then that's true like end to end exploit creation for a bad actor and that's I think what scares anthropic

the most um now I know there's an argument where like Firefox wasn't in the sandbox for that experiment so like it doesn't actually matter but I mean just apply that process to the sandbox and the same thing applies you know it's just I think I wanted to prove a point that it could do that well and also I mean again like as I said many times I can't stand AI companies so I'm not trying to defend them or anything but I'm just trying to point I'm just trying to point out how plausible this stuff is to me from a neutral observer standpoint classic key defending AI companies yeah I know right I know if you think about it it's like look security researchers who do not number that many we're already cranking out zero days that I much too alarming rate for me right

like like you know there's a hack every other day right it's not like CVEs are piling up like there's no demand yeah not all of them are actually all that bad or whatever but like it's not like security researchers were having trouble producing a fair number of of critical vulnerabilities even with the limited resources that they had so it's also not weird to think that like if you had more automation you would find a lot more of them it doesn't like there's clearly just a lot of bugs guys like there's a lot of freaking bugs and it's just doesn't seem that unusual that if you have more sophisticated pattern matching more sophisticated combinatorial checking where the security research doesn't have to spend a lot of time setting up the tool because it can

just kind of ingest the code and it knows what roughly what it means yeah I mean they're they're rates gonna increase if nothing else existing security teams rates of finance weights it has to I mean it just has to unless this thing is just a complete pile of crap it's got it the other thing too we've been seeing from each like new generation of model is that they're getting at least from my experience and what I'm what I'm reading from people and everything they're getting better at calling other tools so like they call out to stuff more regularly right yeah and they can pay attention for a longer right like like recompile this code and see you know make this exploit and run it against this thing or whatever right like those are all things that if you automate them a

security researcher gets much faster at finding but because they're not having to set up the tooling themselves to like go work on this exploit like whatever whatever those steps were they don't have to do them anymore right right so then if you're like oh now it can run instead of like I have to prompt it at every stage for the next thing to do is I can give it 10 rough things say try a bunch of combinations of these and then it runs for 24 hours yeah you're just like a lot it's literally like in my mind some of it is like yeah well we already know fuzzers exist like we use them all the time and they're good right it's like in some ways almost like yeah it's like fuzzers it's like fuzzers squared right it's like a thing now that can like target the fuzzing

at things specifically so that it fuzzers to things that would be very hard for the classic testing to catch because when you have stochastic testing and you have to chain two things together you're never gonna randomly pick the two things that would have to happen for them to work right here is a thing that can like target that specifically go like oh I think combine these two things to be a problem me fuzz that specific path oh yep I got it right that's where it's crazy is like you just have the AI right the fuzzer and then like if you can automate that process you win a lot of the time it's pretty pretty amazing um I do have to go though I've been meeting in three minutes so I got to rip um hopefully you get mythos access congrats yeah

I'd be neat now it's not gonna happen come on guys thank you bye I like you guys but it looks like it's the end of our show unfortunately yeah so true all right thank you everybody I would just like to say that I would I would just like to say that Casey and TJ and obviously Timo Casey that just left commonly known as low-level learning uh you guys you know you make the show magic and and now I'm just gonna go about being lonely again kind of sad oh prime I thought I thought I thought I was gonna get food but I uh I just assumed something's gonna happen all right um the real the good news is guys you can enjoy full episodes of the stand-up now on YouTube if you go to the stand-up pile full which I'm gonna try to rename hopefully at some point

we're trying to work some things out to get it a better name but right now you to you to YouTube am I right um if you go to the website if you go to our website will it have links to these yeah will okay it will and it'll have it spelled out uh we'll we'll make it more clear once we figure everything out over the next week maybe by the time you're listening to this on YouTube at by time listening is on YouTube uh we're gonna upload all of the backlog to that channel as well so we should have every episode on YouTube in one spot very easy to see etc obviously always can you know RSS download download the audio directly don't press the red button on that side of course yeah teach what is that web address that people should go to the stand-up pod

yes the stand go to the stand-up pod.com all the links will be there all the episodes will be there you want YouTube you want Spotify you want downloads you want RSS you got it the stand-up pod.com yeah check this out I'm just gonna do something for the audience look at this if you go here you click trash made a black mirror app you can go and you can listen to it right on the website you can have all the nice information right here you don't even trash his app right there you can go in here we don't even charge you for that we don't even charge you can play on Spotify you can download and just have personally for you to do whatever you do that's for you now that we're and then I'll make it I'll make it so it links to the YouTube there later as well

now that we're gonna have a dedicated YouTube channel for that too so for all of you out there yeah you know the AI companies claim that you're gonna get UBI but we're actually giving you universal basic podcast you just get it you be free UBP UBP you know me yeah UBP yeah I was gonna say well I don't know what I was gonna say that's fine we should just stick a fork in it guys it's done all right thanks good seeing you everybody thank you YouTube thanks again whatever your name is teach you're pretty neat

What Mythos Means for Software Security

Kapitelklicken springt direkt hin

Shownotes

Transkriptvom Podcast bereitgestellt · Zeitmarken springen hin