Memory Safe C

0:00 You guys ready to talk today it's today's a pretty good topic you guys ready for today. I'm ready. Okay. Yeah. I mean I'm not ready. I'm just ready. You're going to start anyway. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah.

0:40 Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah Yeah.

3:40 Yeah. Yeah. Yeah. That's why I see it from that diagram that's pretty funny. Look at all these engineers sitting at their neat little desks. It takes dirty work to keep a code-based clean. Every day sickles are out there committing unreviewed code. And when that happens, renters won't save you. You need someone like me to screw back. Who you calling screw bag?是什麼? What's the You're trying to push, unnecessary comments, global state, nested turnaries?

4:42 Ugh, my bad. I didn't even read the code yet. You disgust me. Step away from the keyboard. Let me explain. Instead of mouse, he's margin-a-prod! You have the right to remain silent. Anything you push to get help can and will be used against you. You have the right to a debugger, but if you can't out of forward one, a public stat phrase will be made available.

5:02 And one more code criminal off the streets where they belong. HR. Look, I didn't... I know I didn't review any of the code, but I was going to have code rabbit review from the start. With one click fixes and style enforcement, I don't need merch cop. I would never merge our reviewed code, but a first pass with code rabbit always makes things go faster.

5:25 Actually, you can try it too at coderabbit.ai. Next week on merch cop. The difflers out there and I'm going to be the one to deprecate them. So to kind of put kind of wrap it all up so everybody understands how Phil C works, the runtime impact of Phil C is going to be somewhere between 1.2 to 4x is what I have seen.

5:50 And then you will see comments on the internet that say something along the lines up. What's 20% these days? Nobody cares if you lose 20% performance, which I know that case you will probably lose some sort of sleeper hair over with that kind of comments on the internet. How do we get from 1 to 4x and go from that to 20%.

6:07 Well, it's 1.2 would be, you know, 20%. Oh, you mean if you just took just the minimum as assuming that you were going to get the minimum for some reason. Correct evidence. Okay, sure. There is another performance penalty. Can we talk about that really quickly? Yeah, yeah, let's see how I do that. I just have to, I'm going to be a little bit on the rust side today.

6:29 So I came prepared. Oh, no. Perfect. Okay. Hey guys, it's Rusty. Checking in today. The other problem is that there's a serious memory, additional performance constraint here in the sense that it's going to allocate a lot more memory for lots of different kinds of programs. So I just wanted to make sure that that's clear.

6:54 It's not a zero cost abstraction. Okay, it's not a zero cost. So probably you may continue. Actually, teachers very, very correct. It is not, in fact, a zero cost abstraction. Anyway, so I just wanted to talk about this because the thing that really sparked all of this was that pseudo RS is kind of this new brand of pseudo.

7:11 It's the new pseudo on the block and and you bun to saying, hey, we're using it. But all the comments are like, why? What's why are you doing it? Like, it seems to be at least it appears. I don't know if it's actually true in real life, but in Twitter life, it appears to be a largely negative experience.

7:26 And so with that in mind, some people are saying, well, why don't you just use Phil, Phil C, the performance of pseudo is negligible. And you can get 90% of the safety without having to deal with Rust in a rewrite and rediscovering all the bugs. And so that's why we have with us a Casey and low level learning.

7:44 TJ is just here for fun. And so therefore, I think it's fun to start probably with with the security side just because I think that that is probably the most poignant or at least the most reasonable place to start when it comes to a memory safe sea. Ed, if you'd like to kick off any any talks about that, I have a lot of questions about it, but any initial statements.

8:05 Yeah, I mean, so I guess to kind of kick this conversation off right what what is spawning this? If you guys aren't aware, pseudo, you know, well-known well-known program that tends to be the target of a lot of vulnerability research because it is one of the most widely available widely used set UID binaries, right?

8:21 It's binary that runs as root and use it to run things either as root or somebody else. And so a memory corruption vulnerability in pseudo obviously violates a pretty pretty hefty, pretty scary security boundary. And so as a result, there's a push to re-rate a lot of benutils, one of them being pseudo in rust, right?

8:38 Kind of a cool idea. This also for some reason kicked off an effort to like re-rate all of the core utils in rust to include like echo, cat, said, which is a whole other conversation. But unfortunately, despite it being written in rust, shocking, I know I think there were two CVEs that came out recently in the RS implementation of pseudo.

8:59 So it kind of begs a bigger question. I think, you know, we talked about this in the previous episode that I was on with Casey. You know, all because you write in rust doesn't mean that you're going to get the exact same behavior. And if you can't confirm by a some set of testing that you have, you know, not regressed at all in your functionality, then you're potentially introducing new vulnerabilities to the code through logic errors.

9:19 And that's I think exactly what we saw with pseudo RS. And so instead of rewriting everything in dollar sign new language, I'm sure we're going to iterate on new language every 20 years. Why not drop in a new compiler that has sanitizers that has intermediate representation checks to put in on an arbitrary pointer, the vector size, right?

9:39 And that's what Phil C doesn't. I have not used Phil C personally. I'm kind of curious prime how it went for you. Phil C is a great idea. There are some compact issues. You have to be compatible with the LLVM tool chain to make it work, which is kind of an issue for some some piece of software, but generally great idea. I love the concept.

9:56 So that's kind of my my first take. Casey about you. I'm on the same page, essentially like rewriting something and rust to me is usually a pretty bad idea because the problem is that your really rust can only guarantee you a couple security things that it's specifically designed to do, but all of the other ways you can have bugs are still there. So it doesn't really handle all of like your logic errors or things like that.

10:24 That could result in your program doing something potentially catastrophic, especially in the case of like a core utility like pseudo. And so really like I would never have like I would be like don't do that like like we know that there are memory, you know, safety problems in the languages these were written in, but they've been kind of beaten out over the years and we can keep looking for them and so on.

10:45 And so rewriting something and rust like that is just very dangerous and I think ill advised. So something like Phil C is kind of a very nice middle ground, I would say, which is that like we hey, we can preserve the logic integrity of this thing that has now been battle tested so that we're pretty sure that we found a lot of the logic errors that I'm sure we're in there originally that we've now kind of gotten out over the years as people found them.

11:08 But now you can also get this added layer of memory checking just to make sure that memory things we haven't found yet, maybe this will protect against those. And then you're really just down to like, okay, are there bugs in Phil C itself, right? And you know, maybe those take a little while to sus out to like maybe there are some edge cases that, you know, we'll be found.

11:30 But until then it's like that seems like a much better that would be true of rust as well, right? Like there's no difference between rust and Phil C in that sense, you can always have bugs and the implementation of the memory safety. So it really does seem like a much more sensible approach if you're just talking about our goal here is to make sure that we're providing the safest possible version of something like pseudo.

11:50 Yeah, okay, so I mean, those are all kind of like they feel like intellectual arguments more than actual like practical arguments. Do you think there's an actual practical argument to say that Phil C is going to be something that is that is like fully usable like Casey, could you use Phil C in a game or would you even ever consider using something like Phil C in a game engine slash game.

12:10 Well, I mean, you'd have to have some reason why you felt like what it's offering you is important. So, you know, there are some ways in which you could imagine this being important. For example, if you have a lot of user generated content in a game where potential bad actors might be trying to use that ability to upload content to create sort of security exploits, right?

12:39 You can imagine something like Roblox or Fortnight or whatever where they're kind of constantly trying to increase the amount to which their users can contribute new games, new modes of play, whatever. You could imagine something like that where there are cases where you're very scared about things like memory protection and stuff like that.

13:01 And so I could see maybe some arguments in certain specific scenarios where you're like, we really need this. So we're going to take this isolated part of the engine and maybe we will use something like Phil C will accept the performance hit, but we just really need this we need is all the security we can get.

13:16 But for the for the majority where you're talking about just running a game in an isolated context on someone's machine, it's unclear what you really need this for right for for the average game development. It's not really and I don't I mean the person created Phil C works at epic right they are they are working on I think they work on verse the epic like epic fortnight programming language thing.

13:42 And so like you know that I would imagine most of like an unreal answer something that wouldn't really adopt something like Phil C but you could see some specific cases where that might be important and for example verse which is their language that would be a case where he probably is employing some of these techniques there I don't have any inside knowledge of that and I don't work on that.

14:01 But so that be my take on it is it you'd have to be in an area where you really cared about it because otherwise I'm not really sure what what you would need it for does that make sense. Yes, but I think that does make sense. I mean where where my big misunderstanding probably just comes from is that I don't understand I was trying to read some of the things with the differences between like a sand and you be sand.

14:25 And how that how Phil C actually can send some of those problems but at the end of the day TJ brought up really good points because he's he's being the resident Rust have there are things that Phil C just can't do and they even they even shout it out which is like file handles. They are something that we can't that can't be represented in the same kind of context.

14:45 I'll give I'll give the example first off that's probably the longest a rust users ever gone without speaking so I just would like that to go into the record as a moment for allowing other people to speak. I do have rust running in production right now so that is something that also most rust users have never done so I would put myself ahead of almost every rust user in the entire world.

15:11 Yeah, so just throwing it out there as two things that make me very qualified to talk about this. Can I need to say one thing though you also did the zero to I use rust by the way at the same standard rate as any rust dev who start speaking within the first like five seconds you mentioned the fact that you use rust by the way.

15:33 Yeah, I don't I thought I wasn't allowed to run rust see unless I did that I still need to run rust see sometimes so I don't understand so okay anyways. So this is on a serious note instead of me being about rust users. The this is not a reason that I would rewrite a program so this is like a different class of things if we want to start we I would we can maybe talk more about like the rewrite situation first and then talk about you know like if you were starting off a new project which one you might want to choose and pros and cons because the thing that I think a lot of people are missing and and case you guys both alluded to this at the beginning I maybe I have to take my rest to offer this to say something nice about Phil C.

16:22 Is like you can't just rewrite everything it actually doesn't work like we don't have enough time or resources or a bunch of other things even just like if you think you can bug for bug remake it exactly right and handle every case it's just not realistic to make that happen like even I like I think probably I could run me of him you know at like even at Phil C's current worst case 4x slower and me of him would still feel fast and good for me right you know I'm saying like even if it was for for time slower and that's assuming and I'm I see as well we got a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a

17:32 little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little

18:02 a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a

18:32 little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little

19:02 bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit

19:32 of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of

20:02 a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a

20:32 little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little

21:02 a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a

21:32 little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little

22:02 bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit

22:32 of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of

23:02 a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a

23:32 little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little

24:02 a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of a

24:32 It tracks essentially all allocated objects of any kind. So any time you actually do any kind of allocation, it's going to say, all right, I'm going to track that in using, I mean, what you would call like memory outside of the addressable space of your language. Meaning inside what, you know, doing what you're able to do inside a FLC program, you cannot access the tracking data if that makes sense, right?

25:01 And so that tracking data will then be used any time you're trying to work with one of these pointers, you know, like a regular C pointer. To you, it looks like you're working with the pointer normally like you wouldn't see, but actually, FLC is going to look to make sure that that pointer originated from the actual part of memory that you are accessing, right?

25:25 So it tracks not just are you in bounds, meaning not just are you within the particular object that you, that you were sort of claimed you were accessing, but are you in the one that this pointer originally came from? And this works even if you like read and write pointers to like blind memory and stuff like this, right?

25:44 Because they're, because essentially what FLC is doing is it's using sort of almost like a pseudo pointer. It's using a 64-bit value that you don't directly use. You indirectly use it by looking at sort of that backing, that tracking information to make sure that you're tagged, that you like match what you're trying to access.

26:05 And so when they do this in hardware, it's sort of a weaker version of this. What they do is for pointers in the hardware versions of this, they just use the top bits of the pointer that wouldn't be used because you don't have that much memory. You know, you don't have 64 bits worth of memory in a machine.

26:22 These the top bits to assign say like a four-bit tag, you know, a value between one and 15, let's say, and zeros usually reserve that sort of stuff. They'll use that tag and then every block of memory that you sort of allocate or rather that you mark, you can say like this region of memory, you can mark regions of memory with tags, whatever you want, some random tag that you associate. Every time you use a pointer to access some memory, it checks to see whether the tags match.

26:50 So that even if two blocks of memory are right next to each other, if you go from one to the other by increment the pointer like low level was saying, if you go outside of like an array bound, the tags hopefully won't match because hopefully any two neighboring regions got different tags, right? But that's really like, there's some other things that you can use that these extensions do, but you know, for the most part it's doing that sort of thing.

27:14 And Phil C is just sort of like the on steroids version of that, right? This it tracks everything about these so that you can't even have accidental conflicts of the tags, it's not just using some simple like four-bit scheme. And it also does like more aggressive use after free tracking and all this other sorts of stuff by using garbage collection to like never remove those things so that it remembers like each region until it's actually not accessible by anyone anymore and so on.

27:38 So hopefully that's just a little bit like that's like so it's a much more complete system than something like address sanitizer if that makes sense. So that's a lot of stuff in there, but is it effectively turning all memory accesses into like a virtual pointer where like yes, it's like a look up into another object table and then it does.

27:57 I mean, that's probably not the that's probably it sounds a little bit more. That might be a slightly more aggressive way to say it. You can think of it more like there's almost like a shadow version of thing. You should go take a look. They have it. I'll show you, but I have to power up first. Where do you have it from?

28:19 Why? Why? Okay. That's pretty good. Now I'm super cyan mode because we have a whiteboard. Okay. We're saying rust user. I like it. Yeah. Okay. Right. We have like pointer X. It's pointing to 40. Right. Why? Four four. Right. You can imagine in memory, we've got these two pointers. Right. What happens in normal C when you access like X five?

28:44 Yeah. You just get zero or whatever. Whatever like the next if element. Yeah. We get over here. Yeah. But in Phil C, it says, yo, this if we did like malloc four and malloc four for both of these, right? In this case, it's going to say it's got a lower bound. A four zero and an upper bound. A four three.

29:01 Yeah. And here's where it currently is. Well, I understand that. I'm working with implementation, right? Like in the context of this example, what is X? Is X a pointer to expand your exchange? To you, the user. Yeah. X is X to you. The user is still just this pointer. Interesting. But this is, I'm assuming this is what this is what I'm going to, I'm, may we can get confirmation as far as I could tell from reading and everything else.

29:23 It gives you back exactly this. This is what you think you've got. I think it's wider. So I don't think that's how it works. And it's not wide. It's wide though, right? No, no, no, no, no. It's the same size. It's 64 bits. Oh, it is. Okay. Yeah. I don't use Phil C. And the first time I heard about Phil C.

29:43 When was when you guys said we were doing an episode about Phil C. Nice. Okay. I apologize for not knowing more about it. If I was a Phil C user, I'd be happy to get the fluctuation. I believe the way that it works is they use a shadow allocation. So every time you allocate something, they allocate twice that effectively, right?

30:01 Yes, you're right. You're right, Casey. And then they use the, so effectively what X is is it's going into the shadow region which then tells Phil C where, like, how to get the additional data, right? Data. In fact, you said the person who wrote it was in chat. Is that correct? Would that be a good summary?

30:18 Because I read about this literally yesterday. So that's all I know. But that I believe was how it was explained in the documentation. Yes. Yes. Yes, I think you're right, Casey. Yes. Yeah. That was my apologies. It doesn't return the wider one. It has one somewhere else that it's holding on to. Yes.

30:36 It's called in visit caps, right? Yes. It has the invisible capability stored off in a region in which you're not allowed to access. Exactly. And which they returned to you looking just like a pointer acts like a pointer smells like a pointer. And see it's just literally a number. And then boom, by the way, it does that runtime checks against it.

30:50 Right. And that's why you can still add to it and work with it like you normally would because because they have a shadow region, they can still like figure out like what's going on there, right? Or things like this. And so I don't know the specifics. I want to now read more about it because I thought it was really cool when I was reading.

31:05 It's like, oh, this is a really neat thing. So I'd like to go read about how they handle all the actual practical details. But I didn't immediately find sort of the document of that. Like there's a there's really good overview documents, but like, oh, how did you handle this? How did you handle that?

31:19 Like that's I didn't find that immediately. So I like to go read like how do you actually use the shadow region? Like how is it doing that and all that sort of stuff? And also the other important thing that they bring up in this documentation, which I think is important to mention as well, is that if like a naive implementation of stuff like this, would have a lot of problems with threading potentially because like different people can sort of mutate pointers and from, you know, and you can have race conditions and sort of stuff.

31:45 And so Phil sees also also very careful to make sure that they get those things right, which again, like if you look at it, it's like, wow, this sounds very complicated. Like why did it have to, you know, why does it have to allocate all this extra memory or things like that? And the answer is because they were they were trying to take this really seriously.

32:02 Like not like, oh, we cut some memory bugs, but like no, we really do provide a secure operating model, which is, which again, if you're drinking the rust-cooled, you have to respect that. This is this is doing it for real and on existing programs. It's pretty great. Yes, we like that. This looks crazy thing for the rust people, even with my rust super, super crazy hair on right now.

32:25 That I don't really get is like, this is good. This stops this from happening right now. If you ran C code and you did this, it would be, it would be chill. You're like, it would let you, it would be chill. It would let you just read from memory. That's not the thing that you're doing. That's bad. We don't like that.

32:41 We want computers to work good. That was the, that's the thing that I don't get. So that part's good. And it seems like for some reason everyone's mad about it. And I don't really get why they're mad. I don't know. Because it's good. Most people really hate it when you introduce something good that's not the thing that they were previously telling everyone was good.

33:02 Right. Like they don't like to hear that someone else made something good that might be a reason not to use their good thing. That's true. Right. Like heaven forbid. Right. But yeah, there's, I do have one question about Phil C that I wasn't able to answer directly from my sort of like five second read.

33:21 So again, like it'd be nice to meet, maybe if, if, if the author is in chat, maybe they can come on sometime and tell us so that they so that people can have a more accurate description of what it's actually doing. Because that would be really nice. And obviously they would not get it wrong because they wrote it.

33:36 But what I was going to say is I don't really know how this works with things like I okay, I create a block of memory that I'm going to sub allocate out of. And I wasn't sure like what it does for that. Like that's what one thing like it's like on my list now, I'd like to go read about like how it deals with that.

33:58 And like, like are there, is that just disallowed? Is it allowed? And we just say, oh well, no sub capabilities right now. What? No sub capabilities right now. Okay. I think that's because if you take a pointer from that region, you technically have a lower bound of the original lower. Exactly. Exactly.

34:17 Of the big upper. And so I was, and so like that's a thing that makes it kind of hard to port a lot of existing programs because a lot of them will do that. So then it's kind of like, okay, those programs kind of have to be rewritten to. Like maybe there's some way that, you know, Phil C could be extended to support that.

34:33 Like, hey, allow us to sort of write a section that's like, here's the, here's how the memory marking is going to work inside our thing. I don't know. But like that would, that would make it challenging to port programs that were written in that style, which is a very good style of writing. Like it's very efficient.

34:48 So it's kind of, I don't know what you do about that. If it's, you know, in terms of, I want to use this in some subset. And I want to just use my existing C code. Effectively, the creator is saying the best thing you can do with your arena allocators is to replace them with individual mallets. Okay.

35:07 I mean, which is not that hard because usually your arena allocations are going through like a pound define kind of a thing to like allow you to switch them for debug and stuff like that. Anyway, or to mark like where they're coming from. And I assume, I guess let me ask another question while we have the benefit of the author and chat.

35:23 So do you then just like how do you, because normally arena freeze are just going to free the whole arena. So is that just taken care of by the garbage collector? Don't call free the deal with it. Don't. Then yeah, then so then that's a not a hard port. Yeah, right? That's not a, that's not a difficult port.

35:41 I don't think for most people, because like at least for my own code, I always have those running through macros so you can mark them with file in line and debug and stuff like that. So that seems pretty pretty reasonable. And if you've been following along, we'll link as well. Pizzleinator, your ex in the like description of the episode.

35:59 But like, Pizzleinator's been working on a bunch of different like very complicated, very real world. See programs that are big that you cannot just rewrite to rust. Guys, you can't do it. And talking about like Emax and Ruby and a few other ones, you can follow along as he's kind of like micro blogging.

36:23 As I say, that's that's in these days, right? Micro blogging. I'm on Tumblr, obviously. You can tell from my hair. So you can follow along there for a bunch of different ones and see what he's been up to and how he's gotten there, which is cool. Cool. TJ, as the resident rust expert. Yes, thank you. Could you could you help us understand things that Phil C doesn't do?

36:46 Or why? Like if you're starting brand new, but you're really good at C, why would you even want to choose rust if you want to type like or at this level of safety? And you're like, well, I could just use Phil C, but why would you want to choose rust? Yeah, a blue field project, if you will, for me. Yes.

37:03 Yes. So a blue sky project. Blue sky project. If I was starting a new blue sky project today, what would I pick? Yeah. Okay, so one reason you wouldn't use Phil C. You don't enjoy writing C. That's one reason. Good answer. That's a legit reason. Okay. There's lots of projects that don't need to be written in C.

37:25 Okay. That's fine. In fact, there's lots of projects that don't need to be written in rust. I almost have to take this off when I say this, right? Yeah. It's just true. It's just true. They lost your blue hair on that one. I know. I know. I think you're going to have to lose the wig, Teage. Yes. You've been removed from your board seat on the rust foundation for that.

37:45 But okay. Yes. But let's say you're starting out a new project. And the requirements are such that rust makes, or some systems language makes a good choice. Right? Or you just have some other requirements. The example for me that, like when I was working at Source Graph, we have a very straightforward, in and out system where we have text come in and we need syntax highlights to come out on the other side.

38:12 It actually matters for that to be really fast. You want it to go really fast. It's always going to look the same shape. You have text come in. You'd like to copy it not so often. You're going to run this for like lots of different files and lots of different languages for lots of different customers.

38:29 You'd like this to be really good and really nice and really, really fast. Okay. You have options for what you could do. Right? You could write that service in C, I guess, right? It's maybe a little bit harder to picture, but there are decent parser generators and other stuff you could use to get you part of the way there.

38:50 And then you could apply some likes and text highlighting from that and all that good stuff. But now if you're going to go and do something with Phil C, you're going to experience a slowdown. And if you don't do that, you're more likely to experience some bad security vulnerabilities on a bunch of untrusted input because you're highlighting random people's code.

39:08 Right? So, oh, they put in a Unicode character that was too wide for your thing and then overflowed this buffer. Now you access this and it's corrupted memory and you screwed yourself over too bad, so sad. Right? So in some cases like this where you need, you need a lot of speed, you're maybe going to work on a lot of untrusted input, you're connected to the internet, you're doing other stuff like that.

39:29 Like I think there's good reasons why you might want to choose Rust and you'd be able to get like faster performance or something like that with Rust. I mean also I think, you know, from a like technical standpoint, a lot of people appreciate the some of the like type system guarantees you can get from Rust.

39:49 That Phil C is not going to give you. You're hopefully now like you won't in many cases have like an actual security vulnerability. You won't have a CVE from these kinds of problems, but it will crash like at runtime, which is good. That's objectively better. Also Rust crashes at runtime for things too. That's also how it solves some problems.

40:10 So that's not like if Rust acts as something outside and it's unchecked, that's just an expector a panic and then it will crash. Sweet, that's memory safe. We are happy. That's actually good. But like there's lots of things. The example you had brought up earlier for like file descriptors, right? In Rust, if I open up a file and I am making my own, you know, like API for it, I'll have the file descriptor like or I'll have the function take a callback, that callbacks one of the parameters will be a reference to the file descriptor.

40:44 In Rust, the borrow checker will prevent me at compile time from accessing that outside of the callback. So you couldn't like save the file descriptor to a global and then try and read from it later. That won't work, right? And most of the time it's like not clonable and not copyable. So you would have to go through like a lot of hoops to try and even get that value out in a way.

41:06 You could even literally like just callback code. The callback code is impossible in Rust. It's so hard to use. The callback pattern in Rust. It's so forgetting. True. Yes. I still haven't figured out streams quite successfully yet. Yeah. But that would be a thing, right? That like if I try and use this value, this file descriptor outside of the callback, the type system and the borrow checker, right?

41:27 The linear type system that Rust has will block you from being able to do that at compile time instead of like runtime, right? That's like for me, I'm a type systems guy like types for me that makes me happy, right? Like of course, especially the blue hair on that makes me really, really happy. So that would be something where like if what you want to do, right?

41:50 Like push left, we're going to push the errors to happen earlier along in the dev cycle. We're going to experience those at the compiler and there's no like 100,000 lines of battle tested code that has a bunch of stuff. You might choose Rust over like Phil C, right? And then you like, and that would be I think a reasonable choice.

42:10 But for me, that's everyone seems to be talking past each other about that on the internet, which is surprising. Can I ask a clarifying question there? I'm not sure 100% follow that. First of all, I would also say that the phrase push left. Yeah. It's strongly like sort of suggests a left to right leading language, reading language, because that's the order you're going in.

42:30 So I think you lose the blue hair on that one as well. But true. I don't say more towards the beginning of a sentence push. Is there a way to say that? Yeah, exactly. What I would say is what is the bug you're trying to prevent with this file handle scenario? Just to I understand what you mean by Rust preventing this better than?

42:49 Like if you if you have some function, right? That like opens up and then closes a file handle for you and says like, Hey, or a file descriptor writer, any kind of this in database connection, any kind of thing where you have like a handle, right? If you're writing this and like, this is not true of just C, but this would be like in Python or something too.

43:08 You could just like save that handle inside of your callback to some global value. It can escape its region, right? So if you like to with open file descriptor in Python and then inside of there, you like save the file descriptor and later in your program after that original things already closed it outside of that region.

43:29 Now you try and read from it again. It'll be an error. It will crash, right? Because that's pretty easy to implement in in C or C++ as well, right? Like especially C++. Like to do what part? You just wrap that wrap file handles in an in an accessor and that will just work, right? So like if you wanted that protection and see you could just get it, right?

43:54 Like you don't need a language being defeated as we speak. No, I mean, I'm not trying to just understand why you couldn't just make that same thing by just making sure that instead of using like an int as your file descriptor, you actually use a struct that does something, you know, or you know, a C++ class for up to any programmers, right?

44:12 But I'm saying it like sort of regardless of that, I think we're not talking about singing. I'm saying like we could prevent in like rust from the with the borrow checker. You can prevent the value from escaping its region, right? Which I don't, which I maybe there is a C++. I'm sure there is a C++ thing that can do that.

44:34 I don't know. I just assume everything is possible. You're just talking about like you want like because they have special case file descriptors with the static analysis in rust or something like this. No, I'm, yeah, I think I'm, I'm just talking about type system. And I'm like, well, the type system in both can do what you're talking about.

44:51 So we're just talking about the type system as opposed to other things, right? Yes, I'm just talking about the kinds of things. Like this is, you know, as like I'm trying to present the case for why someone would say they would rather write this kind of thing in rust, right? Instead of experiencing a crash with Phil C at runtime.

45:09 Okay. We could find that out from the compiler, right? That like, oh, this kind of access isn't allowed. That's not thread safe. That's not that escaped its region. But it shouldn't this kind of value, right? Okay. There are things that we can do there, right? In like rust with the type system that fight the same kinds of bugs.

45:32 You might get, you might get solved from Phil C, right? But instead of, uh, instead of having a crash at runtime, we get, it doesn't compile. And I handle that case. Yeah. The borrowed checker, during these use after free and double free at compile time. That is the objective of the borrowed checker.

45:47 Yes. That's a probably way more succinct way to say that. Good job, Ed. I like to think of it as just like, it's effectively a unique pointer. Yeah. But at compile time. Right. So my point being separately from, I think it's obviously good that there's an option where we can solve a bunch of these classes of bugs in C and prevent it, right?

46:11 In this case of like which language would I pick for certain projects? There are definitely like reasons you would pick rust over just picking Phil C for like a new green field project. Of course, there's other reasons why you might pick C over rust as well. Um, like you might actually ship it and things like that, which is cool.

46:31 Um, so, but that's like those kinds of bugs Phil C does not solve that at compile time. That's all I'm trying to say as like a comparison of the two. And that's like a reasonable trade off that people want to make. Yes. And that's just to be clear on that. That's because of the borrowed checker, not the type system, right?

46:49 Just to be clear on that part, right? Uh, yeah. Or just so I'm just what I'm understanding and I should say, right? Because that is a part of the through me. I was like, why is that a type system problem? Okay. Tag them actually. Okay. So it is actually a type system. Well, the borrowed checker is sort of like a result of the linear affine type system that rust has.

47:08 Okay. And I would just say they, they go together. You would like to talk about type system, the type system being like including the borrowed checker. Yes. Okay. Exactly. That makes sense. So I also have another question, which is that does rust catch all of the same memory errors that Phil C does because I don't know if that's true.

47:32 No. Phil C might catch more. It does. Yeah. So that's it. Like if you really care about security, you probably shouldn't be using rust. Yeah. Rust will catch them, but like it'll turn the condition into a DOS. So it's like it's not an exploitable memory access, but it's also not caught a compile time.

47:50 So like, like you can do even, I mean, even at runtime, does Phil C catch things that rust doesn't catch is my question. I thought I saw a run time. Unfortunately, forgot to save it, but it's a condition in which Phil C does not catch something in which rust does. I really wish I wouldn't say blocks.

48:09 There are, I think, certain things you can do that in rust inside of unsafe blocks. That rust does not check. Real quick. Even all of its stuff. And maybe still I'm not actually a 100 presenter. If it will do a CVE, but inside of unsafe blocks, that Phil C does catch. I do have to go in five minutes.

48:31 It's all related memory safety issues. Yeah. Syscall related memory safety issues. Are you saying, for example, in a syscall handler where it does a copy to user, like outside of the bounds of a memory region? Like what about syscall is particular? I'm curious what you're saying there. And then I do have to run.

48:49 I'm sorry guys. We're going to pay you wait. I just want to say he said he did confirm CVE in unsafe blocks is totally possible. How would Phil C detect that? Yes. And then we'll have to just have Phil come on. Yeah. 100%. The right thing is to have him on the show. If you if song is he wants to come on.

49:09 I mean, he said yes. Okay. Yes. Yes. Yes. There's also. I do think. I'll just mention one other thing as the rust person. I'll just put this back on. Oh. Sometimes 20% slower or up to four times. And 1.5 to two times as much memory actually makes it not possible to do constrained hardware. Or like other stuff like that.

49:33 We'll just literally make it so it's just not possible to accept the performance hit. Yeah. Because some places in the world performance still actually matters. And you can't just spin up 5,000 new lamb does and pay $10 million in cloud costs. Okay. There you go. You've just some you're going to summon DHH dude.

49:49 You're going to summon him. He's going to be like, yes, we can. He's like, yes, we can. We will spin up all of them. No, he's against also true. You got to be careful. You can. Tactically spin up all of them. You just pay a lot of money. Yep. And we got to have it on camera. Like this is all in my mind right now.

50:03 He says that it intercepts all sys calls of the ABI layer which makes sense. Phil, I know what we need. We need a slide show with these examples. And we can work on it together. I'll message you after this on X. And we'll get the setup. Welcome back with a slide show with code examples. And then Ed, you can ask every single question you want.

50:19 I've been wanting to do this a lot. Ever since trashed the the TypeScript presentation. I've been wanting to do more presentations. And then have us all interrupt the presentation. Be like, put what about this? What about this? What about this? And that was such a fun time. Yes. All right. Okay. Well, then let's end it right now.

50:36 We'll go. I do want to say one thing. Oh, go ahead, I'm sorry. I will say that every time I use a union and see, I get so happy and so sad at the same time because it does make me long and yearn for rust. I will say that the automatic tagged unions at the syntax level is just a thing of glory and beauty.

50:52 100% at the time. Yeah. But don't worry. In another 50 years, the C++ committee will finally get one working. We're almost there, guys. Just one more committee. One more version. One more. One more. One more. One more two thousand page revision. And they'll get the discriminated unions working properly instead of sucking.

51:09 Casey, if that were true, then why are some people using like C17 instead of C94? That doesn't even make sense, then. Checkmate. Yeah. Checkmate. It's 94 is bigger than 17. So we should be using that one, don't you think? 20, 20, 20, 24. Yeah. I got to go. I got to go. Bye. Bye. Sounds worse then, really.

51:28 Yeah. All right. Everybody will go as well just because this is happening and already left us. And now the stream is completely screwed up. Thank you, everybody, for joining us for this. Hey, if you are watching on YouTube, you can see everything on Spotify. Click the full episode because you miss a lot of the banter when you only watch it on YouTube.

51:48 So thank you very much for watching. Thank you for all the guests. Ed with low level dot academy. Casey with computer enhance.com and teach with boot.dev slash teach. Yeah, or by the way, or by the way, you want to support both TJ and that because I also have two courses on there as well. I just did like a spider man thing and the name is the spider gen.

52:09 All right. Thank you, everybody.