ChatGPT Just Caught a Spy Stealing From OpenAI

ChatGPT just discovered and arrested a spy in OpenAI's own HQ. And that's the least craziest thing I'm going to say on this entire episode, because this whole story is crazy. It starts off with an unknown individual planting an audio listening spyware device in the back of a television wrapped in toilet paper.

Why it's wrapped in toilet paper, I have no idea. But the important part is the conference room. Why do we care? Well, a bunch of really important executives and business partners at OpenAI meet to discuss proprietary information, secret product plans. So you can imagine if that information got into the wrong hands of a person, a company, or maybe even another country, that could probably become a bit of a problem.

So OpenAI handed over the case to the FBI and their own internal model, who ended up ratting on the individual who had planted that device. Kash Patel, the head of the FBI, said this is the Bureau's utmost priority at the moment, that AI has finally become an importance or asset of national security and risk.

Imagine if James Bond, 007, had access to Fable 5.1 back in the day. He'd be a bit of a beast. But also imagine if his enemies also had access to this model. That's what we're going to uncover in today's episode of In The Loop, brought to you by Qualcomm. So the story starts off two weeks ago, September 30th, when an OpenAI employee discovered a device plugged into the back of a television in an executive conference room.

And it was wrapped in toilet paper for God knows what reason. But upon recovering this device, it was handed over to the security team at OpenAI. They started looking at it and they were thinking, hmm, this looks like a USB stick, but it has this weird circuitry that's kind of attached to it. So it seems like a bit of a hack job. What on earth is this device?

And so they started to examine it and they asked their own internal AI model to uncover what this device might be used to. And they discovered three particular things. One, it had very heavy cryptography designed around its entire thing. So accessing the information that this device had was near impossible.

Number two, the device was designed to be continuously recording audio, that it would also be triggered to start recording audio when someone would start speaking. So it seems very intentional that this device was to try and spy on conversations that were happening in that particular room. And then number three, any kind of data that got extracted from this device that could be transferred to, you know, your computer to view it or review it or listen to it was under heavy encryption as well, which meant that the data transfer layer had been very meticulously designed.

And so OpenAI kind of like looked at this and thought, this seems like an attempt to spy on us. So we should probably hand this case over to the FBI, give them access to this internal model or work with them at least to try and figure out what the hell is going on. So opening i continued their internal investigations whilst they gave it to the fbi and very soon using their own internal model of i don't know gpt 6.5 gpt 7 call it whatever that latest model is the one that solved all the math stuff definitely go check that episode i made that very same model named someone.

Traced back the entire circuitry design of this device and who was most likely to have planted the device and uncovered a name. So Open Air had an idea of who it might have been, but let the FBI continue on their investigation. But while they were doing this, they also decided to plant a camera in almost the exact same position where the USB stick was.

So in the event that someone who planted it came back to look at it, they would be able to capture the suspect red-handed. And that's exactly what happened a few days later when the suspect returned had a look behind the tv. And the camera captured exactly who the culprit was and on the same day that this culprit was captured on this camera security team discovered a bag left behind by the culprit by accident in the printing room they went through it and they discovered a second device that looked very familiar to the one that they found behind the tv so they just took the bag put it in lost and found but they took the device.

They were trying to capture this individual wet-handed. They also wanted to figure out what on earth was going on, what this individual might be using this information for. So they, again, were keeping hush and quiet. And they handed over the device and the information that they had to the FBI. Now, this individual discovered that his bag had gone missing and went to Lostom Property to try and find it. And he found the bag. So he felt pretty good about it, except that he noticed that the second device was missing.

And he also noticed that the device behind the TV when he went to go check it was also missing. So he reported that there had been a theft of his own property. But by that time, the FBI and the OpenAI team was already on him and issued an arrest or warrant for his arrest the exact next day. Upon arresting this guy, they discovered that the suspect, his wife, and another family member of the suspect had a one-way ticket to Saudi.

So he was going to fly out with his family and likely never return because. Presumably he thought he'd been busted so the guy got arrested and he has been internally investigated and being put through court or whatever all that stuff is by the fbi and cash patel who's the leader of the fbi and his team has been handling it since then but open ai has come up with an official statement kind of giving a high level idea of what's been going on but never named any names who has been naming names is pretty much every other media publication out there which gets to the next section, who the hell did this and why did they do this?

The man's name is Abdulrahman Al-Fozan. He's a 35-year-old Saudi national who's lived in the US since 2020. But he's also worked at OpenAI for over a year on the product team. So he's like a product slash engineer. He's worked on multiple product releases at OpenAI, likely a core employee, been there for a while as the valuations have increased, so likely his equity is worth quite a bit.

This is a person that people actually know at the company. If you look at his LinkedIn history, he's worked on things like AI agents, which is important for the next bit of this story. What was he doing? How did he create this device? And why was he trying to use this device in that secret room? Let me take you back six weeks prior to this device being discovered.

This individual, Abdul Rahman, queried an internal personal agent at OpenAI. Guess what he asked that agent? Hey, where do high-powered execs at OpenAI? Have meetings? Also, business partners. Where do these execs have meetings with those business partners? Can you tell me the exact rooms? This guy was literally asking an internal version of ChatGPT, which the security team and any OpenAI employee or anyone with a brain cell could get access to and see who was querying it.

That's the device that he, or rather, that's the agent that he used to try and target and locate an executive room. So he finds this room. He finds the perfect room. It seats six people and more. They only have discussions about proprietary research, designs, product releases, things that you don't want getting out to the public.

But there's a problem. He can't book the room. He can't get access to the room. So that's only bookable by execs and their secretaries. So he tries to then devise a plan as to how he can be a listen-in on the conversations there. And he comes up with a bright idea. I need to create an audio device that can spy and listen in to the conversations that are happening in there, but I don't know how to build that device.

You know what might be useful? Let me ask AI. So he goes to his wife, who has a chatbot account, and he goes on to ask it a series of very specific prompts, which is, I want to create an audio device that continuously listens to the surrounding conversation around it. But also when it stores this data, I want it to be encrypted such that only I have access to this data.

And in the event that someone gets a handle on this device and is able to extract the data, I still want it to encrypt the data so that no one can get access to this thing except me. But I also need to design this device so that it plugs into a television in a room. But I don't want the television to detect the device because that would give it away.

So could you create the device such that it only appears to the TV as a power device, which means that it's charging the device, but it doesn't know what the device is being used for. So a series of back and forth prompts over the next like four weeks from that initial six week kind of like starter, ask my agent, which rooms should I plant this device in?

He starts to plan this device and presumably he ends up building it. This is the thing that is wrapped around toilet paper. Investigators looked into what it took to make this device. It cost all but $24. So a $24 device cost this guy presumably a large chunk of his life. I looked into the prison sentence. It's like up to five years for the particular case. But before I get into that.

The idea here is a $24 device basically caused this guy and the entire company and this entire kind of spy mission, a whole world of pain. Okay, so then we get to September 28th, two days before the device is discovered. Abdul Rahman enters the OpenAI Office HQ building. He walks up to his desk. He chills. He's working.

He's going about his day, seemingly. Keeps glancing over to the executive conference room, where he keeps wandering around, scouting the area. He enters the building at 2 18 p.m and he's there till about 6 30 p.m so whatever a regular working day in his eyes but he's caught on cctv because of course investigators are now onto him and they can see him in hindsight scouting the entire area where he then pops into the room when presumably someone exits and he's able to hold the door open this is something straight out of a spy movie this is ridiculous also why the toilet paper i don't understand okay so then fast forward two days Zeta September 30th and it's the story I just told you.

Someone discovers the device at the back of the television, presumably because the guy was caught on CCTV being shady around the office or the conference room in the first place. Everything comes down around him. He's caught, he's arrested, and they're currently working on what his sentence might be for his particular perjury.

Now, before I get into what the ramifications of this is, I think it's important to spend a short moment how OpenAI's internal model discovered the guy because it got access to the device and it basically reverse engineered the entire thing. And it was able to uncover two things. Number one, how it was particularly designed and in accordance with the CCTV footage, which OpenAI also fed it.

They could then figure out who the person likely was. And then when they had a suspect, which was this Abdul Rahman guy, they started to look at his activity secretly without warning him. And they discovered his conversation with the internal personal agent that he had six weeks prior, basically asking.

Hey, where do all the execs have super important information and high priority product release information so that I can plant a listening device? I'm not a spy, but Spy 101, one. Don't talk to the tools that your companies own if you're trying to pull off this kind of a heist. It's crazy. But then they took it an extra step further.

And this is the part where I'm still trying to figure out how they did it. But they were able to discover that he was having a conversation on another personal chatbot. And he then used this to prompt engineer what that $24 device would look like. He built it himself or he got someone to build it for him.

And then it led to him planting it in the back of the room. So the guy used AI to plan and build and execute the entire thing. And then AI that he used ended up becoming the exact point of reason for his downfall, discovery and arrest. It's just this insane loop where you could have used the AI to do the bad thing, but you can use AI to do the good thing as well.

So it's kind of like, it's not a wholesome story, but it does give me more confidence when, you know, we talk about fearmongering around AI cyber attacks and AI is going to wreak a lot of havoc, AI can be used for a lot of good as well. So it's just this insane story. And I have a lot of takes on this.

So let's get into them on today's episode of In The Loop brought to you by Qualcomm. But the analogy that I would use here is he effectively walked into the building and what would be the analogy here? He asked the office receptionist where the most secretive conversations were happening amongst execs, found out the information from this receptionist, and then went to a colleague and asked them, hey, how do I build a listing device that I could plant in this room?

He basically gave the cards over to people or employees or agents of the building. And he's honestly stupid to have not thought that people wouldn't investigate the tools that he would use, that he wouldn't leave a digital trace. And I guess that's like one of the larger points about this whole thing.

Putting the politics and the crazy espionage theories aside, everyone's digital footprint when they use AI has become massively enlarged. Remember, when you're talking to your chatbot, it may seem like it's quite a personal thing, but all your data is being tracked by OpenAI Anthropic or whoever the model lab is, unless you're using open models hosted at home on your own.

That's not necessarily a bad thing. It's a good thing in this case that they were able to trace it back. If this was an open model, probably not. The second thing is around the sentence. So I think right now, rumors or reports are saying at least that they have him on attempt to overhear or record something.

I forget the term that they were I think it's like oral conversations, something very generic like that. Nothing that would indicate espionage, spying, or a breach of national security, which would presumably result in a much larger sentence. But right now, the max sentence that he can have for recording of oral conversations is five years, which seems honestly surprising to me.

But he should get the charge for maybe like spy stuff. But again, details are unraveling. FBI is on it. That Kash Patel, head of the FBI, is fully committing resources to try and figure out what the hell this is. Which brings me to my third take. AI has officially become. An issue of national security. Now, I've said this many times on previous episodes on this channel, and it's crazy to think that this is happening now, and there's real-life examples of this happening.

I think of a conversation that Jacob Coxon, who is the crazy safety and alignment researcher, remember the one that we spoke about on an episode previously where he left OpenAI, joined Anthropic, then left after three weeks because he did not have any faith that either the company were taking model alignment seriously.

He went on a podcast last week and he said, there's this clip of him that went viral. He said that there is a 100% or sorry, 90% chance that there are spies at the frontier labs. What are the odds of him having that conversation, going live, and then literally two weeks later, opening out uncovers this spy, Abdul Rahman, who had been planted there for presumably a year.

It's becoming all too real, if that makes sense. And it's one thing kind of talking about it to you guys, talking about it on a camera, hypothesizing what might happen. It's another thing, it actually happening. And then it's a third thing, us being aware of it, it being all revealed to us. But can you imagine how many other attempts are currently underway?

If you're Anthropic and you're seeing this news, if you're Google, if you're SpaceX SI or whatever Frontier Lab. You're going to be now on high alert that someone is trying to access or listen to your conversations. Now, with all intents and purposes, this could have been a really bad situation for OpenAI.

This guy could have attempted to do something even worse, not listen to conversations, but try and exfiltrate model weights or model blueprints that they could then use to build a competitor model or use it for another model for more nefarious purposes at the state level or at the company or entity or even the individual level.

Like there's myriad different ways that you could use this thing. Thankfully, OpenAI's investigation has revealed that there's likely not a chance that he's been able to pull this off. And personally, if the guy's wrapping his device in toilet paper. It doesn't sound as sophisticated. But you know, you can imagine that they are high level organizations probably trying to attempt this right now, maybe even still at OpenAI or at Anthropic or any of these other labs.

The fourth take is AI is officially a national security threat. And how we protect those models, how we keep them secure, needs to become a national priority. That's what Cash Patella said. But right now, it's under lock and key for small, well, small private companies such as OpenAI and Anthropic who employ their own security team and who kind of take matters into their own hands.

And they're not necessarily a public company. They don't have public investor interests. They're only beholden to themselves and their current shareholders. So I can only imagine that the government wants to get involved, they are probably placed on high alert now. And Kash Patel's probably thinking, I wonder how I can plant an FBI agent or several FBI agents just in the day-to-day nine to five at OpenAI's office.

I'm almost certain that they're having this conversation. And then it kind of got me thinking about Dario's suggestion of pacing the frontier, about a month ago, where he suggests that we want external parties to get involved and kind of evaluate and monitor the alignment side of AI. I'm noticing a trend, essentially, I guess is what I'm trying to say, where external parties need to start getting involved or will likely start getting involved with these private companies.

Traditionally, it's been at the whim or direction of Sam Altman and Dario Murray, but now they may not have a choice. And I wouldn't be surprised if in the next few weeks, Trump, the administration or whoever announces that there'll be some form of light nationalization of these companies. Because as these models get more powerful. And Anthropic and OpenAI are both rumored to be housing much, much more powerful models than we currently have access to internally, which they can't release because.

The ramifications of that could be quite dangerous. As a government, as a nation state, if that got into the wrong hands of your adversaries, such as China, whoever that might be, you could end up having a massive problem just from a pure security standpoint. So the labs are national security sites now.

And then the other thing I was thinking about is Apple. Apple is currently suing the hell out of OpenAI, accusing them of stealing or reaching out to hundreds of Apple employees trying to exfiltrate their own blueprints and, device architecture. That's it for the AI hardware device. Seems like OpenAI has got their own problems. And if the sophistication levels of their own attacks that they're dealing with is a toilet paper wrap device plugged into a television, it seems highly improbable that OpenAI has a sophistication to be able to pull off a similar attack on Apple, although I hope they're not doing that.

So that's really the episode that is an insane story that i read late last night i woke up this morning i thought i have to make an episode about this if you're wondering where i still am i am still in kyoto japan and yes it is amazing yes if you ever come here get a kimono but i still need to keep putting these episodes out guys it's it's fun and it sounds like all the crazy stuff is happening whilst i'm on this crazy time zone right now i am, currently 13 hours ahead of the world or New York, as I call it, and.

News breaks overnight as I'm asleep. So thank you so much for listening to this episode. I hope you enjoyed it. I hope there was some valuable and useful takes. I would say if you had to leave this episode with one top takeaway, it's that objectively, the DNA or surface area that you have online on digital, surfaces, whether it's your phone, whether it's your computer, whether it's any kind of agent or software package that you use.

Is now being tracked, especially if you use ChatGPT or Claw. And that's not necessarily a bad thing, as we've seen in this instance. And so I'm happy to finally talk about a crazy, dangerous thing that actually had a really good ending. Because at the end of the day, I do think optimistically that when it comes to cyber defense or cybersecurity, using AI can actually end up putting us in a much better world than we exist in today.

And all the doom it takes about AI agents running rogue will definitely happen, but we'll have a way to figure it out. So if you enjoyed this episode or if you disagree with any of my takes, let me know. Leave us a comment. Please share these episodes with a friend, whether you listen to it on Spotify, Apple, or wherever you listen to it. We are anywhere and everywhere.

Give us a rating. Leave us a comment. I read all of them. I try and reply to all of them. Give me feedback. I know this is a solo show now, but we're trying to figure it out, me and producer Luke. So if you have any feedback, it's been great. i've been you know really enjoying the feedback that i've received so far and also follow me on x i kind of post a lot of my thoughts even when i'm you know in japan with my family, um and so you know you can hear more news there more current up to date and some of my crazy takes i guess you can also dm me there as well, but yeah thank you so much for listening and i will see you on the next episode of in the loop brought to you by qualcomm thank you.

ChatGPT Just Caught a Spy Stealing From OpenAI

Kapitelklicken springt direkt hin

Shownotes

Transkriptvom Podcast bereitgestellt · Zeitmarken springen hin