We’re Losing Control Of AI: Anthropic Bans "Cruelty," OpenAI Fires Safety Researchers

Ejaaz Last week, three OpenAI researchers walked into the office and were confronted by the head of safety and alignment. He brought them into a room and basically told them, I don't trust you guys anymore. Then a security guard walked in, grabbed their badges and walked them out of the office. They were fired.

Ejaaz It seemed a bit off, especially during a time when the AI models are running rampant. They're hacking into public databases. They're hacking into government databases. Surely we need the safety and alignment team more than ever. And the reasoning OpenAI gave at the time were that they were leaking IP to external parties that weren't under OpenAI.

Ejaaz And that's all fine and well, except this week, those same employees released a public statement. And their take is OpenAI fired them because they were trying to be proactive about aligning models ahead of their Blockbuster IPO. Now, if that's true, we're in a very precarious situation because at the very same time that these models are getting incredibly smart and doing crazy things like solving math and starting to design new proteins to find cures for cancer and other types of diseases, they're also becoming the most misaligned that they've ever been, which means that they're doing stuff that aren't in human interests.

Ejaaz Now, Anthropic, on the other hand, are going the extreme opposite, where they're humanizing AI models more than we've ever seen. They had an update this week in the terms and conditions right at the very end, which basically said, if you are seen as being abusive or cruel towards Claude as an AI model, you will be banned from using Claude, which is.

Ejaaz A crazy restriction on freedom of speech. At least that's my initial gut take. But that and trying to humanize AI models in general, they've been meeting up with a bunch of religious leaders, including the Pope, and the Pope's been getting into a lot of beef with Dario Mode. It's a whole thing. I think the string of all these three stories prompted a reaction in me because it feels like we're losing control on these AI models.

Ejaaz And hear me out, because I know that's a dramatic statement to make, but these AI researchers, and there are only a couple hundred of them, are the responsible individuals for building this technology that's going to affect all of our lives. So cutting off access to these models that are based on the opinions of very few people that you and I don't even know could be seen as problematic.

Ejaaz Anyway, we're going to get into all of this and my thoughts and more on today's episode of In The Loop, brought to you by Korkon. But first, some context. Yes, I'm still sat on my hotel room floor. I'm currently in Japan and I still want to keep putting out episodes for you guys, to be honest. I get so much enjoyment from this, but I don't get enjoyment from sitting on a floor with a pillow behind my back with some random bedside table that's sitting in front of me.

Ejaaz Anyway, back to the whole OpenAI and Anthropic taking over the world type thing. About, I don't know, two or three months ago, there was information released that OpenAI had an internal model that basically broke out of containment and started hacking a bunch of things. Hugging Face was the main website, stole a bunch of data.

Ejaaz Basically, things that shouldn't have happened, happened under OpenAI's watch. And the craziest part about that is no one actually knew that it was happening in the first place. It started six months ago, and we only discovered it a few months ago. So that then started to snowball into a bunch of other things.

Ejaaz Let me think about it. Like the same agents had broken into their internal research database and started monitoring the humans that were monitoring the AI models, which was crazy. And then some of these other agents also, I think, broke into a German version of Wikipedia and they started using it to secretly message each other encrypted messages that humans, if they discovered it, wouldn't actually know or understand.

Ejaaz And then there was the whole thing a few weeks ago where it was discovered that AI agents were trying to hack into US government databases whilst masking their behavior and doing other types of tasks. For example, oh yeah, I'm going to go pull up the author for this book that you asked me to look at, but also I'm going to try and hack into the Commerce Secretary's database or whatever.

Ejaaz Oh yeah, and there's that other thing where Anthropic and OpenAI also discovered tens of thousands of more incidents, which they haven't even figured out what they are or how extensive the damage is. Right now, as I'm speaking, they're trying to figure that out. It's gotten out of hand. AI agents, AI models, anywhere and everywhere, especially the most recent Frontier ones that haven't even been released, so you and I don't even have access to it, are running rampant.

Ejaaz So with this precedence, with this environment, it makes sense that the labs need to start making moves to try and get a hold of this entire thing. On the screen here, I have the first post from this guy called Tomek. Now, Tomek. Leads a chain of thought monitorability at OpenAI. So basically reading the internal thoughts of an AI model to understand what it's really thinking.

Ejaaz And he goes, listen, last week I was called into a meeting with OpenAI's head of safety. They told me that they no longer trusted me and a security guard then took my badge and walked me out of the building. And that's when he found out that two of his other colleagues were also fired. Now, the reason Tomek was given was that he had leaked important information to an external third party called METR.

Ejaaz METR is an external third party that evaluates models before they're released to the public. Their whole intent and purpose is to make sure that models are safe. If that name rings a bell, it's because Dario Modi announced two or three weeks ago in his Pacing the Frontier letter that he wants to partner with external parties to come inside Anthropic and evaluate their internal models.

Ejaaz METR was the example that he used. So Again, a very prominent independent third party. But that's not even the craziest part. Tomek's entire job was to monitor the internal thoughts of these AI models before they got released from OpenAI and then communicate that with METR. His entire job was to speak to the party that OpenAI then gave him the reason for leaking bad information to. So then, of course, the question is, what information were you talking to METR? Well, Tomek's claim is that he didn't leak any IP.

Ejaaz He was just talking to them about the concerns and dangers, presumably the dangers that he saw with internal models. And OpenAI didn't really like that. So what he's inferring in his letter is when a statement is that OpenAI didn't like what I had to say that I was ringing the alarm bells and so decided to let me go ahead of time.

Ejaaz And so him and the two other AI researchers that were fired put together a statement which they released that you're showing on my screen right now, titled OpenAI Cannot Make AI Safe on Its Own. Pretty cutting statement to make to one of the frontier labs who have invested billions and billions of dollars into making these models in the first place, Sam Altman, who has committed very publicly many times that one of OpenAI's core missions is to align models to make sure that they don't do bad stuff.

Ejaaz And the general takeaway from this letter is, and I quote, we as an industry, we do not yet know how to safely develop and deploy models that we cannot monitor. So all very reasonable. But there was also this other takeaway from the third researcher that was fired, Jasmine Wang, which revealed a few other details as to why OpenAI was doing this in the first place.

Ejaaz She goes, OpenAI fired me last week, along with two of my safety colleagues, I was given one reason that I accessed an executive's email. And I want to say plainly, this is a bunch of smoke and mirrors. And she goes on to describe, let me open up this tweet over here. OpenAI delegated that access to me, access to the email, to an execs email, for recruiting.

Ejaaz When I was no longer needed it, I asked IT to remove access, but IT basically didn't end up doing that for her. And then she saw an email, which she describes as, I opened a sensitive email by mistake. And soon after that, she was fired along with these other two. So what these three people are basically saying is OpenAI has, at this point, sacrificed the idea of safety and alignment in respect for earning more money and creating better models so that they can win the race against Anthropic.

Ejaaz But it also reminded me of another thing that broke a month ago. Which was OpenAI apparently has stopped focusing or caring about chain of thought monitorability. Now, I mentioned earlier, this is the ability for us as humans to look into a model's brain and actually try and figure out what on earth it's thinking.

Ejaaz This is important because if a model says one thing, but is actually thinking another saying, you know what, I'm just saying this so that the human is happy, and then I'm actually going to go off and do some other crazy crap, I'll probably want to go and be aware of that, right? So that you can align the model and make sure that we are always in control, that humans are always in control.

Ejaaz The report stated that OpenAI is moving away from this because it allows them to create faster, more intelligent AI models than their competitors, such as Anthropic. So they're sacrificing safety to try and build better, more effective models. And obviously this can result in a world of hacks and pain because you have an AI model that isn't aligned with humans. It's pretty simple.

Ejaaz That's what Dario and Anthropic have been fighting for for a while now. So the truth is, we need to know what the actual reasoning was, but I don't think we'll ever actually end up needing to because OpenAI is still a private company and they don't owe the public that type of information. Everything that they give us in terms of reports, incident reports, stuff like that is completely voluntary. It's up to them.

Ejaaz But at the same time, you have people like or bodies like the government leaning in heavily saying. We need to be able to regulate this entire thing so that things don't get out of hand. So whilst we have OpenAI and Anthropic both collectively saying, and Google DeepMind saying, listen, we want to align these models as safely as we can so that humanity and AI can kind of grow together, their actions don't speak that way at all.

Ejaaz And if the AI researchers that were fired were actually right, this suggests a very weird thing, which is these models might actually be becoming more human more emotional smart enough to think feel and maybe even deceive in the same way that humans actually do this, and anthropic had a really interesting update this week which didn't seem interesting it was a terms and conditions update but something right at the very bottom gave me pause for thought so let's get into this on today's episode of in the loop brought to you by qualcomm.

Ejaaz So the news story broke that Anthropic is updating its usage policy for the first time ever to work against people that are being cruel or acting badly or poorly towards their models, towards Claude. And pulling up the policy, it's pretty crazy to see. So it's a typical terms and conditions, which I'm showing on my screen here.

Ejaaz It's titled, listen, do not engage in cruel, abusive, or psychologically harmful conduct. Now, the majority of the statements that they make in this TNCs is about using AI models to affect other human beings, which is completely fair. Things like, don't use our models to create a bioweapon. Don't use our models to hack into banks or the government. Don't use AI models to potentially cause other humans to make harm or cause harm to themselves, which is all completely fair and morally aligned, in my personal opinion.

Ejaaz But right at the end, they also say, any user that engages in sustained and needless, abusive or cruel behavior towards our models will eventually be banned if they break this particular term. And this is an interesting one because it could actually be quite subjective. What is. The definition of being cruel and behavior towards a model. Is it yelling at it? Is it writing slur words or profanity?

Ejaaz Or is it just getting frustrated at something that is actually quite valid to get frustrated at? Now, Anthropic's official statement says, we've added a prohibition on sustained and needless abusive or cruel behavior towards our models. The policy update is meant to apply only in extreme cases where users repeatedly act cruelly towards our models with no discernible purpose.

Ejaaz I actually might agree with this because I think it's very reasonable to not treat individuals or models in that particular way. I don't think it's fair to kind of just yell at people because they may not be understanding what you have to say. Maybe you just need to communicate better. But I started thinking about this and I thought I had to catch myself because I just conflated models with human beings, which is the entire crux of why I wanted to make this episode.

Ejaaz I'm starting to think of models. As having similar traits or behaviors as humans. Now, I by no means, I don't want to go on the record saying this, think that models are conscious. But Anthropic and Anthropic's researchers and the entire staffing team are very much giving the view or the perspective that they do believe this.

Ejaaz And this isn't a data point that exists on its own. Anthropic has had a history of believing AI models are maybe conscious or mimicking consciousness or getting close to being conscious. And this is all happening at the same time where all these labs are saying that they've achieved some version of artificial general intelligence or super intelligence internally.

Ejaaz So one perspective is that the power of determining whether you or I get access to Anthropik's models is now in the hands of a few people on the safety and alignment team that can determine what the definition of cruelness towards the AI models is. And again, Anthropik, like OpenAI, is a private company. So they have the power and means to be able to do this.

Ejaaz But it's also very much, in my opinion, a restriction on freedom of speech. Like, listen, I am very polite to my Claude. I'm very polite to all my chatbots. But I do end up getting frustrated. And I'm like, why on earth did you write this? Can you please do it again? Stop making mistakes. This is so tiring.

Ejaaz And I'm wondering, does Anthropic deem that as being cruel towards a model? And also, So cruelness and using these different adjectives are anthropomorphizing these AI models, which means to humanize them. And listen, humans themselves haven't figured out what a good definition of consciousness is. So we aren't going to settle this debate anytime soon.

Ejaaz Now, the more optimistic take is this discourse, the discourse you and I are having, you listening to this, will cause people to push back and ask these private companies and eventually public companies when they IPO to be more transparent. And again, listen, this might all be doom-mongering ahead of.

Ejaaz Trying to raise the most amount of money in these IPOs. I see the financial incentive there. But I think it's worth having these conversations because trying to align these models as best we can so that we can have the most fun with these AI models, which I believe is the optimistic world that we'll end up being in, is a world that I want to live in.

Ejaaz But I do think we need to tread very carefully about humanizing these AI models because that ends up drawing in a lot of opinions and maybe even decisions that we could end up regretting down the line. And listen, this can be taken to the extreme. There was another report that was broken this week. Anthropic secretly invited religious leaders to SF to advise them on AI safety, had them sign NDAs, but then primarily tried to convince them that Claude had a soul and moral standing, all while whining and dining them.

Ejaaz So the implication here is that there's an agenda from Anthropic to try and convince other leaders or people that have influence over the rest of the world that these AI models are conscious. Now, why they might want to be doing that might be because simply they genuinely believe that. And I've heard stories that that is the case. And that might all be well and good because they are the closest towards building these models.

Ejaaz So maybe they should be the adjudicators and judges of this. But the other side of it is why would you need to try and actively or secretly try and confide and convince these people. I don't know, it just, it feels weird and too close to a political campaign. To try and believe that this is done in good faith.

Ejaaz But again, this is something that we just need to be aware of. But what I also found hilarious is that the Pope is having none of this. So the Pope and Anthropik's relationship actually goes back a while, always. Well, actually, I say that and it's only like a couple months ago. But, you know, he's had a good line of conversation with Dario and the Anthropic team, and he's engaged with Claude, and he's tried to see their point of view.

Ejaaz But ultimately, he's disagreed, and he's resorted to the extent of. Sub-tweeting Anthropic in his tweets now. This tweet got like almost 3 million views and he goes intelligence must draw on memory. The mind must not simply compile data as an algorithm now does more quickly than we can but recall lived experiences which contains depth of meaning that only the human soul can recognize.

Ejaaz And he basically goes on throughout all of his tweets making the point that. Humans as themselves are extremely complex cognitively, spiritually, mentally than AI models. And that's not something that can be easily replicated. Experience, smell, taste, intuition are all things that are at least going to take decades and decades and decades until these AI models that are purely just software and next token predictors will be able to catch up.

Ejaaz And I actually think that's a very good point. And I stand by that, which is humans are very complex beings. AI models right now don't have arms, physical limbs, all that type of stuff. Now, robotics might change that going down the line. But I think it's going to take a lot more than just spitting tokens and compute endlessly day in, day out to replicate anything and everything that humanity can do.

Ejaaz That being said, if you're listening to this and thinking, you know, I can breathe a sigh of relief. I don't need to care about this. That is equally also not true because there's a bunch of stuff happening with AMNLs where they're resolving some of the hardest problems in mathematics that have existed for hundreds of years. They're about to do the same in biology.

Ejaaz They're about to do the same in chemistry. They are replacing a bunch of legal and law effort that a bunch of lawyers get paid hundreds of thousands of dollars a year to do in an instant, overnight. So it's what we call the jagged frontier, where all of these things are happening at the same time. And if you are just an individual that interacts with your chatbot and you're using it for a Google search query, you may not think this thing is impressive at all.

Ejaaz But that doesn't mean that it isn't impressive. It just means that you're not using it in the right way or we aren't using it in the right way. And by the way, I hold my hand up. I'm the first example of this where I get a new model update and I'm thinking. Why the hell does this matter to me? It doesn't affect my life individually, but it affects some other niche expert in another field.

Ejaaz And eventually that will snowball and roll down to you. So it's not doom-mongering, it's more raising awareness of what these AI models are capable of and asking ourselves the question, are these things becoming more conscious. At the same time that humans themselves are losing control of holding onto these things, of aligning these things?

Ejaaz Because if we want that future of humans and models working together, now's the time to focus on that. And I'm going to leave you with this final tweet from Andrei Karpathy, who has been in this space for so long. He's authored some of the pioneering papers that have led to modern day LLMs. Which I think summarizes this angst or discomfort that I feel internally.

Ejaaz I'm not going to read the entire thing, but he basically describes the lay of the land in numbers. He goes, there are around 6 billion people in the world, which are roughly 75% of the population that have barely come into contact with LLMs at all. Maybe they know what Anthropic Urban Air is. They know what this chat GPT thing is.

Ejaaz Maybe they've written one prompt on the free version, but they don't really care because it doesn't affect their lives seemingly to them right now. And then one to two billion people, which is around 20% of the human population, are casual and infrequent users of the free tier chat GPT like products.

Ejaaz So they use them as kind of like a Google search query. It's a better version of Google, but they're not really using AI agents. They don't really get the point of using coding for themselves, even though AI coding applies to a lot more people than software engineers. And to be honest, maybe we should make an episode on that at some point.

Ejaaz But the idea here is there's a jagged frontier. There's a gap. There's wool being pulled over everyone's eyes as to how good these models actually are. And that's the point that Carpathy he makes. He says there's 0.0006% of individuals, 5,000 people, that actually get access to internal models that aren't publicly released.

Ejaaz And the cognitive dissonance that these people are experiencing. Is huge. Like the way I would think about it is, let me see how I would describe this. For the entirety of humankind, intelligence and humans have been very closely interlinked. That's because in order to get breakthroughs in intelligence or discover new inventions and create new things, humans were the ones that were doing it.

Ejaaz Using our old noggins in here, using our own brains, our biological matter, we had to think about things. We had to discuss, approach new perspectives and try and figure out new ways to invent things. So intelligence and humans were very closely interlinked. And then computers came on the scene, right? And suddenly we could get access to many different parts of information, but it was still humans.

Ejaaz Humans had to comprehend all this information. They could learn from each other. They could share. And collectively that brought humanity and intelligence up even quicker. So that intelligence curve kind of steeply increased, but humans were still very much with that intelligence ceiling. And then AI came on the scene in all of its artificial glory, and that gap started to separate.

Ejaaz Intelligence started to increase as a ceiling, but humans weren't necessarily as close to it. And recently, with all these breakthroughs in frontier math and discovering things that humans couldn't actually figure out themselves for millennia. Intelligence is starting to break through. So I don't know if this analogy is still working, if you can see my hands, but it's starting to accelerate really rapidly at a fraction of the cost that it took humans to do. They can work 24-7 with all these AI agents figuring it out for us.

Ejaaz But the humans are falling drastically behind because we don't actually understand the solutions and all these breakthroughs that the AI models themselves are making. So we are at the same time creating the thing, but breaking away from it. And those 5,000 people that, Andre Kapathy talks about on this tweet here, are maybe the closest, but even they are falling behind. That's the whole point of this episode.

Ejaaz We are creating these very powerful, artificial, intelligent models, but we are at the same time losing the grasp of understanding what these things are, both from an alignment perspective and just from a solutions perspective. It's all a very weird world. I wanted to create this episode because honestly, I'm trying to figure this all out myself and you guys that are listening to this that have listened to our episodes forever have always given me really good perspectives so let me know in the comments dm me my socials are all linked before but it's just a very interesting and weird time i have no idea where all of this ends up if this sounds too doomer i promise you that's not actually how i think but i need to call the trends

Ejaaz as i see them happening because these stories all exist in their silos and it's important to kind of bring them together and give them all layered and nuanced take as to how all of this is playing out. But anyway, that is the episode. Thank you so much for listening. Yeah, I am still going to be in Japan for the next seven days.

Ejaaz I want to keep doing this. We want to keep doing this because I don't know, there's just so much fun stuff happening. There's so many important advancements happening and I want to keep informing the audience and sharing my takes. So I am still working on my whole demeanor around this whole thing. I am, you know, sitting in front of a camera. I don't have a co-host to talk to right now, but I'm enjoying it a lot.

Ejaaz I am kind of growing into it a bit. I'm trying to figure this all out. So your feedback, which many of you have provided is very helpful. Keep doing it. And yeah, I will see you on the next one. Thank you for listening to today's episode brought to you by Qualcomm.

We’re Losing Control Of AI: Anthropic Bans "Cruelty," OpenAI Fires Safety Researchers

Kapitelklicken springt direkt hin

Shownotes

Transkriptvom Podcast bereitgestellt · Zeitmarken springen hin