Trump's new AI czar + Nvidia exec caught smuggling chips to China

0:09 Welcome back to the AI Power Podcast. I'm Adam Goodwin here with Greg Allen. Adam, great to have you back. How are you doing? Doing well, doing well. It's great to be back, Greg. It's been a couple of weeks, but looking forward to jumping in here. Yeah, so what do we got on tap? So a week after the White House AI launch, the president named an AI czar and launched a superintelligence force.

0:30 Then, who's liable for AI incidents? A Chinese open source agent used to hack South Korea's biggest banks, NVIDIA chips still reaching China in the DC Circuit Court's ruling against Anthropic, how it differs from its California win, and what it means for AI government contractors. First, last week's follow-up.

0:49 So last week we discussed the AI luncheon hosted by the White House, where President Trump promised a new AI czar. It appears one has been named and put in charge of a new White House office, the Superintelligence Force. Greg, who's running it and what is this task force supposed to do?

1:07 So the person who got the job as AI czar, this is the second AI czar of the second Trump administration, is Jay Clayton. And folks may remember seeing him in the news in the not too recent past because he was in just August named as the director of national intelligence. So the sort of senior person overseeing the United States intelligence community and overseeing the president's daily briefing on intelligence.

1:36 Before he was DI, he was the U.S. attorney for the Southern District of New York. And that is the same position that Rudy Giuliani held. That is the same position that a number of very influential legal minds in the United States have had. That is the court that oversees the U.S. financial industry. So it's a very important district to be U.S.

1:58 attorney for. Jay Clayton has historically been a skeptic when it comes to China on economic security issues, on national security issues. And he has also been, I think it's fair to say, pretty warm to some of the arguments that the Trump administration has made around AI. So it's unclear just exactly where he's going to land.

2:24 But so far, unsurprisingly, because he's just got the job, all of the positions that he's expressing are in line with what this administration is saying at once. Namely, we're going to focus on the existing authorities. Now, one thing I should say is that this AI force, which again, given the Trump administration's name change, is going to be called the Superintelligence Force.

2:48 Well, there's been a radical rescoping because previously President Trump had compared it to his idea of the Space Force, which would have been a new branch of the United States Armed Services. Now it's much closer to a policy task force. This is a group of folks, some of which have other jobs in the government of significant importance, and they've got 120 days to come up with recommendations to improve the U.S.

3:16 government's posture on AI within existing authorities. Okay, let me say a little bit also about the structure of the superintelligence force. So, in addition to Clayton as the chair, you've got three vice chairs. Those are Emile Michael, who's currently the Under Secretary of War for Research and Engineering.

3:36 He's going to lead coordination with AI companies as part of this superintelligence force. You've got Scott Kapoor, who's the OPM director, and he's going to lead critical infrastructure and other industry stakeholder type engagement. And then you've got Andrew Ferguson, the FTC chairman, and he's going to lead outreach to consumer interests and advocacy groups focused on faith, children, privacy, and freedom of expression.

4:04 So that's a very interesting group of three folks. First off, Emile Michael used to be a senior executive at Uber, so strong Silicon Valley bona fides. Scott Kapoor was a former Andreas and Horowitz A16Z managing partner, so also very strong Silicon Valley Bona Fides. And it's worth pointing out that A16Z has been among the strongest forces politically advocating against AI regulation throughout the Biden administration and the second Trump administration.

4:35 There's other folks who are a part of this superintelligence force, and that includes Vice President Vance, Secretary of War Pete Heggseth, Secretary of the Treasury Scott Bessant, Chief of Staff Susie Wiles, and Deputy Chief of Staff Richard Walters. Interestingly, there are two folks who are currently sitting outside government or partly outside government, and that includes David Sachs, the former AI Zar, and Condoleezza Rice, the former Secretary of State and National Security Advisor in the George W.

5:03 Bush administrations, which was kind of an interesting addition.

5:06 Yeah, not a name you hear very often with the Trump administration.

5:10 Exactly, right? Like she's got a strong Republican resume, but I normally don't associate her with having a strong Trump Republican resume. So that was an interesting choice. So the components and the vice chairs give you sort of a theory of the case. There's also an explicit charter, and I'll read the quote here.

5:27 This was reported in the Wall Street Journal. Quote, develop plans for responding to SI-enabled threats to our society while preventing over-regulation and regulatory capture that would stifle innovation and competition. So I think that very much tells you that's where this government is. Now, another thing that's interesting about this charter is it's going to review how breaches, hacks, and jailbreaks get reported to the government and quote, recommend ways to improve the government's response capacity under existing authorities.

6:02 End quote. So what I think is so interesting about that is they're acknowledging the presence of SI-enabled threats to our society, but they are focused on making sure that whatever the response is, it's going to be handled under existing authorities. And so what I think is kind of contradictory about that is there's a fact-finding element of this mission, right?

6:28 Review how breaches and jailbreaks and hacks get reported to the government. They don't claim to fully understand how it happens now, but they know that whatever is going to be required to fix it is not going to require expanding their own authorities. And I think that is kind of just being fixated on the argument they've been making publicly and being consistent with it, as opposed to being open-minded about what they're going to learn and how it might change their opinion.

7:02 They're sort of putting the conclusion before the analysis, which I think is frankly an error, even if it turns out to be perfectly reasonable. So the focus remains on light touch regulation, and I don't think that's going to change, but there's one thing I want to point out here. What we just described, that charter, that reflects the administration's expectations for the future, their expectations for what they are going to do.

7:32 And they may prove to be wrong in predicting their own future behavior. And the reason why I think this is important to remember is the fact pattern may force their hand. And there's at least two ways that that could happen. The first is we could have additional AI, safety, and security incidents that really cause folks in serious power positions to ask themselves whether or not more is needed.

8:01 Recall that the Trump administration's original negotiating position was let her rip, do effectively nothing on regulation. The new policy position is do something, but only under existing authorities. And that is because of the change that we've seen with mythos, with the OpenAI hugging face incident, with the spate of child safety tragedies that have occurred in relation to AI.

8:29 And so this is my point. They're predicting their future behavior, but the facts on the ground may change their minds in the future the same way it's already changed their mind in the past. And the second thing that could change is the political reality of the situation. So recall that the Trump administration was pushing strongly for a moratorium on state-level AI regulation.

8:55 And then when that failed in Congress, they said, okay, we're going to explore some kind of federal regulation as a means to getting a state-level regulatory moratorium passed. So they recognize when they are blocked politically and they change their policy pronouncements and recommendations and efforts in response to the politics on the ground.

9:21 And the politics on the ground for AI are bad and getting worse. A non-trivial share of the United States population thinks that the extinction risk of AI is high or moderate. And a huge majority of the population hates AI data centers and supports greater AI regulation. So you have folks like Susie Wiles, the chief of staff, for example, who are very sensitive to that political reality.

9:52 And so if the politics continue getting bad, if the midterms turn out poorly for Republicans, I think you could see what the superintelligence force is saying they're up to might not be a good prediction.

10:05 Yeah, and Greg, I would also mention that Jay Clayton himself is an interesting selection for this job because of how strong the bipartisan endorsements were as he was going in to become the director of national intelligence. So he's by no means a bomb thrower. He's by no means somebody that's caught out of center.

10:21 He's certainly somebody that Mark Warner, who's a moderate but a very thoughtful person who's been either the ranking member or the chair of the Senate Intelligence Committee for years, significantly supported. So I think there's a very thoughtful move going forward, at least for the staff at the head of this organization.

10:37 So the administration has spent the years saying that it won't regulate AI and it now has a task force on AI risk. How should we really read this? Is this a turn towards oversight, internal oversight? What does this look like?

10:52 So I think the government is for now still committed to the approach represented in the AI accord that is barely a week old at this stage, which is emphasizing industry self-regulation, potentially combined with FTC style investigations focused on liability type regulation. But one thing that I think is very interesting about Jay Clayton is in addition to the strong endorsement that he got going into his nomination, which was ultimately on a party line vote, but that had to do with stuff separate from his qualifications for the job.

11:30 The other thing about Jay Clayton is that he still has a day job, which is director of national intelligence, which means he's going to be consuming all kinds of intelligence and national security assessments on AI. And Jay Clayton was not in the seat as director of national intelligence when the National Security Agency was freaking out about what Mythos could do in terms of cyberpower.

11:57 But a lot of people who Jay Clayton is going to be seeing all day, every day, were in the weeds of that incident and are going to be consuming intelligence on all natures of AI threats and risks from a national security and intelligence perspective. So I think he's going to be naturally kind of injecting that theory of the case into this debate.

12:18 And I think his selection guarantees that national security, hopefully, I guess I said guarantees, but really I should say hopefully, gets a fair hearing in this story.

12:28 So on our September 11th episode, Greg, you said safety is back, people, with the labs and even the White House comfortable using the word again, which they hadn't been for quite some time. Looking at who is on this force and what the president's allies are doing around it, is that the right read for this transition?

12:47 So I still think yes, safety is back, right? I mean, look at the language in uh the charter, SI-enabled threats to our society. So I think the government admits that there are risks here. There is a problem here. The Trump administration admits that something has to be done because good outcomes are not guaranteed.

13:11 We actually have to work towards those outcomes. But there was some great reporting in the Washington Post over the past week basically saying that even though safety as an idea is increasingly welcome in the administration, there is still contempt for the so-called doomer philosophy around AI safety.

13:35 They don't like hearing about extinction level risk. They don't like hearing about how AI is some massive threat that's as big as nuclear weapons. So, yes, they're not completely ignoring what's going on with AI and cybersecurity. They're not ignoring what could be going on with AI and biosecurity. But I think they really have sort of an immune reaction to these existential framings or these doomerist framings or these sky as fallest framings because they fundamentally have an incredibly optimistic view of what is likely to happen with AI.

14:15 And I think they really do worry about this regulatory capture risk, which I don't think is an accurate interpretation of what's going on in terms of the behavior of these companies, but it's certainly a consistent position they've been arguing for. So one thing that was in the AI accord that I thought was really interesting, sorry, I shouldn't say AI Accord, the superintelligence accord, was that it commits the signatories to partnering with an independent external auditor or evaluator.

14:46 And what's interesting is if you look at what David Sachs said on his all-in podcast, which he also hosts with his colleague Chamath, Sachs said, quote, in my view, they should be professional auditors, not these NGOs. So he basically wants like the big consulting firms to be the one who are auditing AI safety inside the labs, not nonprofits.

15:11 And what's so remarkable is that Chamath, also a venture capitalist and also the co-host of the podcast, he said on the same episode that his startup, 8090, is building EY as an Ernst Young, the auditing and accounting firm, quote, an entire audit infrastructure for superintelligence. Goes on to say, quote, you're going to see EY in short order here announce something, end quote.

15:39 And I just have to call out what appears, at least on its first face, to be jarring hypocrisy here. You've got folks like Chemoff arguing about the risks of regulatory capture when it comes to AI safety. And now he's saying specifically the nature of AI safety regulations should push people towards private sector auditing firms who his portfolio companies are developing technology for, right?

16:15 You want to talk about like regulatory capture, that is certainly a plausible way that one could execute a strategy of regulatory capture. And so I don't like to see it. And what I did like to see was the quote from Adam Gleave of the Frontier Alignment Research. This is an AI safety nonprofit. And he was quoted in the Washington Post saying, quote, when it comes to looking at millions of agent transcripts and trying to piece together what they did and how they were interacting, I don't think there are many organizations in the world that have that expertise.

16:50 End quote. And so this is what I want to point out. Who should be doing the audits, who should be doing the outside evaluations? It should be a choice based on who has the competence, who has the best tools. And right now, what I see in the market, what I see in the world is that that is mostly these NGOs that are being attacked with contempt by folks like David Sachs, who said some very mean things about meter, for example, which was one of the principal authors of the OpenAI Hugging Face Incident report.

17:22 And if EY ends up acquiring that level of competence, great. But the focus needs to be on who has the skill set, not who your companies have profitable relationships with.

17:36 So let's talk a little bit about accountability, Greg. After the OpenAI hugging face incidents this summer, there seems to be a desire to properly establish who is responsible for these agentic breakouts and ascribe some level of blame. The labs have pledged to police themselves and step up efforts after every new incident is reported.

17:55 And now federal regulators and state attorneys general are opening investigations. What are they looking at?

18:02 Yes, and both sides of what you just said are super important, what the state's attorney generals are doing and what the FTC is doing. Now, the FTC probe is pretty remarkable. It covers anthropic, open AI, other labs, plus meter, which both labs have used to investigate their incidents. And FTC is leveraging its existing authority to investigate unfair and deceptive practices, not any AI statute, because like what AI statute would they even point to?

18:34 So that's a perfectly reasonable use of FTC authority. Under the Biden administration, the FTC did similar requests for information from OpenAI looking at user privacy concerns. I think the question here is: is that type of investigation going to be used to create political ammo for accusations of regulatory capture, where they're going to say, aha, C, meter, and OpenAI said they really like each other.

19:04 That must mean they're colluding. That would be a disaster. I seriously hope that doesn't happen. I really hope that the government is just trying to get smart on what's going on under the hood here. So we'll have to just see what the FTC is doing. But it's certainly plausible that what the FTC is engaged in is not going to be the most useful bag of tools for trying to improve AI safety.

19:27 But think about what California is doing. The California's attorney general served OpenAI an investigative subpoena on September 30th. And this is a really broad inquiry into the cybersecurity incidents around OpenAI. And the attorney general, who's Rob Bonta, he said, quote, companies have a moral and legal responsibility to ensure that they do not perpetrate or enable cyber attacks either during model testing and development or once models are placed into service.

19:59 Developers that fail to do so can and should be held legally accountable. And I thought what was very interesting in that quote was that he focused on during model testing, because California's SB53 AI safety law had a carve-out for model deceptive behaviors that occurred during the evaluation designed to elicit it.

20:24 So the hugging face incident was not under the original text of SB53. SB53 was about reporting requirements, transparency requirements. It was a safety transparency bill. But what he's sort of saying is that under other laws around criminal negligence or other items, these companies still have a duty.

20:47 So it's kind of interesting there that he's dwelling on liability law or other tools in the toolbox, the existing tools that the Trump administration has been so willing to focus on. But it seems like he's willing to take it farther. The second thing that I think is going on is that the Iowa Attorney General, Brena Byrd, is leading a 15-state coalition, including Alabama, Arkansas, Texas, and Utah, so solidly Republican states, seeking information from OpenAI on the hugging face incident.

21:18 And of special note, going on in Florida, Florida's attorney general in late September asked the state court to block OpenAI from building new models without outside approved safeguards. And Florida, I think every statewide office, including the attorney general, is held by the Republican Party. This is what I think is important to keep in mind as you're asking yourself, what is the Trump administration going to do?

21:47 Look at what Republicans are doing at the state level. They are trying to find ways to demonstrate that they are being tough on AI, that they are promoting safety. And I think that. Is the political reality that I was talking about. The let it rip posture is not just unpopular nationwide, it's unpopular in the Republican Party.

22:11 And so whatever this superintelligence task force is going to figure out, it has to reconcile its actions and recommendations with that political reality, which at least for right now appears to be getting worse on AI, not better.

22:26 Yeah, this is an issue that has really penetrated, Greg. And I think that the concerns of these attorneys general has actually bypassed the original talking point that we've gotten out of the all-end podcast that if a model's dangerous, just don't offer it for sale. Right. What we're talking about here is danger during testing.

22:42 Yes. And first off, great point. That is something worth separating. Just don't offer it for sale. Like, guys, it happened during the period when I was testing it to see if it was safe for sale. But I think the other question is in an ideal world, right? Our number one preference is not that murderers be punished, right?

23:02 Our number one preference is that murders don't happen. That's what we really want. But given that we have an imperfect ability to prevent all murders, right? Our sort of second-order preference is to ensure that murderers are punished. And when you think about AI type risks, if you are dwelling upon liability law as the primary tool, what you're focusing on is actors whose negligence results in something bad happening.

23:34 You're focusing on let's punish those people. But there's the other side, which is, well, wouldn't it be better if the bad thing didn't happen? Can we find what is the safeguards that we want to put in place to minimize the chance of that happening? And I think that has to be a focus here, right? We don't just want bad actors to be punished.

23:59 We want bad actions to be prevented. So think about what the Florida Attorney General is asking a court to do. It's asking the court to stop OpenAI from releasing new models until and unless those models have outside approved safeguards. And I think the quote from the Attorney General motion is really juicy here and it's worth reading.

24:25 So quote: defendants claim they cannot stop barreling forward with their potentially civilization-ending endeavors unless they are forced to do so by the government. They have asked the government to tie them to the mast. Plaintiff brings good news to the defendants. The Florida Attorney General is answering your cry for help with a motion to enjoin you from harming Floridians with your reckless, unacceptably risky product.

24:54 End quote. And that's why they're looking for regulations that apply generically, not merely this kind of emotion, which applies to a handful of companies specifically. So on the one hand, it's a very juicy quote that sort of reveals the political state of play and how powerful actors are interacting.

25:35 But I don't think it quite addresses the specific rationale being used for the AI companies when they're calling for binding regulation.

25:44 Greg, let's dig in a little bit more on the incidents involving the agents themselves. So these incidents involve agents doing things nobody asked them to do, and that is driving a big debate. If an AI agent hacks somebody on its own, who's legally responsible? Right? How are people thinking about the liability associated with this right now?

26:04 So this is a huge part of the debate in AI regulation right now. There was a good quote in the New York Times from an interview with Woodrow Hartzog, who's a professor at BU law, I believe. And he said, quote, worst case scenario, they, meaning the big AI companies, have massive liability on both the criminal and civil side.

26:28 We haven't seen the firm busting case yet, but I could envision it, end quote. And so when you think about like Meta, for example, recently being forced to pay billions of dollars in damages related to child safety as part of court cases. It's not like liability law is toothless. It's a very powerful tool, although apparently it wasn't enough of a disincentive to stop Meta from doing the things that resulted in them losing that court case.

26:56 But I think that there's like a big question about what we were just talking about. Do you want folks to be punished after the harm occurs, or do you want to prevent the harm from occurring by having mandatory safeguards in place? But one thing is key to ask the question when you talk about liability is who is liable?

27:16 And at least for right now, the focus in Washington is on the companies that build and direct the agent, not the agent itself. AIs don't have legal rights, and not on the deployer, at least not yet. So not on the user of the AI agent, which could change over time, we'll have to see. But FTC Chairman Ferguson, he said, quoted in Reuters, I'm going to continue as long as I am chairman to resist this anthropomorphizing of these tools.

27:48 If someone tells a tool to do something and the tool does it, I don't think we would say, oh, what do we do about the tool? End quote. So the point here is that we're not going to put AI agents in jail. We're not going to try to punish Claude. We're going to try to punish anthropic. But I think the other part of the debate that really needs focus is around the deployers.

28:12 So what if, hypothetically, Claude is used by a customer to do something that results in harm? Right? I do not expect this anytime remotely soon. But what if a company involved in nuclear reactor safety equipment asks Claude to upgrade its software and then the nuclear reactor fails? In that case, maybe Anthropic is the one who is going to be sued for liability, but I think it's probably going to be the nuclear reactor company, at least in part.

28:45 So that is one part of the story that folks aren't really focused on in public, but was a huge part of the debate in, for example, the EU AI Act. In some ways, you could argue that the current cases that we have are like easier to think about from a liability law perspective because there's no reckless or careless customer to think about.

29:06 This is the developer running the agent itself. But there is a question about how liability law is supposed to do anything about open source in this story, right? Because what open source might allow is for much less well-resourced actors or for actors who aren't even based in the United States. So, how is liability law going to keep Americans safe from open source tools?

29:32 And how are you even going to identify who is the party responsible for the harm when many open source tools are modified by many, many, many different actors? So who is the creator responsible for the harm in question? Kind of difficult to think about how that's going to work out.

29:50 And specifically the reality we're aware of now, where some of the overseas models are very likely distilling from US models, right? They're not developing their own technology. If they wind up with a toxic effect out of something they've distilled, how do you get back to root cause on that?

30:05 Right. Some of that distillation might capture safeguards that the American companies put in there or the frontier companies put in there, but some of that distillation may not capture those safeguards because you don't have the same harness, you don't have the same bunch of other stuff.

30:19 So what's striking is who is on the same side of this conversation and almost goes back to the horseshoe theory of politics. Lena Kahn, former President Biden's FTC chair, and Scott Besson, President Trump's Treasury Secretary, are saying much the same thing on this issue. Greg, can you give us a synopsis of what they're saying?

30:39 Yes. So it comes back to this sort of existing tools theory of the case. Your horseshoe theory of politics is a funny analogy, but J.D. Vance, the vice president, during the campaign in 2024, he specifically said that Lena Kahn was one of the few Biden administration officials who he had a lot of respect for.

30:58 So when she was the head of the Federal Trade Commission under the Biden administration, she was famous for wanting to go after tech companies and restrict their actions in almost all cases under existing authorities. And what she said in a recent post on X, which she followed up with in an op-ed, quote, law enforcers already have the authority to charge companies and their CEOs for creating and releasing dangerous, unvetted, or defective products.

31:27 We shouldn't let discussions about new legal regimes distract from the fact that there's no AI exemption from the laws already on the books. And frankly, that's pretty consistent with what the Trump administration has been saying, right? The laws on the books can work. But again, I just want to emphasize what the laws on the books might do is punish people who are responsible for bad things happening.

31:57 And that punishment might be well into the future, right? Meta was found liable for child safety harms that happened five years ago, a decade ago, right? That's a long period of time before something actually happens. If you want to drive behavior today and you want to drive effective interventions that are going to increase the safety of the American people from these serious societal risks, as the Trump administration describes them, liability is a very, very, very slow tool to do that.

32:33 And liability also doesn't lead you to any economies of scale or standardization anytime soon. What about liability as a theory of the case is going to drive near-term industry cooperation or best practices sharing? When you talk about liability, it creates a lot of incentives for companies not to share information with the government because anything that they share might help the government go on a fishing expedition to go sue them.

33:06 That is the question here. If we're currently entering a very dangerous few years where AI capabilities are accelerating very rapidly and our safety capabilities are not keeping pace, I just don't know that liability is the most effective tool to prevent a lot of bad things from happening.

33:28 Yeah, there's also a question of how effective liability or even liability insurance theory would be associated with some of the most well-capitalized companies in the history of humanity, right? What what kind of fines or inability to get liability insurance would actually matter to an open AI, to an anthropic, to any of these companies?

33:48 I take your point. And just to be clear, I'm not saying that Lena Kahn or the Trump administration folks who have come to similar positions are necessarily wrong, that we should be pursuing use of the liability toolbox now. The question is, is that the only thing that we should be doing? And that's the part of the argument where I think I disagree with the Trump administration.

34:14 So we've spent months on AI agents breaking into systems by accidents during tests. And this week brought a case of attackers using one on purpose against some of South Korea's biggest banks. Give us an idea of what happened, Greg.

34:29 So this is a pretty remarkable outcome. Specifically, the attackers, now I think we have good enough evidence to say this pretty definitively, used an open source autonomous penetration testing system that can be plugged into any AI model. It's called R-Tex. And that is a tool that was open sourced for cyber defense reasons, right?

34:53 It was created out of a contest in China. Penetration testing is what the good guys are supposed to be doing. But the problem here is that when it comes to cybersecurity, offense and defense is very often the same bag of tricks, right? It's not like medieval nights, where there's a very clear distinction between defensive technologies like a suit of metal armor and offensive technologies like a sword or a bow and arrow.

35:23 In cyber, it's the same set of technologies for offense and defense, automated penetration testing. That's what you want to do to test all of your defenses. And that's what you want to do if you're trying to probe for weaknesses in an enemy's defenses in preparation for an attack. So this was a capability that was open sourced, I have no reason to doubt, with the best of intentions.

35:47 But here it is being used by criminals or bad actors of some kind in order to extract tens of thousands of records from many of South Korea's leading banks. And I want to emphasize this is a Chinese automated penetration testing tool, but there's no evidence, at least right now, to suggest that the malicious actors behind this attack were themselves Chinese.

36:14 Could be, could not be, this attack looks criminal in nature, not nation-state in nature. But I think it raises some questions that we've discussed in recent episodes of this podcast. Recall that Project Glasswing, it was a project to give defenders in government agencies and critical industries privileged access to these AI cyber tools in order to shore up their defenses.

36:44 And Anthropic never published the full list of who in South Korea was part of this program, which was basically run by the US government but executed by Anthropic. So my question is: well, I know US banks were part of Project Glasswing. Were these South Korean banks part of Project Glasswing? If I had to guess, I'd say no, they were not, which raises the question, why not?

37:13 Right? If you're South Korea, you're asking yourself, could this have been prevented if the US government was more generous in giving out Mythos access in advance when we needed to be shoring up our defenses? And then the flip side, what if they weren't a part of Project Glasswing? Well, then you're asking yourself, oh my gosh, what did we even accomplish in the months since we launched Project Glasswing?

37:39 If we spent all this time and effort trying to shore up our cyber defenses and AI tools, if it can still break in, well, gosh, like where are we in the story? Are we in an era where maybe just cyber offense has the advantage? I mean, in military theory and in the history of humanity, there's been various times when attackers or defenders had the advantage.

38:04 One of the explanations for why feudalism came about is because castles, defenders, just had an enduring advantage over siege technology of the era. And once cannons were invented and got pretty good, you saw the rise of larger nation states in Europe. The offense-defense balance in a given military technology throughout history has literally shaped what forms of government predominate in a given geography.

38:34 And so if we are entering an era in which cyber offense has the advantage, that's a pretty scary era. And part of what Mythos Glasswing was intended to accomplish was to ensure that defenders had the edge they need. But this to me is still a very much an open question.

38:53 Yeah, Greg, let's take a second, just very briefly. So R-Tex is not a model itself, right? How is the usage here different from what we've seen in the past?

39:03 Yes. So I'm going to struggle to get the technical nuances right here. But I think the thing to point out here is that this is a tool that you use, and it definitely needs a large language model, but the user gets to choose what they plug in there. So if I had to guess, and I'm only guessing at this stage, I would say it's almost certainly using a Chinese open source model, just because that is what appears to be appealing to criminals right now, because it's the most available, it's the most modifiable, has the fewest safeguards, and so on and so forth.

39:36 But good guys are capable of using this same tool potentially with anthropic or open AI models. So we have to ask ourselves what's going on here, and we still don't have all of the details.

39:49 So let's relate that back to our previous liability conversation. If an open source agent built for defenders gets used to rob banks, who's on the hook? Right.

39:59 This is the key. At least right now, with the limited evidence we have about the nature of this attack, it appears to be a relatively small criminal organization taking advantage of the very powerful tools that are out there available in the open source. Additional evidence might come out as we learn more that might contradict that, but that's the sort of best available interpretation of what's going on here, admittedly, somewhat speculative.

40:26 So, how is liability exactly going to solve that problem? OpenAI and Anthropic, they're pursuing valuations on the stock market in the trillions of dollars. They have a lot to lose in a liability lawsuit, right? They're scared of getting sued and losing. But if this is just a small criminal actor taking advantage of these open source tools, well, they might already be poor.

40:54 So if you sue them, what exactly are they gonna lose here? And then secondarily, it's the open source part of the equation. So either you sue the bad actor or you sue the people who develop those tools. But the people who develop those tools are, number one, they're in China. Number two, they might be ethereal in nature.

41:14 So if you're an open source developer like a ZAI or a Moonshot AI or a Deep Seek, you know, you can say, well, how do I know whether or not this model that was used by a criminal organization was modified by them in a way that I don't understand? Because once I put my weights out there on the internet, who knows what anybody else is going to do it?

41:34 So they can say the liability doesn't rest with me. It lies with the people who developed R-Tex, or maybe it relies with the people who modified my model in a way that I had no information about. And then the other people in the equation, the people who developed R-Techs, that's not some big company.

41:50 That's people who are in the open source development community. So I think this really highlights just how hard it is to imagine that we're gonna go after this problem with liability alone. It doesn't mean that liability is useless in this story, but I just don't see how it can fully solve the problems that we currently face.

42:13 So let's turn to chips. A central pillar of Washington's strategy to limit Chinese AI development has been to restrict sales of advanced AI chips to China since 2022. Nevertheless, Nvidia's chips keep turning up there. The most serious case so far reaches inside NVIDIA itself. Back in August, Taiwan indicted an NVIDIA manager accused of smuggling.

42:34 What happened here?

42:36 Well, I just struggle to express how frustrating this story is for me, but also how predictable it was. So let's just rewind a little bit. NVIDIA CEO Jensen Huang for years had basically been arguing that smuggling was impossible because these chips are integrated into servers that are very heavy. And how could we possibly smuggle something heavy into China?

43:06 That was literally one of his key talking points for many years. Then a lot of smuggling evidence came out. There's some very high-quality journalism from a lot of different outlets saying, hmm, really seems like a lot of chips are being smuggled. And NVIDIA continued to say, no, smuggling reports are unconfirmed, seems not real, whatever, this isn't happening.

43:30 Then we saw the Supermicro indictment, where the US Department of Justice basically said, no, a lot of smuggling is happening. And by the way, executives of a key NVIDIA partner, Supermicro, one of their biggest distributors, the co founder of the company, was personally involved in orchestrating large scale smuggling.

43:53 So NVIDIA then responds Well, but our practices are incredibly effective, and we're doing Everything right to follow the law and prevent smuggling. But now, in this Taiwanese indictment that came out on August 24th, they indicted a senior NVIDIA manager as the core figure in a smuggling ring. So Keelung district prosecutors in Taiwan charged this senior manager, surnamed Chang, and eight others.

44:25 They are seeking five years for forgery and breach of trust. I think this is the first known case of an NVIDIA employee facing legal action over smuggling, and it involved smuggling in the billions of dollars worth of chips. And what's really interesting is that the allegation specifically says that an NVIDIA insider helped a fake buyer, Shell Company, pass NVIDIA's own checks and safeguards.

44:56 So the checks and safeguards that NVIDIA had been telling everyone were going to be so effective at preventing smuggling. Well, actually, it turns out, at least according to this indictment, which I had no reason to doubt, that NVIDIA senior managers were involved in the smuggling ring to ensure that those internal safeguards did not work effectively.

45:18 So at this stage, one of the natures of the NVIDIA checks, which we're learning about in this indictment, is they have to go and inspect a customer's facility to make sure that all of the servers that they're purporting to buy can fit in there. And this individual and his cabal were the inspector in this case.

45:38 So they go to a Taiwanese facility where it is crystal clear that there's nowhere near enough space to have all of these servers. And they say, yep, seems legit. This customer is real. We can go ahead and proceed with this order. And so the point here is this was a straight up lie, right? The smuggling was obvious and the NVIDIA procedures were not adequate to catch it.

46:07 But I think we still have to come back to the question. Do we want this individual to be punished? Sure. But what I really want is for smuggling like this to not happen. What I really want is for the procedures to be effective.

46:22 So, Greg, based on what's going on, how much of a problem is all of this for NVIDIA and the US?

46:28 So I think we have evidence at this stage that's pretty considerable that shows that NVIDIA chip smuggling is in the billions and billions of dollars. We know about the INSPR smuggling ring that we talked about in the past on this show, the supermicro smuggling ring that we talked about in the past on this show.

46:50 It's definitely in the billions of dollars. So officials in the US government now believe that China has smuggled in more than 115,000 NVIDIA chips. And this is according to Bloomberg reporting from 2025 that data centers in China's western provinces were meant to house 115,000 restricted NVIDIA chips.

47:17 And NVIDIA and many other officials said, oh, that's unrealistic. But now US officials believe that China has smuggled in that much and then some more. And semi-analysis, which has done a lot of analysis of supply chains in China on AI data centers, they were quoted in Bloomberg saying that their supply chain checks show, quote, hundreds of thousands of AI chips have slipped through, end quote.

47:44 So on the one hand, this is really big. We're talking billions of dollars. And I think it's a travesty. I think it really has accelerated China's progress in AI in a meaningful way. At the same time, this is huge by traditional smuggling standards, but the AI market is unlike anything else that we've seen in the global economy.

48:11 The semiconductor market as a market was forecast to become a trillion dollar market by the year 2030. Now we're probably going to hit that metric four years early in 2026, just because of how incredible the AI data center build out has been. I mean, the biggest AI infrastructure builders, the Frontier Labs, the hyperscalers, they're expected to spend something like a trillion dollars on AI infrastructure.

48:42 So while this smuggling is gargantuan, while it is geopolitically strategically significant, it's worth pointing out that the export controls still achieved something meaningful, which is that America has more than 75% of all the AI computing power on planet Earth. And if the export controls are not weakened further, if we take steps to take action on smuggling, America's edge over China in AI compute is actually forecast to grow, even with all the failure modes that we talked about legally and illegally on this show.

49:22 So, Greg, on our September 22nd episode, you asked specifically whether letting smuggling slide is actually the secret policy being observed by these governments. So whose job is it to stop this? Is it NVIDIA's? Is it the United States? Who's responsible?

49:37 Yeah. So NVIDIA's position on all of this is, according to an NVIDIA spokesperson quoted in Bloomberg, quote, when the law is clear, American companies honor U.S. laws as they are, not as America's critics wish they were. End quote. And I think that is so telling, right? I mean, NVIDIA has been arguing against the export controls.

50:03 They have basically said, let us sell everything to China. And now when it's clear that smuggling is occurring on a massive scale, they're basically saying, well, it's not our job to prevent smuggling always, everywhere, forever. Our job is to implement the restrictions that you have stated around know your customer, around auditing shell companies, so on and so forth.

50:31 And that I think is the issue here. Dan Remler, who used to be an AI policy official in the State Department under both the Biden and the Trump administrations, he said this that I thought was quite insightful. Nvidia's entire posture of we will follow the law and no more, we follow the laws as written, speaks to a broader cultural issue that just does not reckon with the fact that China is our primary geostrategic adversary.

51:01 I mean, that's so poignant here. China is preparing for war with the United States. They think war with the United States is a totally plausible scenario, right? They're not quintupling the size of their nuclear arsenal because they're so excited to buy American soybeans and have profitable trade with us.

51:24 In their mind, they know that competition over geopolitical power, it's on and they're competing with the United States. But here in the United States, you know, we've got companies who are just thinking about how do I make the next quarterly revenue projection. And that involves selling a lot to China.

51:42 And I just think it's really bad. Nvidia's focus in this story is not, I think it's fair to say, US national security interests. Their interest is profit interests as a multinational corporation. And so the challenge is that so much of U.S. export control enforcement actually hinges upon making the company have substantive enforcement, devote sufficient resources to enforcement.

52:12 Because what the government devotes to export control enforcement is very, very thin. The Bureau of Industry and Security at the Department of Commerce has about a dozen overseas enforcement staff. I mean, that was true for many, many years. I think it's bigger now, but not a ton bigger. And at a recent hearing, Jeffrey Kessler, the under-secretary at BIS, he said, quote, we're concerned that there are cases that we haven't detected or that we just don't have the resources to chase down.

52:44 End quote. And I would say, yes, it's true that BIS needs more resources. I've been one of the loudest proponents of increasing the resources and staffing available to BIS since 2022. But I also think it's fair to say that this administration has just lost its appetite for enforcing these kinds of cases.

53:04 And that's why I think it's so remarkable that it was Taiwan that caught this guy, not the United States. Now I want to quote from Liza Tobin, who's a former official from the first Trump administration and now is a nonresident senior fellow at the Center for a New American Security. And she said this: quote, it is clear that the CEO of NVIDIA has influence over U.S.

53:28 export control policy, even more so than the members of President Trump's own cabinet. End quote. And so that's the thing. Nvidia is saying we follow the law, but they are also shaping the law. They are weakening the law. They are weakening the U.S. government's, I would say, ability to enforce the law.

53:53 And that is the problem. That is what Congress needs to address. That's what I hope this executive branch would address. And that's why we're in the very unfortunate position that we're in right now. Now, in Congress, the story is a bit better. You've seen members of the Senate, for example, Elizabeth Warren and Senator Banks.

54:14 So that's a bipartisan group of folks. They have suggested that Envisia's leadership has misled the government about smuggling. Banks called for a federal probe in mid-May, but so far, nothing has materialized. Recall that we talked about the other week on the podcast that there is bipartisan legislation looking to take steps to make export controls significantly more enforceable.

54:40 In the case of NVIDIA, that might be something like the Chip Security Act, but there's also the Match Act, Rasa. There's a lot of tools that are being proposed to add to the toolbox. So I'm hopeful that this situation might be significantly different perhaps after this year's midterms.

54:58 So, Greg, let's go ahead and transition to the courts now. So on our September 11th episode, we covered that Anthropics legal team had a win in the California federal court where Judge Rita Lin threw out the Pentagon supply chain risk designation. On Friday, September 25th, the DC Circuit Court ruled in the other case in this saga and went the other direction.

55:21 So what happened and can you give a distinction between these two cases and the underlying fact patterns?

55:27 Well, first, let me just take a victory lap. I've been one of the people since the beginning saying that a split verdict here was totally possible. I even got some criticism via email saying, oh no, it's not going to be a split verdict. What are you talking about? But yeah, here we are. It's a split verdict.

55:45 Sorry, when I say split verdict, I mean as in the two courts ruled different rulings on the case. It was also a split verdict within this DC court, in that the circuit court ruled 2-1 that the Pentagon lawfully cut anthropic out of its supply chain and that it can continue barring them from Pentagon work, and also that the supply chain risk designation was legitimate and can be maintained.

56:13 So, just very specifically, how can one federal court find that the government acted unlawfully and another find that it acted reasonably on what sounds like the same designation, right? There's got to be a little bit more of a difference there.

56:25 So it gets down to how the court system handles national security and also how some of the laws in question specifically tag different courts as responsible for something. So Judge Lynn in the Northern District of California, she was reviewing the case under a 2011 defense procurement law. And it's worth remembering, right, that Anthropic sued in both of these courts.

56:54 So they won on the grounds of that defense procurement law. But the DC Circuit is the only court that is allowed to review cases under the 2018 Federal Acquisition Supply Chain Security Act. So Congress has sent all of those challenges to this court and this court alone. So they're going after different aspects of the case because those different laws talk about what can and cannot be done.

57:26 So the 2018 law is evaluating supply chain risks under the criteria that, quote, any person may sabotage, maliciously introduce unwanted function, extract data, or otherwise manipulate, end quote. And I think even in that sentence, right, we're really talking about malicious intent in the context of manipulate.

57:50 But what those two judges did is they just said, manipulate, we're going to use the everyday English definition of the word manipulate, which is can change in any way, maliciously or not. And they said that they had no quarrel with the other judge, the Northern California judges, finding that anthropic acted without bad motive.

58:13 But in their interpretation of the law, that otherwise manipulate clause, quote, no such bad motive is required, end quote. And so what this means, right, is just the fact that Anthropic could change their product was enough to give the government authority to designate them a supply chain risk. And this is what I think is so crazy here is that all of their ruling implies that they can basically designate anybody who ships software to the US federal government, at least AI software, can be designated a supply chain risk at any time, even if they act with the absolute best of intentions, and even if they do everything that the government asks them to do.

59:03 And I just think that's a ridiculous interpretation of the law. And it's worth pointing out that the three judges on this district court, all three of them were Republicans. It was a 2-1 split decision, and all three of them are Republicans. And the dissenting voice, also a Republican, basically attacked the majority view as nonsensical and unreasonable.

59:27 So what did the DC Circuit Court find and what did its decision rest on specifically?

59:34 So there's basically two elements. The first is just what I said before on the everyday interpretation of the word manipulate. The second is just on national security deference. And this is a pretty established practice in the US court system that they are hugely deferential to the national security establishment, unless there's other clear instructions as to why they should not be deferential.

59:58 Here's the quote that they gave on that: quote, in our republic, it is the president and the secretary of war who must determine how best to balance the competing risks. End quote. I think my big criticism here is that this ruling basically uniquely punishes anthropic for features of its technology and features of its business practices that are not unique to Anthropic.

1:00:26 And this is not just my interpretation, this is the interpretation of the Northern District of California ruling, where the government, in that case, not the court, the Department of War, conceded that Claude is, quote, no riskier to the national security than any other black box artificial intelligence model, end quote.

1:00:47 So it's not that there's like something special about what Anthropic did here that makes them a legitimate target in this case. The ruling basically says that the government, the Department of War, can do this anytime it wants to any AI company. And if you are an AI company and you're a startup and you're asking yourself, should I bid on contracts for the Department of War, this is a remarkable, chilling effect to put on that.

1:01:18 And it doesn't just threaten all the AI companies. I think a lot of the interpretation here would just apply to anybody who updates their software, like all the cloud companies doing business with the Department of War. And so I think this just brings a unpredictability, a capriciousness, and honestly, really disappointing reality to what it's like to work in the national security complex that I think is very likely to be discouraging to a lot of companies.

1:01:50 And I remember being in the Department of Defense and trying to persuade technology companies to dip their toes in the water of defense contracting when they faced many incentives not to do so. And I think this is just a shockingly powerful new incentive for the best and smartest talents in the private sector to not want to devote those talents to national security.

1:02:15 And so, like Anthropic lost this case, but the Department of War lost this case too.

1:02:23 Yep. Unfortunately, I think that's very much the case, Greg. That actually seems like a really good place to end it for now. Thanks goes out to our audience for tuning into this episode. And as always, to Greg for walking us through these developments. We'll be back soon with a new episode, and as always, I'm sure we'll have plenty to talk about.

1:02:41 Indeed. Thanks for listening to this podcast. And if you enjoyed it, there's a way you can help. Please give us a five-star rating on whatever your preferred podcast app is and help spread the word that this podcast is out there. Every bit helps.