Stealing Reasoning Traces from Proprietary LLM APIs — Ilia Shumailov & Alexander Panfilov

Tim Scarfe speaks with Ilia Shumailov and Alexander Panfilov about their paper, Stealing Reasoning Traces from Proprietary LLM APIs.The core bug sounds deceptively simple: providers return encrypted reasoning state so conversations can be resumed or forked. But those blobs can be replayed across users and sibling models. A smaller model can ask the provider to decrypt the trace, then repeat the hidden reasoning in plain text. The discussion covers leaked private data, a broadly reusable jailbreak, poisoned agent traces, chain-of-thought monitoring, responsible disclosure, and possible defenses.Ilia Shumailov is an AI and security researcher, formerly at Google DeepMind, who completed his Cambridge PhD under Ross Anderson. Alexander Panfilov is a PhD researcher at the ELLIS Institute Tübingen and the Max Planck Institute for Intelligent Systems, working on AI safety, adversarial machine learning, and LLM red-teaming. They close by separating the demonstrated jailbreaking threat from ordinary benign distillation, and by arguing for controlled experiments over sweeping claims.---TIMESTAMPS:00:00:00 Intro montage00:01:33 Portable encrypted thought and decoded reasoning00:24:55 How the attack works and what it means00:39:04 Doom, defense, and scientific restraint---REFERENCES:paper:[00:00:00] Stealing Reasoning Traces from Proprietary LLM APIs00:09:22]" rel="nofollow noopener ugc" target="_blank">arxiv.org/abs/2608.09867[00:09:22] Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety00:11:30]" rel="nofollow noopener ugc" target="_blank">arxiv.org/abs/2507.11473[00:11:30] Reasoning Models Don’t Always Say What They Think00:37:22]" rel="nofollow noopener ugc" target="_blank">anthropic.com/research/reasoning-models-dont-s… PostTrainBench: Can LLM Agents Automate LLM Post-Training?00:41:02]" rel="nofollow noopener ugc" target="_blank">arxiv.org/abs/2603.08640[00:41:02] Large-scale online deanonymization with LLMs00:09:28]" rel="nofollow noopener ugc" target="_blank">arxiv.org/abs/2602.16800other:[00:09:28] OpenAI and Hugging Face partner to address security incident during model evaluation00:10:22]" rel="nofollow noopener ugc" target="_blank">openai.com/index/hugging-face-model-evaluation… Claude, GPT, and Gemini All Struggle to Evade Monitors00:42:08]" rel="nofollow noopener ugc" target="_blank">metr.org/notes/2025-08-22-claude-gpt-gemini-st… Isabelle proof assistantisabelle.in.tum.de/---RESCRIPT: app.rescript.info/share/07fc38276e0823dc9b8986…