EP13: Recurrent-Depth Models and Latent Reasoning with Jonas Geiping

In this episode, we host Jonas Geiping from ELLIS Institute & Max-Planck Institute for Intelligent Systems, Tübingen AI Center, Germany. We talked about his broad research on Recurrent-Depth Models and latent reasoning in large language models (LLMs). We talked about what these models can and can't do, what are the challenges and next breakthroughs in the field, world models, and the future of developing better models. We also talked about safety and interpretability, and the role of scaling laws in AI development.

Chapters
00:00 Introduction and Guest Introduction
01:03 Peer Review in Preprint Servers
06:57 New Developments in Coding Models
09:34 Open Source Models in Europe
11:00 Dynamic Layers in LLMs
26:05 Training Playbook Insights
30:05 Recurrent Depth Models and Reasoning Tasks
43:59 Exploring Recursive Reasoning Models
46:46 The Role of World Models in AI
48:41 Innovations in AI Training and Simulation
50:39 The Promise of Recurrent Depth Models
52:34 Navigating the Future of AI Algorithms
54:44 The Bitter Lesson of AI Development
59:11 Advising the Next Generation of Researchers
01:06:42 Safety and Interpretability in AI Models
01:10:46 Scaling Laws and Their Implications
01:16:19 The Role of PhDs in AI Research

Links and paper:
• Jonas' website - jonasgeiping.github.io
• Scaling up test-time compute with latent reasoning: A recurrent depth approach - arxiv.org/abs/2502.05171
• The Smol Training Playbook: The Secrets to Building World-Class LLMs - huggingface.co/spaces/HuggingFaceTB/smol-train…
• VaultGemma: A Differentially Private Gemma Model - arxiv.org/abs/2510.15001

Music:
“Kid Kodi” — Blue Dot Sessions — via Free Music Archive — CC BY-NC 4.0.
“Palms Down” — Blue Dot Sessions — via Free Music Archive — CC BY-NC 4.0.
Changes: trimmed