Machine Learning Street Talk (MLST)
Subbarao Kambhampati - Do o1 models search?
Donnerstag, 23. Januar 2025 · 1 h 32
Kapitelklicken springt direkt hin
Shownotes
Join Prof. Subbarao Kambhampati and host Tim Scarfe for a deep dive into OpenAI's O1 model and the future of AI reasoning systems.
* How O1 likely uses reinforcement learning similar to AlphaGo, with hidden reasoning tokens that users pay for but never see
* The evolution from traditional Large Language Models to more sophisticated reasoning systems
* The concept of "fractal intelligence" in AI - where models work brilliantly sometimes but fail unpredictably
* Why O1's improved performance comes with substantial computational costs
* The ongoing debate between single-model approaches (OpenAI) vs hybrid systems (Google)
* The critical distinction between AI as an intelligence amplifier vs autonomous decision-maker
SPONSOR MESSAGES:
***
CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments.
Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. Are you interested in working on reasoning, or getting involved in their events?
Goto tufalabs.ai
***
TOC:
1. **O1 Architecture and Reasoning Foundations**
[] 1.1 Fractal Intelligence and Reasoning Model Limitations
[] 1.2 LLM Evolution: From Simple Prompting to Advanced Reasoning
[] 1.3 O1's Architecture and AlphaGo-like Reasoning Approach
[] 1.4 Empirical Evaluation of O1's Planning Capabilities
2. **Monte Carlo Methods and Model Deep-Dive**
[] 2.1 Monte Carlo Methods and MARCO-O1 Implementation
[] 2.2 Reasoning vs. Retrieval in LLM Systems
[] 2.3 Fractal Intelligence Capabilities and Limitations
[] 2.4 Mechanistic Interpretability of Model Behavior
[] 2.5 O1 Response Patterns and Performance Analysis
3. **System Design and Real-World Applications**
[] 3.1 Evolution from LLMs to Language Reasoning Models
[] 3.2 Cost-Efficiency Analysis: LLMs vs O1
[] 3.3 Autonomous vs Human-in-the-Loop Systems
[] 3.4 Program Generation and Fine-Tuning Approaches
[] 3.5 Hybrid Architecture Implementation Strategies
Transcript: dropbox.com/scl/fi/d0ef4ovnfxi0lknirkvft/Subba…
REFS:
[] Monty Python (1975)
Witch trial scene: flawed logical reasoning.
youtube.com/watch?v=zrzMhU_4m-g
[] Cade Metz (2024)
Microsoft–OpenAI partnership evolution and control dynamics.
nytimes.com/2024/10/17/technology/microsoft-op…
[] Kojima et al. (2022)
Zero-shot chain-of-thought prompting ('Let's think step by step').
[] DeepMind Research Team (2023)
Multi-bot game solving with external and internal planning.
deepmind.google/research/publications/139455
[] Silver et al. (2016)
AlphaGo's Monte Carlo Tree Search and Q-learning.
nature.com/articles/nature16961
[] Kambhampati, S. et al. (2023)
Evaluates O1's planning in "Strawberry Fields" benchmarks.
[] Alibaba AIDC-AI Team (2023)
MARCO-O1: Chain-of-Thought + MCTS for improved reasoning.
[] Kambhampati, S. (2024)
Explores LLM "reasoning vs retrieval" debate.
[] Wei, J. et al. (2022)
Chain-of-thought prompting (introduces last-letter concatenation).
[] Barbero, F. et al. (2024)
Transformer attention and "information over-squashing."
[] Ruis, L. et al. (2023)
Influence functions to understand procedural knowledge in LLMs.
(truncated - continued in shownotes/transcript doc)
