Prof. Jakob Foerster - ImageNet Moment for Reinforcement Learning?

Prof. Jakob Foerster, a leading AI researcher at Oxford University and Meta, and Chris Lu, a researcher at OpenAI -- they explain how AI is moving beyond just mimicking human behaviour to creating truly intelligent agents that can learn and solve problems on their own. Foerster champions open-source AI for responsible, decentralised development. He addresses AI scaling, goal misalignment (Goodhart's Law), and the need for holistic alignment, offering a quick look at the future of AI and how to guide it.

SPONSOR MESSAGES:
***
CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. Check out their super fast DeepSeek R1 hosting!
centml.ai/pricing

Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich.

Goto tufalabs.ai
***

TRANSCRIPT/REFS:
dropbox.com/scl/fi/yqjszhntfr00bhjh6t565/JAKOB…

Prof. Jakob Foerster
x.com/j_foerst
jakobfoerster.com
University of Oxford Profile:
eng.ox.ac.uk/people/jakob-foerster

Chris Lu:
chrislu.page

TOC
1. GPU Acceleration and Training Infrastructure
[00:00:00] 1.1 ARC Challenge Criticism and FLAIR Lab Overview
[00:01:25] 1.2 GPU Acceleration and Hardware Lottery in RL
[00:05:50] 1.3 Data Wall Challenges and Simulation-Based Solutions
[00:08:40] 1.4 JAX Implementation and Technical Acceleration

2. Learning Frameworks and Policy Optimization
[00:14:18] 2.1 Evolution of RL Algorithms and Mirror Learning Framework
[00:15:25] 2.2 Meta-Learning and Policy Optimization Algorithms
[00:21:47] 2.3 Language Models and Benchmark Challenges
[00:28:15] 2.4 Creativity and Meta-Learning in AI Systems

3. Multi-Agent Systems and Decentralization
[00:31:24] 3.1 Multi-Agent Systems and Emergent Intelligence
[00:38:35] 3.2 Swarm Intelligence vs Monolithic AGI Systems
[00:42:44] 3.3 Democratic Control and Decentralization of AI Development
[00:46:14] 3.4 Open Source AI and Alignment Challenges
[00:49:31] 3.5 Collaborative Models for AI Development

REFS
[[00:00:05] ARC Benchmark, Chollet
github.com/fchollet/ARC-AGI

[00:03:05] DRL Doesn't Work, Irpan
alexirpan.com/2018/02/14/rl-hard.html

[00:05:55] AI Training Data, Data Provenance Initiative
nytimes.com/2024/07/19/technology/ai-data-rest…

[00:06:10] JaxMARL, Foerster et al.
arxiv.org/html/2311.10090v5

[00:08:50] M-FOS, Lu et al.
arxiv.org/abs/2205.01447

[00:09:45] JAX Library, Google Research
github.com/jax-ml/jax

[00:12:10] Kinetix, Mike and Michael
arxiv.org/abs/2410.23208

[00:12:45] Genie 2, DeepMind
deepmind.google/discover/blog/genie-2-a-large-…

[00:14:42] Mirror Learning, Grudzien, Kuba et al.
arxiv.org/abs/2208.01682

[00:16:30] Discovered Policy Optimisation, Lu et al.
arxiv.org/abs/2210.05639

[00:24:10] Goodhart's Law, Goodhart
en.wikipedia.org/wiki/Goodhart%27s_law

[00:25:15] LLM ARChitect, Franzen et al.
github.com/da-fr/arc-prize-2024/blob/main/the_…

[00:28:55] AlphaGo, Silver et al.
arxiv.org/pdf/1712.01815.pdf

[00:30:10] Meta-learning, Lu, Towers, Foerster
direct.mit.edu/isal/proceedings-pdf/isal2023/3…

[00:31:30] Emergence of Pragmatics, Yuan et al.
arxiv.org/abs/2001.07752

[00:34:30] AI Safety, Amodei et al.
arxiv.org/abs/1606.06565

[00:35:45] Intentional Stance, Dennett
plato.stanford.edu/entries/ethics-ai

[00:39:25] Multi-Agent RL, Zhou et al.
arxiv.org/pdf/2305.10091

[00:41:00] Open Source Generative AI, Foerster et al.
arxiv.org/abs/2405.08597

<trunc, see PDF/YT>