Francois Chollet — Why the biggest AI models can't solve simple puzzles

Here is my conversation with Francois Chollet and Mike Knoop on the $1 million ARC-AGI Prize they're launching today (arcprize.org).
I did a bunch of socratic grilling throughout, but Francois’s arguments about why LLMs won’t lead to AGI are very interesting and worth thinking through.
It was really fun discussing/debating the cruxes. Enjoy!
Watch on YouTube (youtu.be/UakqL6Pj9xo). Listen on Apple Podcasts (podcasts.apple.com/us/podcast/francois-chollet…), Spotify (open.spotify.com/episode/7bmeJQOvXGy4LYl6YoiYY…), or any other podcast platform. Read the full transcript here (dwarkeshpatel.com/p/francois-chollet).
Timestamps
(00:00:00) – The ARC benchmark
(00:11:10) – Why LLMs struggle with ARC
(00:19:00) – Skill vs intelligence
(00:27:55) - Do we need “AGI” to automate most jobs?
(00:48:28) – Future of AI progress: deep learning + program synthesis
(01:00:40) – How Mike Knoop got nerd-sniped by ARC
(01:08:37) – Million $ ARC Prize
(01:10:33) – Resisting benchmark saturation
(01:18:08) – ARC scores on frontier vs open source models
(01:26:19) – Possible solutions to ARC Prize

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit dwarkesh.com?utm_medium=podcast&utm_campai…