Deep Dive: Jeff Dean on Google Brain’s Early Days

In the fifth installment of our Moonshot Podcast Deep Dive video interview series, X’s Captain of Moonshots Astro Teller sits down with Google DeepMind’s Chief Scientist Jeff Dean for a conversation about the origin of Jeff’s pioneering work scaling neural networks.

They discuss the first time AI captured Jeff’s imagination, the earliest Google Brain framework, the team’s stratospheric advancements in image recognition and speech-to-text, how AI is evolving, and more.

For more on X’s AI moonshots, check out Jeff on Episode 3 of The Moonshot Podcast, “Machines That Think”: youtube.com/watch?v=_jLPA8Qpc74

00:00 - Introduction
00:35 - What Jeff’s childhood was like
03:50 - The first “non-trivial” thing Jeff coded
05:02 - Jeff’s “love/hate” relationship with C++
06:59 - The first time AI captured Jeff’s imagination
10:40 - When neural networks fell “out of vogue”
12:24 - How Jeff made a career out of “starting over”
14:23 - The origins of Google Brain
17:30 - Critical components of scaling neural networks
18:43 - The earliest Google Brain framework
19:32 - How the team first built a 100x bigger neural network
22:04 - The cat video that started it all
25:02 - Google Brain’s major progress on image recognition
26:24 - Google Brain’s major progress on speech to text
27:29 - The origins of TensorFlow and TPUs
31:56 - Three breakthroughs in understanding language models
37:41 - Where Jeff thinks the world is headed
40:07 - The shift from “making things” to designing prompts
42:38 - How Jeff uses AI to enhance his own life
43:51 - Safety, security, and privacy in AI
50:26 - The future of understanding neural network behavior
53:34 - When computers are “smarter” than humans
56:52 - Jeff’s next-five-years plan

Learn more about your ad choices. Visit podcastchoices.com/adchoices