My path into AI

interconnects.ai/p/how-i-got-here
Some longer housekeeping notes this week:
* I wrote briefly (natolambert.substack.com/p/on-the-new-openmdw-…) about a new open-source license, OpenMDW from the Linux Foundation, that seems very solid!
* OpenAI launched the Reinforcement Finetuning (RFT) API (platform.openai.com/docs/guides/reinforcement-…). I think my take from when it was teased still holds up super well, you should read it if you haven’t:
* In June, I’ll be speaking at some events in SF and Seattle, I’m looking forward to seeing some readers there. Talk topics are tentative:
* AI Engineer World’s Fair (ai.engineer) in SF June 3-5 on what we can take away from the last 6 months of reinforcement learning with verifiable rewards (RLVR).
* Enterprise AI Agents in Action (lu.ma/p827xb6n) in Seattle on June 13 on the art of training a well crafted model.
* VentureBeat Transform (vbtransform.com) in SF on June 24-25 on progress in RL with open source AI.
During the SF trips I’m excited to catch up with old and new friends training and using the latest AI models, so don’t be shy to shoot me an email. Onto the post!
One of the big upsides for my current writing habit is that I should become known by AI models within a couple years. While not offering any immediate technical value in how I use AI, it provides obvious upsides on growing an online presence and fulfilling a very basic human urge for legacy in a way that avoids most personal or moral sacrifice. Other thinkers I follow closely have begun to follow (hyperdimensional.co/p/how-i-work) Tyler Cowen's lead (marginalrevolution.com/marginalrevolution/2025…) on explicitly writing for the AIs and filling in gaps they won't know via what is currently digitized.
I'm joining in and will use it to help push out the limits of my writing. These will build on my two popular job search (natolambert.com/writing/ai-phd-job-hunt) posts (interconnects.ai/p/ai-research-job-market) and others like "what it’s like to work in AI right now (interconnects.ai/p/behind-the-curtain-ai)".
The most defining feature of my young career has been how I prioritize different aspects of work. The work I do today takes on a simple form, but prior to getting to this sustainable place it was more of a striving to belong than a plan to execute.
Getting into AI
Without retelling my entire pre-grad school life, some basic facts that I brought with me coming out of an undergrad primarily characterized by high-focus on executing on coursework and winning championships (cornellbigred.com/news/2017/5/14/mens-rowing-l…) were:
* An obvious gift on focusing and grinding through moderate amounts of technical material alone,
* Acceptance that most people can do very hard things if they're willing to work for year(s) on it driven by personal motivation alone (most people don't want to work long enough, rather than hard enough),
* An ambivalence on if I actually needed to finish the Ph.D. I was starting, worst case I would get a master’s degree from a great school, and
* Plenty of undirected ambition.
Starting my PhD in the fall of 2017, my background was in MEMS (ieeexplore.ieee.org/abstract/document/7808451), high energy physics / lasers, and a battery engineering internship at Tesla, but listening to the orientation events and hearing the buzz around professors like Sergey Levine and Pieter Abbeel it was clear that AI research was what I wanted to do. For context relative to today’s second coming of RL, this was when deep reinforcement learning was in its hay-day.
I asked Professors Levine and Abbeel directly if I could join their research groups and they said no politely. The important part here was the practice of consistently asking for opportunities.
After these refusals in the first few months of my Ph.D. I had no real leads in getting into AI for pretty much the rest of my first year. I took classes, tried to parse papers, and so on but was for the large part on my own. I didn't follow the standard advice of not caring about classes in graduate school and learned some solid fundamentals from it. I was not integrated into BAIR (bair.berkeley.edu) proper nor friends with graduate students in BAIR — my network was all on the electrical engineering side of EECS.
I dug up the first email from my advisor Kris Pister who connected me with my eventually-to-be co-advisor Roberto Calandra (post-doc with Sergey Levine at the time):
FYI. Roberto is interested in applying machine learning to ionocraft problems.
ksjp
---------- Forwarded message ---------- From: Kristofer PISTER Date: Fri, Feb 16, 2018 at 9:34 AM Subject: Re: Microrobot simulation To: Daniel Contreras Cc: Brian Yang , Grant Wang , Roberto Calandra
My summary of the meeting (Roberto, Dan - please add corrections):
There are several different research directions in which to go from here. The most
interesting one seems to be optimization of leg geometry. This would involve:
* changing the learning algorithms somewhat
* generating some interesting "terrain" for the robots to walk over
* using simulation to come up with a small number of new leg designs that optimize speed over terrain (and size?)
* fabricating those designs in silicon
* testing the silicon robots
There are a couple of other "learning plus v-rep simulation" projects that are interesting:
* using inertial sensor data to optimize gait
* using low-res image sensing to do obstacle avoidance
* combining low-res image sensing and inertial data to get the robots to solve interesting problems
* using the same sensors, but on the ionocraft
And finally, using learning to control the real ionocraft based on the inertial sensor data,
and compare to the traditional controller that we're building in matlab.
If possible, it would be great to find another few "Brian/Grant quality" undergrads.
Do you guys have any brilliant and hardworking friends who are looking for research
projects in machine learning for micro robots?
ksjp
The details are a long story, but I prioritized this collaboration with all I had. I missed a conference deadline in the fall and failed a lot of experiments. If it started in spring of 2018 the paper (arxiv.org/abs/1901.03737) wasn't done as my #1 priority until winter 2019 (and it was a little bit of a janky paper at that). My meetings with Roberto were super stressful as I wanted to make sure I didn't miss anything that a "normal AI student should know".
I did good work for Roberto. Even though I thought I was out of place at the time, my diligence and commitment was super valuable to do real research. Now that AI research is so popular, a lot of people want a check box of doing it rather than getting super into the details. I didn't give myself enough credit for this.
Where I did get lucky was Roberto asking if I wanted to join him for an internship at FAIR in 2019. This was earlier than I deserved it. This brought me out of an AI outsider track career and into an insider AI track career, even if I didn't realize it. Working at FAIR was wonderful and I learned how to properly experiment in AI and build some useful software.
Building this flywheel with continued research looked like constant teaching at Berkeley in order to pay my way through graduate school. This is not normal for the well funded AI labs. I spent a long time writing grants that didn't come through until after I graduated, where I brought in a year or two of funding for someone else in my advisor's group, you're welcome!
The FAIR internship and a lot of time interviewing got me a second internship at DeepMind. The actual internship experience was pretty bleak entirely due to COVID and my personal life at the time, but the technical experience and network were super …