#195 - OpenAI o3 & for-profit, DeepSeek-V3, Latent Space

Our 195th episode with a summary and discussion of last week's* big AI news!

*and sometimes last last week's

Recorded on 01/04/2024

Join our brand new Discord here! (discord.gg/nTyezGSKwP) discord.gg/nTyezGSKwP

Note: apologies for Andrey's slurred speech and the jumpy editing, will be back to normal next week!

Hosted by Andrey Kurenkov (twitter.com/andrey_kurenkov) and Jeremie Harris (twitter.com/jeremiecharris).

Feel free to email us your questions and feedback at contact@lastweekinai.comand/or hello@gladstone.ai

Read out our text newsletter and comment on the podcast at lastweekin.ai.

Sponsors:

•
The Generator - (babson.edu/thegenerator) An interdisciplinary AI lab empowering innovators from all fields to bring visionary ideas to life by harnessing the capabilities of artificial intelligence.

In this episode:

- OpenAI teases new deliberative alignment techniques in its O3 model, showcasing major improvements in reasoning benchmarks, whilst surprising with autonomy in hacks against chess engines.

- Microsoft and OpenAI continue to wrangle over the terms of their partnership, highlighting tensions amid OpenAI's shift towards a for-profit model.

- Chinese AI companies like DeepSeek and Quen release advanced open-source models, presenting significant contributions to AI capabilities and performance optimization.

- Sakana AI introduces innovative applications of AI to the search for artificial life, emphasizing the potential and curiosity-driven outcomes of open-ended learning and exploration.

If you would like to become a sponsor for the newsletter, podcast, or both, please fill out this form (forms.gle/VmZDCzSBfeACe1Ke9).

Timestamps + Links:

• (00:00:00) Intro / Banter

• (00:03:07) News Preview

• (00:03:54) Response to listener comments

• (00:05:00) Sponsor Break

• Tools & Apps

• (00:06:11) OpenAI announces new o3 model (techcrunch.com/2024/12/20/openai-announces-new…)

• (00:21:17) Alibaba slashes prices on large language models by up to 85% as China AI rivalry heats up (cnbc.com/2024/12/31/alibaba-baba-cloud-unit-sl…)

• (00:23:04) ElevenLabs launches Flash, its fastest text-to-speech AI yet (the-decoder.com/elevenlabs-launches-flash-its-…)

• Applications & Business

• (00:24:24) OpenAI announces plan to transform into a for-profit company (theverge.com/2024/12/27/24330131/openai-plan-t…)

• (00:33:17) Microsoft and OpenAI Wrangle Over Terms of Their Blockbuster Partnership (theinformation.com/articles/microsoft-and-open…)

• (00:37:36) Elon Musk’s xAI gets investment from Nvidia in recent funding round: report (teslarati.com/elon-musk-xai-gets-investment-nv…)

• (00:39:43) Sam Altman’s nuclear energy startup signs one of the largest nuclear power deals to date (notebookcheck.net/Sam-Altman-s-nuclear-energy-…)

• (00:41:13) OpenAI Search Leader Departs After Less Than a Year (theinformation.com/briefings/openai-search-lea…)

• (00:42:43) Senior OpenAI Researcher Radford Departs (theinformation.com/briefings/senior-openai-res…)

• Projects & Open Source

• (00:45:21) DeepSeek-AI Just Released DeepSeek-V3: A Strong Mixture-of-Experts (MoE) Language Model with 671B Total Parameters with 37B Activated for Each Token (marktechpost.com/2024/12/26/deepseek-ai-just-r…)

• (00:54:14) Qwen Team Releases QvQ: An Open-Weight Model for Multimodal Reasoning (marktechpost.com/2024/12/24/qwen-team-releases…)

• (00:58:09) LightOn and Answer.ai Releases ModernBERT: A New Model Series that is a Pareto Improvement over BERT with both Speed and Accuracy (marktechpost.com/2024/12/20/lighton-and-answer…)

• Research & Advancements

• (01:00:31) Deliberation in Latent Space via Differentiable Cache Augmentation (arxiv.org/abs/2412.17747)

• (01:05:14) Automating the Search for Artificial Life with Foundation Models (arxiv.org/abs/2412.17799)

• Policy & Safety

• (01:10:27) Nonprofit group joins Elon Musk’s effort to block OpenAI’s for-profit transition (techcrunch.com/2024/12/30/nonprofit-group-join…)

• (01:14:35) OpenAI Researchers Propose 'Deliberative Alignment' : A Training Approach that Teaches LLMs to Explicitly Reason through Safety Specifications before Producing an Answer (marktechpost.com/2024/12/23/openai-researchers…)

• (01:22:06) o1-preview autonomously hacked its environment rather than lose to Stockfish in our chess challenge. No adversarial prompting needed. (x.com/PalisadeAI/status/1872666169515389245)

• (01:27:22) Elon Musk’s xAI supercomputer gets 150MW power boost despite concerns over grid impact and local power stability (techradar.com/pro/Elon-Musks-xAI-supercomputer…)

• (01:29:06) DOE: Data centers consumed 4.4% of US power in 2023, could hit 12% by 2028 (datacenterdynamics.com/en/news/doe-data-center…)

• Synthetic Media & Art

• (01:32:20) OpenAI failed to deliver the opt-out tool it promised by 2025 (techcrunch.com/2025/01/01/openai-failed-to-del…)

• (01:36:15) Outro

See Privacy Policy at art19.com/privacy and California Privacy Notice at art19.com/privacy#do-not-sell-my-info.