#190 - AI scaling struggles, OpenAI Agents, Super Weights

Our 190th episode with a summary and discussion of last week's* big AI news!

*and sometimes last last week's

Hosted by Andrey Kurenkov (twitter.com/andrey_kurenkov) and Jeremie Harris (twitter.com/jeremiecharris).

Note from Andrey: this one is coming out a bit later than planned, apologies! Next one will be coming out sooner.

Feel free to email us your questions and feedback at contact@lastweekinai.comand/or hello@gladstone.ai

Read out our text newsletter and comment on the podcast at lastweekin.ai.

Sponsors:

•
The Generator - (babson.edu/thegenerator) An interdisciplinary AI lab empowering innovators from all fields to bring visionary ideas to life by harnessing the capabilities of artificial intelligence

In this episode:

* OpenAI's pitch for a $100 billion data center and AI strategy plan outlines infrastructure and regulatory needs, emphasizing AI's foundational role akin to electricity.

* Google's Gemini model challenges OpenAI's dominance, showing strong performance in chatbot arenas alongside generative AI advancements.

* DeepMind's AlphaFold3 gets open-sourced for academic use, while new chips from NVIDIA and Google show significant performance boosts.

* Anthropic and TSMC updates highlight strategic funding, regulation influences, and the complex dynamics of AI hardware and international policy.

If you would like to become a sponsor for the newsletter, podcast, or both, please fill out this form (forms.gle/VmZDCzSBfeACe1Ke9).

Timestamps + Links:

• (00:00:00) Intro / Banter

• (00:02:44) News Preview

• (00:03:34) Sponsor Break

• Tools & Apps

• (00:04:36) OpenAI, Google and Anthropic Are Struggling to Build More Advanced AI (bloomberg.com/news/articles/2024-11-13/openai-…)

• (00:16:22) OpenAI Nears Launch of AI Agent Tool to Automate Tasks for Users (bloomberg.com/news/articles/2024-11-13/openai-…)

• (00:19:14) Google drops new Gemini model and it goes straight to the top of the LLM leaderboard (tomsguide.com/ai/google-gemini/google-drops-ne…)

• (00:19:14) Chinese AI startup takes aim at OpenAI's Sora with image-to-video tool launch (cnbc.com/2024/11/14/chinese-ai-startup-shengsh…)

• (00:20:04) Introducing the Forge Reasoning API Beta and Nous Chat: An Evolution in LLM Inference (nousresearch.com/introducing-the-forge-reasoni…)

• Applications & Business

• (00:23:47) OpenAI Discusses AI Data Center That Could Cost $100 Billion (theinformation.com/briefings/openai-discusses-…)

• (00:26:48) Elon Musk's massive AI data center gets unlocked — xAI gets approved for 150MW of power, enabling all 100,000 GPUs to run concurrently (tomshardware.com/tech-industry/artificial-inte…)

• (00:29:34) Newest Google and Nvidia Chips Speed AI Training (spectrum.ieee.org/amp/ai-training-2669810566-2…)

• (00:34:45) Ex-OpenAI CTO Murati’s New Team Takes Shape (theinformation.com/articles/ex-openai-cto-mura…)

• (00:34:45) Amazon Discussing New Multibillion-Dollar Investment in Anthropic (theinformation.com/articles/amazon-discussing-…)

• Projects & Open Source

• (00:37:52) Google DeepMind open-sources AlphaFold 3, ushering in a new era for drug discovery and molecular biology (venturebeat.com/ai/google-deepmind-open-source…)

• (00:41:29) Near plans to build world’s largest 1.4T parameter open-source AI model (cointelegraph.com/news/near-plans-to-create-wo…)

• Research & Advancements

• (00:45:38) The Super Weight in Large Language Models (arxiv.org/abs/2411.07191)

• (00:55:42) Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task (arxiv.org/abs/2310.09336)

• (01:03:47) Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models (arxiv.org/abs/2411.04996)

• (01:08:14) Contextualized Evaluations: Taking the Guesswork Out of Language Model Evaluations (arxiv.org/abs/2411.07237)

• Policy & Safety

• (01:11:14) The Code of Practice for general-purpose AI offers a unique opportunity for the EU (euractiv.com/section/tech/opinion/the-code-of-…)

• (01:15:38) Three Sketches of ASL-4 Safety Case Components (alignment.anthropic.com/2024/safety-cases)

• (01:23:05) U.S Department of Commerce finalizes $6.6 billion CHIPS Act funding for TSMC Fab 21 Arizona site (tomshardware.com/tech-industry/u-s-department-…) , TSMC cannot make 2nm chips abroad now: MOEA (taipeitimes.com/News/biz/archives/2024/11/08/2…)

• (01:26:21) OpenAI to present plans for U.S. AI strategy and an alliance to compete with China (cnbc.com/2024/11/13/openai-to-present-plans-fo…)

• (01:30:42) OpenAI loses another lead safety researcher, Lilian Weng (techcrunch.com/2024/11/08/openai-loses-another…)

• (01:33:00) Outro

See Privacy Policy at art19.com/privacy and California Privacy Notice at art19.com/privacy#do-not-sell-my-info.