Episode 36: About That 'Dangerous Capabilities' Fanfiction (feat. Ali Alkhatib), June 24 2024

When is a research paper not a research paper? When a big tech company uses a preprint server as a means to dodge peer review -- in this case, of their wild speculations on the 'dangerous capabilities' of large language models. Ali Alkhatib joins Emily to explain why a recent Google DeepMind document (arxiv.org/abs/2403.13793) about the hunt for evidence that LLMs might intentionally deceive us was bad science, and yet is still influencing the public conversation about AI.
Ali Alkhatib (ali-alkhatib.com) is a computer scientist and former director of the University of San Francisco’s Center for Applied Data Ethics. His research focuses on human-computer interaction, and why our technological problems are really social – and why we should apply social science lenses to data work, algorithmic justice, and even the errors and reality distortions inherent in AI models.
References:
Google DeepMind paper-like object: Evaluating Frontier Models for Dangerous Capabilities (arxiv.org/abs/2403.13793)
Fresh AI Hell:
Hacker tool extracts all the data collected by Windows' 'Recall' AI (wired.com/story/total-recall-windows-recall-ai)
In NYC, ShotSpotter calls are 87 percent false alarms (bloomberg.com/news/articles/2024-06-20/nyc-sho…)
"AI" system to make callers sound less angry to call center workers (ft.com/content/2827e9b3-0f61-4c44-8395-1d9e32a…)
Anthropic's Claude Sonnet 3.5 evaluated for "graduate level reasoning" (linkedin.com/posts/anthropicresearch_introduci…)
OpenAI's Mira Murati says "AI" will have 'PhD-level' intelligence (x.com/Swilua/status/1804532224559108287)
OpenAI's Mira Murati also says AI will take some creative jobs, maybe they shouldn't have been there to start out with (x.com/tsarnick/status/1803920566761722166)

Check out future streams on Twitch (twitch.tv/dair_institute). Meanwhile, send us any AI Hell you see (thecon.ai/submit-fresh-ai-hell).
Find our book The AI Con here (bit.ly/3OZ9PNx), and MAIHT3k merch here (store.dair-institute.org/category/mystery-ai-h…).
Subscribe to our newsletter via Buttondown (bit.ly/456Q0Ln).
Follow us!
Emily
• Bluesky: emilymbender.bsky.social (bit.ly/3VpFk7H)
• Mastodon: dair-community.social/@EmilyMBender (bit.ly/3yJ6g9z)
Alex
• Bluesky: alexhanna.bsky.social (bit.ly/4c05L9h)
• Mastodon: dair-community.social/@alex (bit.ly/3X4poc1)
• Twitter: @alexhanna (bit.ly/3V6NuQM)
Music by Toby Menon.
Artwork by Naomi Pleasure-Park (naomipleasurepark.com).
Production by Ozzy Llinas Goodman.