34 - AI Evaluations with Beth Barnes

How can we figure out if AIs are capable enough to pose a threat to humans? When should we make a big effort to mitigate risks of catastrophic AI misbehaviour? In this episode, I chat with Beth Barnes, founder of and head of research at METR, about these questions and more.
Patreon: patreon.com/axrpodcast
Ko-fi: ko-fi.com/axrpodcast
The transcript: axrp.net/episode/2024/07/28/episode-34-ai-eval…

Topics we discuss, and timestamps:
0:00:37 - What is METR?
0:02:44 - What is an "eval"?
0:14:42 - How good are evals?
0:37:25 - Are models showing their full capabilities?
0:53:25 - Evaluating alignment
1:01:38 - Existential safety methodology
1:12:13 - Threat models and capability buffers
1:38:25 - METR's policy work
1:48:19 - METR's relationships with labs
2:04:12 - Related research
2:10:02 - Roles at METR, and following METR's work

Links for METR:
METR: metr.org
METR Task Development Guide - Bounty: taskdev.metr.org/bounty
METR - Hiring: metr.org/hiring
Autonomy evaluation resources: metr.org/blog/2024-03-13-autonomy-evaluation-r…

Other links:
Update on ARC's recent eval efforts (contains GPT-4 taskrabbit captcha story) metr.org/blog/2023-03-18-update-on-recent-evals
Password-locked models: a stress case for capabilities evaluation: alignmentforum.org/posts/rZs6ddqNnW8LXuJqA/pas…
Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training: arxiv.org/abs/2401.05566
Untrusted smart models and trusted dumb models: alignmentforum.org/posts/LhxHcASQwpNa3mRNk/unt…
AI companies aren't really using external evaluators: lesswrong.com/posts/WjtnvndbsHxCnFNyc/ai-compa…
Nobody Knows How to Safety-Test AI (Time): time.com/6958868/artificial-intelligence-safet…
ChatGPT can talk, but OpenAI employees sure can't: vox.com/future-perfect/2024/5/17/24158478/open…
Leaked OpenAI documents reveal aggressive tactics toward former employees: vox.com/future-perfect/351132/openai-vested-eq…
Beth on her non-disparagement agreement with OpenAI: lesswrong.com/posts/yRWv5kkDD4YhzwRLq/non-disp…
Sam Altman's statement on OpenAI equity: x.com/sama/status/1791936857594581428

Episode art by Hamish Doodles: hamishdoodles.com