Davidad Dalrymple: Towards Provably Safe AI

Episode 137
I spoke with Davidad Dalrymple (x.com/davidad) about:
* His perspectives on AI risk
* ARIA (the UK’s Advanced Research and Invention Agency) and its Safeguarded AI Programme
Enjoy—and let me know what you think!
Davidad is a Programme Director at ARIA. He was most recently a Research Fellow in technical AI safety at Oxford. He co-invented the top-40 cryptocurrency Filecoin, led an international neuroscience collaboration, and was a senior software engineer at Twitter and multiple startups.
Find me on Twitter (twitter.com/spaniel_bashir) for updates on new episodes, and reach me at editor@thegradient.pub for feedback, ideas, guest suggestions.
Subscribe to The Gradient Podcast: Apple Podcasts (podcasts.apple.com/us/podcast/the-gradient-pod…) | Spotify (open.spotify.com/show/6onNcSqsP6hEEqmZ6TU2g8) | Pocket Casts (pca.st/itunes/1569777340) | RSS (api.substack.com/feed/podcast/265424/s/1354/pr…)Follow The Gradient on Twitter (twitter.com/gradientpub)
Outline:
* () Intro
* () Calibration and optimism about breakthroughs
* () Calibration and AGI timelines, effects of AGI on humanity
* () Davidad’s thoughts on the Orthogonality Thesis
* () Understanding how our current direction relates to AGI and breakthroughs
* () What Davidad thinks is needed for AGI
* () Extracting knowledge
* () Cyber-physical systems and modeling frameworks
* () Continuities between Davidad’s earlier work and ARIA
* () Path dependence in technology, race dynamics
* () More on Davidad’s perspective on what might go wrong with AGI
* () Vulnerable world, interconnectedness of computers and control
* () Formal verification and world modeling, Open Agency Architecture
* () The Semantic Sufficiency Hypothesis
* () Challenges for modeling
* () The Deontic Sufficiency Hypothesis and mathematical formalization
* () Oversimplification and quantitative knowledge
* () Collective deliberation in expressing values for AI
* () ARIA’s Safeguarded AI Programme
* () Anthropic’s ASL levels
* () Guaranteed Safe AI —
* () AI risk and (in)accurate world models
* () Levels of safety specifications for world models and verifiers — steps to achieve high safety
* () Davidad’s portfolio research approach and funding at ARIA
* () Earlier concerns about ARIA — Davidad’s perspective
* () Where to find more information on ARIA and the Safeguarded AI Programme
* () Outro
Links:
* Davidad’s Twitter (x.com/davidad)
* ARIA homepage (aria.org.uk)
* Safeguarded AI Programme (aria.org.uk/programme-safeguarded-ai)
* Papers
* Guaranteed Safe AI (arxiv.org/abs/2405.06624)
* Davidad’s Open Agency Architecture for Safe Transformative AI (alignmentforum.org/posts/pKSmEkSQJsCSTK6nH/an-…)
* Dioptics: a Common Generalization of Open Games and Gradient-Based Learners (research.protocol.ai/publications/dioptics-a-c…) (2019)
* Asynchronous Logic Automata (dspace.mit.edu/handle/1721.1/44914) (2008)

Get full access to The Gradient at thegradientpub.substack.com/subscribe?utm_medi…

Davidad Dalrymple: Towards Provably Safe AI

Kapitelklicken springt direkt hin

Shownotes