A deep dive on AI model distillation attacks

In this solo episode of Risky Business Features James Wilson explores how distillation techniques are both a legitimate way to train smaller models, as well as a way to steal model capabilities. It’s not just a problem for frontier labs! Any LLM-based product could have its competitive advantage stolen through these attacks.

James covers:

• High-level concept of distillation

• Why it matters including close/open-weight/open-source explanation

• Types of distillation and the prompts used

• The distillation pipeline end to end

• Distillation at scale and mitigation techniques

• Hardware resource constraints for distillation

Show notes

• Self-Instruct: Aligning Language Models with Self-Generated Instructions (aclanthology.org/2023.acl-long.754)

• Alpaca: A Strong, Replicable Instruction-Following Model (crfm.stanford.edu/2023/03/13/alpaca.html)

• Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality (lmsys.org/blog/2023-03-30-vicuna)

• Orca: Progressive Learning from Complex Explanation Traces of GPT-4 (arxiv.org/abs/2306.02707)

• Zephyr: Direct Distillation of LM Alignment (arxiv.org/abs/2310.16944)

• Stealing Part of a Production Language Model (arxiv.org/abs/2403.06634)

• Microsoft probes if DeepSeek-linked group improperly obtained OpenAI data, Bloomberg News reports (reuters.com/technology/microsoft-probing-if-de…)

• Detecting and preventing distillation attacks (anthropic.com/news/detecting-and-preventing-di…)