Ep. 034 - Engrams: How DeepSeek Offloads KV Cache to DRAM and SSD (Core Research) | Jordan Nanos, Cam Quilici, Alec Ibarra, Bryan Shan

The InferenceX team is back after two months. Jordan Nanos (@JordanNanos) is joined by Cam Quilici (@noslawextratost), Bryan Shan, and Alec Ibarra to discuss Engram and DRAM offloading, the AgentX benchmark and its profit calculator, TPU v7 on InferenceX, Vera Rubin vs Blackwell, fast tokens, and TileRT on NVIDIA GPUs.

0:00 Intro
0:31 Engrams
12:27 AgentX Benchmark
16:36 Profit Calculator
23:40 TPU v7
32:03 Vera Rubin
40:49 Fast Tokens
49:33 TileRT
54:00 Chip Startups
1:01:31 Wrap Up