LLM-as-a-Judge, Quotation Fidelity, and A/B Testing Models: AI Publishing at Scale

What happens when a data scientist builds a generative AI proof of concept — and it scales to 700,000 articles and 4 billion page views? Recorded live at AWS Summit London, Romain is joined by Lewis James, Senior Data Scientist at Reach PLC — the UK's largest commercial publisher with over 120 brands including the Mirror, the Express, and OK Magazine. Lewis shares the full journey from GPT-2 experiments to a production AI publishing platform called Launchpad that now assists with 20–30% of the portfolio's daily article output. We explore how the team earned journalist trust by focusing on mundane tasks first, how they built multi-model pipelines with quotation fidelity checks to avoid misquoting, and why working backwards from users — not pushing technology — drove adoption where others failed. The conversation covers the technical evolution from prompt engineering to fine-tuning, model distillation, and agentic workflows built with the Strands Agents SDK running on Amazon Bedrock AgentCore. Lewis also introduces the concept of 'vibe publishing' — giving journalists a chatbot interface with more creative freedom — and discusses how evaluation strategies differ when you're measuring editorial tonality versus factual accuracy. Whether you are building AI-assisted content pipelines, navigating enterprise AI adoption, or thinking about how to earn user trust for generative AI tools, this episode offers a rare look at what three years of production generative AI looks like at massive scale.With Lewis James (linkedin.com/in/lewisjames1), Senior Data Scientist at Reach PLC• Reach PLC — UK's Largest Commercial Publisher (reachplc.com)Amazon Bedrock AgentCore (aws.amazon.com/bedrock/agentcore)Strands Agents SDK — Open Source (github.com/strands-agents/sdk-python)Amazon Bedrock Model Distillation (docs.aws.amazon.com/bedrock/latest/userguide/m…)Amazon Bedrock LLM-as-a-Judge Evaluations (docs.aws.amazon.com/bedrock/latest/userguide/e…)The 4 Stages of Psychological Safety — Timothy R. Clark (leaderfactor.com/4-stages-of-psychological-saf…)Werner Vogels — The Renaissance Developer (re:Invent 2025) (youtube.com/watch?v=NHnMk1Yjf4M)