Search engine bots crawled so AI bots could run

Ryan hosts Akamai data scientist Robert Lester on the show to discuss how the growth of AI bots affects internet traffic, the ways these AI bots differ from the original search engine optimization ones, and why you might not want to mitigate AI bots on your websites.
Episode notes:
Akamai (akamai.com) is a CDN, full-stack cloud computing, and cybersecurity company that keeps experiences closer to users and threats further away using the world’s most distributed compute platform.
Connect with Robert on LinkedIn (linkedin.com/in/robert-lester-4a50321a7) and check out his AI Pulse blogs (akamai.com/blog?author=robert-lester).
Today’s shoutout goes to user Evan Phoenix (stackoverflow.com/users/643905/evan-phoenix) for winning a Populist badge for their answer to llvm ir back to human-readable source language? (stackoverflow.com/questions/5180914/llvm-ir-ba…).
TRANSCRIPT (stackoverflow.blog/2026/01/06/search-engine-bo…)
See Privacy Policy at art19.com/privacy and California Privacy Notice at art19.com/privacy#do-not-sell-my-info.