Smooth scaling: System design for high traffic
Smooth scaling: System design for high traffic
Podcast Description
Smooth Scaling: System Design for High Traffic focuses on all things scalability, reliability, and performance. Tune in for expert advice on how to scale systems, control costs, boost availability, optimize performance, and get the most out of your tech stack.
Host Jose Quaresma is the VP of Technical Engagement at Queue-it, working on the frontlines with some of the world’s biggest businesses on their busiest days, from Ticketmaster to Zalando to Home Office U.K. He’ll be joined by experts across industries, uncovering how major organizations design, build, and deploy systems that remain reliable at scale.
Podcast Insights
Content Themes
The podcast centers on scalability, reliability, and performance of systems. Topics include design principles for failure, simplicity versus complexity in architecture, and real-world examples from major organizations like Ticketmaster and Zalando. Episodes like 'Simple is Scalable' with Mojtaba Saroonghi delve into maintaining simplicity to enhance scalability, while 'Design for Failure' with Martin Larsen covers architectural approaches that increase resilience.
Smooth Scaling: System Design for High Traffic focuses on all things scalability, reliability, and performance. Tune in for expert advice on how to scale systems, control costs, boost availability, optimize performance, and get the most out of your tech stack.
Host Jose Quaresma is the VP Customer Experience & Solutions at Queue-it, working on the frontlines with some of the world’s biggest businesses on their busiest days, from Ticketmaster to Zalando to Home Office U.K. He’ll be joined by experts across industries, uncovering how major organizations design, build, and deploy systems that remain reliable at scale.
Suman Karumuri has spent 17+ years building observability systems at Amazon, Twitter, Pinterest, Slack, and Airbnb. He was tech lead for Zipkin at Twitter, co-authored the OpenTracing specification that fed into OpenTelemetry, and has now replaced Elasticsearch twice at high-traffic platforms. In this episode of the Smooth Scaling Podcast, Suman walks host Jose Quaresma through KalDB, the open-source, cloud-native log search engine he is building into a product, which runs at petabyte scale at Slack and Airbnb. They get into why he keeps rewriting Elasticsearch instead of tuning it, how separating compute from storage on S3 changes what a log system can do, and the recovery-task trick that keeps fresh logs visible when volume spikes 10x. The back half turns to agentic AI: why agents querying logs in unpredictable bursts break traditional log stacks, why you can’t sample data anymore, and what engineering leaders should measure before the bill explodes. A concrete, in-the-weeds look at running log search when the primary user is no longer human.
—
- (00:00) – Intro
- (01:04) – The origin story of KalDB
- (05:45) – Why keep the Elasticsearch API
- (07:09) – Separating compute from storage
- (11:07) – Solving the 12-hour log lag
- (14:02) – The journey of a single log message
- (15:56) – What makes KalDB unique
- (18:17) – Why not just use ClickHouse?
- (22:12) – The polystore: search and analytics together
- (23:38) – Native support for traces
- (24:27) – How agents change observability
- (31:24) – Chat as the UI, and what breaks first
- (35:11) – What engineering leaders should do now
- (39:54) – Build for a burning problem
- (41:15) – Rapid fire: Acquired, and “scalability is…”
Suman Karumuri is the founder of KalDB, the open-source cloud-native log search engine he is now building into a product. KalDB runs at petabyte scale at Slack, Salesforce, and Airbnb. He spent 17+ years building observability systems at Amazon, Twitter, Pinterest, Slack, and Airbnb, was tech lead for Zipkin at Twitter, and co-authored the OpenTracing specification under the CNCF, which became the foundation of OpenTelemetry. He is based in San Francisco.
Suman has replaced Elasticsearch twice at high-traffic platforms. And with the rise of Agentic AI querying log systems in non-deterministic bursts, this places a whole new kind of stress on those systems to perform at high scale.
Suman Karumuri: https://www.linkedin.com/in/mansu/
Host José Quaresma: https://www.linkedin.com/in/jose-quaresma/
This podcast is produced and researched by Perseu Mandillo, and brought to you by Queue-it, your virtual waiting room partner.
© Queue-it, 2026
Disclaimer
This podcast’s information is provided for general reference and was obtained from publicly accessible sources. The Podcast Collaborative neither produces nor verifies the content, accuracy, or suitability of this podcast. Views and opinions belong solely to the podcast creators and guests.
For a complete disclaimer, please see our Full Disclaimer on the archive page. The Podcast Collaborative bears no responsibility for the podcast’s themes, language, or overall content. Listener discretion is advised. Read our Terms of Use and Privacy Policy for more details.