In this special second episode, the roles reverse as I share some stories from my extensive experimentation background across tech giants like eBay, Apple, and Yahoo. This is a cut from my first episode with Ishan where Ishan asked me about my experience.

Discover the statistical challenges that plagued major experimentation platforms – from misaligned alpha thresholds and rampant false positives to the complexities of handling extremely skewed revenue metrics. Learn how I rebuilt trust in A/A testing through blind studies, applied multiple comparison corrections, and navigated the challenges of KPI engineering for metrics with distributions spanning from $1 to $8,000. This candid behind-the-scenes look into real-world experimentation reveals why understanding statistical concepts is crucial for accurate decision-making in tech. Perfect for data scientists, analysts, and product managers working with experiments and causal inference at scale.