Marton Trencseni - Sun 19 September 2021 • Tagged with ab-testing, variance, stratification, cuped
I use toy Monte Carlo simulations to demonstrate 5 ways to reduce variance in A/B testing: increase sample size, move towards a more even split, reduce variance in the metric definition, stratification and CUPED.
Marton Trencseni - Sun 05 September 2021 • Tagged with ab-testing, cuped
In this final blog post about CUPED, I will address some questions about CUPED, such as, is correlation between "before" and "after" the same as seasonality?
Marton Trencseni - Sun 15 August 2021 • Tagged with ab-testing, cuped
I use Monte Carlo simulations of A/A tests to demonstrate how Data Scientists can incorrectly skew lift and p-values if they pick-and-choose between reporting traditional and CUPED results after the experiment has concluded.
Marton Trencseni - Sat 07 August 2021 • Tagged with ab-testing, cuped
I use Monte Carlo simulations of conversion A/B tests to demonstrate how CUPED reduces measurement variance in conversion experiments.
Marton Trencseni - Sat 31 July 2021 • Tagged with ab-testing, cuped
I use Monte Carlo simulations of A/B tests to demonstrate CUPED, a method to use historic "before" data to reduce the variance in the measurement of the treatment lift.