Sample Size Estimation in A/B Testing: Easy Explanation for Data Science Interviews

[@soylentpink7845]

Very succinct, but still formally correct and seen from a product oriented view. To achieve this, one needs a real thorough understanding of the matter. Thank you for your great videos.

[@emma_ding]

Glad you enjoyed it! Thank you for sharing.

[@david-olivierpham3364]

Hello, I wanted to thank you, I used some videos as inspiration for my preparation for my interviews as a Product Analyst at a big advertisement and search engine company, and it helped a lot :)

[@emma_ding]

Fantastic! I love to hear this! Please keep me in the loop when you land your dream job! I want to be the first to cheer for you!

[@david-olivierpham3364]

@@emma_ding Hi Emma :) I got the job 🤣! I recommended your videos to a friend who confirmed that your material is the best on KZbin!

[@grabngoinfo]

Great contents and very well explained! I watched your tutorials when preparing for my data science interviews last year and found them super helpful! Got job offers from a FANNG company and a pre-ipo hot startup 😀 I recommended your channel to every of my data scientist colleagues and friends who were looking for a job change, and they all find your channel informative too! Thank you for preparing the awesome content, Emma. I can tell that you put a lot of thinking, research, and effort into each video. Keep up the great work, Emma!

[@emma_ding]

This warms my heart to hear! Congratulations on your success! I would love to share your testimonial. Please connect with me, and let me know more about your new job!

[@yutongyang1667]

Emma, you are amazing! Thanks for sharing all these wonderful contents!

[@Sivaraj-c2005]

i think its emmas fake id ,haha

[@yilicnus]

Really appreciate all your content, very easy to digest, love it!

[@jasonsj]

I’m excited about this series! Thanks Emma

[@emma_ding]

Me too! I can't wait to share it with you!

[@namandoshi4478]

Thanks Emma, you are doing a great job, very helpful!

[@jinglili1905]

thank you Emma. In the real work, would you also consider "baseline conversion rate" into sample size calculation?

[@patrickk6233]

Looking forward to the series!

[@emma_ding]

Thank you Patrick!

[@saramou5262]

Hi, Super interesting content, love your videos, they're generally well thought out and very educational on how to approach data questions. You are talking here about sample size, I imagine it is where the experiment is split 50/50 between test and control. How about if we want to do a 80/20% split or 90/10% split. Do we use the same logic? What percentage of control vs test would be enough to detect stat sig?

[@emma_ding]

Great suggestion! Thanks for watching and asking such thoughtful questions.

[@sheetalborar6813]

sigma square is the variance of the metric or the difference of metric between control and treatment?

[@skvali3810]

I’m waiting for this series!. Emma

[@emma_ding]

Thanks, it's going to be amazing!

[@nitishgalat5483]

Thanks for sharing, your are Amazing Emma!!

[@emma_ding]

Thank you so much Nitish! That is very kind of you! Thanks for watching!

[@leassis91]

very good video! thanks for sharing

[@cindykhuu451]

Question: is this formula only good for a two-sided test since alpha is divided by 2?

[@jerrytai5343]

Question: How do we assume sigma in other cases? Why chose 20 for this particular test?

[@garyboy7135]

the delta in the formula is the absolute difference or relative difference?

[@omalm1782]

Hi, Emma. I guess this is the formula for comparing two means. Which is the formula for comparing two proportions? Many thanks

[@ericliu839]

I got this formula from ChatGPT: (2 * P*(1-P) * (Z 0.5 alpha + Z beta) ^ 2 ) / (P treatment - P control) ^ 2; basically you replace the variance sigma^2 by P*(1-P) where P is the average proportions between two groups. For the minimum detectable effect, we just take the square of the difference between two proportions. Hope it helps.

[@Sivaraj-c2005]

1 , nice to meet you

[@wongkitlongmarcus9310]

how about for ratio metric like CTR?