Easy as ABC: A Quick Introduction to Bayesian A/B Testing in Python (Will Barker)

[@stupedigo]

This is actually the best bayesian ab testing intro!

[@matthewturner639]

True, this is solid and simple

[@olesiaaltynbaeva4132]

100% true

[@indiegrungefolkstuff1446]

Interpretation is important to clients who don't remember stats. This is why Bayesian can be so useful.

[@kougamishinya6566]

The advantage of Bayesian approach vs frequentist is that you don't have to waste time and money testing more than you need to reach the most effective solution. You update the distribution as you go along and eventually the less effective options are selected less frequently leaving you with the best performing option. In frequentist you have to test each option X number of times. Even if option A is performing badly after X/2 number of trials, you can't start selecting it less like happens in Bayesian automatically. So Bayesian is more efficient.

[@Stefan111189]

Very nice and brief introduction. The relative increase calculation based on Monte Carlo is something great that I haven't seen before.

[@kinwong3224]

Beside the AB testing, Will has a very nice and simple explanation of Bayes

[@somalkant6452]

Excellent explanation. Loved it

[@albusdd1225]

Very informative and an amazing session

[@fanyin9056]

Very nice tutorial. I guess my question is (5:16) - if eventually we are gonna link the posterior probability with p-value, why do we want to conduct Bayesian A/B test at the first place?

[@sarahwilde3501]

I found this really helpful! I haven't dove in too much to Bayesian inference yet, but this is the best intro i've seen since others are too complex to really get me started. one question I have tho....the way the 98% probability was likened to a p-value of 0.02....that's incorrect, right? they suggest similar things, but they are not the same. based on the speaker's comment at the end that he doesnt know frequentist approaches, i'm going to assume i'm right here....but i'd love other comments.

[@russellwarden2899]

You are correct. A P value is not a probability

[@arturleperoke3205]

@@russellwarden2899 hey guys.. I did not watch until the end, yet I skimmed through some comments.. I am fairly certain that we do can interpret the p-value as a probability. It is the probability to see a difference in the statistical summary between two groups as or more extreme under a specified statistical model(Wasserstein RL, Lazar NA. The ASA's Statement on p-Values: Context, Process, and Purpose. Am Stat. 2016;70(2):129-33.) Though his comparison of p-values to the 0.02 seems to be wacky.

[@TK-pe7sf]

Good job, very followable presentation!

[@estanislaomariaferrer9]

Loved it!!! Thanks for sharing this content

[@eightrice]

5:00 what does the y axis refer to? density of what?

[@sangramkapre]

posterior prob for metrics from variants A and B

[@BillusTinnus]

Great presentation !

[@leassis91]

can you explain futher why the parameters of neutral and skeptical were halved?

Where’s the code ?

[@brittosabu07]

Please provide the source code