AB Testing Case Study | Google and Spotify Data Scientists

AB Testing Case Study | Google and Spotify Data Scientists | Data Science Interview

Рет қаралды 29,672

Күн бұрын

Пікірлер: 24

@Bethechange766 2 жыл бұрын

Hi, Please share the remaining part 4. Experiment Design (MDE, Power, Alpha, Sample Size) 5. Running an Experiment (Ramp Up, Validation Checks) 6. Launch Decision (Decision Tree, Post-Launch)

@rafaBR 3 жыл бұрын

Playback Speed 1.5x recommended

@olhakyrychanska5254 10 ай бұрын

Incredible! Professional yappers, very useful for practicing corporate English✍️😁

@brietje123456 3 жыл бұрын

Not really sure why we need the "first session only" constraint? If you just sum/average per user you can also get results "promptly", right?

@flyyang001 2 жыл бұрын

I agree, the first session only would also be impacted by the novelty effect.

@garyboy7135 2 жыл бұрын

On 24:30 about not using CTR due to violation of iid, I think you’re missing some important info, would love your feedback: 1. If you’re aggregating CTR to per user across session, your unit are users and still iid, assuming users are independent 2. Even if measuring at session level, you can still use delta method to estimate variance where otherwise variance would be underestimated due to sessions are not iid

@kristiyanto 3 жыл бұрын

What's the purpose of having a guest speaker if you don't let her talk?

@squeeps33 3 жыл бұрын

Well they made the presentation together and she had quite a bit of engagement imo. He asked her to talk about the hypothesis statement and they had some solid dialogue... I guess the idea was that he would guide the presentation passively, and she'd give great insights/alternate. I learned quite a bit

@ankitbiswas8380 2 жыл бұрын

thats your whole take away from the video ?

@AlirezaAminian Жыл бұрын

You mentioned that CTR is not a good metric which is understandable, but how about CTP? In CTP, we are looking at unique page visits and unique clicks, therefore, if a user has multiple sessions and multiple visits, they will all be counted as one.

@xxd3030 3 жыл бұрын

Hey Dan, I really love your courses on KZbin and purchase the monthly subscription a week ago. Now I could not login due to Google's prevention: "Attackers tried to steal your information"

@samw2066 8 ай бұрын

Question on metric tested in the hypothesis test - avg. # of clicks per user. Why wouldn't it be avg revenue per user per day? I believe the user could click the "buy now" button and still exit, or they may buy but spend less money with the new button,etc....

@garyboy7135 11 ай бұрын

Use delta method to account for the repeated measures

@htf523 3 ай бұрын

User clicks in the following session is not a violation of IID… your analysis unit is ‘user’… if you are only measuring user clicks uplift in the first session, how could you confidently say that once launched the feature is actually increasing user clicks? This just doesn’t make sense…

@iqless7313 Жыл бұрын

A user would be way more likely to click a 'buy now' button for a cheap, insignificant item. I think this definitely needs to be controlled for in the experiment.