Ace A/B Testing Interview Question: A Data-driven Approach for Data Scientists

[@emma_ding]

If you prefer to read rather than watch this video, check out the blog post I wrote on this, and other, topics here ➡ www.emmading.com/blog/acing-a-b-testing-interview-questions-with-no-experience

[@杨玄-v4k]

The total length of the video is 10 minutes, but it's not until the fourth minute that it starts to cut to business...

[@pikakac2441]

Thanks for the video Emma. As someone who’s learned AB testing on the side (thanks to your video and some courses I took) but haven’t done it in a job, I’ve encountered job applications that asked if I had real-world A/B testing experience. I believe answering no would lead to automatic rejection. How should we deal with this type of questions?

[@brunovieira9512]

I get the point that anyone can learn A/B test stuff by themselves, but It is such an awkward situation when the interviewer asks to tell some real situation and you just can't because you have never come across this problem. what do you recommend in these situations? Is it something that people can add to their portfolio with a toy problem? What do you think?

[@quishzhu]

Actually A/B test can be everywhere from your previous experience but people just don't call it A/B test.

[@brunovieira9512]

@@quishzhu yes, I know It. But It really depends. In the First two years as a data scientist, in two of the biggest fintechs of Brazil, I did not have the chance to perform an A/B test properly. Not every Company provides the proper infraestructure to do a/b tests for example.

[@ElvisSCL]

Your videos are amazingly clear on everything!!!!!

[@EmilyHoughKovacs]

What is the link to the Data Science Interview 2023: How To Prepare When Time Is Tight video explain data-driven methods? I can't seem to find the exact video on KZbin. Thanks.

[@JJZhou-e7l]

Hi, Emma, thanks a lot for sharing the information, as i observed you categorized your videos into different playlist, for example, this one under A/B testing, inside the playlist, is there any order we need to follow to better gain the insights?

[@yuanzhang_mia]

Hi, I like this video! But the A/B Testing Cheatsheet link doesn't work. Could you please update the links, thanks!

[@arjun-bali]

Great video Emma :)

[@operagreen7581]

想给emma提醒一下小红书上有人把你总结的材料到处发 不知道算不算侵权 ‘jasper老师-ds求职’

[@dikshitapadte2809]

I am not able to get access to the cheatsheet after visiting the website. Can you please guide me how to do that?

[@mandeepgulati82]

It feels more like a promotional video for the channel. no real content

[@lenka4662]

hi Emma关注你有一段时间了，也看过你的个人网站 请问你的目标人群是留学生还是非留学生也可以。我在国内想直接申请国外的数据科学家，请问可以提供帮助吗？没有留学经历，谢谢！

[@rudroroy1054]

A/B testing seams like a resource consuming testing. Are there any alternatives? maybe something more automated?

[@jimbocho660]

Propensity score matching. Difference in Differences, discontinuity regression etc but these are observational methods and are less reliable than A/B testing which is experimental.

[@austinhoag5130]

If we are A/B testing a referral program, like you suggested, and the metric is the rate at which new users sign up, the idea of a control group is somewhat confusing to me. If the treatment group is some users to whom we give the ability to refer users, then the control group must be a group of users to whom we do not give the ability to refer. Let's say we run the test for 2 weeks and measure a rate of X new user sign ups per hour with the referral program active, we could just compare that to the baseline new user sign ups per hour before we ran the test. But this does not use the control group at all. I don't see how to turn this scenario into an A/B test.

[@momoko7537]

I think there are 3 points why we use A/B test here: 1. It's better not to deploy a new system to all users, there might be problems(bad route design, technique problems, server overloads and so on), A/B test allows us to make a small scale users to test all the functions and to determine how many users will utilize the referral system; 2. This referral system may do bad things to current users. For example, they may find the referral system hard to use and eventually leave the whole application. It is what we want to compare with the control group; 3. For the new users, what we usually do is tag them as the refferred ones and track their behavior for a relatively long time, in order to know if they are going to turn to long-term users or just use the application for once or two. If we finally find that the benefits (the profit made from new users or something like this) outweight the costs (bad things from the treatment group, the extra cost in running the new system, and so on), the new referral system can be considered successful.

[@Kheekhee_khakha]

Really bad video. Keeps iterating the same thing.