Crack Metric/Business Case/Product Sense Problems for Data Scientists | Data Science Interviews

In this video, I'll share three frameworks that I used to answer product sense/business case/metrics questions that help me get job offers from Airbnb, Lyft and Twitter.
3 Categories of Questions & Frameworks:
- Diagnose a problem
- Measure success
- Launch or not
🔗 Slides www.emmading.com/blog/the-ult...
🟢Get all my free data science interview resources
www.emmading.com/resources
🟡 Product Case Interview Cheatsheet www.emmading.com/product-case...
🟠 Statistics Interview Cheatsheet www.emmading.com/statistics-i...
🟣 Behavioral Interview Cheatsheet www.emmading.com/behavioral-i...
🔵 Data Science Resume Checklist www.emmading.com/data-science...
✅ We work with Experienced Data Scientists to help them land their next dream jobs. Apply now: www.emmading.com/coaching
Lyft/Uber Metric Interview Question and Answer
• Lyft/Uber Metric Inter...
Facebook Metric Interview Question and Answer
• Cracking Facebook (Met...
LinkedIn Metric Interview Question and Answer
• LinkedIn Metric Interv...
// Comment
Got any questions? Something to add?
Write a comment below to chat.
// Let's connect on LinkedIn:
/ emmading001
====================
Contents of this video:
====================
0:00 Intro
1:22 What are product sense problems
2:54 When you don't have any experience
3:28 What are companies looking for
4:17 Diagnose a problem
8:01 Measure success
10:52 Launch or not
13:42 Tips to ace the interview
15:08 Outro

[@emma_ding]

If you prefer to read rather than watch this video, check out the blog post I wrote on this, and other, topics here ➡ www.emmading.com/resources

[@leigao5727]

I just wanted to say your videos have really helped me pass my Data Engineer interview at Facebook recently. Thank you for all the great content!

[@emma_ding]

Congrats! Best of luck with your new job!!

[@daradadachanji5670]

@@emma_ding Facebook now actually recommends this video to Data Engineer candidates. Really great video!

[@rajatgupta9713]

underrated channel. most of the candidates do excellent in the technical round and statistics round but fail miserably in the case study/product round. These videos are spot on and fill that gap perfectly.

[@sinnottboy]

Amazingly helpful video and content! I was able to easily move onto the onsite round because of the great content you’ve provided on this channel!

[@user-xy6ru7jl7k]

This is the best video I have ever watched for the product & metric interview preparation! Thank you so much!

[@dunjianxiao4105]

First ever video that motivates me to play again and again while making notes carefully. Thank you so much Emma!

[@dougmax120]

Great content. I'm only on video two but I love the breadth and depth or the content with specific examples and real interview questions. I appreciate all the hard work you went through to provide a simple, easy to follow, concise video but I am sure it took hours of reading and note taking. Found you through KD nuggets.

[@yonahgraphics]

Wow! This video has really helped me. I was preparing for an IBM data science interview. Thank you a lot. I also do appreciate that you used real-world questions, which were asked by big names. Other people usually just make up their own. Thanks a lot.

[@takozhangsbox3414]

非常有用，解开了我之前面试时候遇到的好多困惑，期待更多！

[@erza4876]

Best video ever on product sense, saved ton of time to go through other articles/ videos...

[@franchiniitalo]

Amazing Job! You are helping so many ppl landing their dream job! Thank you for your work!!

[@weifengwu5675]

每期都质量很高，学到了很多，非常感谢！

[@utkarshnigam9146]

Video content is structured, comprehensive, and practical.

[@borngenius4810]

To say that this has been helpful is an understatement. Thanks!

[@ruiliangliu7913]

Comprehensive and well-structured way to answer product sense questions. Emma you deserve a big thumb up!

[@emma_ding]

Glad you find it helpful, thanks so much for watching!

[@guanhangchen1852]

Really like the structure and tips in this video and I'll try using these framework to prepare for my next interview. Thank you!

[@aakashdusane]

I've been looking everywhere for metric and product sense DS questions. Thank you! Hopefully these will help for the LinkedIn DS interview

[@shelllu6888]

A flawless framework and extremely applicable

[@collegebgi8167]

谢谢emma老师！快面试了祝我成功～

[@ramamuge3380]

Loved the breakdown...very simple and comprehensible.

[@yournameormyname8708]

I’ve watched almost all of your videos and I learned sooo much. Thanks for sharing this information for free! You are very talented at describing complex problems with very few words and give the exact right examples. You could be the most popular instructor at any universities.

[@emma_ding]

Thank you for your kind words! I'm glad that you're learning a lot from my videos! :)

[@halflearned2190]

This was very informative, thanks!

[@fobaogunkeye3551]

Thanks for the hidden gems Emma!

[@annicai4757]

This is super helpful, thank you Emma!

[@lebronx9516]

Amazing content as always!

[@leavonfletcher4197]

Excellent job. Greatly appreciated. Right now I am doing the University of Texas Certificate in Artificial Intelligence and Machine Learning. Hopefully, this will prepare me for interviews after I have completed it. Thanks again.

[@qiwang5]

This is underrated channel, so many awesome videos!

[@rachitsingh3299]

Subscribed within 5 minutes in the video. Very helpful

[@rewiringafrica7794]

This is the most helpful framework I have seen regarding product sense questions. Thanks a lot. Please do more of these

[@emma_ding]

More to come!

[@Squiderify]

This is pure gold.

[@niranjan08538]

Simply amazing explanation.

[@nancywei6998]

Thank you for posting this very useful video! Cheers

[@vlkhvan]

Thank you a lot! This is an awesome outline

[@lexichen4131]

This is so helpful thank you!!!

[@cql8878]

This is one of the best product sense videos

[@adamyatripathi2743]

This is amazing!!! Thanks a lot!

[@Bookerer]

This video helped me to pass the DS interview at TikTok! Thank you so much emma :)

[@emma_ding]

Great news Bookerer! Wish you every success at TikTok! Please feel welcome to email me to share more details! This kind of news really makes my day!

[@Bookerer]

​@@emma_ding The problem they gave during the interview fell exactly into the "Diagnose a problem" category and they were quite impressed with my response even though I was merely following your framework :) understanding how commonly-used metrics can lead to success was also really important!

[@alpapaca9332]

Much better than most DS interview videos. You deserve more subscribers!!

[@emma_ding]

Wow, thanks!

[@nikhilniket]

Well explained !! Loved the framework. Thanks

[@emma_ding]

You are welcome!

[@Mohib283]

Thanks for making this video. Very helpful to me.

[@shikharjoshi6349]

Giving reference to those interviews questions is really helpful. Also learned some new keywords. Thanks 👍

[@emma_ding]

Great to hear!

[@PeterHontaru]

This was such a great video, thank you very much!

[@poopah4497]

I am taking notes for all your videos.

[@aeoliaasa3634]

best channel of DS

[@yin8180]

感谢分享！最近在准备面试，很有帮助～

[@khizarsultan3316]

Thank you for sharing amazing content.

[@yudhiesh1997]

Emma thank you so much for this video. It helped me answer the Product question during a DS interview and I managed to secure the job.

[@emma_ding]

Glad my videos are helpful. Congrats!

[@andrioktavianto7604]

Honestly, this video is really helpful for me! Thanks (again) for sharing this, Emma. A step-by-step video on how to solve a business problem using data science with real industry case would be intresting

[@emma_ding]

Will do!

[@user-hl7ii5uh6s]

It's really awesome video! Thank you so much!

[@padmaswethak8547]

well organized information, thank you

[@oliveryuan1483]

Love the content. I think a case walk-through video would be even more helpful. Keep up the good work!

[@mr_jacklu8014]

Yes! Please and thank you kindly.

[@emma_ding]

Stay tuned. More to come :)

[@wangwendy8690]

I just get asked the "how to measure success" question in my interview! This is very helpful, thank you for creating the amazing content!

[@emma_ding]

You're so welcome! :D

[@PATRICKCHUAD]

This will help in future data science interview. Thanks

[@nidhiathawale391]

Hello! Thank you for such a helpful video. Made a lot of sense to me. I was wondering where I can learn more about the different types of business metrics? Like conversion rate, etc. I come from a Statistics background and I am pretty unfamiliar with these metrics. Would help me a ton if you could suggest some resources to me. Thanks!

[@chilukanand]

Awesome video, thanks!

[@moutaouakkilyoussef5825]

Thanks a looot for this video! Got the job thanks to it :D I was struggling with business case studies especially in consulting firms interviews but this video made me so, so much better at it.

[@emma_ding]

This is awesome to hear, Moutaouakkil! Congratulations on securing your new role! Feel free to share some more details with me if you'd like; what's the position and company?! So excited for you, and happy to have been a small part of your journey. 🥳

[@luckykukreja95]

Heyy! Really helpful, as these topics aren't covered by anyone properly. Thanks! And Keep making such videos ;)

[@alexz7032]

I hope in the future you could release some walk-through videos targeting at real product sense questions, and discuss the possible answers using the framework. But this is already great content and very helpful! Thanks Emma

[@weiyangshi4729]

Totally agree! More walk-through videos please!

[@emma_ding]

Stay tuned! More to come!

[@ronzhang9500]

感谢分享，最近正在准备Product sense的面试

[@emma_ding]

谢谢观看 😄

[@harish00784]

This is the best video regarding data science. Thanks Emma mam. I'm following you right from towardsdatascience article.

[@emma_ding]

Most welcome 😊

[@siddharthvij9087]

Excellent video

[@yalili1600]

Best video ever!! Thanks, Emma. Could you please make an example of the "measure success" problem following your framework?

[@emma_ding]

Check this one out kzbin.info/www/bejne/gJukZ5usosZ-bQ where I share how to measure success for a LinkedIn interview question. Let me know if it helps :)

[@marciama9917]

Hi Emma, thank you so much for the video! I have some questions about some of the steps you mentioned: 1. I was wondering what was the website in the video that has those sample case questions? 2. For the first category of questions which is diagnose a problem, the second step was analyzing the internal and external cause based on if the change happened suddenly or progressively. I feel like the examples for internal and external causes would lead to sudden change. What would be some external and internal reason that would lead to progressive changes? 3. What are the differences between the second category and the third category, which are measure success and launch or not? I feel like for launch or not type of question, we do need to measure the success first and then decide to launch it or not. So my understanding is that launch or not has the extra final step then measure success.Please correct me if my understanding is wrong. Thanks Emma!

[@youqingzhou9037]

Awesome!

[@TJ-wo1xt]

great video, thanks.

[@yilinyang7560]

This is an awesome video! I also like cracking DS case series a lot! I have a question. When I had onsite interview with some tech company, sometimes the interviewer will ask, "how will you collect the data to measure this?" or "What data will use to investigate some issue/measure the impact" all of these applied data questions. How should I ace these types of questions?

[@jonsings]

Great video!

[@yashovardhan9841]

I would tweak the framework for "Diagnose a Problem" a little bit. I believe decomposing the metric is very important, specially when it is a ratio. It helps to identify if the numerator went down or if the denominator went up. The framework I like to follow is - 1. Clarify 2. Time 3. Decompose the metric 4. Segment 5. New features 6. External influence 7. Summarize

[@jieyuwang5120]

Great Video!

[@tawiwutcharuwat8999]

Thank you!

[@yida]

Very helpful!

[@emma_ding]

So glad!

[@ansarbutt7665]

Excellent content - just subscribed. You mentioned you went through a bunch of materials yourself - is there any content in particular you would recommend? Such as certain books, websites, videos etc Thanks again for putting this out! Looking forward to your future content

[@emma_ding]

There are a lot of materials I benefited from and I listed them in my blog post towardsdatascience.com/how-i-got-4-data-science-offers-and-doubled-my-income-2-months-after-being-laid-off-b3b6d2de6938. Hope it helps!

[@cool_videos6016]

you are the best

[@ananthapadmanabhanss4553]

Thanks it's useful :)

[@iy3987]

Thank you for sharing the framework. I wonder if there's an option for incorperating experimentation testing into the second "measure success" question? And when we don't want to incorporate experiment testing into the second part if it's an option?

[@ilikefrenchtos5401]

I absolutely love your videos (thank you for making them!), but in case you were not already told, your slide at 8:38 says "calcification" , I think you meant "clarification", hope this helps!

[@MrArijeet_]

Thanks a lot for the wonderful content. The framework is really well structured and I am sure it will help me prepare for my interview. But I was curious to know, what sort of questions should I start practicing. Since I am pretty sure most of us will be applying for different roles/industries, so it can seem a bit overwhelming and hard to decide from where we should start and what kind of questions should we focus upon.

[@emma_ding]

You can check out this video for requirements of different roles. kzbin.info/www/bejne/omqWmYmQj5yko5I

[@haoranqiu7217]

Hi Emma, I've watched all your videos related to product sense problems including the videos of the real interview questions. They are all so helpful! There seems no sample question for Launch or Not problems though. Could you please also make a video on this type of question and give a sample answer? Like the LinkedIn, Lyft and Facebook ones. Thank you so much!

[@emma_ding]

For sure! It's on my list but I did not have time to make it yet, stay tuned!

[@rajatgupta9713]

on the spot

[@yuhan7320]

总结的很好，谢谢分享

[@emma_ding]

谢谢观看 😄

[@emma_ding]

Hey guys, I'm so sorry for the typos in the slides, below are the corrections. In section "Measure success": calcification (clarification), sutable (suitable), Cancelation (Cancellation)

[@sakuralin2056]

Hey Emma, ty for the content. The video is so helpful that these small mistakes don't matter!

[@charleschen5194]

很清楚 能让我边学边记笔记的视频不多啊哈哈

[@emma_ding]

感谢观看 😄

[@tinniwu]

Would you consider to go through a real interview question and answers for "Launch or not" category? The videos for diagnose a problem, measure success, and improve a product are very helpful!! If the interviewer is a Data Scientist, should we emphasize more or lead/expand the answers to AB test or ML modeling? How to do it? It seems like most categories can be expand to AB testing, but less to ML modeling.

[@rashmikulkarni9102]

Hi Emma - First of you all THANK YOU for the amazing content. I do have a question for you. How to answer questions like what should be the KZbin daily view time? How to answer such questions?

[@mijikim7115]

Excellent video. Just sharing the next video idea(1): Several videos of... you are using/modifying your frameworks to answer the actual interview questions? (2) mock interviews video.

[@mijikim7115]

Because while product sense questions can be found easily, those sample *good* answers are hard to see!

[@emma_ding]

Thanks for the suggestions! You may find this playlist helpful kzbin.info/aero/PLY1Fi4XflWSvtu963rZpfH6WeX54vSrDW. It contains answers to real interview questions.

[@dongqingxiao4523]

Hi Emma, I love your video so much and it's really helpful. I have a quick question to ask: For the launch or not questions, I'm wondering if the engineering team has already finished developing the feature for you to do the experiment, why not just launch the feature on the product directly? Also, if I have a question like "I have 2 features in the backlog, can you help me decide which one should go first", how to think about a question like this? Thank you so much!

[@emma_ding]

1. There are many factors influencing whether a feature should be launched or even rolled back after being launched. In fact, "launching" is a complicated process. This is a great question to ask the interviewer at the end of the interview. 2. This is similar to the "measure success" problem. You can refer to this video kzbin.info/www/bejne/gJukZ5usosZ-bas

[@juliar5741]

Thanks for this post! So are you focused on analytical type of ds positions? Since you have listed different kinds of data positions, how did you decide to focus on any one of them? Have you also interviewed for ML focused roles?

[@emma_ding]

I'm working on covering more aspects of DS interviews. Stay tuned!

[@xiaoqianzhang1934]

This video is very helpful! Thanks, Emma! When we look at other features launched at the same company, is there any way to narrow down the features we need to look into? Since for big companies like Facebook, it might have hundreds of features changed within one month.

[@emma_ding]

The interview time is short, you can give 1 or 2 examples of features and have good reasons to justify why they would affect the current product. You will be evaluated on the thought process and structure of your answers instead of knowing the exhaustive list or some particular features.

[@junyanyao6977]

Hi Emma, can you please help talk some products questions related to Opportunity sizing and Enter new market or not

[@emma_ding]

Love this idea! Will consider it in future videos!

[@tingxu3365]

Thank you for the amazing framework! I think there is a typo in the slide Framework(launch or not), 3. recommendation based on experiments result. I think it is "conversion rate" instead of "conversation rate".

[@emma_ding]

You are absolutely right! Thanks for the feedback, and I'm glad you enjoyed the video!

[@nidhisanghavi2165]

Thank you for your clarifying videos! I never could understand where to even begin, but now I have some structure. Can you suggest any books or resources for specific questions that I can use for practice?

[@emma_ding]

Check out my latest masterclass here. www.datainterviewpro.com/services

[@yourkarma7012]

Need some more insights on launch or not framework. Any good articles or books on it would be helpful. How can we test the efficiency of the new feature before launching it? Once we launch we could use success and guardrail metric as you suggested.

[@emma_ding]

Typically, A/B testing is used to measure the effectiveness of a new feature. You may want to check out this video kzbin.info/www/bejne/jmnYZ56oacurjsU. I recommend two resources at the end of the video.

[@rubyjiang8836]

Hello~ i love your videos so much~ i'm wondering what is the website that you got these interview questions?

[@emma_ding]

Those are from glassdoor.

[@rongli9875]

Hi Emma~ Thank you so so so much for providing these free high-quality content for us! And I want to know what's the website you used to find the interview questions shown in the video! Thank you in advance.

[@songxiyou2347]

Hey Emma! First of all, thank you for this video! It's very clear and well explained, so helpful! -- Change to Chinese mode -- 感谢视频，最近在准备产品题，反复观看您的视频，觉得很有收获。但是因为没有任何case study实战经验，也真的特别期待您出一期例题实战回答或者mock interview+点评，（e.g. diagnose problem，why engagement is down）。我也看过一期不错的PM mock interview，也是类似的问题。但是非常好奇，一个DS回答此题的角度，跟PM应该有何不同呢？DS是否应该更侧重metric本身？更用数据说话？我看PM回答问题的framework也是clarify - time - segmentation等等。那期mock interview最后的solution就是发现其他team launch experiment/feature影响到了engagement,于是跟其他team讨论要不要继续。好奇从DS的角度，要如何侧重给一个solution呢？

[@emma_ding]

请问你可以分享下你看到的PM mock interview的视频吗？关于DS是否跟PM的答案一样 - 这个要看具体的问题了。PM跟DS都应该重视metric都应该用数据说话，这个好像不是区分的关键，而且我个人的经验是很多这一轮面试就是PM来面的。能从PM的角度回答问题也是充分说明一个DS的产品知识很扎实啊。但是有些公司的面试会把产品跟experimentation结合起来，一些问题会问到A/B testing的细节，这个就是DS的专长了。

[@songxiyou2347]

@@emma_ding 谢谢回答，原来如此。kzbin.info/www/bejne/f3Wbiod8ZpVjd5I 这是我之前看到的视频，也是回答dignaose类型的。另外我也确实看到一些资料或者公司面试题要求从产品框架延伸到experimentation，这部分的mock interview见到的不多，感觉有些难度。我自己到定义metric之后，就答不出了。

[@nikhil_joy]

Good job👍Thank you so much. Can you suggest some helpful tutorials for learning this? Am a beginner with an eye on this field.

[@emma_ding]

Thanks for the suggestion. I'm working hard on more contents! Stay tuned!

[@zhefeijin9627]

Hi Emma, really like your videos! I have a question about your last slide, in which you mentioned 'write down thought process with bullet points'. I am wondering how can I do that? should I propose to the interviewer that 'can I share one doc with you to write down my thought process ?' Thanks a lot, I believe it would be really helpful for my interview with FB next week.

[@emma_ding]

That pullet point is for (pre-COVID) onsite interviews that you can use the white board in the room. For possibility of sharing a doc or other ways of presenting, I would recommend communicating with the FB recruiter. If it's not possible to share any docs, the best way is to verbally communicate your framework or thought process, say "This problem we have 3 things to consider. First.. Second.. Third...". Good luck to your interview!

[@zhefeijin9627]

@@emma_ding Thanks so much for replying to me! That's a good strategy.

[@emmama1598]

Hi Emma, I had an interview ask if a feature can improve retention of their product. I followed the framework to ask interviwer to clarify the senario, but the interviewer was very impatient to clarify and couldn't wait to ask me how to use history data. I don't have chance to define the metrics (or follow other steps in the framework). I was very confused.

[@takosuzuki5685]

I don't know if it's too late but I read this book called "Lean Analytics". It categorizes different business models (E-commerce, two-sided market, SaaS) and what metrics to use at each stage of the business

[@lingli8999]

小姐姐，想问一个关于measure success 的问题。1）launch 之后就不能做ab test 了吧？ 2） 如果不能做ab test, define 的metrics 要和谁比呢？ 整个ecosystem 的metrics 比如DAU 什么的可能还有baseline，像feature specific metrics, 比如新feature 的time spent, adoption rate 以什么为baseline 呢？还是只用整个ecosystem 的metrics 就可以了？

[@seant7907]

Very helpful. I was once asked a question like this, "data analysis result varies depending on the person who analyze it. If I ask a group to analyze a data, how can I know who has the most 'correct' analysis result?". Can you or anybody who sees this post give me a good brief answer, link, or video regarding the question?

[@emma_ding]

Hmm.. It's hard to tell what the interviewer is looking for.