Netflix Product Manager Interview: Inactive Users

[@tryexponent]

Don't leave your product management career to chance. Sign up for Exponent's PM interview course today: bit.ly/3JVhkRr

[@scriptkiddie6151]

He's directly looking into our souls lmao

[@jeetjain579]

Hahaha felt the same

[@kushagrakukreja9722]

You blink you lose the role🤣

[@tommynguyen6399]

@@kushagrakukreja9722 LMAOOO

[@rajesh9206]

XD

[@petrbelousov1441]

@@kushagrakukreja9722 lol

[@rawkindepic]

1. Whether this inactivity has been a trend or a sudden drop? 2. Was it web vs mobile specific? 3. Was it android vs ios specific? 4. Was it category-specific - TV series vs movies? 5. Any demographic specific? 6. User journey- from login/signup to discovery to actually clicking the content? Is there a drop in those parts of the funnel? Where are all these questions? People getting bored doesn’t cause a sudden drop. And if it’s a trend, the period of the trend is important to figure out.

[@lukexuanliu]

Dude does not blink.

[@mahesh26sai]

I sometime think the video is frozen

[@lvernon9471]

I think your eyes are broken, he blinked mad times

[@t00862]

He doesn't need to

[@ashwinkumar675]

Thanks for these videos Kevin/Exponent! There are a few key points missed in the above solution : 1. Is the 10% of inactive users in a particular age group? - This question could lead to insights to lack of content aimed towards that user group 2. Is the inactivity / drop over a week? Or a month? - there is no time frame considered 3. As some folks already mentioned below, "past" watch behavior of the users could point to a lot of useful insights.

[@jasper5016]

All valid points.

[@abhinavshrivastava7508]

Great points. Another one can be - are these legitimate users or fraudulent accounts. Highly unlikely that 10% will be fraudulent account base though.

[@abhishekrbhat8919]

@@abhinavshrivastava7508 what do you mean by fraudulent accounts?

[@tobitosho]

Exactly what I was thinking

[@aniketsavardekar2359]

+ common insights from Inactive users (content watching genre, engagement pattern, watching patern etc.. )

[@aparnasinghchhibber7520]

My takeaway from this mock interview (Case of Netflix inactive users) - Define the problem. What are inactive users? - Ask questions to build hypothesis/identify the why behind the problem. In this case, try to get answers to questions like - Have the users watched all the shows on My List?, Is there any negative news about Netflix that's kind of floating in the market?, Is there any new content that is published or we have old shows only?, Is this happening for any specific region or globally?, Is there any new product that's kind of giving competition to Netflix? - if yes, what's unique in there? - Is it their pricing strategy/discounts or new shows being published going viral that kind of drifted away our users attention? - Based on the insights we got by asking clarifying questions, come up with solutions. - Decide what metrics you'll track to gauge improvement. Now metrics here can be - 1. North Star Metric - How many users are logging into the app?, 2. Engagement as in are they taking up the quiz or kind of reading the show description, sharing shows with friends, checking out what's new - adding it to My List, etc. 3. Acquisition 4. Retention Note - Throughout the process, do keep checking if the solution proposed is aligned with the problem we are solving.

[@leroyjenkins5679]

One question I would have loved to hear was whether 10% inactive users is higher or lower? What is the seasonal pattern. During summer, you will definitely see a drop in usage due to vacations. Second thing is what happens to inactive users, what percent cancel after certain time or what percent come back at different time of the year.

[@illstinguu]

Also the weekly usage pattern, on what days and what time ranges they use the product more.. push notifications can be leveraged then to pull back the users with 3 top suggestions based on their usage history

[@AakashMehta]

Ways to make this answer better: 1. First understand the trend of inactive users - did it go from 15% down to 10% or up from 5% to 10%? 2. Set the objectives and metrics that a PM for Netflix would care about; then define if inactive users is important or not 3. Before getting into solutions, think about potential causes that could result in the 10% inactive users 4. Do all the thinking in a structured way by sharing your screen and writing everything in a Google doc 5. Talk about Netflix’s SWOT and then come up with solutions 6. Prioritize solutions and then pick one that will improve the inactive users metric 7. Define next steps, metrics, experiments, etc towards execution

[@tryexponent]

Thanks for sharing your thoughts, AakashMehta!

[@fullnesters]

the way I would approach this are: 1) Define the inactive user: I would define active user by # of minutes watched. I don’t be log in a good indication of engagement. For example, me logging in and find all the shows no relevant, and not watch anything is probably bad. Since the value of Netflix are the content, and inactive user would limit to cancellation. 2) Link inactive to business problem: users may not watch Netflix daily, so it’s important define this further. Perhaps, inactive users are those who watch less than 60 minutes in 5 days. Additionally, I would tie this to the business problem. Because if 95% of user who watch 60 minutes in 5 days will keep their subscription may be acceptable activity. I would leverage data to figure this out. 3) Mutually exclusive collectively exhausting categorization of possible inactivity, things like stale content, possible UX changes that drove it (unlikely), possible recommendation algorithm changes, competitive actions that draw eye-balls, possible macro changes, etc. See if there are any clustering of inactive, age group, geography, etc. Then see what the cost benefit trade-off is in trying to change move the needle from “inactive” to “active”.

[@tryexponent]

Hey fullnesters! So great to see you interacting with the content and sharing your thoughts!

[@jainakhil]

I would have done the same thing myself.

[@abhinavtthakur9675]

Would be intuitive to break down the problem at hadnd here, especially drill down on the user cohorts for the inactive users. Could braodly be divided into below categories : 1. New inactive users ( D7 logins which havent been activated yet- not watched any videos ) - could hint at an issue with onboarding ; lack of default recommendations basis region 2. Old in active users ( previously engaged but recently inactive ) - Coud hint at : A. inferior recommendations - check increase in homepage bounce rates ; soln - try push notifications to re-engage basis genre/director/actor prefernces - Note B decrease in subscription renewals - low intent to renew subsctoption-soln - can make case for offers for high arpu watcher who are now inactive/churned

[@TheMarketingMan4U]

May I know what was the solution to the problem? I couldn't understand bcoz interviewer hypnotised me.

[@agujoseph3542]

Hehehehehehee Hypnotized you with what. Magic or science or religion???

[@AdityaSenarchadii]

I too need a video "how to blink"

[@TheMarketingMan4U]

​​@@agujoseph3542 I wouldn't have hypnotized if one of the persons was a jihaadi radical.

[@Laura-sg6ss]

Aha! She is gorgeous to be fair!!! 💃🏾

[@Laura-sg6ss]

​@@TheMarketingMan4U😭😭 jihadi radical loool whatttt😂😂

[@moderngekko562]

Thanks for the video! Few missed points according to me: - Defining inactive and active customer initially - Defining age group, watch time, device used

[@tryexponent]

Hi Modern Gekko! Thank you for watching and sharing your insights!

[@sachin96488]

It was a great approach. I would like you to narrow down the scope of Question. It would give you the direction where you should be heading to. Asking more questions to the interviewer. ( Looking Broadly for External and Internal Issues) ie - How long have the user been inactive( After the proper definition of Inactive users) - Was it an Age Group issue. - Has the same behavior seen by same set of users in past.( the binge watch and remain inactive most of the time) particular language group is major in the list of inactive users.

[@vinitmahale]

I had these exact questions in mind. Was going to write them down in the comment section. I feel these will give us much more clarity, especially about the demographic. Also, does inactive users are the ones who are paying up for the month and not using the app or they just have the app downloaded and the account but not subscribing after their usage. This would be critical in my thought because then we can also narrow down in understanding the habits, thoughts and liking of the users. One more interesting thought struck my mind after she mentioned the recap was, exclusive content release for certain shows, maybe a bonus episode but will only be live after end of the 2nd month of the series release. This will keep the hype up and might also push users to re-watch few episodes in case they don't remember what happened.

[@justcurious2439]

Interesting approach. I think the interviewee could have asked if there were any issues users have reported having with the service or any major trends from user satisfaction surveys.

[@Trex13]

I understood the question as the users are already Inactive and it wasnt a drop. So from that perspective it makes sense to analyze these user personas and then figure out solutions to convert the inactive users to active users. Also its crucial to define what's an acceptable inactive user percentage, maybe lets say below 5% is acceptable. Then the solution should be bringing down inactive user percentage to 5%.

[@anshu5337]

Diagnosis of Behavioural aspects of inactive users are very important. Diving deep into past data will give good insight on what they actually enjoyed or watched, Average time they spent on Netflix ( Past) and how many times they logged in but didn't watched anything. Are they get overwhelmed with choices or they get bored with existing content?

[@justcurious2439]

The first two options you mentioned are great data points to consider. I think the last one about being overwhelmed would call for a survey to get user sentiment

[@matthewnazlie5812]

Great interview @Exponent and Chloe! I would like to add few more questions: 1. Is this 10% inactive users mostly came from smartphone or tv or laptop users? 2. The 10% come from which type of shows? Cinema, TV series, docuseries, standup, etc?

[@Brand_Blend_Bloopers]

Hey, great session. Just one question, the solutions mainly talks about how users can login and then check the wraps/new features, but isn't that something which comes after user logs in. The possibility of user logging and not watch a video is very less if he/she is bored on the platform already. IMO, before any of these features, what helps is probably providing them a notification message saying that they watched a certain movie at certain time (that nostalgia bit), to bring the user on the platform (which is the first step in activity) and then probably having further engagement.

[@ruchikajangid9408]

points missed: 1) issues in user loggin in ? or issues in user watcvhing the content? - could be a bug where user got logged out and couldn't log in back. 2) was it sudden drop or gradual drop ? 3) was the drop in new users or old users ? power users or casual users ? - users who were using trial version or paid users ? (not sure if netflix has trial verson) 4) related to particular geography or demography?

[@illstinguu]

- Conversion metric: How many people actually used that solution out of the total who received the notification of the new feature (via email, or push) - Retention metrics Measuring that if the returned users continued to be there week after week, or not. - Measuring if the active users per week have changed or not after this feature.

[@A_Shreef]

It would be great if you can turn these interviews into a podcast

[@ayenisedaghat4667]

this makes so much sense

[@Singularity12321]

Great feedback at the end. I am sure at the end, by "activation", Kevin actually meant "re-activation"

[@SS-sy4qw]

Is there any model she followed to ask these questions

[@monikakanomata4508]

Loved Kevin's follow up questions!

[@adorablecheetah2930]

She didnt ask if there was a language factor? People arent watching shows in hindi maybe? or a platform factor? maybe people are not using on iOS frequently enough? Is usage down on movies or TV shows? where is the watchtime down? from recommended section or from search? She did a good job but so many details that were missed in diagnosing the problem I think :(

[@meghnasoni]

The interviewer said that the drop happened globally. So it's not necessarily related because of language related content, although he did say that there weren't many new shows that they had released.

[@adorablecheetah2930]

@@meghnasoni Yeah but i listed like 10 different variables.. can think of more

[@gpuwar1]

Main thing is discovery and algorithm feed. U don't find thing to watch. 2nd, is language availability as most album details don't even tell you language, u click title and see its Greek, and back off so click event triggers, but let's say 10% completion of a show doesn't. 3rd thing is pricing changes and how many profiles who can watch in parallel involved - Netflix changed pricing tier, should take in consideration rolling out of that and correlation.

[@xcellerate8787]

I didn't understand from the problem whether there was any increase in inactive users or is it a segment size that we want to work on. I would clarify that first.

[@amitbenski9140]

Many questions are missing by the interviewee, but the biggest one is - Do we even care? If they pay and the churn rate isn't bleeding then it's low priority.

[@barakhazan26]

This was a good structure, Thanks! I would like to clarify some questions, what I didn't understand is why she promotes new features and improvements? the question was about drop-in metric of inactive users - what she would do? Second, if she answered about improvements - I think she should talk first about the different user segments and their pain points. please share your thoughts with me - I would love to hear some insights :-) Thanks

[@jasper5016]

@Vaishali Kanyal - If you dont know the problem why users are "bored", there is no point simply providing more and more content.

[@coolwaterz]

she was great and natural. so happy she didn’t reply initially with the robotic “do you mind if i take a moment to gather my thoughts”. like an interviewer would reply yes i mind do not gather your thoughts.

[@vcconnector605]

Why does the interviewer look mad?

[@tripnation9659]

He’s hypnotized

[@vcconnector605]

@@tripnation9659 😂 😂 😂

[@Kyridiculous]

Would this classify as an ideal approach? 1. Define what exactly is the problem 2. Define causes for the problem at hand 3. Scope in one a specific problem which you feel is most important 4. Solutions to problem at hand 5. Assess different metrics to check whether your solution is effective (Do check in with interviewer as to whether the approach sounds good)

[@olamideakomolafe802]

I think it is a great approach. Finding out what the problem is and possible causing is the fastest way to find out what the solutions will be.

[@justcurious2439]

May consider adding an MVP after potential solutions to the problem

[@cap4life1]

Great job, Chloe and thanks Exponent for the interview.

[@iCyberZombie1]

Can we get contents like this for Product Designer 👩‍🎨

[@tryexponent]

Great suggestion! What kinds of interview questions would you like to see?

[@DeepakSharma-jy9bo]

@@tryexponent say (1) how does the mismatch between rationality of a solution being delivered and the feeling of its Information, visual and flow design - is related to a products metricised performance success ...all through the product life cycle phase. (2) How does a Product designer applies deductive & inductive integrated thought process during the design phase of a product

[@MrElderwand]

That guy does not blink at all??!!

[@TheJerryjj99]

Well done Chloe, great video :)

[@tryexponent]

Glad you liked it!!

[@yogindertickoo]

Bro forgot to blink haha. Amazing content though! :D

[@tigerrx7]

I feel like this is a product execution question and not so much a product design one (this gets rolled up into figuring out what you would do)

[@nikilragav]

or you share your wrapped with friends and they have a show that you haven't seen but would like -> oh that looks cool. let me log in and search for it on netflix -> possibly becomes active again. Can netflix track friends sharing the recaps? if so then netflix could add those friends' shows into your suggestions

[@justcurious2439]

That's an interesting idea because it opens up the door for sharing your top shows via social media like people do with their Spotify recaps. Then you can add a direct link from their story post to the Netflix login page.

[@vermillionvideos]

Whats the reward for users to share the wraps? A hook has 4 steps : Trigger -> Action -> Reward -> Investment. The loop would fail without a reward.

[@supradipmuralidharan6923]

What exactly is an RPM? Full form of this would help.

[@beohlai]

Rotational product manager. It's a 18 month program encompassing, 1 month of training, 3 rotations across different product teams, and a 2 week global research trip

[@tryexponent]

That's right!

[@sid4752]

@@beohlai does it count as full time employment? Sounds like a training program instead..

[@chloeene7895]

@@sid4752 I can confirm it’s full time employment :) with a boat load of training too

[@Flyer4EVER7]

I think the host should become a magician instead..

[@ziaulsiddiqui8528]

Can someone help with this questions that what would happen if Netflix doesn't address their inactive users issue?. One answer is that the user eventually cancel their subscription.

[@RealProductManager]

Honestly it felt like a really weak response.

[@Fnyvids]

I will install any app/video which does not support my language.

[@niyatigosalia8050]

Sorry but the mock interview was sad. I wouldn't hire her. Some of the clarifying questions were repetitive and dumb. the solutions were dumb too. your first solution can't be to create new content. try notifying, SSO, different type of recommendations, etc.

[@alexfriedman2175]

Hmm I think they did a below-average job? Not super structured and jumped to conclusions, little meaningful discussion about user segmentation.

[@Muchomorekable]

Very leaky framework, lots of guess work. Should focus more on analyzing user behavior across time and other dimensions.

[@manishkumar-qn6lx]

I gained much more insight if the problem, required analysis and potential solutions from the comment section as compare to her interview! She missed some major big points and analytical questions which she should have asked. Sorry if I'm being rude, but her untimely smile in serious discussion made me feel uncomfortable!

[@Amin-wd4du]

10% inactive sounds normal. not every user is an active one

[@kymzTech]

She is a Nigerian wow

[@tracy7275]

Corona wow

[@awesomechannel7713]

Great job, Chloe! Black women are definitely underrepresented in the space, so you would have an easier time getting into top companies if you are just somewhat potent, but you speak like a great professional as well. Kevin, can you pick an LGBTQR+ person next time?

[@venchingfu]

Netflix revenue is going down Y/Y and people are getting laid-off, so clearly this interview style is not working

[@imraanmoh770]

KPF is Killer... KPF stands for Kevin's Poker Face

[@LB-jw3ly]

She missed low hanging fruit. Netflix spends a ton on new content. How many users are only logging in to watch new content (new series, new seasons of current series) vs getting bored of binge watching? She lightly touches on it with a slowdown due to pandemic but there is a lot of meat on the bone there.

[@Amin-wd4du]

No wonder there are many failing products out there, This guys has no social skills and she doesn’t know product management.

[@chloeene7895]

Thanks for your feedback!

[@rhezarkhani]

Pretty dumb question. Why even care if they're inactive? It's free money.

[@jasper5016]

There are lot of things crucial things that she missed. Hard to believe that she is a product manager.

[@chloeene7895]

Hard to believe I'm one myself haha!

[@Youthemoneyguy]

yoo this dude hahaha

[@praveensg]

Meh. Her elicitation questions were quite general. You need to provide your own inputs. Cannot ask questions like are there any new services in the market. You need to know your market and make some basic assumptions.

[@aj2291]

Dude is frozen in time.

[@satiyapa]

Worst interview.

[@TJ111188]

This is a very shoddy interview! Probably this is why DEI recruitment is getting stopped.