Meta/Facebook Product Interviews: Execution Tradeoff Questions

[@cutzmusic]

I dont think people understand how many gems and isight Dianna is giving here. These are things people pay for. Great work and thank you Dianna!

[@diannayau]

So so happy to hear it's been helpful!

[@OnionKnight541]

these are absolutely the best youtube videos for PM interview preparation. so good.

[@redbaron007-e5p]

Good video. A few comments: since Meta makes virtually ALL its revenue via advertising as per their annual reports, I feel Ad impressions (for impression-based ads) and actions (for action-based ads) should somehow be part of the metrics tradeoff discussion. For example, if Time spent of FB/day goes down, and we see a corresponding decrease in # of impressions/actions per day, then DON"T ship, even if you see broader participation going up. Even considered by itself, a decrease in # of impressions/actions per day attributed to the experiment would be enough for the DON'T ship decision. What do you think?

[@Fedkar]

Really cool explanation and way of product thinking. You demonstrated much deeper understanding of product management than those FAANG PMs doing mock interviews here on KZbin. Thank you for sharing! Cheers

[@YourJustin]

Liking, commenting and subscribing to give you maximum engagement in KZbin! Thanks so much for creating these videos Dianna. I feel more confident in my upcoming Meta interviews because of them 👏🤞

[@diannayau]

Thanks for all the support Justin! 🤩 I’m cheering you on for your upcoming interviews!

[@deepmp1]

Great video! Before you get into the hypothesis, it might help to also talk aboutvthe difference in the value that feature a (reactions) has va feature b (conments) since you use those implications in the hypotheses

[@coffeewithsimon]

Hi Diana. What if the tradeoff question simply asks you to prioritize.? "You are the PM for Facebook live - what features would you prioritize?" My initial assumption is to ask the interviewer to specify the features like A vs. B. If the interviewer tells me to decide, I'll have to make some assumptions. I am just confused about how I would approach this question. Do I list all the features from the creator and viewer side and then prioritize? Thank you so much!

[@satyajit9]

Thanks for sharing this. The list (of metrics) is quite comprehensive and is quite easy to follow the flow. My goal now is to internalize this for my FB interview in 2 weeks :)

[@diannayau]

If you get familiarized with the products beforehand aka using the product and thinking what value you get from it, it'll be easier to internalize...cheering you on for your upcoming interviews! And if you need extra support through coaching, don't be afraid to reach out!

[@PushpaRakesh7]

great explanation! This is really a great stuff. I have two questions though. Is it ok to ask the north-star metrics to the interviewer during such a conversation? Is it necessary to always connect all the metrics in discussion to north-star metrics? Thank you

[@markchao4848]

really like your video. the structure and framework are very easy to understand.

[@ankittandon222]

Hypothesis could be on the lines where we expect the number of post to increase, which in turn will increase the new feed for the user and ideally from Impact perspective: 1) Time spent should be almost the same 2) #session should increase (craving for new content) 3) DAU could be more or less the same? Need some insights on No Ship. Lets say we have Social interaction increase (maybe 5% up), no. of post is almost the same but DAU have a slight dip maybe 0.1%-0.2%... What will be your call post that. Any suggestion?

[@ritikadhiman7459]

Hi Dianna, Amazing video but I have couple of questions. 1. Shouldn't we prioritize our metrics? 2. Can we also bring demography, platform, etc. into the picture in addition to this framework. If yes, when? Thank you :)

[@vidushikshatri718]

Hey Dianna , thankyou for all the content you have created and shared . Do you have an example video of how to tackle same surface, different products ? (Beisdes the general guidance that you have provided in one of the other videos )

[@tolulopeaduramo2039]

Diana you are super fantastic. I will share this video to aspiring product managers.

[@meili5826]

When making the ship or no ship decision, should we link back to the business mission?

[@samw2066]

Outstanding content and presentation. Look out Emma Ding, there's a new sheriff in town and its Dianna Yau!

[@subratadazz]

Hi , this is great stuff . I just could not help but think that should we not consider whether the comments and reactions are going down across FB or is it a localised issue or limited to a specific channel ( mobile / web) . Also in absolute numbers should we not go into how much is Comments as a % of total reactions . That should give us some more insights

[@danfran4856]

I love how you approach these execution/metric questions, so surgical! Kudos! liked and subscribed However, if it was not a launch, I would love to know how you would differ the approach if it was after 1 year or 2.

[@diannayau]

Thanks Dan! Haha love the adjective “surgical” makes me feel more doctorly than I am. Unless the results of the test change after 1 or 2 years, whether it’s a launch or not will change accordingly

[@danfran4856]

@@diannayau you're welcome! I was thinking instead of "ship or not to ship" if it's in later years "keep or sunset"

[@danfran4856]

@@diannayau Where also your experience shows that you may take for granted, is choosing the apt metrics. I am learning a smart trend. You focus on quality of the metric(Reaction may be Vanity vs Comment is deeper), who it is affecting (is this leading to more or less effect wise on a poster or viewer to partake or is it creating more posters/viewers) and at what rate.(is the total posting changing/ is the total viewing increasing, then per poster/viewer effect) You don't overtly say this, but I think it is important to flesh out the minutia a bit regarding thought behind the choice for the less advanced looking on lol.

[@jamesshastry6118]

What a response! Thank you Diana!

[@renewu3677]

Hi Dianna, thank you so much for the great videos! Quick question, you mentioned users value, metric, hypothesis from three sides ( supply, demand, FB). Due to the time constraint during interview, should we ask the interviewer which side he/she would like us to focus on? Or we should pick 1-2 metric and hypothesis from each side? Thanks!

[@diannayau]

Ideally you pick a few metrics from at least the supply and demand side, the FB side can be a nice to have if you're running out of time

[@neelkumar7932]

I love the way your constructing your thoughts

[@diannayau]

Thanks Neel! Let me know what other content could be helpful :)

[@nguyenhuycuong.freddie]

Hi Dianna, this video is really very helpful. I have a question: "How long should we measure for these metrics to decide ship/no ship ?". I hope to receive any recommendations from you.

[@diannayau]

It depends on the test, but you’re not expected to answer that in an interview because it requires some data science. It could be anywhere from 1/2 weeks to a few months, really depends on how many data points you can collect and how much data is required to get statistical significance

[@carlosc6799]

Thank you. I love how are you articulated this point!!

[@amirfakhim7797]

Hi Dianna, a wonderful video! thank you. As I was watching this video i was thinking about, like other videos you did define the north star metrics. The reason I say this is because metrics movements are not always enough to take any action as action costs money and resources for the organization. Would you please help me to see if the following though process makes sense to you: I will gauge how much these two metrics have a positive or negative impact on the north star metrics. if the movement of these two metrics holistically ( aggregation ) moves the north star metrics up, I don't need to do anything (if we are close to the target we have defined for that metric), but if it drives the north star metrics down, then we need to do something about it. Here I will branch my thoughts to two branaches: 1- How big is the impact of reaction? 2- How big is the impact of the comments? Which one is hurting us more and then define action items to rectify the situation? Does this thought process and analysis makes sense? Many thanks! Cheers!

[@vivek2675]

@Dianna - This is great and super helpful. I got an observation. Can you verify if that is true. If you were a PM for facebook comments section then I would assume in their context that - this is a Root cause analysis question. If you were a facebook PM for reactions then I would assume this is a design question with trade off decisions when they were asked to design reactions in the first place?

[@mag3351]

Hi Dianna, so would you say the clarifying questions on geo and device type etc really don't matter? I remember form your other video you tappen on the fact that it would not really change someone's answer.

[@diannayau]

If it’s not going to make a meaningful difference in your answer then yes it doesn’t really matter

[@ashwinkumar8245]

Thanks for the video, Dianna. Follow up question. Suppose this is not happening during launch but after then shouldnt we first establish that there is causation here instead of random correlation? For example, what if there was an issue with posting comment that decreased the total number. In interest of time should we ask the interviewer if we have established causation or is that something we should do? If yes, what is the best way to establish causation? Do we simply identify all hypothesis similar to a diagnosis question and eliminate?

[@diannayau]

Good point, you could ask the interviewer whether it’s direct causation and offer you can check causation with a holdout group whose not getting reactions, whether you’re seeing the same effects. I wouldn’t recommend treating it like a diagnosis question

[@sauraodalvi2502]

Thanks a ton for this video. Just one question, as this is the live scenario of FB how does ship or no ship come into the picture, shouldn't we be thinking of how to convert this situation into our favor?

[@DG-yr1tk]

Thank you so much for posting this content. This has been super helpful in interviews.

[@waz306]

Another great video Dianna , keep posting ☺️

[@Leon_1218]

And I have a question. When interviewers ask this type of question, shouldn't we focus on these two metrics only? As you elaborated on the metrics from the viewer side and poster side, it makes sense but I feel you are introducing more metrics to assess the impact of the increasing interactions rather than answering the original question. Besides, for the three hypothesis you presented, it seems only the first one deals with the %comment going down which got me confused if those last two are legit hypotheses. I'm really interested in this question and I might be wrong but would like to hear your thoughts. Thank you!

[@bhavadeepreddy8804]

Same question I have Could you please elaborate

[@MC-wp2vz]

I think the full question for this interview would be you're the PM for FB reaction, after you ship the product, you see X go up, but Y go down, what would you do?

[@TheKmisra]

I think the key point here is that reactions / comments are both sub-metrics within a Facebook product (News feed / events et al). They are not the north star metric or even a key metric, that point to the overall health of that product - I'd argue that something like reactions / post / day or overall reactions / user / week are more relevant. Therefore evaluating these metrics in isolation is unlikely to tell us anything - up-leveling it to a more important set of metrics is the right way to do it, IMHO. On a separate note, one challenge that I have with this answer is that it does not address the RCA piece at all - it seems to assume that the drop happened after launch of reactions on FB (which is at-least 5-10 years old by now). I don't think that's practical and therefore, RCA would be an important step (Step # 2) to my mind.

[@Leon_1218]

I come across the video after I got tripped by this question in interview lol...

[@anvikaanvika9834]

Shouldn’t we be asking a few clarifying questions as well ? Or did you skip it for simplicity ?

[@diannayau]

Depends on what clarifying questions- the most helpful clarifying questions is probably clarifying the product. Other stuff like "is this on mobile or desktop, which country" won't make a big difference. Are there clarifying questions that you've asked that have helped you in these type of questions?

[@anvikaanvika9834]

@@diannayau i was thinking more like when did you start observing this dip, did we recently launch something around those features and is there a particular demography that this is happening in. Have we observed this on other platforms like insta as well ?

[@diannayau]

@@anvikaanvika9834 yess the "when did you start observing this" is especially useful. And asking for more context like, did this happen suddenly or is this from experiment results

[@kakabuca]

Hey Dianna, another great video! Thank you! Quick question, I often find the "X metric up and Y metric down question" as a follow up question after success metrics question. When you only have 5-10 mins to answer this trade-off question, how can we best tailor this framework?

[@diannayau]

good point! then you'll prob just want to cover 1-3 of the framework and skip section 4 and no need to mention setting up an experiment.

[@balakrishna197]

@Dianna Yau, could you also plesae make video about "How would you tradeoff between increasing ad revenue and decreasing retention on WhatsApp?"

[@tofulover09]

The two goals you have listed appears to be more business-centric and would ideally be decisions that are taken on the business level. If your question is about how to choose between them, a good approach would be to conduct prioritisation while keeping the business goal in mind. Is the priority of the company increasing ad revenue or increasing retention?

[@AnjaliGola]

The music is sooo distracting .

[@OnionKnight541]

FIRST (: (: