Facebook/Meta Product Interviews: Success Metrics (Execution/Analytical)

👩‍🎓 Book 1:1 Coaching With Me! 🧑‍🎓 @ calendly.com/diannayau
If you're interviewing for a product manager or data scientist role, you'll be asked to define success metrics and set goals for products.
👋 Connect With Me 👋
► LinkedIn: / diannayau
► Blog: diannayau.substack.com/
► Discord: / discord
► Linktree: linktr.ee/diannayau
🎓 Interview Questions, Company Guides, Mock Partners 🎓
www.tryexponent.com/?ref=dian...
📚 Books to Prep for Product Interviews & Careers 📚
www.amazon.com/shop/diannayau

[@diannayau]

🤩Comment below: ✍what other products do you want me to set success metrics for? 🥴Struggling with product interviews? 🚀Book 1:1 COACHING @ superpeer.com/dyau

[@LovelyisYOURstory]

I can’t thank you enough for all of your tips and for you taking the time to teach and prepare me for the FB interviews! I’m so excited to say that I got a FB offer and I certainly credit it to you. Thank you for your personal sessions with me and creating relevant and real content that can’t be copied. I love your authenticity and your passion to help others excel.

[@cjb5000]

Still here for Meta interview prep. It's intimidating and intense preparation, but if you want to get the bag, you gotta put in the work.

[@yairgraif7342]

Thank you! More videos like this would bring a lot of value.

[@yujeongkim9409]

Very helpful. Thanks so much Dianna!

[@xiaoweiyan9050]

Love the examples, Thank you so much Dianna!!!! 😃

[@shifia9]

Thanks for the amazing videos Dianna. Your framework and examples are super useful to prepare for the interviews. Please keep creating more such content.

[@OnionKnight541]

This framework is great --- the setup feels almost like a Generative Adversarial (Neural) Network because you're pointing out both the success and fail cases of the product. FB Marketplace might have a user who lists a dirty sock, and you want to be able to quantify that. That's a great insight. And it reflects the range of user behavior within the system. Versus assuming that all the listings will be Eames chairs. :P

[@Pinktan800]

Yes, it would be great to see you walk through examples so that we can see the application of the framework. Love your energy!

[@diannayau]

Great to hear you're liking the format of going over examples. Let me know if there's other products you'd like to see me cover!

[@Mariethegoat]

Hey friend glad you found value in her channel!

[@airliu1]

I liked this example. super helpful to see the framework in action!

[@DarshanShah1994]

Thanks Dianna, even after almost a year, this is super helpful in my prep. Keep posting!!

[@diannayau]

So happy it’s helpful! Let us know what other content could be useful!

[@dmytrogalakhov5366]

Great stuff! Very helpful! Keep going!

[@lydiayuna9155]

Really helpful content with examples, thank you!

[@kumudineee4618]

Hi Dianna, glad to have landed here! Love your videos and the brevity and precision with which the frameworks are explained! This channel has been the most helpful source for me.

[@diannayau]

so happy you found it helpful! Keep watching ;)

[@fabslousnature9788]

Super helpful to see the framework in action through examples. Thanks Dianna!

[@diannayau]

Glad it's helpful Fabio! Yess, show don't just tell :P

[@devar90]

This is very helpful Dianna. Please keep on making videos with examples as it allows us to first do it ourself and then compare with yours. :)

[@ipoojasharma_]

Awesome Dianna. This was really helpful :)

[@inspired784]

seriously brilliant content and can't thank you enough, helping me prepare FB onsite interviews

[@diannayau]

greatt! I hope you aced it! Let us know if you made it!

[@nyntein]

Thanks Dianna, this was quite helpful.

[@OnionKnight541]

very helpful. this is some of the highest value and most relevant PM content. (:

[@Mariethegoat]

It would be really helpful to see you do a mock interview; to give example of questions to ask and how that interviewer would quiz you on your metrics and ask why or why not.

[@vikashkumardce]

amazing, more examples like these please

[@sauraodalvi2502]

This was super helpful!

[@rachelma6192]

Super helpful video! And yes, more examples of how you execute these frameworks would be amazing.

[@diannayau]

Thanks Rachel! More examples coming up for you! :)

[@ruturajpatil51]

Would you be able to do similar video on Reels goals and success metrics. Reels/tiktok is growing alot so it might be super helpful to understand whether to focus on growth, engagement or monetization

[@gungunare]

Thanks for this video Dianna! Videos with examples definitely help more than the videos that just show the framework since the application of the framework is what matters the most. Another great aspect of this video was the product that you used. Marketplace is becoming such a common model but information about it from a PM's perspective is too little. I'd love to see more such uncommon product questions being talked about by you. The clarity with which you solve problems is amazing. Looking forward to more content!

[@diannayau]

great feedback Gunjan! I'll definitely have more videos going through examples coming up for you! What are other uncommon products you'd like to see me cover?

[@productmanagementbyutsav3069]

Please write a book on big tech interviews 👍 . The content is brilliant

[@diannayau]

haha, if i start writing a book I might not have time for KZbin content anymore :P which medium do you prefer?

[@pratik0300]

@@diannayau Definitely youtube, can run it at 2x, cant read at 2x

[@emhtptc]

@@diannayau classic PM negotiation/prioritisation trick right there 😄

[@diannayau]

😆 The PM thinking isn't just for work

[@productmanagementbyutsav3069]

@@diannayau I will prefer KZbin for now, but in long run if you find time you should write a book 🙂🙂

[@jadeclarke4116]

LOVE THIS

[@bvelazquez11]

You are the bomb! Keep it up.

[@prblsoni]

Thank you so much mam. Loved this example. Request your help on sharing few examplea from Program management perspective

[@abhilashaanantharamakrishn3958]

Hey Dianna, thanks for this! Super helpful. Question on the NS metric - curious why you chose # of interactions between buyers and sellers vs number of completed transactions? My take on this is a lot of interactions often go unanswered on marketplace or there is not continued engagement from either side which then makes one or both parties lose interest. So curious why you chose that as your NS metric?

[@Nicolenyc]

You make it look so easy ;-)

[@sahilsingh7891]

Really good content.

[@khaterehkhodavirdi6536]

Thanks Dianna for all the useful content. Question for you: Currently you only need to message the seller for local listings and for majority of FB MP listings you can buy them through online check out wo even messaging the sellers. Do you still think that this north start metric is relevant? bc more sellers will provide online check out option so the # of messages might go down which is not a bad things. Let me know your thoughts. thanks!

[@junqichen6241]

Thank you for another awesome video! With so many metrics though, how do we or should we narrow down to a few for formal tracking?

[@diannayau]

Hey Junqi, thanks for your question! you'd have your northstar metric and then the other ones on the list you'd ideally track too

[@vishalshah9619]

Hello Diana, great videos. 2 Questions. 1) Yours doesn't necessarily follow a framework but it sort of does as in first discuss product, second talk about users and stakeholder(FB) and why and how this product adds value to all, and then move on to NS and then talk about metrics in detail, etc. However, would it be ok if I were to answer the question this way? Or would it sound like the guy is randomly throwing ton of metrics? 2)There is no certain goal that you talked about such as Engagement, Retention ,etc. Is it necessary to mention that Goal prior to answering since if you don't know what the goal is how do you know what to focus on?

[@sohailpuri8338]

Great content! I think I like your approach of coming up with a NS metric that touches each user segment. Also, I wanted to check if you think we should work with the interviewer to prioritize one objective (either of adoption or engagement or monetization) for the given product instead of talking about metrics from all different stages/cycles of the product?

[@diannayau]

you can ask the interviewer but the best candidates i've seen usually cover all those objectives to show they can think holistically about how to define if a product is succeeding or not.

[@angelajo5733]

Would you say WAU could be a topline metric for FB ecosystem assuming more people are buying/selling products on marketplace and therefore spending more time on FB? e.g. Responding to buyer inquiries is another way a seller can hop back into the FB ecosystem frequently.

[@Kb777]

Hi Dianna, while considering breakdown of the Northstar metric, is it ok to just identify the levers and how they’ll impact the NS metric? Or do you recommend that we necessarily come up with a formula for the breakdown?

[@diannayau]

No formula needed, just the levers and how it’d impact the NS.

[@yairgraif7342]

Why didn't you mention a timeframe for the metrics. Like: Meaningful conversations per week or per day?

[@aasthathakur8704]

What would an active listing mean? Also, do we need to track number of active listings/ month or no of new active listings per month?

[@Mariethegoat]

I would like to know when to you %vs #of when explaining users of the ecosystem

[@user-id9gj1gc1s]

Hi Dianna, thanks for the awesome videos! After having watched lots of mock interviews on youtube and feeling lost, your tutorial videos are really helpful in gathering my thoughts ;) here are my questions. 1. How long are we allowed to come up with our ideas? less than 5 min? To be honest it took me really long to come up with the ideas! 2. how specific answers should we give? (of course more detailed answers are always the best!) It will reaaally be difficult for me to give an answer with so many details like yours, especially during the interview😭 Would you be able to give us some tips when preparing for the interview? 3. If we are asked to suggest metrics of changing a specific feature (e.g. search toolbar) should we clarify the purpose of the change in the beginning and how it's changed, since... there should be a reason for the update? 🤔 (e.g. to provide better search results vs. to improve user engagement)

[@diannayau]

Hey Miji! Good luck on the upcoming interviews 1. 2-3 mins feels appropriate! 5 mins feel a bit long. But if that's how long you need to come up with really good ideas, you might ask the interviewer if that's ok 2. How much detail-more detail isn't necessarily better but you should push yourself to go deep. Most candidates give shallow answers which means they don't stand out. So during the interview, give yourself space to think silently, jot down some bullet points 3. Yes! lead with your hypothesis on why you think they want to make the change

[@rajamuthuraman1704]

@dianna Thanks for your service to the PM community! A tactical question for clarity. Could you please help explain how you differentiate counter vs. downstream metrics? A downstream metric could be negatively impacted if we optimize for the north star metric. This means one of the downstream metrics could be a counter metric. In this FB Marketplace example, if we improve north star, # of items sold via Buy/Sell groups could decrease. However, you called this metric under Downstream metrics. Why not call it under counter metric? (Do you define counter metric strictly from the product-in-focus perspective e.g. marketplace, and call all ecosystems metrics whether they are impacted positively or negatively as downstream?)

[@diannayau]

Hey Raja! Good question- a countermetric is what happens if a North Star is optimized and what negative consequences could happen. A downstream metric is either positive or negative metric effects-so a negative effect could be cannabilization of another product say marketplace’s effect on buy sell groups decline or a positive effect could be helping drive growth for another related product say marketplace growth on the usage of Facebook Pay.

[@kailfu]

Thanks for your great video! Just subscribed and liked. I have one small questions though. In 8:50 you mentioned another downstream effect is "increasing the time spent on facebook (especially for users who don't use Fb anymore, we create a use case)." How could that be a downstream effect? I suppose creating new use cases for users would be a good thing. Maybe my definition of downstream is wrong. Thank you so much!

[@diannayau]

Downstream effect means what are the dominos effects that’ll impact the rest of the product ecosystem. By having ppl use marketplace, it gives another use case for ppl to spend time. Eg I don’t post much on fb anymore BUT I do spend a lot of time on marketplace (unfortunately 😂)

[@utube4vrao]

Dianna this is just phenomenal but the text is hard to see on the phone. Can you maybe use a presentation or Google slides to show text so it shows clearly on the mobile phone screen? I am sure this looks great on desktop but on phone I can barely read the metrics.

[@diannayau]

tks for the feedback! let me experiment with this!

[@donmeswimee]

Dianna, can you explain why this is such a marked departure from your other execution metrics video re: Airbnb? Specifically, why we don't break down "# transactions with 3 convos" into "# transactions per user x # users", then continue distilling that down? This seems more like brainstorming, which is what you said not to do. Thanks!

[@Laxmisaagar1]

great! is it not necessary to define further the user segments in such cases ? like what kind of buyers - the way it is done in design questions? if yes - to what level shall one spend time on that ?

[@diannayau]

nope you don't need to further break down user segments in execution questions

[@Laxmisaagar1]

@@diannayau thanks

[@asutosh123]

Hello Dianna - Thanks a lot for this model response. It was really clear and easy to follow. One question. what's the difference between "set goals" and "define metrics" types of questions? are they are the same or different? how do you propose to tackle the "set goals" kind of questions

[@diannayau]

They’re the same :) well they want you to define a qualitative goal first and translate that into a quantitative metric

[@asutosh123]

@@diannayau thanks, that helps. Goals sound so much like setting a target to reach. As an example, should the qualitative goal (e.g. for FB stories) then be to "increase creator engagement" in the next quarter and a corresponding metric be "# of creators who created a poll atleast once a week". Are we expected (For FB product execution) to say increase the same metric by 20% or not mention the increase %?

[@raghavendrandharani52]

👍

[@numtwo1]

I'm a bit confused.... the question is about setting a goal for product X. Now the metric is about what we will measure to know it's working but what about the goal? It feel like it's not answering directly to the question (but just to the parts about the metrics). But all in all great videos, super helpful!

[@diannayau]

When they ask what the goal is, they’re essentially asking what metric you should be using to measure the goal

[@numtwo1]

@@diannayau Thanks! I also found a similar question on one of the other videos where you wrote: "When they ask about the goal, they want to hear you come up with a qualitative goal and translate that into a quantitative success metric." which helps all the pieces fall into the right place. I really like the approach you are taking with making the answer a walkthrough story which gives much better clarity and helps to communicate better than a lot of the other frameworks. Thank you!

[@karthiksureshlbs]

Talking about north star metric before analyzing the user journey and the subsequent metrics is like coming up with a solution first as an anchor and then analyzing the user segments and problems in a product design interview (you would never do that) - That's one thing I wouldn't agree with this framework otherwise looks great!

[@diannayau]

As an interviewer, I've seen candidates take either path (northstar last or northstar first) and still come up with great answers. Why we start with the northstar first is because strong product thinkers are able to identify the main value of the product to users early on- you don't need to go through the entire user journey to know the main reason users are using your product.

[@karthiksureshlbs]

@@diannayau I guess let's agree to disagree here. This assumes you are already fully familiar with the product which may not be the case (and also where the product is in its lifecycle). Strong product thinkers spend time analyzing the product before coming up with a north star metric which is literally the goal of this interview. It is not just about identifying the value prop but considering various other factors as to where the product is in its lifecycle, and where should the team be focusing their efforts to help drive product usage.

[@bubblelanlan]

Although Meta is doing quite bad with their ecommerce products, this video accurately pointed out the key metrics for a merkatplace. Great video.

[@adil1335]

more examples please!

[@diannayau]

More examples of goal setting questions or what type of questions?

[@adil1335]

@@diannayau thanks for your reply Dianna, in the video /watch?v=RpAgs1z4K_0&t=63s at 5:28 you mentioned sharing a framework for "should youtube buy Netflix" question. It is a strategy-type question. Can you make a video on that, please? thanks

[@ourlifeensemble6727]

Is "Number of goods marked as sold on marketplace " a good north star metric?

[@diannayau]

Yep that’s a solid North Star

[@ourlifeensemble6727]

@@diannayau Thanks you so much for your content. Has been very help. It would be super useful if you can also share videos answering strategy / execution questions for non-Meta products.

[@gogozheng9353]

✋Have a quick question: what does "MP tab" in "active=in the MP tab" mean?

[@diannayau]

Marketplace tab :)

[@yairgraif7342]

Doesn't Facebook make money from the Ads in the marketplace? Shouldn't that be addressed in the example and maybe even be part of the north star?

[@e.w.6470]

I still doubt the north start metric is the meaningful conversation. There is no value for seller or buyers without transaction. No one comes to the marketplace just to have meaningful conversation without deal. Completed transaction drives sellers and buyers to FB marketplace vs other marketplaces.

[@gogozheng9353]

✋Have another question: in the video, it is not so clear which one, by you preference, is your specific goal since several of them could be, like creating a healthy ecosystem, maintaining active interactions, or increasing meaningful conversations? Or do you mean the northstar metric should exactly be the goal and other metrics are categorized as success metrics?

[@diannayau]

your northstar should be the key goal and the other metrics are supporting metrics

[@MMSGEE]

I wonder what you work on! hahaha

[@chieduagain]

it would be easier to follow if what you're writing on pad was easier to read For a PM, I'm surprised you have noticed it hard to read