You are a true gem, already been following lot of your contents in youtube. You articulate the system design cases and solutions so well with in-depth explanations of the technologies used.
@jordanhasnolife51632 жыл бұрын
Thanks Surendra ♥️♥️
@tyson962 жыл бұрын
This video randomly popped up in my recommendations (pun intended) and it was a very good recommendation. Really like the detailed explanation. Follower += 1
@jordanhasnolife51632 жыл бұрын
Thanks Rahul! Funny to see the system in action haha
@ayanSaha13291Ай бұрын
Great video Jordan! Nicely explained.
@AidamanTV22 жыл бұрын
You're the Yin to my Neetcode :)
@avirajkumar41157 ай бұрын
Have You thought about how you will work with a recommendation system if we have addition data on users such as age/ location/handset/Time based recommendation and their activity as well as adding different types of tags such as Content/ category/ genre/ actors/ director/ content creator etc.
@jordanhasnolife51637 ай бұрын
How would you say this would deviate from our traditional recommendation system pipeline? At least to me, these just seem like more features with which to make decisions!
@clara.hwajoon6 ай бұрын
Hi! what are the ranking models exactly taking in and putting out? I thought that from the retrieval, nodes containing the lists of candidates would have scores associated to each candidate based on the neighbors being close. And then filtering happens with bloom filters. thank u!
@jordanhasnolife51636 ай бұрын
Hey Clara! So in the retrieval phase, we hit our embedding caches to find all of the possible suggestions that we may give to a user. Then we filter, to ensure that we don't show the user any content that they've seen, or shouldn't be shown to them for whatever reason. In the ranking phase, we have room to rank a more limited search space of potential items (because we limited them before) according to user specific criteria. If we retrieved items that already had a score, it would mean that we would either need a copy of the embeddings per user (probably impossible) or that we couldn't account for any user specific information when making our rankings (not ideal). Hope this makes sense!
@clara.hwajoon6 ай бұрын
@@jordanhasnolife5163Thanks for the answer + thanks for the help! love your videos.
@himanshukriplani68673 ай бұрын
I personally liked your ipad notes and prev method more than this.
@jordanhasnolife51633 ай бұрын
My iPad notes is my current method - that's why I'm remaking them
@user-fl9pg1zb2v2 жыл бұрын
Hey just a silly question are you same guy in your dp , if yes you are a gem and a meme legend too
@jordanhasnolife51632 жыл бұрын
No I'm not haha, I just also enjoy the meme - it may be time for me to change it tbh
@harshbhatt_4 ай бұрын
If the embedding model is trained offline, how will it recommend most up to date data? (ie a new video got created 10 mins ago) In that case, I'm a bit confused on how it's real-time? Also do we not need to use Spark Streaming anywhere in this design? Thanks!
@jordanhasnolife51634 ай бұрын
It probably won't. That being said, you can combine those suggestions with things like subscriptions (see Twitter design) to deliver videos of creators that you subscribe to to your news feed
@aryamansinha29324 ай бұрын
hello.. all model training seems offline with the 2nd approach..so how is it real time?
@jordanhasnolife51634 ай бұрын
The real time part is just asking for more videos. That being said, I suppose you could use stream processing to create embeddings of newly created videos on the fly and then include those as candidates for new suggestions.
@LeoGoldenLabrador2 жыл бұрын
Can you a video to talk more about KAFKA? why should we use it and when in SD?
@jordanhasnolife51632 жыл бұрын
I recommend watching my stream processing video - it's a log based message broker.
@harshbhatt_4 ай бұрын
One more question: Does the embedding model + index only look at the item embeddings? (videos recently watched) or does it take user information as well (I set my preferences as liking action movies) Also how would it worked for a cold start user? (user does not have any "'last 5 videos watched" to get 1000 nearest neighbors?
@jordanhasnolife51634 ай бұрын
Embedding model doesn't have any user info or else we'd have to duplicate all the embeddings, for the first open we can just show popular videos or something
@jordanhasnolife51634 ай бұрын
Embedding model doesn't have any user info or else we'd have to duplicate all the embeddings, for the first open we can just show popular videos or something
@Prem-xe8hk2 жыл бұрын
Can you please elaborate more on the neighbouring index?? and how the vector model linked to this index? Anyways Great content (y)
@jordanhasnolife51632 жыл бұрын
It's a map from (vector) to [closest neighboring vectors]. Feel free to elaborate on your question and I'll do my best to help!
@muhammadhabibullah12052 жыл бұрын
great content!
@anahata.kaalkiАй бұрын
I saw comments and thought it's a good video but It was a huge waste of time. Maybe interesting for people who have Zero idea about tech.
@jordanhasnolife5163Ай бұрын
Kek
@Gerald-iz7mv11 ай бұрын
what are item change events?
@jordanhasnolife516311 ай бұрын
Mind giving me a timestamp to reference?
@Gerald-iz7mv11 ай бұрын
@@jordanhasnolife5163 in the architecture diagram here at timestamp 306 sec
@user-vv8fw4fj5i11 ай бұрын
@@jordanhasnolife5163 top right in diagram at 5:13
@jordanhasnolife516311 ай бұрын
@@user-vv8fw4fj5i Ah ok - just describes any changes made to the set of items that we could possibly be recommending (adds, removes, edits of an existing item) as resulting from user action
@Gerald-iz7mv10 ай бұрын
@@jordanhasnolife5163 could that be a good usecase to store it in a feature store? could the user activity also be stored in a feature store? in case of a music recommendation system are those infos regarding: song titles, text, images, artists, music genre, audio analytics?