Real-Time Search and Recommendation at Scale Using Embeddings and Hopsworks

[@XxXx-sc3xu]

Wow. Amazing presentation on Machine Learning Infrastructure at scale! Thank you.

[@bigger302539]

That was so good. Nice detail and example thank you so much for putting this up.

[@chrisogonas]

Incredible lecture! Thanks

[@haneulkim4902]

Great talk! I've got two questions: 1. Is it a real-time recommendation because upon user query it is enriched with historical data as well as real-time data(stored after each interaction)? Or something else like you are training embedding at near real time? 2. How often do you train Ranking model and does it only train with candidates?

[@MrNagano00]

From my understandings: 1. It's real time because it's capable to do one recommendation per user query without much delay. Instead of spotify that just gives you one recommendation once a week. So it's essential the same thing except the infrastructure required to be able to provide "real time" recommendation is different. But the ML models and processes you'd use to do one or the other would be largely the same. It's not about training it's about how speedy your evaluation is. Also, adding filters and so on, so overall it's a more complex form of interaction with the ML system. 2. You'd want to train your ranking model whenever you feel you've gotten enough data. There's not really a magic number here. As users interact with your website you'll get better data for positive/negative examples; it also depends on how much it costs for you to re-train your algo as well; if it's too expensive then maybe you just want to be re-training when necessary.

[@Gerald-iz7mv]

@@MrNagano00 how is the infra different between batch recommendation and realtime recommendation? also what is the model serving doing in the real-time recommendation system - is it for ranking only? why you need a vector database for the embeddings? dont you need to train an embedding model too?

[@EranM]

Can't you get the score (ranking score | similarity score) while fetching items from the Vector DB? ..

[@Gerald-iz7mv]

what model does the batch recommendation use? some content based, collaborative filtering model?

[@Gerald-iz7mv]

how does the model get generated for the batch recommendation service? does the embedding model for the realtime-recommendation only use userdata and not item data?

[@lifeconfused52]

Hello, could you share the code with me? There is no such codes in the github now

[@quanghieuvu1012]

Did you have the code?

[@EranM]

can someone explain to me, how come you calculate USER embedding when training. And when searching for similar embeddings, you actually get ITEMS embeddings???

[@drhxa]

Two-tower model. He explains it briefly in the talk but the user and item embeddings are trained to have very similar dot product. Use data from what people actually ordered

[@amantandon-ln9xx]

Amazing, thank you