Real-Time Search and Recommendation at Scale Using Embeddings and Hopsworks

The dominant paradigm today for real-time personalized recommendations and personalized search is the retrieval and ranking architecture based on embeddings. It is a fan-out architecture where a single query produces a storm of requests on the backend. A single query will search through millions of items to retrieve hundreds of candidates that are then enriched by a feature store and ranked so only a few recommended items are presented to the user. A search should return in much less than 1 second. Retrieval and ranking architectures need significant infrastructure - an embeddings store and a feature store - to provide both the required scale and real-time performance.
In this talk, we will introduce an open-source, scalable retrieval and ranking serving architecture based on open-source technology: Hopsworks Feature Store, OpenSearch, and KServe. We will describe how to build and operate personalized search and recommendation systems using a retrieval model based on a two tower embedding model, and a ranking model gradient boosted trees. We will also show how you can train your embeddings and build your embeddings store index using Hopsworks and Apache Spark.
Attend this session to learn:
* how to to build a scalable, real-time retrieval and ranking recommender system using open-source platforms;
* how to train item/user embedding models and ranking models;
* how to put all these pieces together in an end-to-end solution for training and operating a scalable recommender/search engine.
Connect with us:
Website: databricks.com
Facebook: / databricksinc
Twitter: / databricks
LinkedIn: / data. .
Instagram: / databricksinc

[@XxXx-sc3xu]

Wow. Amazing presentation on Machine Learning Infrastructure at scale! Thank you.

[@amantandon-ln9xx]

Amazing, thank you

[@haneulkim4902]

Great talk! I've got two questions: 1. Is it a real-time recommendation because upon user query it is enriched with historical data as well as real-time data(stored after each interaction)? Or something else like you are training embedding at near real time? 2. How often do you train Ranking model and does it only train with candidates?

[@MrNagano00]

From my understandings: 1. It's real time because it's capable to do one recommendation per user query without much delay. Instead of spotify that just gives you one recommendation once a week. So it's essential the same thing except the infrastructure required to be able to provide "real time" recommendation is different. But the ML models and processes you'd use to do one or the other would be largely the same. It's not about training it's about how speedy your evaluation is. Also, adding filters and so on, so overall it's a more complex form of interaction with the ML system. 2. You'd want to train your ranking model whenever you feel you've gotten enough data. There's not really a magic number here. As users interact with your website you'll get better data for positive/negative examples; it also depends on how much it costs for you to re-train your algo as well; if it's too expensive then maybe you just want to be re-training when necessary.

[@Gerald-iz7mv]

@@MrNagano00 how is the infra different between batch recommendation and realtime recommendation? also what is the model serving doing in the real-time recommendation system - is it for ranking only? why you need a vector database for the embeddings? dont you need to train an embedding model too?

[@lifeconfused52]

Hello, could you share the code with me? There is no such codes in the github now

[@Gerald-iz7mv]

what model does the batch recommendation use? some content based, collaborative filtering model?

[@Gerald-iz7mv]

how does the model get generated for the batch recommendation service? does the embedding model for the realtime-recommendation only use userdata and not item data?