NEW Transformer for RAG: ModernBERT

[@code4AI]

With the automatic audio dubbing from KZbin /Google you hear a synthetic voice in your regional language. To hear my original voice in English, switch to "Default" or "English" in the settings. Thank you.

[@CharlotteLopez-n3i]

Modern BERT = game-changer! Flash Attention and savvy design make it a hot topic for info retrieval and rec systems. Kudos to the team for this awesome open-source release.

[@malikrumi1206]

3 burning 🔥 questions: 1. What about DSPy? I haven’t seen a lot of conversation or adoption of that idea. How does one prompt modernBERT? 2. With NVIDIA in the mix, what are the chances we can get this on Apple Silicon/MLX anytime soon, if ever? btw, I can use some small open source models on my Silicon Mac without MLX… 3. Did you say modernBERT doesn’t need or use a tokenizer?!?

[@andrewandreas5795]

Nice video! could you please explain where exactly in a RAG pipeline could this new model be employed? Not in the generative part, or?

[@chegouparalutar]

You use it in the R of RAG :) Explicitly, 1) to embed your documents into a vectorstore and then again 2) to embed your query to calculate its similarity to the ones in the vectorstore. Now, you can RETRIEVE the most "similar" documents, to AUGMENT the context of your GENERATIVE model.

[@andrewandreas5795]

@@chegouparalutar Thanks for your answer. I am a bit confused, so is this an embedding model?

[@chegouparalutar]

@@andrewandreas5795 Shortly: Yes, for a typical RAG-Chat Model, yes. Longer answer: Typically, one would use models with causal attention (tokens only attend one direction = backwards) for **auto-regressive** generative tasks (chatting), since it is way more efficient (check out KV-caching if interested) compared to bi-directional attention but theocratically you may also use it. Models with bi-directional (both backwards and forwards) attention are way more powerful in understanding the text, since the context of a word does not only depends on the words behind it, as well as the words in front. Therefore many generative tasks that do not really on an auto-regressive pattern like translation, summarization are popularly done via bi-directional models, as well as text embeddings. Therefore, you may also use ModernBERT not only as your embedding model but also as a generative model in your RAG pipeline whether to summarize text or answer questions. My take: As far as I understand, the current ModernBERT release is not fine-tuned for any of those tasks. Looking at the models they are comparing to (like Nomic and GTE) they are aiming for an embedding model, and the Hugginface team are releasing notebooks to use it as an embedding model. IMO, it is time for a bi-directional model that generates short and structured answers and I am kinda sad that they are not going that direction.

[@carlhealy]

Oh wow, this is really exciting. Thank you for sharing!

[@wdonno]

Thanks for covering this!

[@IvarDaigon]

I'm waiting for the Quantum Enabled Version called Q*Bert.

[@aliettienne2907]

Building a reliable LLM architecture that can retrieve information faster with every unit of information is just the ideal conditions to obtain. It's like having an interpreter that can interpret a foreign language between you and a foreign person much faster. And if you can receive the interpreted language faster with quick listening skills then even that will be a plus for you. 😎💯💪🏾👍🏾

[@irbsurfer1585]

YES! Sweet! Beautiful!

[@asimabusallam3147]

❤ thanx

[@john23232]

English only for a RAG… Same problem as the initial BERT. Good to show off the technics but useless in practice by most of us. Llama 3.3 70B makes a better candidate to be honest, speaking of only open-source models.

[@code4AI]

You mix encoder-only with decoder-only transformer. And you do not speak for "most of us".

[@raymond_luxury_yacht]

What about dogbert

[@raymond_luxury_yacht]

Bruh you're not helping by not linking!

[@NE0123456789]

hi @code4AI hast du eine kontaktmöglichkeit bitte 🙏