Generate LLM Embeddings On Your Local Machine

[@moumniable]

i just love how diverse your videos are ! even when i don't particulary look for something your videos drives me to learn more. thanks ❤

[@rons96]

Not bad, but if i may say any tip, i would say to use a sentence-transformer from hugging face for embeddings and then use a llama like llm only to customize the answer, because models created just for embeddings seems to be more accurate for this task. Also, langchain module is easier and useful than using numpy and requests, with lot more features. I use this setup most for RAG and seems to work pretty well.

[@henrischomacker6097]

Very interesting hint. Why would you suggest to use a sentence-transformer from hugging face for creating the embeddings instead? Which method does a sentence-transformer from hugging face use to create the embeddings and which one does ollama use?

[@hackdonalds]

I tried llama2 and mistral embeddings through ollama embeddings api. The similarity search results were sht compared to Xenova/all-MiniLM-L6-v2 or gte-small

[@rons96]

@@hackdonalds yes, llama for embeddings is not good, with sentence-transformers i mean that one you mentioned, then use llama to elaborate the sentence. There's another model better for embeddings but it will require more resources and i don't remember the name now.

[@rahulmakwana663]

@@rons96instructorembedding

[@godwinntowdanso4111]

Spot on. Simplified presentation

[@user-td4pf6rr2t]

Cool guide. Very well explained. +1

[@mohammadalibazyar5079]

thanks, bro... really helpful ❤

[@Darkev77]

Powerful video! Guys, anyone knows how I can generate these embeddings if I were to deploy my app remotely?

[@iamreallybadatphysicsbutda8198]

Great video! 😃

[@EliSpizzichino]

that's very interesting! I imagine you can build your local knowledge base in this way... I need to make one for code-snippets that store knowledge bits find around.... Is `d` dimension fixed by the model? does it mean I have 4096 bytes to store my embedding?

[@dangalimov7435]

Brilliant!

[@ddschaefer]

Great video! But where comes faiss into play?

[@peterparker5161]

I tried this with LLAMA3 8b locally. It can work if the sentences are short enough. But when I started plugins in long paragraphs (youtube transcripts) it becomes basically useless. Transformers that are creating for embedding (BERT for example) seems to work better. They also have lower computational cost compared to LLAMA. I tried again with "nomic-embed-text-v1.f16.gguf" and it works much better.

[@all-in-one-890]

First comment ❤ and ur videos are fantastic

[@JuanDiegoSalamanca-oy6xs]

if you do it in Colab what url do you use?

[@JJTradess]

🔥

[@roberthenry7283]

where is the source code

[@AGASTRONICS]

comments[-1] #FirstComment😅