The Ultimate Getting Started with Local LLMs Guide

[@WillWehi]

I wish i came across this 2 days ago. I spent nearly 10 hours reading Reddit to research this stuff to try on my still-waiting empty GPU system. You covered much in 20 minutes! Subscribed, and would love to see more.

[@MemphisVon]

Thanks for this and your writingtools video. Happy I found your channel.

[@DaveEtchells]

PHENOMENALLY useful! Absolutely what I’ve needed to get started with local LLMs! One question: Doesn’t the embedding algorithm for RAG have to match the specific LLM you’re using, meaning that you’d need to use a different embedding for each model? Also, when does the embedding run? (Upon import, manually, or at runtime for each chat session?) How do you handle updated documents, can you just have the embedding run periodically, such as every night? Thanks for this reference, you saved me dozens of hours figuring this out - or more likely made it possible for me to do it at all!

[@GosuCoder]

My understanding is that RAG doesn't require you to match the encoder with your language model, though they should be semantically compatible. You can mix and match (like using BERT embeddings with GPT models), as long as the encoder creates high-quality embeddings that capture the meaning of your documents effectively.

[@GosuCoder]

Thank you for the kind words!

[@DaveEtchells]

@@GosuCoder Thanks for that info, it’s a little confusing to me. My conception of it was that the embeddings needed to translate into vectors having the same dimensionality and “meaning” for lack of a better term as the LLM coordinate space they’d be used with. OTOH, maybe there are universal encodings for input vectors that the LLMs are then trained against. - I guess I need to spend some time chatting with ChatGPT or Claude to learn more on the subject. (It’d make sense that embeddings would be at least somewhat universal; it’d be a PITA if you had to re-encode your whole dataset every time you wanted to try a different model.) Thanks for taking time to answer!

[@PerFeldvoss]

Great, but I really wonder what you are on to regarding "rumtimes"... I never ran into that 😉in this context and I don't know how to get access to "Mission control" in LM Studio is it a hidden feature or not part of LM Studio?

[@GosuCoder]

I probably should come back with another video to explain in depth what runtimes actually are.

[@anatoliypodkladov2173]

RAG please 🙏

[@anatoliypodkladov2173]

and deepseek

[@GosuCoder]

I'm working on a RAG video now, hopefully i'll be done in the next few days!