LM Studio + AnythingLLM: Process Local Documents with RAG Like a Pro!

[@nikhils7583]

Great one , seriously a proper human teaching us ai😅, subbed

[@PallaviChauhan91]

Great teaching style. Please also share more tutorials on model finetuning and agents.

[@danielmacedo2154]

Thx! very useful for personal projects with very specific information. Amazing video, great work

[@Jzguan]

Omg I've found u !!!! Been searching over the net. None of it are legit. Yours are true value

[@coding-ai-now]

Thank you for such encouraging words. It makes the effort worth it.

[@Jzguan]

@@coding-ai-now please do not give up. cant wait for more. im learning and implementing what you have taught. been lost .. now im clear =)

[@arooly]

Thx great video, very clear it worked perfect.

[@marshallodom1388]

Thanks for such a great video, Can I use this method to read construction documents (pdf) and have it answer questions about the content relevant to specific trades? I understand it can't read Autocad construction drawings but can it do take-offs? (enumerate quantities across multiple drawings labeled with text). Can it count or do addition? Can I load multiple files? What are my limitations; total token count of the docs (4096) or is that the question-response size limit? Or must total size of doc loaded be no greater than some certain amount? Which LLMs are best for reading docs; Vision models, Coding models, Math models, or Instruct models? Why can't LLM studio handle this on its own? If it CAN do all of this how much longer before I'm forced into retirement? Got any more vids about this?

[@Mixdreamer]

How can you check the model performance in the anything llm gui ? Like tokens per second I know LMStudio interface has it

[@Mixdreamer]

How can you check the model performance in the anything llm gui ? Like tokens per second I know LMStudio interface has it but anything did not

[@anonymous_friend]

I tried this, but Anything says 404 Failed to load model "nomic-ai/nomic-embed-text-v1.5-GGUF". Error: Model is not llm.

[@alqaimyouth]

hi and thanks for the info, quick question why using llm studio while you can run ollama and anythingllm?

[@coding-ai-now]

It's just a nice GUI for managing your models. Plus you can get other models as well. If you're just using Meta's models and are not going to use LLM Studio features then you don't need it. This was just a continuation from my LLM Studio video.

[@2LlamasENT]

Is anyone having issues with Anything LLM? I enter the LM studio base URL and max tokens. However, under LM Studio Mode. it says "Loading available models" I am running the server on LM studio. but the models do not load on Anything LLM. Thanks!

[@sathishchinthana]

hey what happen if i trun off the server .can you explain how close these safely and restart

[@coding-ai-now]

If you have you're volumes set up then It's just like restarting your computer. Everything that was save will see be there. If volumes aren't set up then the memory is ephemeral and everything will be gone.

[@jordonkash]

Seems that the retrieval worked flawlessly despite you using the smaller llama3.2 3B in LM Studio, do you see any RAG performance difference with larger models?

[@coding-ai-now]

I haven't seen a big difference but I haven't done a lot of testing to compare either.

[@wgabrys88]

I encourage you both to try 14b qwen instruct 8kversion - the difference between 7b models and that one is amazing, for me it's the same as chatgpt and the others $ models

[@Mixdreamer]

So the way this works is: AnythingLLM does the API query to vector database which has your files and pipes it through LLama server ? Let me know if I stated it right 😂

[@KenBob52]

it would be great if your monitor was in focus, we could then see what you are doing

[@rikmoran3963]

I could see it fine.