How to Implement RAG locally using LM Studio and AnythingLLM

This video shows a step-by-step process to locally implement RAG Pipeline with LM Studio and AnythingLLM with local model offline and for free.
🔥 Buy Me a Coffee to support the channel: ko-fi.com/fahd...
🔥 Get 50% Discount on any A6000 or A5000 GPU rental, use following link and coupon:
bit.ly/fahd-mirza
Coupon code: FahdMirza
▶ Become a Patron 🔥 - / fahdmirza
#lmstudio #anythingllm
PLEASE FOLLOW ME:
▶ LinkedIn: / fahdmirza
▶ KZbin: / @fahdmirza
▶ Blog: www.fahdmirza.com
RELATED VIDEOS:
▶ Resource lmstudio.ai , useanything.com/
All rights reserved © 2021 Fahd Mirza

[@bomsbravo]

You are my favorite KZbin! This is amazing. I got to know about the LM studio through you and now I am going to try this out. I was trying to RAG llama3 but I ran into a lot of errors. But I think as this is a simpler method, I would be finally able to chat with my pdfs

[@fahdmirza]

Yay! Thank you!

[@publicsectordirect982]

Another useful and informative video thank you

[@fahdmirza]

cheers, thanks

[@PushpendraKumar-it4wf]

Getting error while uploading a pdf or other file...any remedy

[@FrankSchwarzfree]

Great video TY!

[@fahdmirza]

You're very welcome! . Please also subscribe if you haven't already, thanks.

[@Alex29196]

In my opinion, the latest release of Msty is much more functional and has a better UI. AnythingLLM advantage is that connects to LMStudio.

[@maximt1401]

I have a large json file I would like to extract insights from. What is going to be the best way to do this ?... Msty + Which LLM ?

[@fahdmirza]

I have also covered it today. thanks

[@frosti7]

Is there a way to have Vision as well ? that would be amazing!

[@fahdmirza]

would need to check when time permits.

[@blazar8565]

Hi, Thanks for the tutorial. What did you choose under setting: Chat.... and Agent:... LM Studio or Llama?

[@fahdmirza]

its been sometime I did that would have to revisit this

[@blazar8565]

@@fahdmirza I ask the creator. He said to use Ollama.

[@dinamohamed13600]

Thank you , it's exactly what I was searching for. I have a question, Is there a local way to make my model search and use data inside a SQL database ?

[@NextLevelSpaceExplorer]

Chat2DB

[@SolidBuildersInc]

Very nice solution for RAG and using a local model. I was attempting to do this with Streamlit but this appears to be very clean approach. How can we use Colab to point to a public URL with Localtunnel. I seem to have a challenge in getting that working ? Thanks for Sharing

[@fahdmirza]

thanks

[@sadekf-l9u]

think you are AnythingLLM IS safe?

[@fahdmirza]

sorry you would need to do your own due diligence.

[@JohnPamplin]

I'm trying to hook AnythingLLM into a Slack chatbot, because you can use multiple models for docs and websites (even Google search, I think). While LM Studio has a server port, I don't think AnythingLLM does, does it?

[@fahdmirza]

would need to check

[@dennisg967]

Thanks for the video. Exactly what I was looking for. Question: can we still use lmstudio interface for to chat with a model after we added new content to it or do we need to exclusively use anythingllm? In other words, does lmstudio see the added content, or does only anythingllm see it? Thanks again!!!!

[@fahdmirza]

once you've added new content to a model using AnythingLLM, you can still use the LMS studio interface to chat with the model. The new content should be visible and accessible through both interfaces. LMS studio and AnythingLLM are designed to work together, so changes made to the model through AnythingLLM should be reflected in the LMS studio interface as well. However, it's possible that there might be some delay or caching issues, so if you don't see the updates immediately, try refreshing the LMS studio page or checking the model's configuration.

[@dennisg967]

@@fahdmirza , thanks for the info. I have watched a few more of your videos. They are really helpful. Thank you for making them!!!!

[@skillsandhonor4640]

thank you

[@fahdmirza]

You're welcome

[@jesustejerinarivera4627]

but lm studio rag workspace?

[@bhawanirathore3105]

Sir nice vedio , can u pls tell me sir what are the benchmarks to calculate LLM model performance and compare with other LLMs in term of performance and privacy

[@fahdmirza]

I have done few videos on benchmarks, plz search the channel.

[@longboardfella5306]

It's been reported that AnythingLLM has a critical security flaw. Just fyi

[@fahdmirza]

thanks for info. could you please also give link to the source of this info?

[@longboardfella5306]

@@fahdmirza It was a Medium report titled: A Critical Vulnerability at AnythingLLM - Understanding and Mitigating CVE-2024-0765

[@fahdmirza]

@@longboardfella5306 Thanks, thats very helpful. github.com/advisories/GHSA-f7cx-hq8m-95w6

[@publicsectordirect982]

Thanks for sharing 👍

[@shatfield]

Hey there, this vulnerability was actually patched months ago and the Medium report was a report showing how the vulnerability was fixed :)