ChatGPT for your data with Local LLM

[@delhitepawan5804]

nice explantion for ChatGPT for your data with Local LLM

[@jesusisc0mings00n3]

Liked and subscribed. Very useful! I'm going to try to get it running on my M1 as well. That's the question I had and you answered it, what kind of rig you were running. Thanks!

[@jj09]

Thanks for the sub! I used Apple M1 Max

[@BayesTheorem78]

Unfortunately runs too slow on Mac Mini M1

[@ngjenny769]

I follow the step but it seems that the privateGPT setup is different as before?

[@pondyanand]

Very helpful Jacob 👍.

[@jj09]

Glad to hear it!

[@levoniust]

I understand that you are using a Mac on this video, but could you please make a video for a Windows PC with an Nvidia GPU and show us how to run privateGPT on GPU? I haven't seen any other videos on this topic yet.

[@jj09]

Check this documentation: docs.privategpt.dev/#section/Installation-and-Settings

[@benpoz4023]

​@jj09 what kind of deployment would be secure? Would it be possible to do a video that shows how to deploy where the data given to the ai is not used in training and not exposed in any way publicly?

[@jj09]

You could consider deploying to AWS, or you can use Azure Search (they would manage deployment for you): kzbin.info/www/bejne/eJWcfYqNacire7c

[@swaroopas5207]

Can we ask question out context of the local data it was trained on? Suppose i train it over a pdf, can it give me more contextual answers from the internet?

[@jj09]

Nous-Hermes-13b is fine tuned Llama model. So it's trained on the data from internet. By providing your own data you are just supplying it with additional context.

[@haack79]

Are there ways to improve the model or would it take building out your own?

[@jj09]

You can either provide better data (more precise), or fine tune the model.

[@harisytac]

Great Video!!! However, If I follow this video as a tutorial, most of the files do not exist when in clone the GitHub. ingest.py, constants.py, etc. Can some one help please?

[@jj09]

privateGPT repo was updated since I recorded this video. Check out this documentation for latest steps: docs.privategpt.dev/

[@ah89971]

Thank you. Wondering if more powerful model like llam 2 can do better?!

[@jj09]

I think so! Haven't tried yet!

[@thevishalln]

How large can the data be ?

[@MANA-84]

Hello Mr. Jacob is it possible to show us how to add agents to privategpt with out api?

[@jj09]

What do you mean by "adding agents"?

[@benpoz4023]

Are the data sources secure. E.g not madr public in anyway.

[@jj09]

It depends on your deployment, but there is no need to expose data to the internet. Local LLM model doesn't call any external APIs.

[@DePhpBug]

anyone ran into error like : error: metadata-generation-failed ??

[@mdzaman3573]

Hello! Your video content is great, and your thumbnail is perfect. However, your video SEO score is only 19 out of 100. and Performance 0.0 That's why you are not reaching the right audience. Do you know perfect SEO?

[@belcebuu]

It is so slow that is useless

[@jj09]

It's just a matter of the machine you run it on. I ran that on my personal Macbook. Additionally, LLMs are becoming more and more efficient, and require less compute over time.

[@rakancon2215]

thanks for the awesome content Jacob. but i would prefer if you could show how we can download the NousResearch/Nous-Hermes-13b model but thanks a lot for the awesome and amazing content and ill always support the channel

[@jj09]

You can download it here: huggingface.co/TheBloke/Nous-Hermes-13B-GGML/blob/main/nous-hermes-13b.ggmlv3.q4_0.bin Check out my blog post for more details if you need it: jj09.net/chatgpt-for-your-data/