Easiest way to deploy privateGPT app! Ideal for Businesses & Organizations

[@mikew2883]

A Render deployment video would be very helpful. Thanks!

[@VaibhavPatil-rx7pc]

Yes

[@firetownplatformfinders3996]

I've been watching you videos a lot lately and you are really helping me make progress in learning to develop the tools. Thanks fo the videos. They are super helpful.

[@menloparklab]

Thanks a lot! 🙏

[@newtech9541]

Hi . Bro could you please provide an video where we can automate via front end on custom data ingesting from front end and then use LLAMA2 getting trained on those documents and then provide . Like full automation on localgpt

[@joser100]

Very clear, thanks. I have been working with PrivateGPT from the early days, where ingestion was a real issue, now quite successfully solved, they have made significant improvements, but the inference is still an issue, with 1000 docs that I added (I am assuming that this, plus my laptop CPU limitations) might be the reason for the slow response, but I get over 15 min wait to get any response in my case. So I am really looking forward to improvements on that side. The other issue is the completion of the inference stage, I am assuming that they do a similarity search on the Chroma DB and apply a Top-K (or other algo) to get relevant chunks to use to build the final response, but I have seen this failing, again, likely due to the dataset being quite raw (redundant data, etc) which impacts the quality of the final output at the end. All in all very glad to see the project progress and response, but wanted to share with you on those points and check for your feedback as well about those

[@menloparklab]

I agree, the ingestion have improved quite a lot. But inference is very slow. And 15 mins is a looooong time for inference. Yes they do apply similarity search, which has it's own limitations, as you mentioned. Do keep us posted if you find faster infrerence solutions. I am also on a lookout.

[@Psychopatz]

may i know why you choose groovy instead of mistral?

[@aspringer3960]

Good Day, no fcntl for Windows, any alternatives?

[@cooltrucs8813]

Thanks for your tuttuto! How to make the interface of reporeresponse like chatGPT i.e. the text wraps and not fit on a single line like what I see there.?

[@menloparklab]

Might need to update streamlit.

[@olegxander]

Thank you for more great info! In the future, can you please show an example with a multidocument retrieval QA chain and perhaps some chain tools involved as well. Cheers!

[@CREWorxMedia]

Another great video! Yes, please do a video for deployment on render!! 🙏🏽

[@srirams1865]

hi..it runs on offline or online network?

[@jasonbrown-bz3pf]

Love your video! Calm and soothing voice :) Would you be able to do a video about GPTEngineer?

[@menloparklab]

Thanks 😊 Yes GPTEngineer is something I wanna get to once I complete recording course videos. Hopefully in the next few weeks.

[@joaocardoso560]

Nice video. Is it possible to run the FastAPI backend in Windows using waitress-serve, since with gunicorn I receive the error no module named 'fcntl' ? Thanks a lot

[@menloparklab]

Actually I missed to try this on Windows. Might have to install some additional dependencies.

[@Live_draw_today]

​@@menloparklabsir please tell, how to install it in Windows.

[@mrasoahaingo]

Very clear and simple explanations! Thank you! You really helped me to understand how those things work together!

[@ChuckWilliamsTechnology]

Great..yes please make a video on deploying to Render as well....thanks a ton

[@menloparklab]

Sure, will do. ;)

[@ImadAhmad-j7s]

how to run on gpu i have gpu installed. there is no explicit mentioning of cpu or gpu use in code?

[@chikosan99]

Thanks a lot! great video can i ask about multiple uploaded files? Thanks

[@menloparklab]

Thanks! Yes you could upload multiple files and ask about them.

[@RohiniSharma-q6i]

How to do the same on windows. As it is returning an error : No module named 'fcntl' while executing gunicorn app

[@Justme-dk7vm]

Can't I deploy this local chatbot on my webiste?

[@rodrigorubio3498]

Nice! Do you see any issues for this to run in Docker?

[@menloparklab]

Actually this should run well on docker. Lemme know how it goes. 😉

[@delafinn899]

I’m curious if there are pre made vector databases for privateGPT. So that way you can use your own documents and data from the internet. Asking for a friend

[@menloparklab]

You could use any of the open source vector databases like Qdrant, Weaviate, Chroma etc for the PrivateGPT

[@delafinn899]

@@menloparklab thank you 🙏

[@khalidbouziane4005]

I have seen the most of your vodeos, they are really great and easy to follow, since your videos are one of the few videos available in youtube that go through almost all the process from prototyping to deployment (full stack + main), it would be really nice if you create a video or a tutorial where u can explain how to build a multitenant SaaS around the whole process of Q&A of docs. Thank u again.

[@menloparklab]

Thanks Khalid! Yes, building a full course around this topic.

[@Gokulhraj]

How many requests can it handle at a time?

[@menloparklab]

One could use gunicorn to run the app with multiple workers based on the available compute

[@mohamedzekkiriya4854]

Is it possible to run the FastAPI backend in Windows using waitress-serve, since with gunicorn I receive the error no module named 'fcntl' ? also i tried to use waitress but i am getting below error ERROR: Could not build wheels for hnswlib, which is required to install pyproject.toml-based projects. I couldn't install hnswlib in windows. can you please help me on this?

[@snehaarora-f8y]

i wants to deploy it in my organization for all users, when there are multiple queries at a time more then 10 private gpt got stopped, how to do it

[@MANA-84]

Thanks a lot, kindly if you can provide the Windows commands , it would be perfect :)

[@komalgupta1846]

Can you help by Making a video on how to handle multiple queries with private gpt. I have deployed private gpt on flask application but when in private network some are trying to ask query at same time , It shows error.

[@darknessguy4221]

Hey man, can you do a video of FlutterFlow + Flowise?

[@menloparklab]

Oh yes that’s on my list. Will try to get something out soon. 😉

[@nishitmaheta]

with this latest version uploaded documents are not loading in dropdown

[@menloparklab]

Oh oops, will have to look into it

[@TrackHawk7]

Thank You , was looking for something like this, hope someone makes a telegram bot out of this

[@YoungVow]

When I deployed the cloud deployment version just to test it out, I uploaded a document and hit embed, but the document does not show up in the retrieval section of the UI after uploading... I did get a "Documents embedded successfully!" message though so I don't understand why it's breaking.

[@menloparklab]

A few times, memory issues have led to such a behavior from what I noticed.

[@rrrila8851]

Why when using it on a Vertex machine, it does not make use of ALL the cores and threads it has?

[@menloparklab]

That's interesting. Haven't used Vertex, but most likely there is some way to trigger it.

[@rrrila8851]

@@menloparklab If you could help me out, I would appreciate it, as I have NO clue whatsoever on how to make it work myself.

[@photon2724]

i wonder if there are any startups giving PrivateGPT solutions to buinsesses.

[@menloparklab]

I came across a few, but can't remember their names. Some are part of the latest YC batch.

[@DarkOceanShark]

I have been working with PrivateGPT, and whenever I give it queries, it ends up hallucinating with some relevant content in between, picked rrom the document.

[@menloparklab]

Yes still not the best with the current setup. There are some other models that might do better. Will look into it.

[@satriavi]

Hi.. Does privateGPT works with language other than English?

[@menloparklab]

It might work with some of the Latin languages. Let me know what you find out.

[@SylvanSequester]

can you please share more about how privategpt compares to chatgpt with gpt3.5 and gpt4 in terms of power and price? Also can you input the API for privategpt you created into flowise? Sorry I’m a novice with this stuff!

[@menloparklab]

So PrivateGPT keeps all of your data local, so nothing gets sent to OpenAI or other providers. It is quite slow as of now. Since PrivateGPT works locally, there is no cost, except for the harware cost. One could connect the APIs to bubble frontend and run like our previous videos. Using in Flowise would be a bit tricky.

[@ComicBookPage]

This looks very cool. I'm trying to get this running on Windows but I get a "ModuleNotFoundError: No module named 'fcntl'" message which trying to run the backend.

[@menloparklab]

Haven't tested this on Windows myself. you might need to install a few additional modules. Please check for any discussions on PrivateGPT repo. Let me know if you find something.

[@GyroO7]

Your commands doesn't work on windows :/

[@menloparklab]

Oops missed to mention windows commands

[@VaibhavPatil-rx7pc]

Can you make a custom app to read pdfs any format having information and through chatgpt response required information as output in any form. For example invoice pdf as input and required customer identification details in government regulations format Thanks in advance, I'm following your channel

[@menloparklab]

Yes, you could read pdfs and respond back with specific information you are looking for. With privateGPT it might be slow, GPT models will do a quick job.

[@sprekkatsiu779]

Can we get this in JavaScript?

[@menloparklab]

Original PrivateGPT was in python, so kept it in python. Not sure if GPT4ALL is supported in JS. You could use APIs built in this video as a backend and call those APIs using JS frontend as one option.

[@ikjb8561]

Not sure if it is easiest. The answer part is slow as... Also if you read their website they say this is not for prod but for evaluation. They will get there no doubt but last time I tried it took too long to get an answer.

[@menloparklab]

I agree, the answer generation is very slow as of now. Most likely will improve in the near future. I was referring to ease of installation; ease of use is not yet achieved by the open source models. ;)

[@rev.jonathanwint6038]

I do not want bill gates getting his hands on my Data

[@RayHorn5128088056]

Private GPT is not exactly "private". Be careful with your data. You would be better served by using analogous data for this technique if you care about the value of your data.

[@photon2724]

why is it not truly private? would love more info about what makes it 'not so private' if it works completly offline

[@RayHorn5128088056]

@photon2724 Because the data goes where the model runs. OpenAI model run in their data center and that's where your data goes. Keep your data abstract and clean of anything proprietary if you want privacy.

[@photon2724]

@@RayHorn5128088056 Got ya! I'm aiming to use this with open source models running on my GPU. Hopefully that make's it completly private.

[@harrykekgmail]

great. thanks