Private AI Chatbot on Your Computer - Step by Step Tutorial

Here is a simple 5 step process for installing and running a local AI chatbot on your computer, completely for free. I made this for non technical people, and you should be able to follow along step by step.
Step 1: Install Ollama ollama.com/
Step 2: Copy and Paste Llama 3 install command using Terminal
Step 3: Add other LLM models (optional) ollama.com/library
Step 4: Install Docker www.docker.com/
Step 5: Install OpenWebUI docs.openwebui.com/getting-started/
Login to OpenWebUI and start using your local AI chatbot.
We just released a brand new course on our E-Learning platform that dives much deeper into running a private ChatGPT alternative on your computer.
You can learn more here: bit.ly/skillleap

[@SkillLeapAI]

Get access to our new Private AI chatbot course as well as twenty other AI courses here: bit.ly/skillleap

[@js6909]

Thank you so much for this incredibly helpful tutorial! I really appreciate you taking the time to walk through all the steps in detail. Running a private AI chatbot on my own computer is going to be a game-changer. Can't wait to dive in and try it out!

[@sweetbb125]

Thank you. At last a tutorial about setting this up, that worked for me. I have zero experience with docker, and now I have it running on my computer! I'd like to learn even more about the usses of docker (just for hobby purposes at home).

[@nicolaspietri4730]

I have this setup behinds a VPN on my local machine at home and it works like a charm on my phone remotely!

[@masdiqie]

Any tut on how to do that?

[@chefbrdunk]

Loved this video! I will be leveraging your instructions with my students as I introduce AI into their course content!

[@OriginalDramagirl]

Downloaded and working perfectly. Thanks.

[@MrLimo217]

Very thorough breakdown, thank you!

[@ASHRIC07]

Thank you! really like the simple step-by-step approach in an easy to follow ways. Managed to successfully set this up! Subscribed to your channel to support you!

[@SkillLeapAI]

Thank you

[@__SKYNET__]

I just installed the 7B 4 GB model, very lightweight and useful in grid down scenarios.

[@georgebalisacan3517]

This is great! Thanks for sharing!!!

[@vpillajr]

Thank you very much for this comprehensive and handy tutorial.

[@SkillLeapAI]

My pleasure

[@CM-zl2jw]

Very very good. Thank you.

[@oldmantimgaming8297]

It took a few tries but I finally got the Linux items and Vm to run.. Could not find the VM setting in the bios without a few videos and one finally showed on a mother board close enough to mine and poof, one more restart and docker is running Llama3!!! Thank You!

[@EdHohlbein]

Super helpful tutorial! Worked like a charm! Question - I'm on a windows machine. What are the BASE specs and BEST specs for a windows machine to make this process as fast/efficient as possible? Thank you. Thank you. Thank you.

[@joeypeleg152]

I enjoyed your video. thanks and kudos. What about updates? Should we uninstall and reinstall for an update?

[@DeepFactsfy11]

You are amazing!

[@SkillLeapAI]

Thank you!

[@DeepFactsfy11]

@@SkillLeapAI Yw, keep making these!

[@MindBlowingXR]

Thanks for this great video!

[@johngluker9777]

Thank you very much for such a clear and concise guide. I wish to use this to analyse confidential interview transcripts as a way to check my own manual analysis and interpretation of the data and this needs to be done offline so this is perfect. Please can you suggest if there is a model that would be best for this kind of analysis? BTW watching you use terminal took me back to my youth using MS DOS!

[@SkillLeapAI]

Llama 3 or Mixtral are the two models I recommend. You can use both at the same time and compare results

[@akshaybholee3263]

Can it connect to sharepoint sites to read from documents on the sites?

[@edgarl.mardal8256]

It was really easy to install, can you show how you use this with gcolab, using ngrok as a tunnel, and dockdesktop running locally and ngrok endpoint being the openWEBUI? This would allow everyone to use it on mobile as well, and also include setup to talk directly with the different models that support it, and explain how uncensored modell can effectively help you in ways that chatGPT cant..

[@robwin0072]

Hello, does this Private GPT have something similar to ChatGPT’s custom instructions feature, or can I create separate specialty setups for statistics, Authoring, and/or Coding (COBOL, C%, Fortran, etc.)? Can writing (authoring ) samples of mine be uploaded for specific chat sessions?

[@ktajax]

Great video,never even knew about this stuff! Only problem, my computer memory isn't large enough. Where do i even start?

[@SkillLeapAI]

If your computer doesn’t have the specs to handle it, you’ll be limited to using this on meta.ai. Running it private requires a beefy computer right now

[@hemanths2186]

If i host this in a private server for my team .. will my private files and data i used for this llm gets stored or trained by the models of the other person using the llm other than me and my team..?

[@EyeOfStormProductions]

I just bought an HP laptop the day I saw this video. I'm stuck after downloading llama... it won't open after download for me to continue...

[@dom12splayground]

I have terminal on my phone, so actually it would work then to

[@Irykamii]

is it possible to use this uncensored?

[@zipporapper]

what kind of laptop hardware do we need? will it work with normal office thinkpad without a high end gaming gpu for the laptop?

[@SkillLeapAI]

It works with a lot of laptops. But the speed will vary. I think if you have a gaming gpu, it should be decent speed. The big speed issue I noticed is if I upload a lot of files to my knowledge base

[@madnessmaker1215]

⁠yes any laptop with gaming gpu on it , ai loves nvidia gaming gpu even if you have 4gb vram like me , im able to run ai models but make sure if you have less then 8gb to download 1.5b and 3b and 7b other then that you 'll see alot of problems .

[@cerebralvision]

Why wouldnt you use something like LM Studio instead of doing this via localhost and a docker instance?

[@SkillLeapAI]

That's a very good option too. I was more familiar with this option and it's the version I've set up on my computer.

[@tiffanyw3794]

can you connect stable diffusion to lm studio

[@jcdenton1664]

What’s the limit to the number of documents you can upload? Is there a limit on size or count?

[@SkillLeapAI]

I don’t see any limit but it hasn’t worked for me with more than 10. It lets me upload them but doesn’t give me useful responses. I’ll see if they have documentation on this.

[@jordambee]

before I take your course: can I use this in a server environment, so all of my employees can work with the chatbotˋ?

[@SkillLeapAI]

I don’t believe you can with this set up. The only LLM options I know for server based environments are the name brands like Gemini, and ChatGPT on azure OpenAI service. I’m sure a developer can set this type of thing up on a private server but I don’t get that far in my course

[@DeanCassady]

Would you make a video about interact with documents, Pls

[@SkillLeapAI]

You got it

[@fooku5508]

Windows Powershell update requires installs alternate powershell.

[@SkillLeapAI]

Does Window Terminal work the same? I'm going to make a new video tomorrow with my Windows. My main computer is a Mac and the documentation said it was the same process.

[@ccarl0123]

How do you get the files for knowledge base . If yiu want your bot to focus on certain things . Or overall knowlwdge

[@SkillLeapAI]

the knowledge base if your own documentation. Like blog posts your wrote, company policy, etc. It already has knowledge without your own documentations.

[@ccarl0123]

@@SkillLeapAI thank you. Ok so I need to research my own knowledge base. I'm guessing perplexity is good for that or any other

[@DeanCassady]

Oh, man , final solution!

[@SendiFazri]

Windows only ?

[@SubinKrishnaKT]

When I open the web UI along with my local LLM Lllama 2, I can see GPT-4 and GPT-4o in the choose LLM dropdown. Do you know what is causing this?

[@SkillLeapAI]

If you add OpenAI api under settings, those appear in the menu. That’s the only way I know of

[@SubinKrishnaKT]

the API key added itself automatically. It somehow picked the api key from somewhere.I didn't do anything. 😂 I was searching for this so that I could host an LLM locally and do organization-related tasks. Now I'm having doubts.

[@garyhonas1848]

So interesting thing just happened ...Chatbot got math wrong when I was trying to ask it an easy conversion of how much of a 3qt bag of activated charcoal to add to a 2.6 pound bag of coco coir/perlite 79/30 mix to get a 70/30/10 mix ..the answer is the entire bag ...first it kept thinking it was 2.6lbs of coir and perlite each ...but the big thing was it tried to tell me 3 quarts is equal to 2.086 pounds ...the. it admitted it made a mistake...I didn't think thay was possible?

[@BigMac-eq4kv]

is the MacBook Air m3 10 core gpu good enough for the last thing?

[@SkillLeapAI]

Yea I have M3 and have had no issues yet

[@BigMac-eq4kv]

@@SkillLeapAI thanks a lot

[@dbreardon]

Ok, so once everything is install and you play with it and it all works, what do you do the next time you want to run it? Let's say, my computer has been turned off and then on again. What do I need to run to start the ai again? Just docker? I would like instructions in how to restart everything. What are the step?

[@SkillLeapAI]

Launch docker and click on the port to launch it. You can also press stop on docker to stop it

[@dbreardon]

@@SkillLeapAI Thank you! Just got Docker up and running. Alright, everything is up and running! Unfortunately on my 2019 ryzen apu even phi3 is slow as molasses. Oh well, I thought that might be the case. I have to give you great props on this tutorial. I've seen a number of these olama-webui install tutorials and your tutorial was the most clear. The best! Thanks for your help!

[@dakshinamurti]

Im so frustrated... when i run the app after intalling it, nothing happens! nothing opens! what is going on?

[@VideoExpert666]

I believe installing these apps turns your computer into a remote zombie that receives instructions to run a web based network. So in short your power is contributing to the network with out your knowledge. This is what bitcoin is doing… people think their computer was solving math problems but in reality they were part of a huge network that’s creating value with bitcoin…. I tried telling people but nobody understands how information works.. if you ever get bored look at what info is being shared from your computer without your knowledge…. Now imagine that times 1k! I have a computer that will turn on I the middle of the night if I sneeze… thats how sensitive it is.

[@pilotpawanc]

Install a ton of apps to get the functionality. Why would I not go directly to chatgpt?

[@SkillLeapAI]

Privacy. Lot of people can’t just upload documents to ChatGPT. It’s either for privacy, work rules, sensitive info, etc. this is local and private. You’re not sharing anything with any company

[@SkillLeapAI]

I made this video for those people that need a private solution due to work rules or privacy concerns

[@maikvanrossum]

Of course it’s easy to follow your good step by step instructions, but the real magic is as usual happening behind the curtains… 🪄 So how’s ones privacy guaranteed and kept private in the long run…?

[@Darksagan]

Docker doesnt work on older macs.

[@jeremy6384]

turn off the wifi to make sure this data isnt going anywhere...until you reconnect to the internet.

[@KnowingItForYou-Tutorials]

can u upload pics?

[@fooku5508]

Not even close to the same install/setup process on Win 11 PC.

[@SkillLeapAI]

I’ll run it on my windows machine tomorrow

[@1984Dystopia]

I have it running in windows 10.. It's so easy even I did it in minutes and wrote half a book of 20 chapters. Life is simple when you think and believe it is simple.

[@karlpedersen3342]

how beefy should my computer be?

[@aintneveryours4659]

Meta and Privacy don't go hand in hand.

[@Nahsh5ba]

Completely private chat client inside spying Chrome inside spying macOS. Nice.

[@fooku5508]

A "Windows Preview" version that only installs Ollama over and over again. Thanks for catering to Mackies only. UNsubscribed.

[@1984Dystopia]

Are you a troll. I installed it on windows 10 easily. Running perfect and fast.

[@ASHRIC07]

Using the same steps, I successfully installed and run it on Windows too. Hope this helps.

[@mirtafagundesdossantos2326]

@@1984Dystopia Yes, Win10, but not Win11. Doesn't work for me either

[@danielriveradeltoro4262]

“you just lost an important customer” ahh