How To Host AI Locally: Ollama and Open WebUI

  Рет қаралды 12,875

Naomi Brockwell TV

Naomi Brockwell TV

Күн бұрын

Пікірлер
@raphaelcazenave-leveque7991
@raphaelcazenave-leveque7991 2 күн бұрын
Cool Geeky video. Thanks Naomi.
@NaomiBrockwellTV
@NaomiBrockwellTV 2 күн бұрын
Glad you liked it! 🤓
@natemarx4999
@natemarx4999 3 күн бұрын
Naomi makes light blue look extra elegant.
@markldevine
@markldevine 3 күн бұрын
Best content for Nov 1, 2024. You get my vote. Great job!
@NaomiBrockwellTV
@NaomiBrockwellTV 3 күн бұрын
thank you for watching :)
@asan1050
@asan1050 3 күн бұрын
Naomi Brockwell Thanks for posting this video
@NaomiBrockwellTV
@NaomiBrockwellTV 2 күн бұрын
Thanks for watching!
@GregBressler
@GregBressler 2 күн бұрын
Great video! Kinda glad this was recommended by Google...wonder how Google knew I was interested.
@Zane-e8b
@Zane-e8b 2 күн бұрын
This is a HUGE. Thank You!
@tikishark1
@tikishark1 2 күн бұрын
I’ve done this with locally based AI image processing software like comfyui, can’t wait to now try chat as well. I really appreciate the background and tutorial.
@NaomiBrockwellTV
@NaomiBrockwellTV 2 күн бұрын
I'm glad you found it helpful!
@johnlegend1360
@johnlegend1360 3 күн бұрын
Omg I forgot about this! Glad you made another video for it!
@ronm6585
@ronm6585 Күн бұрын
Thank you Naomi.
@terrorbilly1
@terrorbilly1 2 күн бұрын
I honestly can't appreciate this enough!
@Saintel
@Saintel 3 күн бұрын
I've been waiting for your take on AI and I was not disappointed. Great work as normally. Currently using Msty locally.
@jozsefizsak
@jozsefizsak 3 күн бұрын
Always a pleasure. (Forgive my lack of originality!) 😊
@johnlegend1360
@johnlegend1360 3 күн бұрын
Love it! Using this I won't need to agree to terms that allow someone to own MY data. It is mine.
@Warframeplayer-s4l
@Warframeplayer-s4l 3 күн бұрын
This right here can replace the internet
@ninjanape
@ninjanape 3 күн бұрын
YES! This is literally on my to-do list for next week, and now I have a great source of info to start from. You're the best Naomi :D
@karlamellow1655
@karlamellow1655 3 күн бұрын
I love this channel. Clear and easy instructions, even for people that aren't amazing with computers. I now have a private Ai!
@NaomiBrockwellTV
@NaomiBrockwellTV 3 күн бұрын
That's wonderful!!
@Justmylukc
@Justmylukc 3 күн бұрын
KZbin should be sued for all the lies in the ads.
@RandyV2max
@RandyV2max Күн бұрын
great video!
@lendaquino979
@lendaquino979 3 күн бұрын
wow , you smashed it ,
@BogDrakonov
@BogDrakonov 3 күн бұрын
I’d love to see a tutorial on running your own StableDiffusion as well
@bravohomie
@bravohomie 3 күн бұрын
I'd like to make an honorable mention for GPT4All. Turn key UI with CPU based models for more compatibility
@jwb6583
@jwb6583 3 күн бұрын
Very nice! Have a nice weekend!
@theaugur1373
@theaugur1373 2 күн бұрын
If you have an Apple Silicon Mac, LM Studio is another option that also has a CLI tool (lms). It supports MLX (optimized for Macs) and GGUF model formats.
@samo-zaposao8757
@samo-zaposao8757 3 сағат бұрын
So helpful,thanks
@TazzSmk
@TazzSmk 13 сағат бұрын
very well made tutorial! :)
@marcus_cole_2
@marcus_cole_2 3 күн бұрын
You should talk about the several.AIs worth considering for llm art and more especially those using the Pinocchio installer engine for PC. Installing on a secondary drive, like D:, is recommended. For Android, the 'Local AI Offline Chatbot' app is available as a fully offline option, with no internet required. Although this AI is limited to a G-rating, it offers a solid offline experience for those seeking a private, all-ages model. No offensive material allowed. for a private AI.running local"
@itchylol742
@itchylol742 3 күн бұрын
1:03 "a tutorial made by The Hated One" is such a hard line out of context
@lufeacbo8
@lufeacbo8 3 күн бұрын
Great video. Please do another about local LLM on Android. Do not know if it is possible at this time.
@berylliumdotaep
@berylliumdotaep 3 күн бұрын
there are people doing it but kind of complex
@LivingLinux
@LivingLinux 3 күн бұрын
You can install a Linux terminal with Termux. From there you can install Ollama. But there are some things that don't make it easy. I tried with Debian and the directory was not added to the path. And I needed a second session, as I needed to start the Ollama server in a separate session. But it can be done.
@istuff4137
@istuff4137 17 сағат бұрын
2:35 the most statistically probable responses ;)
@GGrev
@GGrev 3 күн бұрын
I love you Naomi!
@ZavakShadow
@ZavakShadow 3 сағат бұрын
"Timeless Naomi"
@wolfgangk4952
@wolfgangk4952 3 күн бұрын
Thx`s for the video. I missed any words about the Hardware i need for the docker lxc`s or the backbone where i install the Librarys for all the stuff. Also in this context the special " NPU " which are necessary for the OP`s . ( TOPS )
@RarelyCorrect
@RarelyCorrect 3 күн бұрын
I was hoping to hear more about hardware requirements. What kind of GPU do we need for 40B models?
@PyCoder82
@PyCoder82 3 күн бұрын
It can run from cpu too it's just slower. Ram is based on the model. Small once are fine with e.g. 2 - 5gb and relly big ones are 40gb+.
@PracticalPcGuide
@PracticalPcGuide 3 күн бұрын
40B Q4_0 is going to be around 20GB. so 3090-4090 is needed, my friend!
@ScriptureFirst
@ScriptureFirst 23 сағат бұрын
Quantization is shrinking all the models quite nicely. Kind of the only way to go w light consumer scale models
@MrSnacko
@MrSnacko 2 күн бұрын
How do you protect your computer from viruses when you download things from the command prompt?
@ScriptureFirst
@ScriptureFirst 23 сағат бұрын
Depends on threat model, but good start is sticking with the big name sources that have tens of thousands of dl’s
@pezz779
@pezz779 2 күн бұрын
Thanks Mate Respect From Oz 🇦🇺🦘
@macbitz
@macbitz 3 күн бұрын
Fascinating stuff, and I can see why a local model is good for privacy. However, if I've registered with an AI system (say ChatGPT) with a masked email address, I always use a VPN and I never use personally identifiable information in chats, what is my privacy risk?
@NaomiBrockwellTV
@NaomiBrockwellTV 3 күн бұрын
1) chatgpt requires an active cell number (not voip) 2) Everything you do through your acct is tied into a single profile and kept forever. I wouldn't bet against the ability for that information to eventually be tied to you. 3) if you don't reveal sensitive information in chat, I think using chatgpt is fine. It's just about understanding what information is collected, and making smart usage decisions accordingly.
@ricochet287
@ricochet287 2 күн бұрын
Naomi be know that ollama LLM are not really Open Source, Meta like others (Google recently corrected this) are abusing the the Open Source definition standard (OSI), this LLM are trained by private data and none of the sources are explained or given to the users.
@108u9
@108u9 3 күн бұрын
Hi NBTV, curious how would users know if any telemetry-esque connection doesn’t happen in the background when the device connects online with the offline options? That prompts and inputs, end user info can still be collected via so?
@thegreyfuzz
@thegreyfuzz 3 күн бұрын
I thought the same thing, I've set up multiple instances of different AI chat and Image generation on separate machines that are in a sandbox'd VLAN that was blocked from internet access and firewall logs monitored at the gateway, so far none of them have tried to call home or off network for anything prompt/query related. For queries that needed very new information that was not available when the model was built it did not call out instead reported it didn't have the info to answer. Additionally, I didn't see it covered in this video but there are uncensored models available that you can use, other public models/engines tend to censor very useful info, quick example - in a SHTF moment how to make useful compounds, for example KNO3CS.
@NaomiBrockwellTV
@NaomiBrockwellTV 3 күн бұрын
I run little snitch so I see every connection being requested
@tiagoreis5390
@tiagoreis5390 10 сағат бұрын
The only problem is that all these modules use a lot of resources (some even GPU)... I think that until we could run a decent AI model locally that does not require a modern CPU and RAM, this will not be feasible... I tried already and it was to slow, or the "lower" models are to bad.... :(... Let's see in the future :)
@Un_Pour_Tous
@Un_Pour_Tous 2 күн бұрын
msty works well for the windows user.
@alieskandari6863
@alieskandari6863 3 күн бұрын
Can we train the models and add parameters to them (forking our own model?)
@لاتشبهنفسكبإنسانوأفعالكأفعالحي
@لاتشبهنفسكبإنسانوأفعالكأفعالحي 3 күн бұрын
Thanks Naomi but is their a way to jailbreak AI or ChatGPT for privacy or like if I ask AI itself to encrypt our chat?
@PracticalPcGuide
@PracticalPcGuide 3 күн бұрын
you have to run uncensored models locally..
@لاتشبهنفسكبإنسانوأفعالكأفعالحي
@لاتشبهنفسكبإنسانوأفعالكأفعالحي 3 күн бұрын
@@PracticalPcGuide Thanks for the answer
@dotaallstars8081
@dotaallstars8081 3 күн бұрын
Honestly seeing the hated one using nord or the pIrate bay undermined his credibility
@DecentralizedJournaling
@DecentralizedJournaling 3 күн бұрын
Is the Grass DePin Project safe?
@aXimo
@aXimo 3 күн бұрын
7:50 ekhem, what is that strange website you've showed here? I have never ever seen it before.
@Un_Pour_Tous
@Un_Pour_Tous 2 күн бұрын
The piratebay was the shyt in its day.
@Hindsight101
@Hindsight101 3 күн бұрын
How do you update Open Web UI? Thank you
@NaomiBrockwellTV
@NaomiBrockwellTV 3 күн бұрын
GREAT question, going to do a video about it!
@Hindsight101
@Hindsight101 3 күн бұрын
@@NaomiBrockwellTV Thank you
@SirWolf2018
@SirWolf2018 2 күн бұрын
I don't really like docker, it's still not user friendly enough. It's not like a regular application that you just install, you still need the command line.
@scotttovey
@scotttovey 3 күн бұрын
The only problem I see with your tutorial is that it, like most tutorials that tell people to sue docker, is that there is no instructions on how to install docker. OH sure, I could do a web search like I have many times before, but they all tell me to fun a command in my prompt that my computer says; does not exist. My biggest frustration with techies is that they make generalized assumptions that are only true if the given computer happens to have the latest, greatest, most advanced everything, including a monitor set to the highest resolution possible, which I can not see and therefore must set mine to 1280 x 768. Other than that, good video.
@AnythingGodamnit
@AnythingGodamnit 23 сағат бұрын
Great video. I must admit I gave up on local LLMs about 6 months ago, due to it being *so slow* to run them, even on my beefy desktop machine. I don't have an NVidia GPU, though. I was hoping to see more info on using GPUs in this video, but it's probably a bit advanced/out of scope. I've been meaning to go back and try again and this video has further encouraged me to do so. Thanks!
@Kebekwoodcraft7375
@Kebekwoodcraft7375 3 күн бұрын
Bonjour gorgeous 😊 I read a study about the crakeling voice it’s only happening in USA so far I remember 😊
@elvendragonhammer5433
@elvendragonhammer5433 2 күн бұрын
Would have been nice if you had linked the actual LLM video- couldn't find a single vid anywhere in his playlist (The hated one) about anything AI, much less LLM's. Also 99% of his content looks like it's ,ore likely to get you put on a gov list first. You also didn't even mention TechXplainator which seems to be where most of the video content is actually from, (like the Docker clips), although all 8 of his video's I clicked on have a female ai as voiceover, so I don't know where the male voice came from.
@NaomiBrockwellTV
@NaomiBrockwellTV 2 күн бұрын
What actual LLM video? This is a project we put together in collaboration, this IS the video :)
@KomradeMikhail
@KomradeMikhail 2 күн бұрын
How about instructions on removing AI slop from our lives, instead of adding more to it ?
@Un_Pour_Tous
@Un_Pour_Tous 2 күн бұрын
ppl don't care about their data until they are caught with a crime.
@rogernevez5187
@rogernevez5187 3 күн бұрын
[Title] How To Host AI Locally ? OLLAMA ???
@krismorgahan8589
@krismorgahan8589 3 күн бұрын
Why are you fo focused on running it on a laptop?
@Bzl-ak
@Bzl-ak 2 күн бұрын
Thank christ I have a brain and don't need these tools.
@nicku8642
@nicku8642 3 күн бұрын
I don't trust it
AI Platforms The Respect Privacy
23:08
Naomi Brockwell TV
Рет қаралды 34 М.
No SIM? No Problem!
22:00
Naomi Brockwell TV
Рет қаралды 1,8 МЛН
Will A Basketball Boat Hold My Weight?
00:30
MrBeast
Рет қаралды 141 МЛН
amazing#devil #lilith #funny #shorts
00:15
Devil Lilith
Рет қаралды 18 МЛН
бабл ти гель для душа // Eva mash
01:00
EVA mash
Рет қаралды 7 МЛН
Это было очень близко...
00:10
Аришнев
Рет қаралды 7 МЛН
host ALL your AI locally
24:20
NetworkChuck
Рет қаралды 1,2 МЛН
MINDBENDING AI ADVANCES DON'T STOP! - Latest AI News & Tools
33:24
MattVidPro AI
Рет қаралды 35 М.
The 10 Trillion Parameter AI Model With 300 IQ
33:45
Y Combinator
Рет қаралды 38 М.
Should You Remove The SIM In Your Car?
19:56
Naomi Brockwell TV
Рет қаралды 384 М.
Is Skynet watching you already?
1:04:00
David Bombal
Рет қаралды 1,1 МЛН
The AI Search Engine War (And Other AI News)
31:23
Matt Wolfe
Рет қаралды 60 М.
Most PRIVATE Password Manager
22:23
Naomi Brockwell TV
Рет қаралды 283 М.
Run your own AI (but private)
22:13
NetworkChuck
Рет қаралды 1,6 МЛН
Use an Email Alias!
13:52
Naomi Brockwell TV
Рет қаралды 496 М.
I Have 2 Weeks to File a Dispute for this Scam TV
25:35
Linus Tech Tips
Рет қаралды 536 М.
Will A Basketball Boat Hold My Weight?
00:30
MrBeast
Рет қаралды 141 МЛН