1 Command To Bring Home Your Very Own Robot Overlord - Text Generation Webui & LLaMa.cpp

[@lochnessmonster4518]

This is awesome. Thanks for the tutorial Don. Hopefully you will do a more in depth video on the different features and extensions that are available. Speaking of extensions, any chance you could tell me where I can add elevenlabs? Would love to have TTS available.

[@yerunski]

Just installed it on my Ubuntu server and it seems to have gone well. Except the web URL can't be reached. I can't use the localhost IP (since the server doesn't have a GUI) so i'm trying from a remote system connected to the same network. . It says the site can't be reached. I have UFW turned off. What am i missing here?

[@NovaspiritTech]

edit the webui.py file on line 146 add "--listen" after "--chat" to enable listening on local ip

[@yerunski]

@@NovaspiritTech Awesome, thanks for the quick reply!

[@HasanAYousef]

Awesome Don, can You show us how to use it with our own data, I've a couple of pdf and text files, how can I use them to answer my prompts?

[@jameswubbolt7787]

Great stuff😊

[@robertjamessteeleiii]

Where can i go if i am having problems with this install scrypt?

[@ORCdot]

I installed oobabogaa webui with docker-compose and llama 4 bit model is about 4gb im using the gpu 1070 ti on linux. What i noticed is that is to slow and when i try to run alpaca model runs out of memory error. Maybe if i use cpu is going to be faster? I have i5 8400 and 16gb ram.

[@shrirangkahale]

Awesome video :D

[@skullpoly1967]

Albony

[@shrirangkahale]

@@skullpoly1967 y e s ?

[@ronm6585]

Thanks Don.

[@ElectrogicsPH]

Can we use coral ai accelerator to RPI and run llama?

[@IronZk]

How much vram of GPU do in need to run this maybe 16gb ??

[@NovaspiritTech]

Yea at least, I was having trouble running 7B with my 8gb 1070

[@phasechange5053]

Sadly im running AMD on both my good rigs so its probably going to be a pain. Someone was telling me it will chug without decent GPU acceleration and a GPU with a fair amount of memory also.

[@NovaspiritTech]

Not wrong, 8gb 1070 still not enough ram to run 7B

[@skullpoly1967]

😢

[@schrodingersmechanic7622]

But will it run on a Pi? I'm thinking the answer will be "yes, eventually."

[@NovaspiritTech]

yes, acutally tested some stuff on rpi and on khadas edge2. if it wasn't for the limited ram, you are stuck on 7B or lower

[@caibi_]

wait rpi got vram?

[@DerekJoinerPRESTONEZ]

@@NovaspiritTech Which executable/zip file did you use for your khadas? I'm attempting this on the orange pi 5b :b

[@shannonashley7224]

Yep like all these chat ais, its broken, the chat ai community has a lot of catching up compared to art ai