Wizard-Vicuna: 97% of ChatGPT - with Oobabooga Text Generation WebUI

In this video, we explore a unique approach that combines WizardLM and VicunaLM, resulting in a 7% performance improvement over VicunaLM. We will be running the model locally using the powerful Oobabooga text generation webui.
LINKS:
--------------------------------------------------
WizardVicunaLM: github.com/mel...
Model on HuggingFace: huggingface.co...
Oobabooga WebUI: github.com/oob...
-------------------------------------------------
☕ Buy me a Coffee: ko-fi.com/prom...
Join the Patreon: patreon.com/PromptEngineering
-------------------------------------------------
All Interesting Videos:
Everything LangChain: • LangChain
Everything LLM: • Large Language Models
Everything Midjourney: • MidJourney Tutorials
AI Image Generation: • AI Image Generation Tu...

[@shotelco]

Been waiting for this to be released. You will have plenty of new videos based on this platform. Hopefully, the next is how to train on internally created datasets, and more importantly, how to set up automated continuously updated training. Thanks!

[@seldian]

Loving these local guides!!!! Thank you! Edit: Thank you for including the uncensored recommendations too, they are hilarious to mess with

[@sayantandas7544]

The more I watch your videos, the more amazed I become. It's incredible how much AI has already evolved, so many things are already available for free to play around. Your videos are always very thorough and easy to understand. 🚀🚀

[@engineerprompt]

Thank you, its just getting crazy day by day!

[@TH-nr2sp]

すばらしい！簡単に導入することができました。ありがとうございます。

[@ekstrajohn]

Vicuna-WizardLM trained on GPT4 outputs, so it makes sense that it would be closer to what a GPT4 judge would prefer. It's not a very useful test imo ...

[@reinerheiner1148]

How dare you question the neutrality of the judge! 😁

[@anispinner]

Good point.

[@hfislwpa]

Thanks for the video! Would like to see how to fine tune as you mentioned please

[@wilfredomartel7781]

Wish to see the text summarization performance.

[@jayhu6075]

This streaming is so useful for everybody to start how to install and exercise with this to understand the fundamentals. Many thanks

[@yassinebouchoucha]

it' s always great to have a video explanation with step-by-step even if it's simple clicks and cmd commands !(Personally I get lost navigating in the everyday new tool/framework of AI models) Looking forward for your in depth Video about Oobabooga and if we can integrate it API with langchain !?

[@seldian]

Can you make a video explaining training, and how it works?

[@madelynmills712]

Thanks for this. Very helpful!

[@nasiraden4628]

did i miss something - if you chose B (AMD) it will tell you GBU is not supported . does anyone have the same issue ?

[@ultimatefaux]

Grat tutorial. Thank you! 🐙

[@TheRMartz12]

The one click installers are not available anymore

[@SB-uh8ik]

I have the text generation web ui and I want to train the llama2 model of 7b with my own data in txt but I get an error, has anyone already done it or can give me the instructions? Because I have been trying for a week and nothing so to see if someone would help me with this, whether we make a call where they show me how to do it or here

[@lovol2]

Thanks. Yes please show training Also, please show api. Wish i was at my pc now to try it!

[@ModelAIweb]

Following your instructions llama2 get stuck after frist prompt, can you help with a video to tweak the parameters to enhance code generation and longer conversations

[@janalgos]

would like to see how does wizard vicuna performance compared to stablevicuna

[@dodeekdd961]

Hi, For some reason when I download ooba booga I have different files from the ones in the video, also when I try to start it up, I only get one file named installer _files and there is no file text-generation-webui anyone knows what might be going on?

[@sahanfernando4414]

same did you figure it out?

[@dodeekdd961]

Yes

[@dodeekdd961]

I deleted the oobagooba file and reinstalled it, i also swithed from option a, NVDIA to d, internal cpu

[@sahanfernando4414]

@@dodeekdd961 I figured it out already, I just deleted the installer files folder and ran it again. I made sure to use the right capitalisation for the options this time and it ended up working. Might have just been an error might have been capitalisation (which would be really stupid) but it works as intended

[@sahanfernando4414]

@@dodeekdd961 kept it on Nvidia btw

[@flyLeonardofly]

This is a very misleading metric. It suggests that vicuna is as "good" or very close to chatGPT, but I tried it for the things I use chatGPT for like, learning how certain APIs work (programming) and it's pretty useless for that.

[@EngkuFizz]

Fact

[@ShaunPrince]

So much misinformation in this vid, no explanation for your choices, and GGML models don't load in the GPU. If you use a huggingface format or GPTQ model it will run on a GPU, and the 1080ti will output almost the same token per second as a RTX 3060. Also, TheBloke repos usually contain many unneeded files, and are confusing for new users.

[@KwissBeats]

I highly disagree with that last comment, The bloke provides un-broken files that don't need the stupid merge with facebooks LLama

[@josenerydev]

How I know if this use GPU? I setup to NVIDEO, but use CPU

[@liquidsnakeblue1]

Same problem

[@ChessScholarOfficial]

Set the n-gpu-layers based on your GPU. I'm on the RTX 4070 and have it set to 33. Make sure your model isn't too big for the GPU, otherwise speeds will be very slow.

[@aashas8553]

Its so weird to me I get insane ammount of error messages and cancellations

[@SAVONASOTTERRANEASEGRETA]

Can you give me some advice: my Alpaca-13B-GGML model is fairly fast but responds as it wants. I tried to change parameters but to no avail. What do you tell me to do?

[@McVerdict]

Can you help me, im trying to use the api for a program but it doesnt seem to let me use the web ui character's through the api

[@didiervandendaele4036]

Thank you ❤

[@behroozhussaini2339]

Hi, How to share the Vicuna GUI with a local computer. I installed on my server now I want to access in my phone or other computers but I can't find any way to do that. please help.

[@conradtwonine9414]

how could I add my own database or file library to this model to create my own version?

[@SAVONASOTTERRANEASEGRETA]

HI . I need to know something I don't understand. If I want to give my assistant a new book text, I have to put it in the file. Bin form or in a webui folder? Thank you

[@muazashraf409]

if we have to host the text generation web ui, we have to download it manually?

[@pangalord]

Does it run actually on windows? It seems u started installing on windows then finished on Linux

[@engineerprompt]

It does but I dont' have a GPU on my windows machine. That's why switched to linxu, mentioned that in the video.

[@michaelroberts1120]

@@engineerprompt But isn't this thing supposed to be able to run on CPU? You're even given that option during installation!

[@WildMidwest1]

No joy for me under two Linux versions: Linux Mint 21.1 and Manjaro (current). I have 64 GB RAM and a Ryzen 7 CPU with a lowly integrated AMD GPU. The script fails at different points with each OS. In Linux Mint the start script puts up a bunch of errors after downloading four 8.1 to 14.6 GB files from TheBloke_wizard-vicuna-13B-CGML, declaring “error loading model: unknown (magic, version) combination: 67676a74, 00000002; is this really a CG!L file?” Various tokenizer errors and assertion errors follow before giving the message Done! In Manjaro, the start script terminated after “Building wheel for llama-cpp-python (project.toml)… error. Subprocess exited with error.” Further down CMake complains “49 (message): Could not find compiler set in environment variable CC: gcc-9”, then “Trying Ninja generator - failure.” Further down, “scikit-build could not get a working generator for your system. Aborting build. Building Linux wheels for Python 3.10 requires a compiler (eg gcc).” Manjaro Linux has gcc 12.2.1 preinstalled. Downgrading gcc 12..2.1 to gcc 9 is a questionable idea, at best, and not easily accomplished by most of us. (I read a bunch of threads before abandoning this effort.) ---- ADDENDUM: Posting an update for the benefit of others. The challenge I was having with oobabooga in Linux Mint (an Ubuntu derivative) was due to recent removal from the llama.cpp of bit-shuffling, requiring re-quantization of the base model AND updating llama-cpp-python to version 0.1.50 or greater. The command that got Wizard-Vicuna running for me was: > pip install llama-cpp-python==0.1.50

[@Cashvib-f4w]

How to fix 'cadam32bit_grad_fp32' not found, need help

[@JuliusCaezar-d8x]

Why is it not doing anything when I run the start-windows.bat? Need a little help here thanks.

[@davidstone2700]

How can we give it langchain and online access? With EdgeGPT Plugin? Please help.

[@greendsnow]

finally.... a prompt tutor with no gpu!

[@SAVONASOTTERRANEASEGRETA]

Why don't you see the transformer parameters in my latest version?

[@patrickmchargue7122]

Miniconda won't start for me. I ran the installer for Miniconda, but no joy. Running Python 3.11 on my system

[@MrTPGuitar]

Why are you saying compared to chatgpt this chatgpt that when the readme specifically compares it to GPT3.5

[@jodter1]

How do I use that and other models without using the terminal or the chat website, but directly in a Python or JS project?

[@hafizkk99]

Does this work on my 4gb RAM system. I dont have a gpu

[@itsalivevideo]

What hardware are you using in your demo?

[@xilix2770]

Is there a way to make the chat logs .txt instead of .json? ``` I would love the chat log format to look like this name1 "Hello." name2 "Hi." name1 "Nice weather today." name2 "Indeed." ```

[@chrisbraeuer9476]

Can someone guide me to an uncencored version? This one seems to work but refuses to reply to many questions.

[@kethtemplar8989]

Does the AMD option work?!! Anyone?! Please! Tell ME! I am DESPERATE for faster than 398+seconds cpu thinking time!

[@jichaelmorgan3796]

Is there any llm that could feasibly work on an i5-4440 with 16gb of ram?

[@Atom5355]

For those wondering I installed on pc with 7900xtx and 7950x on cpu mode works fine

[@engineerprompt]

Nice!

[@henriquemiranda3762]

Hi, how i install more language support in this model? eg: Portuguese languages

[@engineerprompt]

You will look at specific models that are multilingual.

[@harris6363]

Waiting for the video on how to use the chat setting

[@michal5869]

Ehh, I don't see any better difference than with the old Vicuna or Wizard. For example, asking for code to be written in C++ is a waste; if I want more, I'll switch to Java. The answers are short and they misunderstand my questions. I compared all models.

[@valdesguefa9513]

what is the caracteristics of your computer ?

[@homosuperior1337]

Not working on my m1 machine.

[@greendsnow]

How can I make an API out of that?

[@mygamecomputer1691]

Can confirm, this webui from Oogabooga does indeed enable you to have spicy role-play off-line. Thank you for covering this.

[@DrGuy118]

spicy role-play 🥰

[@chrisbraeuer9476]

How. Mine refuses to do uncencored stuff. Is there some setting i miss?

[@cleverestx]

How spicy we talkin'? LOL

[@mygamecomputer1691]

There is no limit to how spicy you can make it if you’re using the uncensored version. fantasy at its finest.

[@mygamecomputer1691]

Do you have to make sure you’re using the model that specifically says it’s uncensored and ideally also use the storyteller preset.

[@lukacosic1353]

Hello everyone, I've only recently started dealing with AI, so I'm still a beginner. Thank you for the video, it helped me a lot to set up the system. My question is, how can I share the WebUI output with other computers in the network? I installed the system under Proxmox in a Servers without a desktop and would like to access the WebUI from my workstation. Can anyone help me set this up? Thanks in advance and best regards Luka

[@RpgBlasterRpg]

How to make the Text faster?

[@engineerprompt]

Better hardware :)

[@sjtrader1363]

If that's the performance on 1080 then what can people do who have 1050

[@aresdeus2996]

Pensei isso também kkkkkkkkkkkkkk

[@reinerheiner1148]

cry? ^^ but more seriously, you can always use google colab and use a cloud gpu thats about as fast as a 1080ti for free, or use cpu, depending on your hardware. besides, a 1050 may not even have enough vram to run it.

[@sjtrader1363]

@@reinerheiner1148 thanks bro. My desperate comment didn't go in waste after all.

[@annwang5530]

Can you analyze freedomgpt?

[@withcarename]

97% of chat gpt ? What a bold claim.

[@mbrochh82]

I believe it is 97% of GPT-3. So yeah, still not very useful.

[@simonalkemade]

@@Utoko kmsys is down probably to busy

[@gileneusz]

got error on mac m1: ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects

[@WildMidwest1]

Same issue here building in Manjaro Linux. The start script terminated after “Building wheel for llama-cpp-python (project.toml)… error. Subprocess exited with error.” Further down CMake complains “49 (message): Could not find compiler set in environment variable CC: gcc-9”, then “Trying Ninja generator - failure.” Further down, “scikit-build could not get a working generator for your system. Aborting build. Building Linux wheels for Python 3.10 requires a compiler (eg gcc.” Manjaro Linux has gcc 12.2.1 preinstalled. Downgrading gcc 12.x to gcc 9 is a questionable idea, at best. I do not have these issues building wheel in Linux Mint, however I have other apparently insurmountable problems installing in Mint. Huge time waste but maybe educational?

[@WildMidwest1]

Posting an update for the benefit of others. The challenge I was having with oobabooga in Linux Mint (an Ubuntu derivative) was due to recent removal from the llama.cpp of bit-shuffling, requiring re-quantization of the base model AND updating llama-cpp-python to version 0.1.50 or greater. The command that got Wizard-Vicuna running for me was: > pip install llama-cpp-python==0.1.50

[@michaelroberts1120]

Maybe the creator of oogabooga for Windows should contact the creator of Koboldcpp, and ask him for lessons in how to code a program for windows. Because when you see how much the creator of Koboldcpp has crammed into a single file less than 20 megabytes in size, versus the monstrosity that is Oobabooga, it is quite clear that the creator of Oobabooga for Windows doesn't know what he is doing.

[@hemzabomb]

nothing I try to install works

[@michaelroberts1120]

Don't waste your time downloading and attempting to install this thing (oogabooga for Windows). You need Microsoft Visual C++ 14.0 or greater; no mention of that is made in the text files. On top of that, even if you specify CPU only mode during installation, the program still fails to initialize because numerous modules won't start since they have been compiled without cuda support. The whole thing is a hot mess, and reminds me of why I hate Linux, and Linux mixed with Windows is ten times worse! Until someone comes up with a pure Windows version, don't waste time with this unless you're a complete masochist!

[@Spock69able]

I tryed to install it on my windows pc with having tons of errors and not running! I guess you are right!

[@ScreamCheese13]

This post was a million times more helpful than anything else I've found. I've been trying to make this work for days and it absolutely refuses to. While it would be nice to find a comment that solves the problem and saves you time, the second best thing is to find a comment that tells you to just stop wasting it on a horribly designed product with pitiful instructions.

[@yassinebouchoucha]

Same issue it won't compile if it build with CUDA env

[@randomman5188]

its not that difficult and if you cant download microsoft visual c++ 14.0 or greater than you shouldnt even be using an llm and and it specifically says for windows not linux so thats why you are having issues because the oogabooga installer is simple and straightforward

[@ScreamCheese13]

@@randomman5188 Congratulations on not even reading my post. That's not at all where the problems were.

[@NebulousTwist]

6:20

[@marcosbenigno3077]

Muito obrigado Prompt . Prevejo que logo teremos uma forte oposição ao uso das AIs estou atualmente baixando e guardando todos os modelos disponiveis (1 tera quase). De todas as interfaces que testei (sou leigo) o oogabonga é o mais promissor. Se puder explicar a diferença entre os tipos seria agradavel de ver (*.pt, pytorch*.bin, ggml*.bin, *.safetensors). Até então só tenho conseguido rodar arquivos *.bin unitários. Desculpe minha burrice mas é para isto que veio as AIs: tornar generalistas especialistas e vice versa. Esta é a ultima fronteira e será barrada antes de 2030. Grato pelos vídeos. Marcos. Sul América. Brasil. Minas Gerais. Betim

[@AaliDGr8]

That's very stupid thing you all people just making videos without knowing everything listen it would be very great and amazing as good with chat GPT if this also could edit images same like GPT4 can do if it's not edit pictures then it will not be amazing or cool

[@CCCW]

I get a "KeyError: 'serialized_input'"

[@jlpfytb]

I followed the steps but gave these errors when starting, can someone help me? INFO:Loading the extension "gallery"... Traceback (most recent call last): \server.py", line 885, in create_interface() \server.py", line 472, in create_interface with gr.Blocks(css=ui.css if not shared.is_chat() else ui.css + ui.chat_css, analytics_enabled=False, title=title, theme=ui.theme) as shared.gradio['interface']: \blocks.py", line 1261, in get_config_file "input": list(block.input_api_info()), # type: ignore blocks.py", line 1285, in __exit__ self.config = self.get_config_file() serializing.py", line 40, in input_api_info KeyError: 'serialized_input'

[@jlpfytb]

@@reinerheiner1148 thanks a lot

[@lacknerish]

If anyone else comes across this problem, the fix is to install a newer Gradio. $ pip3 install gradio==3.28.3

[@opitts2k2]

if you are like me and was wondering where you place the llm model if you ran start_windows.bat and was greeted with a url instead of the command prompt:(. You place it here: oobabooga-windows └───text-generation-webui └───models └───anon8231489123_vicuna-13b-GPTQ-4bit-128g └───Wizard-Vicuna-30B-Uncensored.ggmlv3.q8_0.bin

[@fakeaccount-t3r]

excuse me but would anyone know how to fix this problem. !! self.initialize_options() installing to build\bdist.win-amd64\wheel running install running install_lib creating build\bdist.win-amd64 creating build\bdist.win-amd64\wheel creating build\bdist.win-amd64\wheel\sentence_transformers creating build\bdist.win-amd64\wheel\sentence_transformers\cross_encoder copying build\lib\sentence_transformers\cross_encoder\CrossEncoder.py -> build\bdist.win-amd64\wheel\.\sentence_transformers\cross_encoder creating build\bdist.win-amd64\wheel\sentence_transformers\cross_encoder\evaluation copying build\lib\sentence_transformers\cross_encoder\evaluation\CEBinaryAccuracyEvaluator.py -> build\bdist.win-amd64\wheel\.\sentence_transformers\cross_encoder\evaluation error: could not create 'build\bdist.win-amd64\wheel\.\sentence_transformers\cross_encoder\evaluation\CEBinaryAccuracyEvaluator.py': No such file or directory [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for sentence-transformers Running setup.py clean for sentence-transformers Failed to build sentence-transformers ERROR: Could not build wheels for sentence-transformers, which is required to install pyproject.toml-based projects Command '"C:\Users\???\Downloads\AI\oobabooga_windows\oobabooga_windows\installer_files\conda\condabin\conda.bat" activate "C:\Users\???\Downloads\AI\oobabooga_windows\oobabooga_windows\installer_files\env" >nul && python -m pip install -r extensions\openai equirements.txt --upgrade' failed with exit status code '1'. Exiting... Press any key to continue . . .

[@kiarasha1106]

I had the same issue, your file is in a file, just take it out.

[@marcosbenigno3077]

Thank you very much Prompt E. I predict that we will soon have strong opposition to the use of AIs I am currently downloading and saving all available models (1 tera almost). Of all the interfaces I've tested (I'm a layman) oogabonga is the most promising. If you can explain the difference between the types it would be nice to see (*.pt, pytorch*.bin, ggml*.bin, *.safetensors). So far I've only been able to run single *.bin files. Excuse my stupidity but this is what AIs came for: making generalists specialists and vice versa. This is the last frontier and it will be stopped before 2030. Thanks for the vídeos. Frames. South América. Brasil. Minas Gerais. Betim 🦾

[@WESTSIDE15]

how i can run this program again?

[@Dragons_Armory]

6:28