How to Make Your Own AI Waifu Virtual VTUBER or Assistant

[@ryusuikagaku]

Change your voice to AI Waifu Virtual VTUBER or Assistant using this free program → bit.ly/47XNc3e Remove vocals from songs for free → bit.ly/3RVbfep A must-have free screen recorder → bit.ly/3te8SK9

[@Angelicastories]

can i ask ? program name for make waifu virtual vtuber? or were u create waifu virtual vtuber?

[@OlasodeSidiq]

can i chat to you privately i don't understand this i have an offer if you explain well

[@ryusuikagaku]

Update : Now it's supports TTS on English, Spanish, Russian, etc. It's also support Twitch streamer now. If you find this was helpful, please consider to supports me on ko-fi.com/ardhach Thanks

[@Kalyancheg]

can i use 3d model in, for example, VseeFace? And if i can, so how?

[@ryusuikagaku]

Just use vb cable input on vseeface

[@AshikuzzamanAngkon]

Seems very informative.I will save this video in my playlist ❤

[@ryusuikagaku]

i try my best to explain the code in English :)

[@evieyorisou2586]

You did an amazing job. I was wondering if this code can be made to use 11 labs instead of VOX

[@ryusuikagaku]

Yes you can

[@evieyorisou2586]

Also I don't know what I'm doing wrong I think that twitch part of the program is broken is there anyway you could do tutorial video or do some one-on-one show me how the heck do you use it sort of thing v.v

[@ryusuikagaku]

​@@evieyorisou2586 did you follow the step on my github?

[@evieyorisou2586]

@@ryusuikagaku Yes I think it's just the underscore in my twitch channel name so I'm going to remove the underscore in my channel name and see if that fixes it aright a couple of the problems a lot of people having similar problems and people are suggesting just don't use underscores

[@zeove]

Dude your API Key can be seen at 1:44 (If slowed), I suggest you change it quickly. I don't want you to getting the short-end-of-the-stick for providing us with quality info.

[@ryusuikagaku]

Ohh i didnt see it, thanks

[@zeove]

@Ardha Pina Ch. Hello again Ardha, I was wondering how to fix the problem ⚠ "Import "pyaudio" could not be resolved from source Pylance." I'm also still missing "keyboard" and "emoji" I remember you saying in the beginning video, that we had to install all the libraries but I'm not sure where to get them. This is my first time using Visual Studio Code and only learned recently how to download from GitHub. Thanks in advance! 👍

[@ryusuikagaku]

I put the tutorial on my github. You can install all the library by running 'pip install -r requirements.txt'

[@zeove]

Thanks for the reply! All I had to do was pip install the other missing modules, same with the "pyaudio".

[@OlasodeSidiq]

can i chat to you privately i don't understand this i have an offer if you explain well

[@RsVs2304]

amzing stuff dude, the video and code comments are really consise and every question i would of asked is already answered in the comments!

[@nayu2942]

Thank you so much, I tried myself to create an AI Programm. But Iam to inexperienced with Pyhton so thank you for sharing your knowledge and work. I wished that I had found you earlier. It´s so nice, I can´t imagine how it was without an AI Waifu before. Finally free from the rabithole of normal vtuber...

[@mimalhaar8581]

thanx ... you explained the program so well that i ran this program and now making changes into it. Really thank you.

[@OlasodeSidiq]

can i chat to you privately i don't understand this i have an offer if you explain well

[@azy3929]

-some people put rocky videos for motivation -I put this Edit: Im new to programming and is such a pleasure sneaking around your github repository in free time. Everything is well organized and simple to understand.

[@hashashin-qe4ph]

bro i dont stream i just want to build this for desktop assistant only how?

[@忘狐]

So amazing, thanks for your tutorial!

[@orcaolivegames]

Thanks, I had a few issues, but I actually got it working!

[@supershow7110]

The ultimate test is doing this without open ai whisper

[@ShinzoShi]

(note it costs money around $5 is needed to use your ai as if not you can only send 3 messages)

[@braydensurvivez]

can we get a step by step tutorial involving downloads and set ups etc? you mention a lot of things I dont see listed of understand what is virtual cable?

[@kongouch.9458]

Sumpah bang, suara lu enak banget didenger bang, semangat terus bang ngontennya

[@legan1557]

mantap tutorial nya, makasih bang! lu dapet 1 subscriber baru!

[@OlasodeSidiq]

can i chat to you privately i don't understand this i have an offer if you explain well

[@johnwatts5411]

does anyone else not see the lore.txt when they are in the file?

[@rizqyfirdaus1049]

When I copy the code at minute 9:44 and enter it into cmd it has to download. Is that the same for you?

[@aburi_ru]

Thanks for the tips and tricks bro this is insanely easy, kalo boleh tau belajar dari mana bre? gua pengen banget belajar bikin ginian apalagi python language🔥🔥

[@KEN-nh7om]

Thank you, I got very close to making it work but not sure what's the issue with youtube it detects the first comment but then starts spazzing out. Appreciate your guide though! Merry Xmas!

[@OlasodeSidiq]

can i chat to you privately i don't understand this i have an offer if you explain well

[@me-cm8or]

Can we actually use the local "Text generator" instead of OpenAI api? and also the local "TEXT TO SOUND" this way it will be completely free

[@KommissarAlexander]

Amazing tutorial, thank you so much!

[@DrunkyBearAslaxVODs]

amazing tutorial, would love to follow the steps and make my own AI assistant to make my stream more fun, only one problem, it doesn't support/read twitch chat. Can you please make it /show us how to make the ai read twitch chat? also it would be super cool if the AI not only response to twitch chat, but can also response to you at the sametime/ or after another, like mode 1+2. Looking forward for your reply.

[@ukn4]

ask to the assistant how to make the code changes

[@ryusuikagaku]

It now can also response twitch

[@Cradd]

Thank you ardha, i really need this. POG

[@flyann2]

I solved issues from ealier and its working fine now. Thank you! I was wondering what would the steps would be to fine tune the model? Or is it only limited to a prompt based system?

[@ryusuikagaku]

You can check on openai website to finte tune their model

[@flyann2]

​@@ryusuikagaku Thank you, I'll look into that. Another question, I noticed that when the ai responds with a long answer, the text is really hard to read on the screen. Is there a way to break it up into multiple lines of text instead of a large paragraph?

[@ov_rindustry]

Really Great Video, I was wondering if you could help me for a moment. I successfully installed everything, theyre working fine now. However, when I change the get_model = “gpt-3” to gpt4, It kept on using gpt-3 even though my API has access to GPT-4 and 4 Turbo. Is there any way to fix this?

[@ryusuikagaku]

Try to look at openai api documentation, maybe to call gpt4 you need different function

[@ov_rindustry]

@@ryusuikagakuI have tried this by creating a new minimal script. I called GPT-4 Model in this new script and It worked really well. This means the problem is probably not from the API Key or the function, but rather the main code in the main folder. Really appreciate your quick response! But is there any other way to fix this?

[@ryusuikagaku]

Maybe the openai library? You is yours up to date?

[@ov_rindustry]

@@ryusuikagaku This is possible, perhaps I should check If I have installed the lastest openAi Library in the main folder. I will respond as soon as possible If it works. Thank you!

[@ov_rindustry]

@@ryusuikagaku Turns out the problem wasn’t the library of openai. Google Translate 4.0.0rc1 seems to be outdated, and isn’t the newest version of google translate official. 4.0.0rc1 is rather the newest version of the older google translate. The reason why GPT-4 Model doesnt work in the script is because it automatically falls back to GPT-3 as I am using httpx 0.13.3. This version of httpx is the only one that is compatible with both Google translate 4.0.0rc1 and OpenAI 0.28.1. Problem is OpenAI 0.28.1 can only request GPT-3, and higher openAI version requires httpx 0.23 and above. The only way to fix this is figure out a way to deal with google translate.

[@HakiouDerion]

Yoo thanks, i will try it❤

[@GelasPecahh]

can you run it?

[@HakiouDerion]

@@GelasPecahh i cant, i have bad device 🤣🤣 i forgot about that

[@Yume_Ai-1]

Thank you for the knowledge.

[@stepphriw]

good explain , i have subscribe both channel❤

[@diradosveta]

Hi, does this still work? I would not want to spend another day debugging just to find out it doesn't work anymore. Thanks!

[@ryusuikagaku]

might need to edit the library version

[@diradosveta]

@@ryusuikagaku does the bot react to twitch or youtube comments on stream? Or is it only reacting to voice? It's hard to find a good working vtuber ai for streaming

[@ryusuikagaku]

Last time i used it, it works on both. If it error, it should be lib version related error

[@diradosveta]

@@ryusuikagaku is there a way to contact you? I can pay :D

[@flyann2]

Hey the [VoiceVox Colab] runs an error everytime i run it, " ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. librosa 0.10.0.post2 requires numpy!=1.22.0,!=1.22.1,!=1.22.2,>=1.20.3, but you have numpy 1.22.0 which is incompatible." do you have a fix for that by any chance? my windows is super old so i can't run docker through desktop. Thank you

[@ryusuikagaku]

i didn't maintenance the voicevox since it's not mine and i don't know the code. I think it's becaues voicevox now have a newer version and it makes error on colab

[@flyann2]

@@ryusuikagaku Ahh I see. The code allows me to use old VoiceVox engine versions, do you think changing that could fix the issue? I'm not sure which version would work with this

[@ryusuikagaku]

Im not sure

[@dusty1654]

Can it be used in discord call? Is it possible to make it answer the interlocutors sometimes, or when she is asked a question? Not as a separate bot (although it's probably even cooler this way), but on behalf of the user

[@thohacahyaashshoddiqy2394]

mantap bang, lagi kepingin buat dan butuh dasaran dulu

[@luxz7117]

buset bang , coba bang kalo dh bisa kabarin ye😂

[@daguni2]

You're a genius Thank you for making me a cute AI I have a question. After converting my voice into text in real time Can't the voicevox character read the text automatically? In the order of speech->text->speach

[@ryusuikagaku]

yes if you speak in japanese or translate your speech to japanese

[@Labligko]

Bang, aku mau buat ginian buat projek sekolah. Tapi saya masih bingung itu gimana, dan karakter nya cuma bisa di vtube studio aja kah bang? Mohon penjelasan 🙏🏻

[@aliaanimations]

I think i am at my lowest point, because I am watching this video

[@AntoniusJeffryGunawan]

Brader, videonya keren sangat!!! gimana buat AInya jawab agak panjang dan lebih deskriptif ya

[@papian1]

Can you make a version that installs locally on the pc using llama?

[@animusty5717]

bang serius nanya if I want to input the language to Indonesian and output the voice to Indonesian, which part should I change?

[@andrekwok1188]

Absolute mad lad. Wish openai allowed less moderation. Is it possible to use another language model such as Pymaglion?

[@ryusuikagaku]

yes if your computer strong enough to host it or host it on colab and get API from there

[@noirx1]

@@ryusuikagaku would you like to explain how i can host it on colab and get api , how can i do this ?

[@sho5600]

参考にさせて頂きます！！！！！

[@Fat1413]

Thanks for sharing 😊

[@defory3997]

Nice video bro, keren banget

[@beyabay]

thanks you soo much for this amazing video, just one quick question, how do you make the subtitles show in english?

[@huynjoon878]

Hi I ran into some kind of a problem like sometimes Live2d just doesnt recognized any input from the Virtual Cable even though there is hence the model won't lip sync anything at all. I tried looking into other output file to check if the audio is in there and yes everything else works fine, it's just sometimes live2d doesnt recognized the input at all.

[@ryusuikagaku]

Usually by default live2d model only accept mouth movement from camera. You need to change the setting/parameter for each model so it accepting mic as input for mouth movement

[@huynjoon878]

@@ryusuikagaku Thank for replying, I got that part, I did change the setting like you did in the video for both the Mouthopen and the Micriphone setting. The problem is sometimes or many times live2d for some reason cannot recognized any input from the virtual cable even though there is and I know there is because I play the sound from the Output Virtual Cable to my speaker too. But it's just that live2d keep saying that there is no input.

[@huynjoon878]

Also, do you have another tutorial for the twitch function I would love to see that. You did a terrific job for the documentation and the video btw :))

[@MiminiWarriorsmasters]

Is it possible to use this Ai Vtuber for twitch? to interact with live chat

[@vgx-sam8164]

saved for later , good stufff hehe

[@nashitakhtar]

I made joi from blade runner

[@dyar666]

Thank you so much, Is it very difficult to use Kick chat instead of Twitch?

[@Syrup__Maple]

Oof, everytime i try to run with twitch chat it says "Error receiving chat: 'NoneType' object has no attribute 'group'"

[@ryusuikagaku]

did you follow the step on github?

[@Syrup__Maple]

@@ryusuikagaku Which step? (Sorry, i'm new to github)

[@ryusuikagaku]

@@Syrup__Maple changing the twitch_config

[@Syrup__Maple]

@@ryusuikagaku Yeah i changed

[@ryusuikagaku]

@@Syrup__Maple have you try it to other streamer that doing livestreaming? By changing the twitch channel on twitch_config

[@litati1662]

Hello! I have a problem with dubbing of the translated text. The translation itself from English to Russian works, but I can't do the voiceover, I get an error "Error printing text: can only concatenate str (not "NoneType") to str" or sometimes a message appears with a list of voices from en_0 to en_117. I use Seliro TTS

[@ryusuikagaku]

github.com/ardha27/AI-Waifu-Vtuber/issues/24

[@hitujisya-o6p]

I received a lot of help from your project. Thank you so much.

[@IsJustJed]

For some reason, the pip libraries for Openai and Googletrans dont want to work together. Keeps giving me a httpx version incompatibility error.

[@ryusuikagaku]

Yes, because gooletrans library no longer updated. If you still want to use it, you need to lower openai version

[@noleezyadvance983]

question is there a way to change the art even if its just to a still image

[@haixirtcrack3118]

Garis besarnya cukup jelas Thank you

[@Mika_Zuki7]

Can i do it in my phone?

[@YesIBullyPeopleInTheComments]

We used to bully plankton for having an AI wife, look at us now.

[@damn_bro2203]

I have an issue, i just want the AI for myself so i cut some line from the main code. Turn out that made the code doesn't work anymore, can you help me, i just want to stop at Vtube studio, i dont use OBS. Thank you for your work

[@RyoumaDazaiWorks]

How I wish you'll add a feature where the AI could sing for an AI cover!

[@TheoneandonlyGIFT]

check out a guy by the name of vedal he made and ai vtuber and it sings and much more >

[@HeraOlulu]

shhesh thx for d explanation, poggers.

[@OniiChan-se2zn]

You are the best!

[@VaibhavShewale]

looks amazing, but didnt like the api part of openai cause i exhausted free credits!

[@cositapoca2576]

amazing work, thanks for explaining too. One question, can it be use in Twitch? If I change the stream Id will work? Thanks and good luck

[@ryusuikagaku]

You must find another library that work for twitch, because the library that is use is just for youtube live stream

[@cositapoca2576]

@@ryusuikagaku ok get it now, thanks a lot great job

[@ryusuikagaku]

Ur welcome

[@maihieu5395]

thank you very much , it really works and i created my wife:))

[@rioharta526]

thank you om da sharing

[@MelodyLyrics23]

cara instal librarynya bagaimana kak 1:10 ? API openai nya ditaruh dimana?

[@kurokashi_7545]

min yang bagian script text di githubnya nggak ada

[@panwangyy]

is it already like in live when you have to ask the ai you have to use #

[@KaiK4Isa]

hey friend very nice explanation indeed, but can you answer one detail that i'm constantly trying to find, there is a way to develop a more human Vtube ? i mean i generate the poses and then put them to work ?

[@ryusuikagaku]

i dont understand what do you mean by poses?

[@ElFaridzi]

Bang alternatif pake api key selain pake openai apa bang? Yang bener2 open source, gapapa walaupun ga sebagus openai, asalkan dia bisa respon

[@SrMaio0]

Too much information for my dumb head. Is it possible to do it on Android? Has anyone tried it? Could you send me a link?

[@davidGA殿]

EDIT: I've imported it into PyCharm and after changing PyAudio version to 0.2.13 and torch to 2.0.0 now it doesn't error. It still can't install the requirements with pip. I can't get to install it because it always shows this error: ERROR: Could not find a version that satisfies the requirement torch==1.12.1 (from versions: 2.0.0) ERROR: No matching distribution found for torch==1.12.1 I've tried installing it first with "pip install torch" and I've also tried to install it with version 2.0.0 but then I get a PyAudio error (I've also tried to install it with pip install). It's a cool project and I'd love to try it.

[@copiumkiller]

Did you solve it?

[@MAki-gl9xq]

Error transcribing audio😢

[@Ninpo444]

hey can i get some help please?, i try to run this in my command prompt and i get this error- Error translate Error translate Error printing text: can only concatenate str (not "NoneType") to str

[@Ninpo444]

also is there a way to make this completely local? i love the template of this.

[@nexor7809]

ok im getting OSError: [Errno 5] Input/output error

[@houxia]

una pregunta puede funcionar sin microfono? mi microfono se daño y la camara me sigue funcionando?

[@MelonBreadVR]

how do u get it to track the face mo cap in vtube studio? Is that something they do in vtube studio themselfs?

[@ryusuikagaku]

I record motion on vts and set it as idle motion

[@bandoll9137]

everything works for me, except the part where its supposed to speak. it gives "Error transcribing audio" after the JP Answer line.

[@ryusuikagaku]

Hmm, for now i just can think 2 reason. 1. You are not adding the OpenAI API Key yet 2. Or you send zonk audio input (not speaking anything)

[@bandoll9137]

​@@ryusuikagaku I filled out the openai.api_key, and I can see its being used since there are requests in the Usage tab. The input.wav is being replaced with my audio, but i noticed that the output.wav isnt changing.

[@ryusuikagaku]

@@bandoll9137 can you try this code tutorial? If you can run this kzbin.info/www/bejne/q5nUhIWEqLqKocU You should can run my code too

[@bandoll9137]

@@ryusuikagaku yeah, i was able to do what the video showed, but still getting "error transcribing audio" for the AI xd.

[@ryusuikagaku]

Hmm, what about try to check translate.py, is all the library already installed? Because on transcribe function it's call function from translate.py

[@itskiggu]

Is it possible to use our own local LLM with Oobabooga instead of OpenAI?

[@konstantinrebrov675]

What is the background music?

[@쾌걸문제맨]

I want to input in Korean, where should I edit it?

[@フジュキサイカ]

Is it necessary to use openAI for speech to text transcription? Or can I just use google's speech to text?

[@ryusuikagaku]

Openai is more accurate

[@RC822Y]

I have getting this error “dll load failed while importing _mecab:”, and I have already installed visual c++. How can I fix this?

[@ryusuikagaku]

If you still facing issues with mecab, you can just skip this function in run.py because it's optional

[@puputimumu]

Wow.. I want to try the Ai asistent. Hope I can do it.😅

[@leonrrat]

Hi, I'm trying to use google translator by replacing the code line translate_deeplx with the one from google, but the program throws the error ("detect_google" is not defined) what should I modify to make it work?

[@ryusuikagaku]

You need to clone the new one

[@Yopara]

I’m very amazed by what you have done. I want to try your way but I don’t have a PC yet. Is Mac M1 possible to do the job? If no, can you share your the spec of your PC so that I’m investing in the right one. Thank you again for you excellence

[@ryusuikagaku]

I don't know much about Mac spec since i didn't have it. I'm using laptop Asus TUF FX505DV

[@graysterboys4322]

Hi can you make realistic version using image like livepotraithuggingface

[@triasaputra4943]

Bahasa inggris nya sangat mudah dipahami karna sesama orang Indonesia gw juga kalo ngucapin bahasa inggris kaya gitu

[@Regis_TV]

Can the livestream ID be changed to Twitch as I wanna add it in my Twitch chat?

[@ryusuikagaku]

You need to add twitch library/api for that

[@nateholden7598]

I still don't understand how you got the code to run. I am new to python.

[@jazfdz840]

im getting a problem when using silero , i tried to select the models and it keeps saying missing 3 required positional arguments: 'language', 'model', and 'speaker'

[@ryusuikagaku]

You need 4 parameter to use silero, example silero_tts(tts_en, "en", "v3_en", "en_21")

[@jazfdz840]

@@ryusuikagaku ty very much for the quick reply :D unfortunately i dont know what im doing wrong :C im putting the following # Silero TTS, Silero TTS can generate English, Russian, French, Hindi, Spanish, German, etc. Uncomment the line below. Make sure the input is in that language silero_tts(tts_es, "es", "v3_es", "es_2") but still keeps coming up with the missing 3 required positional arguments

[@ryusuikagaku]

Try to delete the old model.pt and rerun your code

[@devory2044]

Does code or openai have a limit on symbols? Or why the answer from openai in the code is cut off. For example, I ask him to tell a story, and he cuts it off halfway😵‍💫

[@ryusuikagaku]

I think they should have max token parameter

[@devory2044]

@@ryusuikagaku My mistake, I should have read the code more carefully, thanks for the help, it helped me. And thanks for such an interesting code👍

[@yumakuga2328]

would it be possible to use alternative API keys instead of gpt 3?

[@programateiro]

I was able to configure everything but got just one doubt: if I set my output audio I cannot hear what the characters says anymore. How did you managed to set the output to vbcable and also hear the character at the same time?

[@ryusuikagaku]

Set on windows setting to listen to spesific audio driver

[@programateiro]

@@ryusuikagaku Thanks man, your a lifesaver 😁

[@programateiro]

​@@ryusuikagaku great tutorial btw

[@Itsme-wt2gu]

How to restart the voice vox thing?

[@AI-mona]

do you know how program silero?

[@lujen0001]

i keep getting an error "no module named "openai"?"

[@Fidseen]

how to change the voice to Indonesian?

[@LairroTerro]

I liked the video, but I see some changes in the run .py code... do I add them by myself?

[@ryusuikagaku]

Use the updated one, tutorial is on my repo