A ChatGPT Voice Assistant You Can Talk To - Open Source: Vivy

Other Tutorials:
Download Git and Vscode - • How to Install Python,...
Github Issues - • Getting Help On Github...
Links Mentioned:
Python - www.python.org...
OpenAI API - platform.opena...
Eleven Labs API - beta.elevenlab...
Github - github.com/Jar...
Come join The Learning Journey!
Discord - / discord
Github - github.com/Jar...
TikTok - / jarodsjourney
If you found anything helpful, please consider supporting me and the content I am trying to produce!
www.buymeacoff... |
Hardware for my PC:
Graphics Card - amzn.to/3pcREux
CPU - amzn.to/43O66Ir
Cooler - amzn.to/3p98TwX
RAM - amzn.to/3NBAsIq
SSD Storage - amzn.to/42NgMFR
Power Supply (PSU) - amzn.to/3NBAsIq
PC Case - amzn.to/447499T
Mother Board - amzn.to/3CziMXI
Alternative prebuilds:
Corsair Vengeance i7400 - amzn.to/3p64r22
MSI MPG Velox - amzn.to/42MnJHl
Cheapest and minimum specs recommended:
Cyberpower 3060 - amzn.to/3XjtZoP

[@Jarods_Journey]

I know a lot of you have had some issues with the exe files and the best solution to that right now is to just run the Python scripts. If you're not familiar with python, the updated github readme section goes through briefly how to do this (as well as this video)! I've had some recent issues with the EXE files as I made a switch over to Windows 11 and so I am hard-pressed to believe that Windows 11 could be causing this issue since when I port the exe files back to my windows 10 device, they work just fine.

[@nottomayo]

12:20 I was looking through the files and "gpt_ assistant py" is missing from the folder ...(*￣０￣)ノ I also looked in the github did you take it out? i was following along and testing

[@babupekora]

@@iamasoloplayer its open source u can reasearch it

[@TommasoLintrami-q3u]

Ahahahah! Why, do you really think that since Windows 2000 (more than 20 years) there was some important technology change in the operative systems ? Windows is so damn pathetic.-...

[@MrOneTango]

Jarod! awesome work man. I am highly intrigued. I am doing some research right now on the possibility of a voice assistant in a "closed" environment, offline. I would love to connect with you to get more insight and maybe build something for a pretty large business segment.

[@Jarods_Journey]

Appreciate it! Discord or linkedin are the easiest way to connect with me, but by far I'm more active on discord. Feel free to connect anytime! 🤟

[@MrOneTango]

What is your discord user/tag? I will definately look you up.

[@Jarods_Journey]

@@MrOneTango If you join my discord group chat, I'm the owner of it @jard in the description

[@sammi012345678]

Hey! Awesome video and really love this and what it does! And I've got a quick question if you don't mind, is it possible to set it up where I press a button to make it start listening? and then a button again to make it stop listening (use case is for talking to the bot, but then being able to talk to the camera without having the bot listen to all of that aswell) Not really familiar with Python scripting so I'm just following along the tutorial, thank you!!

[@s.akbar1898]

bro the package of gpt_assistance are no longer on the repository wasn't it?

[@johnewiseman1]

I get this error when i try and run it, any ideas what is wrong or where I could go to try and fix? "ImportError: attempted relative import with no known parent package PS C:\Users\MonkeyBiscuit\Desktop\Code\Vivy>"

[@klikfloristpalembang4053]

how to change "hei there", not working when i speak "hei there"

[@chinonoob3219]

i have the exe but it only shows initiated: and nothing happens, do i need to do something else?

[@Genny_Woo]

place the .bat in the Vivy directory

[@jayr7741]

​@@Genny_Woo Whenever I speak "Hey there" it worked but after that immediately come "no longer listening" problem. What should I do? Plz help me

[@Genny_Woo]

@@jayr7741 so you say « hey » then it listens after that you say what you want then it will stop

[@Genny_Woo]

@@jayr7741 make sure to have setup your microphone properly

[@Genny_Woo]

@@jayr7741 you also need python and you need to run the requirements bat file

[@TheWiggleMama]

it always seems to get to a initiated state for me and goes no further, I say "hey" but nothing happens :( any assistance?

[@Jarods_Journey]

Try speaking closer to your microphone and speak a bit louder to see if it initiates. If it works, I'll have to lower the threshold for which it activates since it takes in your ambient noise to gauge when your speaking or not. Also, you might wanna open up an issue on GitHub so that we could track it as well!

[@TheWiggleMama]

@@Jarods_Journey I managed to get it working once, but shortly after it stopped working But I have to admit it's amazing to see it operate :D i'll be sure to open a issue on GitHub for you!

[@jayr7741]

​@@TheWiggleMama Whenever I speak "Hey there" it worked but after that immediately come "no longer listening" problem. What should I do? Plz help me with this

[@TheWiggleMama]

@@jayr7741 I wish i could asisst with your issue but to th ebest of my knowledge it would appear your microphone may not be working with the program

[@jayr7741]

@@TheWiggleMama but whenever I speak "hey there" its response "I m listening" but after that whatever I ask it, it suddenly stop responding. Hence proved its not microphone problem. (my english is weak plz understand my feelings, I want to try it)

[@tribbz1329]

The script looks different from what the video shows. I see ''kokoro'' instead of ChatGPT for a lot of the code written and I don't know Python so I'm not sure if I can follow through without messing up. I'm stuck at 13:50 where I try to type in ''chatbot.assistantP'' and nothing shows up. Does this tutorial still work or do I need to do things differently?

[@Jarods_Journey]

A lot has changed now and it's currently going through some refactors. I've been working on Vivy recently and have plans to update the usage of it in a future video

[@airuler4118]

this is easier and better alternative than talking with real people lamo

[@茲鉧]

it's such a good video, but i got stucked in the middle of it, i dont think my chatgpt was working, my ai voice was connected successfully, and recognized my [ hey, there ], and reply, but when i said next sentence, nohing came out, just waiting for next [ hey, there]

[@g-dojjo]

Thank you so much!! everything working very good ! :D Just one question X_X can we using Voicevox's voices for this ? Elevenlab's price so high for my country :D

[@peuug]

Hi , Jarod . Awesome video and tutorial i think i watched the video 10 times haha and i'm willing to use the content for a personal project but i wanted to ask you is it possible to merg/associate your project with another one ? i mean using the code of someone else and add it to yours ? the idea is to create a Vtube Avatar and make it speak all of this associated to twitch .

[@Jarods_Journey]

Haha appreciate it! The project is open source so use it to your hearts content. You can either download manually or clone the repo to add it to your own project, or you can simply fork it and start working on that repo. I reccomend you get an understanding for how my gpt_assistant works so that you could incorporate it into your own project

[@rookiec9655]

First of all, thank you for providing this tutorial. However, when I say the keyword, it keeps outputting the following error message. I would appreciate it if you could let me know how to solve this issue. Error message: ``` Listening: Done listening. Google couldn't process the audio. Adjusting to environment sound... Listening: ```

[@Jarods_Journey]

What operating system are you using? My finding is that there is some type of issue with Windows 11 that is causing this issue as I encountered the same issue when upgrading to windows 11. Haven't found a fix for it yet. You would have to run it as a python script manually

[@syntheticstories]

Just ran through this tutorial on Win 11 and running my test script (not the exe) I'm seeing the reply from google, but it heard me, and prints out my words as txt in the terminal, but there is no reply from the chat bot. Any idea if this is a Win 11 issue also? Edit: I actually was able to get this to work but have a different issue when trying to use ElevenLabs. I've submitted a bug on Github for that, but I was able to get this to work with the default Microsoft voices.

[@M4rt1nX]

Thank you!!! is working.

[@Jarods_Journey]

Glad it works :D!

[@M4rt1nX]

@@Jarods_Journey I was feeling special, but I just realized that this was auto generated.

[@Jarods_Journey]

@@M4rt1nX haha not yet, anything that's auto generated on my channel with have Jarods-AI-Assisntant before it lol

[@edosetiawan9589]

Very cool 😮, how to custom data version?

[@lizardmutt]

Hi! I've run into an issue with the exe. It responds with "I'm listening" after I've said "Hey there", but after that it doesn't respond anymore (except when I say quit). So those two commands work, but nothing else.

[@chinonoob3219]

Same here, i cant make the exe file to work so i always use the py file 😢

[@Jarods_Journey]

Hmm yeah, I'm not sure what's happening in the exe version of it, I recently lowered the energy threshold for the mic in v0.2.1 so it should pick up better, and the fact that it hears you initiate it as well tells me your mic is working and it's picking up just fine... I'll have to open up some variables to the exe so you guys can change them

[@KoalaCrockie]

same problem for me

[@Jarods_Journey]

@@KoalaCrockie Quick question, so your able to get past the I'm listening part? I can imagine there might be an issue of where the microphone is generally not being heard. Would you be able to run the chat.exe and see how if it's able to register voice?

[@chinonoob3219]

@@KoalaCrockie try the py file , that one works right, i dont know what is the issue on the exe file , it works on his computer.

[@DanielJones-wg7eg]

Hey mate cheers for introducing me to this really cool project. With the exe files they all seem to very rarely get the initial keyword input and start listening. Sometimes they work on the first try after being extracted from the zip folder, but if I change the prompt they'll stop working again. However, I'm not facing any issues with the code version so that's really odd. Best of luck with the project mate, I'm gonna keep an eye on this one!

[@Jarods_Journey]

Thank, I'm glad you're taking a liking to it! I'm not entirely sure on what's occuring between the EXE version and the python build, in theory they should be the same. I do have the min threshold pretty high though so I will probably have to lower that for the exe build. It's the same value currently, but I couldn't reproduce the issue on my end but I also have an XLR mic so that probably contributes to the audio clarity as well lol!

[@DanielJones-wg7eg]

@@Jarods_Journey ​Thanks for getting back to me mate. Out of curiosity, what steps would I need to take to build the example vivy code from your github into an executable like the quick use ones on your github? That way I could give it a try with the lower threshold as it does work flawlessly within visual studio. Sorry for the absolute beginner question, before this my only experience of coding has been simple scripts for unity games lol

[@Jarods_Journey]

@@DanielJones-wg7eg Haha its all good, we all start somewhere! I actually made it very easy to build all of them for my convenience sake and that's to run the build.py in the main folder. Just run that and it'll create the files you need.

[@jamezon3136]

do you have any plans to update this and make an updated tutorial? I have noticed alot of differences and for the life of me can't figure it out with the differences lol.

[@Jarods_Journey]

Yeah, 100%. Just haven't gotten around to making a stable version of it yet, but when I do, I'll have a video for it lol. There's a lot going on that's new.

[@jamezon3136]

@@Jarods_Journey Awesome! I plan to continue to poke around and see what i can do. I'm excited to see the new update as well!

[@gatsbyliu1084]

Hello, I made it talkable and I felt geniunly happy from following you! However, the response speed is quite slow and I am trying to integrate it with TTS that you introduce and some local LLM (so its free maybe?). Would you please make a follow up video on these? My ultimate dream is to make this run locally or on cloud for free or low cost so I can have a true AI companion, will support any creator who contribute to this!

[@Jarods_Journey]

Appreciate the super! Response speed is dependent on two factors, LLM response and TTS generation speed, so if it's all local, it's hardware dependent. I haven't looked at it in awhile but eventually I will be working on getting local LLMs into Vivy as time allows, so stay tuned till then!

[@edosetiawan9589]

Why Initiated: (cant type keyboard)

[@edosetiawan9589]

Not working plis help

[@kurotesuta]

I trained a RVC voice , do you happen to know how I could integrate that in this repository?

[@gilbertocisneros85]

Is it possible to use this with different languages? Like Spanish.?

[@Jarods_Journey]

Entirely, you just need another text to speech engine so it can speak back to you, currently just English though.

[@rahavartsi8250]

that's a super cool video, though i have encountered some problems, what did you do to move the code from the virtual environment back into your main pc?

[@Jarods_Journey]

Ah sorry, could you give me more context of what is happening? The only code that's in the virtual environment are the python packages you pip install, everything else should be in your computer

[@rahavartsi8250]

​@@Jarods_Journeyaa nevermind it was just a little problem with the python version I used, switched it to the one you used and it works just fine, thank you so much dude :D

[@Jarods_Journey]

@@rahavartsi8250 glad it got resolved!

[@jayr7741]

Whenever I speak "Hey there" it worked but after that immediately come "no listening longer" problem. What should I do?

[@Jarods_Journey]

Could be an issue with threshold value that I set for the code. If you are able to get past the first listen, it should be fine afterwards though. Try speaking very closely to your microphone to see if it picks up on things a little bit better. But another release should be out soon where I lower this value.

[@chimlimhao]

I'm really appreciated this awesome video, though it only shows "initiated: ". I tried to speak louder to the mic but it still does nothing. Anyway, thank you so much for this amazing video.

[@chimlimhao]

nvm it's work now that i use my headphone instead lol

[@chimlimhao]

though it's working but do you have any idea on how can i improve the wake up call voice? i mean it would be great if we dont have to struggle waking it up by screaming "hey" xD

[@Jarods_Journey]

I'm glad you were able to get it working eventually lol, as for the wake up part of it, I lowered the energy_threshold value in the 0.2.1 release so it should be set at its default value. That being said, if you're running the code of it, you can set this value even lower and see if that works. If you're running the exe version of it, if you're comfortable rebuilding it from the code after you modify the values, that could work. I am working on trying to get a GUI application up for it that should be a bit more customizable for users to use.

[@totoma3297]

is there any way i can add custom commands to the ai? I have a cli i would like to integrate.

[@Jarods_Journey]

You would need to run things in different processes and threads, that integration would probably be either really easy or difficult

[@hemanthedaoo8601]

Hello, it is indeed a great project. I have some sample voices. How could I implement the sample voices instead of "ElevenLabs"? Could you please tell me? It will be very helpful. Thank you.

[@Jarods_Journey]

I'm assuming you mean some alternative TTS system? There is a method in the code that handles audio generation so you would have to swap out the logic that uses eleven labs as the TTS system and swap yours in there.

[@hemanthedaoo8601]

@@Jarods_Journey Great, I will give it a try.

[@billyindrajaya]

Do you have google colab for share?

[@aerx13]

anyway can we switch the language?

[@AraragiKunHiPach]

Help please, I also want to make a language model, but it is not in my language, I will be very grateful if you help, or just improve yours :)

[@blazingsun394]

Greatly appreciated the how to, I was already coding my own version but this just saved me so much time, I'm loading it up with all my books on brewing beer to have a brew a.i assistant. However I'm having one issue with response time taking 20-40sec Is there anything I'm missing that could be causing this , I would imagine I have a driver somewhere that needs to be updated.🤷🏼‍♂️

[@Jarods_Journey]

Aye that'll be awesome, one that is able to help you through the brew making process. Most of the assistant is handled through openAI, meaning if it's taking awhile, it could be due to the length of the text you are speaking since that audio file gets uploaded to openAI for transcription, and then the text gets uploaded back as well. So this is most likely just how quick you can upload and download from openAI's servers. It'll further take longer if you are using eleven labs as well.

[@blazingsun394]

@@Jarods_Journey that explains a lot actually and yes I'm using eleven labs ,Because having a brew a.i is one thing....but having it sass me in a real time voice when it gives me my answer is priceless 😂

[@Ctesting]

This is awesome. Any way to set it up with Alpaca instead of GPT?

[@Jarods_Journey]

Yes, nowadays there are a bunch of open source LLMs. Only issue is getting the time to implement it :)

[@masuramaru]

This code is interesting, there is no money to test, but there are some functions there are some functions that are not in the video, at least now. and how do you add more voices in elevenlabs?

[@Jarods_Journey]

Add more voices by cloning in Eleven labs

[@anon00s]

Hello everything is working alright but I think it's not responding to my voice what should I do?

[@Jarods_Journey]

Try saying "hey" closer to the microphone and see if that does the trick. I have the threshold value set a bit high, I will be reducing that in the next update

[@thekasanegros8508]

Good video bro, but I have a question. Is there a way to make the AI ​​speak in another language? Sorry for my bad english

[@Jarods_Journey]

Thank you! It's possible, but there has to be a speech to text engine that supports it. I know some that are Japanese as I know some Japanese, but for other languages you'd have to do some searching. As well, if you just wanted to use the system voice, you could download the standard microsoft system voice and then specify that voice in speech recognition, if you wanted to do this, chatGPT could probably guide you through it!

[@joaquinbuenocaro4345]

Hello. Thanks for this very good video. But I have a problem, I have entered the elevenlabs key, but if the chat is not chatgpt or gpt-4 and I don't have a key, how do I do it? thx.

[@Jarods_Journey]

Unfortunately, you need an OpenAI API key for Vivy to work. Without it, it will not function as it's a key component of the project

[@joaquinbuenocaro4345]

@@Jarods_Journey Thanks

[@usamaadeel9346]

Hay Jarods Awsome work brother, I just using your buil aplication but I''m facing a problem and that it is taking long pause then listen and after give response, is there any help form you.

[@jayr7741]

Whenever I speak "Hey there" it worked but after that immediately come "no listening longer" problem. What should I do?

[@Jarods_Journey]

Thanks man, appreciate it. Could you let me know after what part it's pausing, after listening, or no longer listening? If after no longer listening, that means it is no longer due to the local file being ran, rather the upload speed to openAI as it contacts their API. With whisper on, this compounds as well as you must upload that audio file up to the whisper API.

[@usamaadeel9346]

@@Jarods_JourneyIt takes a pause when it has to give a response, after a long time it gives a response.

[@alvie2000]

I wonder, do you it is impossible to make a discord bot which can join a voice chat and talk to it like that?

[@Jarods_Journey]

Possible, I think some people have done it, but I have no experience with discord bots :/

[@odarshan..]

does it still work ?

[@Dardyysan]

Can this Ai using chat box for asking?

[@Jarods_Journey]

Unfortunately, not out of the box. If you wanted to change the voice recognition to user input, you could inside of the code but Vivy is setup as a voice Assistant

[@Dardyysan]

@@Jarods_Journey Can you help me why My API Key Quota Exceed

[@Jarods_Journey]

@@Dardyysan Your chats might be longer than 4k tokens in which case it'll reset the chat. If it's quota exceeded, that means openAI isn't able to bill your account, so you might have to check your billing. If you could open up an issue on GitHub, describe the issue, and provide some screenshots, I could probably help you out better there!

[@pekogimso76]

OpenAI API and Eleven Labs are not Open Source

[@Mario85832]

Hi your project is great , but i am getting errors in my code can you please help me with it ,I will give you the errors below error 1: line 4, in from package.gpt_assistant import ChatGPT ModuleNotFoundError: No module named 'package error 2: Traceback (most recent call last): File "d:\code\Vivy\assistants\assistantp.py", line 35, in chatbot = kokoro.Kokoro(personality=personality_dir, File "d:\code\Vivy\assistants\package\kokoro.py", line 23, in __init__ with open(keys, "r") as file: FileNotFoundError: [Errno 2] No such file or directory: 'd:\\code\\Vivy\\assistants\\keys.yaml' guys if any one know how to solve this errors ple help me out

[@Jarods_Journey]

You never set up your openAI keys yaml file. Make sure to rename it properly

[@dayssnow1049]

Hi, I found that this script includes def listen_for_voice(self, timeout:int|None=5): and the "|" operator for type hinting is not supported in Python 3.9 or below. I suggest you could add note that python=3.10 is required~

[@Jarods_Journey]

I didn't know this haha, now I do. Ill update the repository for this

[@ChampionsOfTheTable]

Is there a way to change the default tts without using ElevenLabs, could I put in the Uberduck API instead?

[@Jarods_Journey]

Yup, you just have to modify all of the code where audio is generated which is just 1 function in the class. Then you have to update all of the logic in the assistant functions to choose that instead

[@ethanmarshall-n9v]

am i able to change its name and characteristics?

[@migoratstorm305]

Hey. thats Voice assistant sound rly nice. what do you think about adding a function that lets you open games or other programs? or how could i add something like that? would it be possible if you create a list with paths of the programs and key words and if you say "open Firefox" it will open it for you? Im at the learning stage of programming and was experimenting with chatgpt so i not very expirenced at this stuff^^

[@Jarods_Journey]

You could do this and this is pretty possible IMO. You could use something like subprocess in python and simply create the list of all of the program path exe files. Then you could call a function say "open_program(name_path)" where you feed in the path of the program you want to open. This is actually pretty intriguing and I actually have an idea on how to implement this so stay tuned to my channel because I might actually add this functionality...

[@jarodmica]

Hello Me.

[@Jarods_Journey]

Hello! How may I assist you today?

[@jarodmica]

@@Jarods_Journey Problems with everything lol

[@Jarods_Journey]

I'm sorry to hear that you're experiencing problems with everything. Can you please provide more details about the specific issues you are facing? Once I have more information, I can try my best to assist you in resolving your problems.

[@jarodmica]

@@Jarods_Journey Absolutely not

[@Jarods_Journey]

I'm sorry, but I'm not sure what you are referring to. Could you please provide more context or information about your issue or inquiry so I can better assist you?