Hands-on Tutorial: Build a Google Gemini-Powered Voice Assistant on Raspberry Pi

[@estugit]

I've watched almost all of your videos, great work! Please make more useful videos.

[@TechMakerAI]

Thank you, will do!

[@videoserver1]

Good video and explained well! Will try with my Pi 4 B.

[@TechMakerAI]

Thank you for your feedback! Good luck and have fun!

[@Alshawafi1744]

Thank you, brother, for what you are doing. Is it possible for me to apply it and have it be in Arabic?

[@TechMakerAI]

Hi, thanks. The Google Gemini can work with Arabic text input, see here ai.google.dev/gemini-api/docs/models/gemini. The Python Speech Recognition package can also handle Arabi audio input. I think the gTTS can handle Arabic too. please check this video for more detail on how to set it up, kzbin.info/www/bejne/nYqpeHuvd8SJgJI

[@Negreen-i8b]

Really good video, if I use a usb speaker do I still need the audio amplifier ?

[@Negreen-i8b]

Also, I do not think I am going to use the LEDS, would the usb speaker and not using the LEDS make me do some changes in the code or no ? Thank you

[@TechMakerAI]

@@Negreen-i8b, if you use a USB speaker, you don't need the audio amplifier. You may not need to change anything in the code for the USB speaker. For LEDs, you can comment out all the lines that toggle the LEDs, such as "gled.on()."

[@jan3019]

subscribed! i love this kind of projects! thanks for sharing! i wonder if you can also add those tiny black and white oled display to display the text that you're asking and the Ai's answer.

[@TechMakerAI]

Glad to hear that. Thanks for your feedback! Yes, it is possible to add an OLED display to show the text. That will be the topic of another video.

[@-IramNishat_2010]

Hey (Pi 4 case with cooling fan and heatsink) where is this used in the video?

[@TechMakerAI]

It is not used in the demonstration, but for long-term use, you need them to keep the RRi cool. If RPi gets hot, it will show more delay when responding to your questions.

[@-IramNishat_2010]

@@TechMakerAI oh okay got it! and also is there any alternate for the audio amplifier?

[@TechMakerAI]

@@-IramNishat_2010 Yes, you can also use a USB speaker

[@neethumohan1352]

Hi This project worked for me. Is it possible to reduce the response delay ??

[@TechMakerAI]

I have optimized the code to run on multiple cores so that it will have less delay. To further improve the performance, you can add a fan to your Pi to make sure it is running at its best condition. Also, a fast wifi connection will also help.

[@cenzo-x]

does it work with raspberry pico 2w?

[@TechMakerAI]

It is a possibility but I cannot tell if it will work like Pi 4.

[@Yelibots]

are there any other microphone modules that work with gpio pins with rp4 and other audio amplifiers that can communicate with the pins without having to attach audio cables

[@TechMakerAI]

It is not easy to acquire audio input via GPIO pins. You probably need to purchase extra components for that. It is a little better for audio output, see here raspberrypi.stackexchange.com/questions/49600/how-to-output-audio-signals-through-gpio. The easiest way is to work with USB speaker and microphone.

[@haribabualapati8780]

Sir please guide me how to get the updated data after the cut off date and time.

[@TechMakerAI]

This is a limitation of the large language model. The dataset used to train the large language model always has a cut-off date and time. You can try to use a newer model. It will have a newer cut-off date and time.

[@niyasnowshad3773]

Hi Great video, Can we use this project offline or does it require internet connection at all time?

[@TechMakerAI]

Hi, @niyasnowshad3773: it needs internet to call Google Gemini API. But there are some offline options available which are not as powerful as Gemini.

[@WaterfrontConcerts]

Would this work on an ESP32 S3 chip board? or perhaps two of them togehter connected via uart? for tts/stt.. ? which version of the pi is the lowest one (cheapest) that would work for this.?

[@TechMakerAI]

I am not sure about the ESP32 S3 board. In this video, I used Pi 4 model B 2GB. I think 1 GB version may work too.

[@Yelibots]

Any decent touch displays that can work with rp4 that communicate with gpio pins for animations?

[@TechMakerAI]

Have you checked on Amazon? Will this work for you: amzn.to/4hD0JmC

[@HananPHashim]

Hi sir , How to get live feed like time , weather etc in gemini ai

[@TechMakerAI]

I do not think you can get the live feed from Google Gemini, because the Gemini model was trained by using a data set that was collected before a cut-off date and time. So it does not have the live information we have today. However, you can get live time information from RPi and the weather information from internet. I will add those to a new version of the Python code.

[@pixyful7025]

hey ! already thank you for sharing a file that really works 😀👍 I try to launch your python script at raspberry startup but impossible to start it with the virtual environment. I have an error message "import google.generativeai as genai" except if I enter the line manually everything works do you have any idea how to do it?

[@TechMakerAI]

Hi, @pixyful7025, I am glad it is working. It is possible to automatically launch this Python program at the startup of RPi. I have a video on this topic coming up very soon. Please subscribe to my channel as you will be notified when it is out.

[@pixyful7025]

​@@TechMakerAI I found a little thing that seems to work by tinkering. I wrote it in .bashrc and told the raspberry to log in automatically

[@pixyful7025]

@@TechMakerAI I will wait for your video to compare the most optimized 😉

[@DoctorRobot-e4w]

Can I train the ai to produce me a particular types of answer as we can train chatgpt ?

[@TechMakerAI]

That is a great question. The answer is Yes. Large language models (LLMs) like Gemini are trained on massive data. While you may not be able to train AI from scratch, you have the option to fine-tune the LLMs. You provide additional data (answers) tailored to your desired questions, and the LLMs adapt.

[@DoctorRobot-e4w]

@@TechMakerAI This is my School Project. After A long search I found this video and Thank you that this works. Now Can you help me more please? I want to train the bot that when I ask the bot a question, it answer me such a format that I trained it. How to do that. please help me

[@TechMakerAI]

It sounds like you want to fine turn the LLM. Please have a look at this article: www.datacamp.com/tutorial/fine-tuning-large-language-models

[@MrBluedrive]

can you also make a tutorial to work with a bluetooth speaker and without the leds. or wouldnt it work that way?

[@TechMakerAI]

Yes, it will work with a Bluetooth speaker and without LEDs. I have used the Bluetooth speaker before, see this video kzbin.info/www/bejne/pHnTe5R7appsqJY. But it requires me to run some commands to pair the Raspberry Pi with the Bluetooth speaker each time. So I switched to wired speaker after that.

[@Leith79]

I am working on a python jarvis project and i want it to work with a raspberry pi zero 2w because of the small formfactor do you think this is possible?

[@TechMakerAI]

It may work, but you will experience a delay in the response

[@Leith79]

@@TechMakerAI yeah thats what i thought i will probably Switch to a better Raspberry pi

[@Leith79]

@@TechMakerAI thank you though

[@TechMakerAI]

@Leith79: it will be a good idea if you switch to a better RPi. But if you manage to make it work on Pi zero 2 W, please let us know too.

[@HananPHashim]

Hi sir , how to change raspberry pi wifi while ssh

[@TechMakerAI]

Please check out this video: kzbin.info/www/bejne/gXjXdGp3bdCChLs

[@arunpanangad]

Is sound amplifier really needed?

[@TechMakerAI]

It is needed if you use a passive speaker like in the video. However, if you use a USB speaker, then it is not needed.

[@TheFamilyHub1]

Hi sir me and this person is working on same project same class

[@mukagedek4010]

hey, thanks for the video. I just want to start project exactly the same with yours, mine is kind a companion robot. but the question is, is the service from google just like gemini ai or TTS are completely free? or there's additional cost for that?

[@TechMakerAI]

@mukagedek4010: The Google Gemini API and Google TTS are currently free to use. The TTS will most likely be free in the future too. However, it is hard to say about the Google Gemini API. Given that the ChatGPT API is not free, Google may charge a fee in the future for Gemini or they may offer a limited version for free. Anyhow, I would say go ahead and build it. There will be some free options available. Even if Google charges a fee, I think it is still worth it. By the way, in this channel, we are also building companion robots. Please subscribe as you will see in future videos.

[@TheFamilyHub1]

Sir if we use rasbery pi 3 any change in code

[@TechMakerAI]

It should work with Pi 3 too but maybe a little bit slower. If you run into issues, please report back here.

[@TheFamilyHub1]

Thank you

[@TheFamilyHub1]

I am making this for school project

[@TheFamilyHub1]

How to code it can u make a explained video on coding pls

[@TheFamilyHub1]

I have also subscribed to u and watched all ur videos soo good ty

[@charanartsandexperiments]

is there any way to communicate with you

[@TechMakerAI]

Yes, you can find my email address on my KZbin channel page at: www.youtube.com/@TechMakerAI

[@m9playz239]

Hello....I have a doubt...if you could help... what if i don't use audio amplifier and directly attach a usb speaker....will it work????.....also can this work on raspberry Pi 3B? Also, what is the wake word for it and the word to stop it from listening??

[@TechMakerAI]

Yes, a USB speaker will work too. I am working on a version of the hardware and will have a video about this topic soon. RPi 3B may work but will be slower due to the small memory size. The wake word is "Jack". To end the conversation with AI, you just need to say "That is all". It will continue to listen but will not respond. Just like Siri, it is always checking to see if you are saying the wake word. If you do not like to see the printed output, just comment all the print() statements.

[@m9playz239]

@@TechMakerAI can you help me change the wake work if you can?

[@m9playz239]

@@TechMakerAI can you make a video on usb speaker sonn cz I want to make this and present it in an exhibition after 2 weeks....I will be very grateful!

[@TechMakerAI]

Just do a search and replace of all the "jack" with any name your like.

[@TechMakerAI]

@m9playz239: You can use the current version of the code with a USB speaker. It should work too.

[@xpectz]

Script crashes from time to time, FYI. Something regarding the chat history and/or communication with the google gemini AI library. I looped the .py script to restart every 2 min as a workaround. Luckily, these sorts of errors are not too common. Also please tell me how to make it stop listening. I tell it "that is all" and it still picks up what I say.

[@TechMakerAI]

Thank you for your feedback! I am aware of the communication issue with Google Gemini API. It can cause crash sometimes. I have seen less issues with the OpenAI ChatGPT API but that is not free. I made some changes to the code on line 259 in this version. When you say "That is all", it should stop interacting with Gemini API but it is still printing out what you are saying. You can comment out all the print() since you do not need it to print transcript when using RPi (It is there only for debugging purpose) or you can move lines 259 to 263 after line 270 (if "jack" in text.lower() and slang == "en-EN":). Note that this code was written so that users can interact with AI anytime they want. They can wake up AI and put it in sleeping mode. This voice assistant should be on all the time like Siri on your phone. If you stop it from listening, then how do you wake up it next time?

[@xpectz]

@@TechMakerAI Got it. I have follow up questions. If I do not say "That is all", how many minutes until it times out which will require the wake word? Which line can I modify this? Also, I already have an OpenAI API Key i paid for. How can I switch over to Open AI???

[@TechMakerAI]

@xpectz: For your first question regarding how many times it takes for it to require the wake word, I think you can add an integer variable to count the times and then set the AI to sleeping mode after it reaches a threshold. For the OpenAI version, please see this video: kzbin.info/www/bejne/oXqumJ9pn6mcors

[@xpectz]

@@TechMakerAI How does it work now? When I stop talking to it, it goes to sleeping mode. What triggers it? I have not timed it. Will check out the video, thanks.

[@TechMakerAI]

If you add an integer variable to count the times and then set the AI to sleeping mode after it reaches a threshold, then, it will be triggered by your integer. To wake it up, you just need to say the wake word again.

[@GauravPotbhare]

Sir please make more tutorials on raspberry pi and Arduino

[@TechMakerAI]

Sure, I have new videos about Raspberry Pi, Arduino, and STM32 coming up soon.

[@GauravPotbhare]

@@TechMakerAI sir could you make full tutorial on Open cv because I need use on my project but still I don't know how to use on project and also make tutorial on how to install open cv on raspberry pi it's most important part sir please try to make as soon as possible I'm waiting sir

[@xpectz]

How many minute until it stops listening? Is there a way to adjust it? Or to tell it to stop listening?

[@TechMakerAI]

Please see my reply to your newer question.

[@keithng4209]

I am a total noob with a 2nd had pi4. i followed all your steps but encountered the following after pip install -q -U google-generativeai: WARNING: Retrying (Retry(total=4, connect=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', remoteDisconnected( 'Remote end closed connection without response'))': /simple/protobuf/ Does this mean the loading of the AI failed? How do i rectify this, please?

[@TechMakerAI]

It means that you did not install the Google Generative AI package successfully. I would recommend that you try it again. If it fails, you can always format the micro SD card and start from the beginning.

[@keithng4209]

@@TechMakerAI Thank you. I'll dp that and get the hardwares so I can do the voice test, like you did in your earlier video.

[@TechMakerAI]

Yes, that is a good idea.

[@icyuniverse5141]

Hi Great video, Can we use this project offline or does it require internet connection at all time?

[@TechMakerAI]

Thanks. It needs internet to communicate with Google Gemini API. There are offline large language models but those models often require high end GPU to run smoothly. You may be able to find some lightweight ones to play with on RPi.