How I CREATED My Voice Assistant Device With WAKE Word Using ESP32 And Home Assistant

[@kirilldyatko3978]

Great tutorial! I've spent less than an hour to repeat your project on bread board and everything works! Thank you very much!

[@SmartHomeCircle]

Glad it all worked 🙂

[@fpvclub7256]

Very cool! I remember writing this myself back in the day from scratch - this is much, much better!

[@SmartHomeCircle]

Thanks a lot for such kind words 😄. Glad you liked it.

[@hebe1792]

I recently tested this (prior to watching your video). The ESP32 randomly stops working due to memory buffer limitations; it will recover but it is inconsistent. I am testing the Wyoming satellite project with a Raspberry pi 3 and same mic and amp, and it seems to be more reliable. Any raspberry pi supporting 64 bits and wifi should work.

[@SmartHomeCircle]

do you mean the memory buffer while playing the media or the output of the voice assistant? then yes. I do face the same issue. I however dont care about the the output as long as the command is recognized and executed successfully. I am still using the device as my daily driver and since my setup is completely local, I just love it. 😍

[@malballuk]

Not sure if you can help with this one, I built one into an old Echo device and implemented more pins on the ESP32 to use the buttons on the echo. I added a red LED to show if the mute mic button was pressed and code to turn off the wake word if its pressed. However, if I say Hey Jarvis, turn on the lights, it also turns on this LED # Red LED - platform: binary name: "Mic Light" id: mic_light output: light_output output: - id: light_output platform: gpio pin: GPIO19 # Turn on off the mic binary_sensor: - platform: gpio pin: number: 18 mode: input: true pullup: true id: "Mic_pin" disabled_by_default: true filters: - delayed_off: 10ms on_press: - light.toggle: mic_light - if: condition: light.is_on: mic_light then: - switch.turn_off: use_wake_word else: - switch.turn_on: use_wake_word

[@SmartHomeCircle]

Your code seems logically correct. May be you could add a delay after the light.toggle to allow the light state to be in the on state, so that the next if condition gets the light.is_on is true. Let me know if it works.

[@andrejsersen59]

Very well done! Great project! Thanks for the video! Keep up the good work!

[@SmartHomeCircle]

Thanks a lot ☺️.

[@JoaquinMoreiraVideos]

Thank you for your great contribution, I look forward to your next video.

[@SmartHomeCircle]

Thank you 😄

[@othman2206]

Thank you for all this great effort, eagerly awaiting the next video

[@SmartHomeCircle]

Thanks a lot for the continued support.

[@Foggy3657]

Super Job. Well done. I too look forward to your next video.

[@SmartHomeCircle]

Thank you 😄

[@sej84]

really cool project. Beautiful product good job.

[@mikesmods317]

I built mine this weekend following your video and it works great!! I do notice a couple things I wanted to ask you about 1. the low volume level is really low and at full volume it is not very loud is there a setting for this in the yaml? 2. When I ask it something and it has a follow-up question it is not listening for my reply is this an easy fix?

[@SmartHomeCircle]

1. In my config, which is there in this article: smarthomecircle.com/created-voice-assistant-esp32-with-wake-word-in-home-assistant I am able to adjust the volume in Home Assistant , because I am using the media_player component. 2. I haven't asked it a question yet, for which it asks me for another question. 😃 Are you using AI in your assist pipeline?

[@222inverter]

this is a great home brew device well done..!! I'm working on something similar..👍

[@SmartHomeCircle]

Thanks a lot. Hope yours also turn out to be great. Happy Building 🙂

[@drumslapper]

Built this and it works great except I can't get any audio out. The ESP media player entity exists but no audio occurs. Tried two amps, same results. Thanks for the project.

[@SmartHomeCircle]

Does it absolutely not play anything? meaning the output from the text to speech?

[@drumslapper]

Thanks for getting back to me. No output at all. I have built two with the same results. The voice input works really well, just no audio output. Tried different pipe lines and still nothing. Speech output from other sources is ok in the HA setup. Cheers, Bill

[@cvasantrao]

I am having the same issue. Everything works fine except the audio. Replaced the MAX98357A but still no luck. Currently re-routing it to my echo dot as in your other video.

[@drumslapper]

Actually it is intermittent. Log file says UDP does not stop.

[@kevinw925]

Fantastic work! Just subscribed.

[@dilnuwanchathuranga1007]

Good job. Keep up the good work

[@SmartHomeCircle]

Thanks a lot 🙂

[@Attila01236]

I really like your work. I was wondering, can I give commands in Hungarian too? Because if I can, it would solve my Home Assistant voice control problem with Hungarian commands. Thanks for your reply.

[@SmartHomeCircle]

It might be possible.There are some Home Assistant community forum articles where people are using other languages other than English

[@hi-ccowboy7983]

Woah, I didn't realize this could be done with the Arduino framework. So this can also be used as a media player? For example, I could send a text-to-speech message to it?

[@SmartHomeCircle]

Yes you can. It had tested the setup using text to speech. And it works.

[@hi-ccowboy7983]

@@SmartHomeCircle thanks! That’s definitely worth a Super Like! (or whatever it’s called. 😀)

[@SmartHomeCircle]

@@hi-ccowboy7983 Wow. Thanks a lot for this. That is such a huge appreciation. This keeps me going. 😍

[@dkickelbick]

Very cool

[@peterszarka3497]

Thanks for your works. Great tutorialt. Short question: can I use ESP32 S3 N8R2 for this setup?

[@SmartHomeCircle]

I haven't tried it. But checkout my latest video wherein I used N8R2 with on device wake word detection. kzbin.info/www/bejne/qIXComOdbcmno7s

[@whatwhat-777]

Well I need the .stl file for 3d printing but it is not available in the blog. Hopefully you can add it.

[@SmartHomeCircle]

I am waiting for it to be published. I'll update you here again.

[@SmartHomeCircle]

I published the 3d model and the article is updated with a link to it.

[@whatwhat-777]

oh cool...thanks heaps for your help and reply. Much appreciated.@@SmartHomeCircle

[@mejnour1]

gracias!!!

[@JohnMacGuy]

Is your system totally local? No subscriptions (nabu casa?) The reason I ask is that lots of people are having issues with a requirement of HTTPS to use the mic with Home Assistant ESPHome browser UI. This is a huge problem since the local URL for HA is HTTP. You seem to make it work without this requirement. My particular setup uses the M5 Atom, which is an integrated ESP32, mic, speaker and button. Never been able to make it work due to the HTTPS requirement of the browser UI to use a mic.

[@SmartHomeCircle]

My setup is completely local running on docker images. The HTTPS requirement is of the browser to use the microphone. In case you are using atom echo, this is not required. You can use the assist feature without HTTPS if you are using an external device and not using the browser and you dont even have to open the assist window. So you cannot use the microphone option on the assist window on the browser without HTTPS. However if you have the mobile home assistant app, you can use the assist window microphone option. If you want to know how to work with Atom Echo, check my video here : kzbin.info/www/bejne/hpDJYqqaaNurmKM

[@JohnMacGuy]

Thanks so much for your prompt reply. I will checkout your Atom Echo video and hopefully get a setup that works with the M5 Atom without use of HTTPS!@@SmartHomeCircle

[@JohnMacGuy]

@@SmartHomeCircle Wow it all worked with your video!!!! I have asked everywhere and you are the only one that provided a real reply that worked. Thanks so much. I am going to get the word out on your channel!

[@JohnMacGuy]

Just a note....there was a recent update to ESPHome on HA. It now needs this to get the right OTA version 2. Get the update, go to ESPHome and UPDATE the M5. Works

[@manfredzimmer1199]

👍👍👍 good Job 👍👍👍

[@cvasantrao]

How have you connected both the speakers.? The MAX98375A module has only one output? A great video again. I plan to make a wooden enclosure. Will share it when done.

[@SmartHomeCircle]

I had connected both speakers to the same output. You can connect one speaker if you want to. Share it when you make one. 😃

[@cvasantrao]

Thanks. Will try it when the components arrive.👍

[@pietgrobler1014]

Thanks for the tutorial😊... Does this project use local wake-word, processing on the ESP32 rather than on Home Assistant? Sorry that I'm asking but I'm new to all of this, and I'm looking for a local esp32 controlled wake-word project.

[@SmartHomeCircle]

No. This one is processing in Home Assistant. But I am working on the local wake word project on an ESP32 .. video will be out soon.

[@DanTheMan3]

Can you please slice the STL to be 3 separate prints? Not one large one?

[@Exparda1]

Hi, is it possible to have multiple microphones streaming to wake words?

[@SmartHomeCircle]

With the new micro wake word, yes you can. Check my other video on using micro wake word..

[@MarkAlmeida-Cardy]

Firstly, thank you for the project. My son and I are building our first local assistant. We have an issue with static come through the speakers when wake word is enabled. Is anyone else had this issue?

[@SmartHomeCircle]

This static sound is one of the issues that is still a problem. I am not able to figure it out yet. The only way to get around it is to give a voice command and wait for the output from the speaker.. It then stops.

[@giantbrain5793]

well done.thanks 🙏

[@SmartHomeCircle]

Thank you 🙂

[@liljim001]

good project! can we use the speaker for multiroom?

[@SmartHomeCircle]

If you specify a command with a room, it would work.

[@shannonbreaux8442]

can you change the voice for the response?

[@SmartHomeCircle]

Of course you can.. in the assist pipeline, change the voice for the text to speech .

[@hi-ccowboy7983]

Thanks!

[@SeverFerdi]

Hello, on_stt_end green light Doesn't work. If you change effect: "none" with "Scan Effect With Custom Values" working only 2 LED's are green after success stt end

[@SmartHomeCircle]

this usually happens when the tts is fast or is cached. You might see the green color. if you want the green led to glow up for a longer time, you can add a delay of 1 second for the green led to glow up.

[@b4iwakegaming]

Okay so I wired everything up but the microphone and the speakers do not seem to work. I tried using the wake word and even changing it, but there is no detection happening, the speakers do not play any sounds either. I used hot glue to secure all of the jumper cables into the components. Maybe I destroyed the microphone or amplifier? I'm not sure how to fix any of this.

[@b4iwakegaming]

everything else seems to work, the lights can be turned on and off, and the board is recognized and shows all the entities.

[@SmartHomeCircle]

Can you play a media file using the amplifier? If so then, then you are pretty close. Try to see if the connections are correct for the microphone and also check if you have the set the preferred assist pipeline in the ESP32 device page.

[@b4iwakegaming]

@@SmartHomeCircle I will have to try tomorrow as the cables I was using were not really appropriate for the hookups. I had to do a lot more splice than I would have liked and I did not have female to female dupont cables which would have made my work a lot easier. I will wait for the new cables and then I will try to solder the pins to each component and then use the female cables to connect everything, then I will try again. Thank you for all your help. The code installed without any issue to the esp32.

[@b4iwakegaming]

@@SmartHomeCircle I don't think it worked, but I cannot test it out on any other of my devices since the only other media players I have are actual alexas and they do not support TTS without the notify call service

[@SmartHomeCircle]

@@b4iwakegaming just want to know what you tried. Did you use the tts service in the developer tools and selected the target as the media player from esp32?

[@seyitbural8184]

Tolles Projekt läuft gut nur das Problem mit den lauten Geräuschen ist nicht toll ich habe das ganze in eine alte Bluetoothbox gepackt

[@SmartHomeCircle]

Do you mean the cracking noise when the device starts? Ya. I am facing the same thing. There is no fix to it yet. But I am working on a solution.

[@NoName_silent]

Sir good work ❤, Sir have my task I wand read firmware my Huawei wifi please help me how to read firmware without open ic

[@SmartHomeCircle]

Sorry.. but I dont know that. 😄

[@martinzipfel7843]

Awesome Video, just tried to recreate it but with slightly different hardware and without the speakers. I have everything installed and the LED's are working but it looks like I have trouble getting the mic to work. The ESP home logs just constantly repeat this: [12:29:24][D][light:036]: 'Voice_Control' Setting: [12:29:24][D][light:085]: Transition length: 0.5s [12:29:24][D][voice_assistant:200]: Requesting start... [12:29:24][D][voice_assistant:412]: State changed from START_PIPELINE to STARTING_PIPELINE [12:29:24][D][voice_assistant:433]: Client started, streaming microphone [12:29:24][D][voice_assistant:412]: State changed from STARTING_PIPELINE to STREAMING_MICROPHONE [12:29:24][D][voice_assistant:418]: Desired state set to STREAMING_MICROPHONE [12:29:24][D][voice_assistant:519]: Event Type: 1 [12:29:24][D][voice_assistant:522]: Assist Pipeline running [12:29:24][D][voice_assistant:519]: Event Type: 9 [12:29:29][D][voice_assistant:519]: Event Type: 0 [12:29:29][D][voice_assistant:519]: Event Type: 2 [12:29:29][D][voice_assistant:609]: Assist Pipeline ended

[@SmartHomeCircle]

This log is perfectly fine. Since the ESP32 does not support VAD(voice activity detector), the pipeline is invoked every 5 secs to try and detect the voice. Now there is one way to see what is recorded from the microphone for every pipeline run. You can set :- assist_pipeline: debug_recording_dir: /config/shared/assist_pipeline This above yaml section in your configuration.yaml file and restart. All the recordings will be recorded at the location. But please remove it after you debug the issue or else it will fill up the disk space in Home Assistant and crash it. So be very careful with this option.

[@martinzipfel7843]

@@SmartHomeCircle Thank you for your answer. I thought something was wrong with it because I didn't see your esp doing that and nothing happened when I say the wake word. Ok, at least I know the microphone should be working, I know the LED's work as well because I can turn them on and off from HA so need to figure out why nothing happens on wake word. I use an ESP32-C3 Supermini board, could that be the issue? I ordered some dev boards but they are still on the way.

[@SmartHomeCircle]

@@martinzipfel7843 it could be. I did see someone complaining about c3 not working as a voice assistant. I think it was a comment on another video. I had ordered a c3 super mini and was about to test it for a voice assistant because of its size. but I am currently in my home town.

[@martinzipfel7843]

@@SmartHomeCircle I'll send you an update if I get it to work with the dev board, once they arrive (ordered on Aliexpress)

[@malballuk]

I do have an issue now, as its been mentioned somewhere else here, when you plug in you get the nasty sound coming out of the speaker. I now have that all the time, funny thing is, if I put my finger on one of the esp32 pins it stops, could there be some sort of grounding issue somewhere?

[@SmartHomeCircle]

I was still doing some research somewhere. It could be the pins are not set in the right state and cause this problem. I think it could that the amplifier pin is not set in at right state and causes this cracking issue. I am still working on a solution. Once I find one, I will be making a video for it.

[@malballuk]

Great work, got mine done today, had to wait for the container to be printed as I dont have a 3d printer. One issue I have, when a command is done I get the done message but the lights done turn green. However, if I click on the toggle in the interface they turn green. Any ideas?

[@SmartHomeCircle]

This is correct.. the reason is the time is so short between text to speech, it doesn't show up. This is because after the first media file generation, the media file is cached. If you try to clear you media cache, and give the command again it will take a little time to generate the mp3 file and then you'll be able to see the green color. If you really want that green color to show up every time, you can put a delay statement of 1-2 secs, after the green light turns on. Also I am really glad I could inspire so many to create their own voice assistant 😄

[@malballuk]

@@SmartHomeCircle As I am not a yaml programmer I wouldnt know how to do that but thanks for the reply

[@malballuk]

@@SmartHomeCircle would this be right? on_stt_end: - light.turn_on: id: led_light effect: "None" red: 0% green: 100% blue: 0% - delay: 2000ms

[@SmartHomeCircle]

Yes.. that also works. Or If you want you can also add it here: on_end: - delay: 2s - light.turn_off: id: led_light

[@manitdangal]

please help me !!! can i use lm386 audio amplifier

[@SmartHomeCircle]

I am not sure about that. Sorry :(

[@vinhphamquang8209]

For me I tried esp32s3 dev kit N16R8 and a Esp32 D1 mini, and they are very unstable. It crashed a lot, sometime it is because of brownout, sometime is because of the media_player component, which took too long to process and not reset the watchdog timer. What about yours?

[@SmartHomeCircle]

For me I am using the normal ESP32 dev board. I started having problems with my batch of USB C based ESP32 related to wifi.. So I switched to the micro USB ESP32 dev board. Its pretty stable. Only in like 1 out 7 times the response from the command does not get played on the speaker.. Otherwise pretty stable..

[@manfredzimmer1199]

I build this device.working good, but some times the Device dont „speak“ to me.

[@SmartHomeCircle]

Do you mean sometimes it does not speak out on the speaker to the commands you have given?

[@manfredzimmer1199]

@@SmartHomeCircle yes.

[@SmartHomeCircle]

And this happens sometimes I guess. Its a problem that exists. Sometimes the speaker buffer get too full to play the output. I also have the same problem in my current smart speaker. Check my latest video related to on-device Wake word with ESP32 S3. As of now its working better. I will be creating a new device with this new setup.

[@PheggasSVKVideos]

Hey, is there higher wattage amplifier with I2S? I'd like to use more powerful speaker but i didn't find I2S amplifier with higher supported wattage.

[@SmartHomeCircle]

As far as I know its not available.

[@PheggasSVKVideos]

​@@SmartHomeCircleAlright, np, as it is just AMP with DAC, i split those two to individual components so i have DAC and AMP separated. With setup like that i'm able to power stronger speaker with those I2S components.

[@SmartHomeCircle]

Ohh.. that's what you would have to do. There are DACs for I2S which I think would work. But ya separating them is the way to go.

[@c.cipryan]

How many hours takes to print the 3d parts?

[@SmartHomeCircle]

It depends on the printer and the print quality. I used the creality ender 3 and it took about 6 hrs for all..

[@techworld24300]

Can i use Any led strip instead of LED Strip WS2812B

[@techworld24300]

please replay me

[@SmartHomeCircle]

If you use other types of led strips it would not work. It needs to be an individually addressable led strip.

[@techworld24300]

@@SmartHomeCircle it is expensive in India

[@techworld24300]

Is there any option for using led strip

[@SmartHomeCircle]

I just checked on amazon, it's available for 450rs for 1 meter.

[@Thebyeman]

So my instance is all connected correctly and works fine. Only issue is when first turned on I get a buzzing from the speakers that only stops after my first voice command and I get a voice response. Any ideas why this is?

[@SmartHomeCircle]

Ohh yes.. its one of the problems that still exists. I mentioned it in the video and haven't got any work around for it. The first command solves the issue. And I am currently using it that way. 😄. If I get a solution, I'll make a video about it.

[@Thebyeman]

Thank you for taking the time to respond. Love what you’ve doing for the Home Assistant community. I’ve built mine into a Dr Who TARDIS toy.

[@SmartHomeCircle]

@@Thebyeman ohh lovely. If you are on twitter, please share a photo. There would be many who would love to see what you created. 😃

[@Thebyeman]

@@SmartHomeCircle I’ll make a little video and upload it somewhere

[@SmartHomeCircle]

@@Thebyeman Niceeeee...

[@amritsinghmoyal]

how can i get key api?

[@SmartHomeCircle]

It is present in the yaml .. i am assuming you are talking about the api encryption key.

[@meowhub3002]

Can i use v1

[@SmartHomeCircle]

I didn't understand the question..

[@billsoper1028]

2024.5 ESPhome update seems to have broken this. Works now only once after reboot and then stops responding. Anyone have an idea why?

[@SmartHomeCircle]

Can you tell me what is broken?

[@SmartHomeCircle]

Ahh, I see it stops working after the first command. Ill try to find a fix for this. I am currently using the on-device wake word detection which I felt is much more reliable. Here is the link : kzbin.info/www/bejne/qIXComOdbcmno7s

[@kampa674]

Thanks for you work man. Unfortunately I am experiencing the same issue; it works only once and then stops. I am forced to reload the integration. Than there is that terrible sound.. I'm interessed also to your project with ESP32-S3.

[@tim-x5345]

Having the same issue as well, stops after the first command. Tried several things with no luck, turned on debugging, did not see any issues. I have an S3 and can make your new design but this device was working perfectly with my custom wake word, no false positives. If a new release of ESPHome does not come out soon, I will move to on-device S3. I see M5 Atom Echo users are having issues too. Hope you can figure it out.

[@SmartHomeCircle]

There is some good news, there is already a fix on the way for this. Will update once its released.

[@JarvisUdeh]

Can this configuration also play notifications or other local media files or just tts?

[@SmartHomeCircle]

Yes.. you get a media player entity in home assistant which can be used to play local media files.

[@francisudeh1393]

​@@SmartHomeCircleI saw the entity too, but the speaker won't play any media or tts. Any ideas why?

[@SmartHomeCircle]

@@francisudeh1393 try playing a media from the devloper > services and then choose play media service. Also the media should be accessible over your wifi network to be played. Thats a pre requisite to play any media on a media player.

[@francisudeh1393]

@@SmartHomeCircle thanks. I'll try it out in the morning

[@francisudeh1393]

@@SmartHomeCircle this worked, thanks.

[@guylast9516]

came across an issue with the latest update of Esphome. It does not like "mode" here speaker: platform: i2s_audio id: va_speaker i2s_audio_id: i2s_speaker dac_type: external i2s_dout_pin: GPIO8 # DIN Pin of the MAX98357A Audio Amplifier mode: mono update: swap "mode: mono" with "i2s_mode: primary"

[@SmartHomeCircle]

the mode is replaced with channel. I updated my article to reflect this.

[@cyber-dioxide]

This is very but i dont think it is capable of responding to questions or playing music like a smart speaker besides of home automation

[@SmartHomeCircle]

Yes.. its still in its early stages. These features might soon be available in the future.

[@b4iwakegaming]

Do I need a breadboard to do any of this?

[@SmartHomeCircle]

If you have one, it would be helpful. But if you have jumper wires then you can connect them directly.

[@b4iwakegaming]

@@SmartHomeCircle okay thank you! Great work with this by the way, the unit looks really nice!

[@SmartHomeCircle]

@@b4iwakegaming thanks a lot for such kind words 😄. I love making things and want to share it with everyone so they can also make them. If you are interested you can make one by yourself. Everything is present in the article in the description.

[@SmartHomeCircle]

Also working on a new V2 version of this one. So I will make a video about it also.

[@b4iwakegaming]

@@SmartHomeCircle I placed an order as soon as I saw your reply. The components wont be here for at least another week, maybe 2, but I plan on building this for myself (: Thank you again. I'm going to check out v2 now.

[@prantikmedhiiiii]

Code??

[@SmartHomeCircle]

Check the description.