This version is dedicated for MicroPython on ESP32.
@jomfawad92557 ай бұрын
Can you explain how dfrobot voice recognition module does recording and training model on the module itself?
@user-ws5id8et2g8 ай бұрын
Do this module works offline?
@user-ws5id8et2g8 ай бұрын
Do this modul works offline?
@sahinahadli868110 ай бұрын
thanks, but speech_model library is nowhere. where can we download it?
@tinkerdoodlediy6 ай бұрын
you'll need to download and flash micropython firmware. see the notebook for instructions.
@ebrahemkhalifa367510 ай бұрын
how can i run this command "%flash esp32 ~/Shared/Junfeng/speech-commands-firmware.bin" using esp32s3
@SuperSmosh12311 ай бұрын
There's no connect button on the page. Is it a problem with my browser or is it something else? I tried following a tutorial on installing the firmware and on that page there's the connect button but no progress on installing the firmware even after following the instructions.
@tinkerdoodlediy11 ай бұрын
Can you share a screenshot of your page?
@aboudezoa Жыл бұрын
This is definitely something I want to learn more.Can you explain more about the setup and hardware ? Trying to do robot car with rpi4 or jetson nano
@tinkerdoodlediy Жыл бұрын
I tested the code on a MacBook, but it should work on rpi. Hope it helps!
@elafefy Жыл бұрын
hello , I tried again and again from more than one computer and also using a VPN, but nothing works. I press the start audio recording button, but nothing happens. I hope to fix the problem
@tinkerdoodlediy Жыл бұрын
Can you use Chrome browser? If that still does not work, go to menu "More Tools" -> "Developer Tools", and check if there are any error messages in the dev console.
@elafefy Жыл бұрын
@@tinkerdoodlediy yes i use chrome , and the error is speech-commands.html:392 Uncaught (in promise) ReferenceError: tf is not defined at updatePrediction (speech-commands.html:392:23) at addSample (speech-commands.html:542:5)
@elafefy Жыл бұрын
@@tinkerdoodlediy any update please?
@tinkerdoodlediy Жыл бұрын
It works on my browser. Can you check if you can open cdn.jsdelivr.net/npm/@tensorflow/[email protected] in your browser?
@elafefy Жыл бұрын
@@tinkerdoodlediyThis service is prohibited in my country But I use VPN so CDN service works for me after using VPN but your tool still not working😢
@marwan.naasan Жыл бұрын
Great work 👏 .. but I wondering why there is so many delay ? I mean when boy get left after 2 seconds game get left ?? ... can we fix it ?
@tinkerdoodlediy Жыл бұрын
Are you talking about delay in this video, or delay with your own M5StickC? The delay should be much less than 1 second. Check to see if your M5StickC is fully charged, and your browser does not run any other heavy web sites at the same time.
@tinkerdoodlediy Жыл бұрын
It is not Bluetooth latency. Delay is mostly from web app.
@jonathanrodriguez8219 Жыл бұрын
Very nice, thanks for sharing~~~ 😃
@hazimalias Жыл бұрын
who came after 2 years? Great work. do any update on board ESP32 S3? support wake up word.
@ians6674 Жыл бұрын
Love the outfit
@aboudezoa Жыл бұрын
No code or tutorial?😢
@tinkerdoodlediy Жыл бұрын
Haven’t got a chance to clean up the code. Will share later:-).
@tinkerdoodlediy Жыл бұрын
Hi the code is shared at tinkerdoodle.cc/user/_/tree/Shared/Tinkerdoodle/Chat%20Box. Hope it helps!
@aboudezoa Жыл бұрын
@@tinkerdoodlediy will try it out , thank you so much
@chorbil Жыл бұрын
You should measure the supply voltage and add it to the spread sheet. This way you can see when to replace the batteries.
@dans-designs Жыл бұрын
this is great! will the firmware you compiled work with any esp32? for example i have an esp32-wroom-32e, will it be compatible?
@tinkerdoodlediy Жыл бұрын
It should work for any esp32.
@dans-designs Жыл бұрын
@@tinkerdoodlediy I keep getting this error, any idea how to fix it? type object 'I2S' has no attribute 'RX'
@tinkerdoodlediy Жыл бұрын
It seems your firmware version is different. Can you try flashing the firmware I published in the same folder as the notebook?
@saranyaasuresh5710 Жыл бұрын
Hi Nice demonstration, am using esp32 s3 korvo2 board which has mic embedded on it, how can I give live audio input to it and display on pc.
@aboudezoa Жыл бұрын
very nice work ! The firmware doesnt work on ESP32C3 , any recommendations ?
@tinkerdoodlediy Жыл бұрын
@@aboudezoa I don't have ESP32C3 so cannot test. You may want to read the MicroPython documentation on how to build a new firmware. Maybe their instructions changed recently.
@jasonhedtke6592 жыл бұрын
Could this in theory be used to make a very basic AI? I want to make a pair of smart glasses and have a "virtual" assistant that can do basic requests like responding to a text, allow me to decline a phone call, music controls, ect You think this could help me achieve that?
@aatifmohd86782 жыл бұрын
Hello, can I use the same example with M5stack-core? It has a built-in mic too! If it is possible, can you make a tutorial video of it?
@tinkerdoodlediy2 жыл бұрын
It should work. You just need to update the pin number for the mic. I don't have a M5Stack core so cannot verify. Let me know if you make it work!
@altitude10392 жыл бұрын
OMG This is GREAT! :) Thanks for posting
@tinkerdoodlediy Жыл бұрын
Glad you enjoyed it!
@JeremyCook2 жыл бұрын
Wow, amazing work.
@TheIronHeadRat2 жыл бұрын
Great 👍
@SA-oj3bo2 жыл бұрын
Did you mean you need to use words that exist in the google speech data set and this is combined with the Delta model? Is very interesting if ESP32 can accurately recognize sounds and speech. Thanks for sharing. I want to learn this too ... :-)
@tinkerdoodlediy2 жыл бұрын
You can use words outside of google speech data set, or even different language. I haven't tried to recognize arbitrary sound. The accuracy probably won't be good, I think.
@user-ws5id8et2g8 ай бұрын
Do this module works offline?
@SA-oj3bo8 ай бұрын
I think it does, will check again and try it myself.@@user-ws5id8et2g
@SA-oj3bo2 жыл бұрын
Amazing, so simple and yet so powerful! Great tutorial and achievement! If I want to detect 1 specific sound, for example my own dog barking, what other sounds should I upload to the model then? How to get the best results to detect 1 sound only? Thank!
@spacecdr2 жыл бұрын
Well done. I tried it on my M5StickC Plus and it works (some changes just for new display) I'm not using MicroPython, but analyzing your code it seems the "main library" is the "speech-commands-firmware.bin" (i followed your link to compile it on python). Do you know how to use this "engine" on ArduinoIDE/VisualStudio? I suppose i can't "include it"... any alternatives solutions? Let me know please! Thank you ;-)
@saj_zamani2 жыл бұрын
Hey, thanks for your great work and this video I just made this with ESP32 devkit and an INMP441 and it works :D I just started learning micro python and got some questions... 1.well there must be a limit in uploading the trained model. I mean how big can it be or like how many samples? Does it save into flash (or SPIFFS)? if so it shouldn't be more than around 1.5MB 2.Got a problem with the I2S microphone. the voice captured is clear but too weak and needs to be amplified(digitally in code). I saw some examples in C++ that shifted the data in the buffer to amplify the voice. but since I'm new to python I don't know how do it right now... so I tried a dedicated way to capture some samples (1sec, 16bit, wav ,16000rate) with this gear in C code to make the model and I thought I can upload them for training but I couldn't. Just figured it out that the Upload Samples button needs to be the former samples in .txt. well my only option now is to stay away from the PC microphone and capture some weak data and test it.(cuz idk how to play with the last cell (that java script thing) you made) :D Thanks again and Tips and ideas are welcome :D
@tinkerdoodlediy2 жыл бұрын
Glad to know the tutorial works for you! For you questions, 1. The trained model is of fixed size, no matter how many audio samples are used in the training. The more audio samples you have, the better model you get. 2. As long as it is clear, the sound volume does not matter in recognition. The speech model takes spectrogram as input, not the raw sound data. If you want to prepare audio samples using your own program and upload to the training page, you can refer to the generated speech_model.py on how the audio samples are captured in MicroPython, and convert that into C/C++.
@aboudezoa Жыл бұрын
How did you wire your INMP441 & initiate I2S in the code ? I'm getting error " TypeError: extra keyword arguments given " at channel type , can you share your code please ?
@alzalame2 жыл бұрын
Perfect , thank you very much .
@alzalame2 жыл бұрын
Nice , good guide
@cleverdickrick2 жыл бұрын
Cool stuff. Seems like a wake-word would be useful. It would be a shame to turn off the lights just because I used the word "dark" in a sentence.
@tinkerdoodlediy2 жыл бұрын
That is doable. You can train the wake word and command words in the same model. Then in the code ignore command words unless wake word has been spoken previously.
@frankvanhooft58492 жыл бұрын
Looks amazing - I'm trying to build one. It appears M5Stack may have changed the LCD. When I run your code, I get junk on the LCD. The code does: lcd = m5stickc_lcd.ST7735() But the M5StickC-Plus units now shipping use a ST7789v2 controller IC. How do we support this display? Thanks.
@tinkerdoodlediy2 жыл бұрын
It seems this might work for you: github.com/russhughes/st7789_mpy. But I don't have a M5StickC-Plus to test. Let me know if you make it work. I'll be more than happy to update the Tinkerdoodle shared notebook to include a section for M5StickC-Plus.
@jayjoshi39022 жыл бұрын
How to usr micropython with new BLYNK 2.0 ??? The old blynk library ask for auth token but the new one doesn't.
@sltechgalaxy16772 жыл бұрын
can i use tensorflow for it plzz reply soon
@tinkerdoodlediy2 жыл бұрын
No the model that runs on M5StickC does not use tensorflow library.
@sltechgalaxy16772 жыл бұрын
@@tinkerdoodlediy ok thanks friend
@sltechgalaxy16772 жыл бұрын
@@tinkerdoodlediy are there have any methods to use tensor flow plz reply
@sltechgalaxy16772 жыл бұрын
can I make it using INMP 441 I2S microphone module ?? plz reply soon
@sltechgalaxy16772 жыл бұрын
can I use ESP 32 for it?? plz reply soon
@tinkerdoodlediy2 жыл бұрын
Of course.
@sltechgalaxy16772 жыл бұрын
@@tinkerdoodlediy thanks
@selahattinbabadag28042 жыл бұрын
Add audio Sample button doesnt work ?
@tinkerdoodlediy2 жыл бұрын
Fixed! Turns out the latest tensorflow.js package was bad. Rolled back to use a previous version.
@poojadubey81722 жыл бұрын
im not able to run "%flash esp32 ~/Shared/Junfeng/speech-commands-firmware.bin",plz help
@tinkerdoodlediy2 жыл бұрын
What is the error you got? Make sure you use a M5StickC. The flash functionality is experimental. If it does not work, use esptool to flash the firmware.
@dans-designs Жыл бұрын
@@tinkerdoodlediy can you provide a download link for the firmware please? I'm having trouble finding how to download the .bin file to flash via esptool, thank you
@tinkerdoodlediy Жыл бұрын
@@dans-designs there is no direct download link. You'll need to log in first, and download at tinkerdoodle.cc/user/[email protected]/tree/Shared/Tinkerdoodle. Find the bin file in the list.
@jacobsbladder69342 жыл бұрын
Thank you very much .. good explanation
@HALISHRICHARD3 жыл бұрын
Sir my account have not available weather station file
@tinkerdoodlediy3 жыл бұрын
The tutorial is at tinkerdoodle.cc/user/_/notebooks/Shared/Tinkerdoodle/Weather%20Station%20on%20Blynk.ipynb.
@yasirali91903 жыл бұрын
can i use it esp32 module with external microphone?
@tinkerdoodlediy3 жыл бұрын
Yes external I2S microphone should work. Refer to tinkerdoodle.cc/user/_/notebooks/Shared/Tinkerdoodle/Speech%20Commands%20Model.ipynb for source code.
@christianmarquez31433 жыл бұрын
Do you have to pay to post your data to your gSheet?
@alzalame3 жыл бұрын
Perfect work, well done .
@Friendroid3 жыл бұрын
Never knew this was possible, very cool!
@MrCutter3 жыл бұрын
Great video, hope to see more :)
@OldManSparkplug3 жыл бұрын
This is excellent. Is this an opensource project? I'd like to learn how to build this sort of tool.
@tinkerdoodlediy3 жыл бұрын
Yes. If you want to learn how to build the custom speech commands model, and how to use it in MicroPython, then just follow this video, and the link in the video description has the code and instructions. This article talks about the implementation details, www.hackster.io/tinkerdoodle/deep-learning-speech-commands-recognition-on-esp32-b85c28. The base model training and building MicroPython firmware is a lot harder. I can give another tutorial if more people are interested.
@OldManSparkplug3 жыл бұрын
@@tinkerdoodlediy very interested.
@nickdaves34672 жыл бұрын
@@tinkerdoodlediy me too
@hasibal-ahmed73852 жыл бұрын
@@tinkerdoodlediy Interested
@SA-oj3bo2 жыл бұрын
@@tinkerdoodlediy Great project, yes if you have links to good tutorials they are vry welcome! Thanks!
@hummmingbear3 жыл бұрын
Awesome work, thanks for sharing this. Is it possible to record samples longer than 1-second for multiple word commands?
@tinkerdoodlediy3 жыл бұрын
Thanks for your interest. The model is trained to take 1-second audio only.
@dans-designs Жыл бұрын
@@tinkerdoodlediy Is it possible to combine multiple models? for example, speak 1 of 5 commands from first model, which then opens 2nd model and waits for next 1 of 5 commands from second model?
@tinkerdoodlediy Жыл бұрын
@@dans-designs it should work if your esp32 has enough memory. Try renaming the second model file as speech_model_2.py, and invoke it as speech_model_2.predict() in the code.