Broadcasting Your Voice with ESP32-S3 & INMP441

[@ThatProject]

Check More Projects - youtube.com/@ThatProject Github Repository - github.com/0015/ Join FB Group - facebook.com/groups/138965931539175

[@WillianMai]

nice but your capture microphone seemed croppy, your broadcast inmp441 mems microphone seemed good as it was being heard in the background with an 500ms delay, correct me if I am wrong. By the way, why didn't you record that one? the output seemed to better represent the quality of the microphone, or you could capture the raw data straightaway and output to the audio channel on the video editor that you use. it would be a better representation of the audio as it doesn't go through a second media (the speakers) to then get captured by a third media (the capturing microphone) I particularly think the galaxy s20+ mems microphone has a astonishingly good audio quality. I'm gonna try it and share the results as well as soon as I do it. (if someone wants to try as well shall google for "s20+ mems microphone aliexpress").

[@ThatProject]

@@WillianMai I just wanted to show it as it is. Trying the galaxy s20+ mems microphone is a good idea. I will try it too when I get the chance.

[@julietonduso7974]

Can you please help me with the code.

[@testdev1804]

Can you please point out the repo for webrtc based esp32 microphone

[@ThatProject]

@@testdev1804 I don't have a project for WebRTC using ESP32 microphone.

[@obria1]

i love your videos, keep up the good work man. Thank you for getting my passion back for such projects with micro contolers

[@testdev1804]

waiting for your webrtc variant of esp32 and this microphone real-time audio streaming with good quality

[@snopz]

it worked for me fine but there are two problems 1. the volume is very low i have to stick the mic to my mouth so i can hear it normally. 2. the stream stops after like 30 seconds and for it to work again i have to reset the esp32 and on the browser click stop playing and start playing. hopefully you can solve these two problems and it will be the ultimate way to test the microphone

[@rbbr08]

Hello. Have you managed to solve this? I'm having the same problem. The microphone output is very weak and has some weird intensity waves. I don't know if this is normal or not.

[@snopz]

@@rbbr08 i didnt tried to do anything because i used this project just to check if my microphone is working.

[@noweare1]

Is it possible for the esp32 be an audio server. That way clients can connect directly to the esp32. Maybe that allows only one client because of resources but that is ok.

[@dkAyhan]

Great project, thanks for sharing with us.

[@medienmond]

At which Bitrate and samplerate is a 2 way communication to/from an esp32 s2 possible If used for nothing else than audio processing?

[@ThatProject]

Bitrate and sample rate are not easy to calculate. At least I don't think there will be any problem receiving data through i2s. It seems that the bitrate/sample rate can be adjusted depending on how the data in the i2s buffer is handled. If you can empty the buffer as quickly as possible, you may be able to have a higher bitrate/sample rate.

[@paarthsstudio]

Amazing video sir!!

[@SA-oj3bo]

Hi Eric, glad you continue I2S streaming projects. Could you make a demo that shows how to ask questions to ChatGPT with the ESP32 and I2S mic and show the answer on OLED or even better read the answer out loud by ESP32 I2S speaker? Thx.

[@ThatProject]

Oh, that's an interesting idea.

[@aliyunoura]

@@ThatProject Yes sir, it will be very nice

[@snopz]

thats what im working on

[@snopz]

@@ThatProject please make a video on it please

[@haydenkress]

Has anyone found a way to do this? Would love @ThatProject to try this

[@noweare1]

@3:33 Number of samples in one second, you have 86. Your sample rate is 44.1 KHz so you sample rate should be 44100 samples/second. EDIT: That 86 must be 86K bytes as that would make sense. It looks like this is in the server code outputting to console but could not find it. I like how you are using the web technologies with the esp32. It is not easy to pull all that together as a maker. These are fantastic projects. Thank you.

[@Dewal4]

Ok.I have sound on my mac browser, but no luck on iphone, safarafi chrome and firefox tested on ios. Also graph doesn't show up anything, only sound

[@ThatProject]

That's weird. Have you checked the console of your Mac's web browser for any errors?

[@Dewal4]

@@ThatProject 192.168.0.112:8000/js/worker.js 404 (Not Found) I get that

[@Dewal4]

​@@ThatProject I had missing this file. I admit it. Now it's on the place, but still error

[@Dewal4]

@@ThatProject OK, I found what was an issue. I run node from terminal by: node /user/etc/etc. Now I did cd to that direction and then node server.js

[@Dewal4]

@@ThatProject Still no sound on mobile browser thou

[@jackflash6377]

Perfect timing. I'm doing a project to use the INMP441 as a vibration sensor. I wanted to send the signal over WiFi. Now I have a basis to start on. Thanks!

[@kibandesteven5117]

hi there, did you get it working?

[@jackflash6377]

@@kibandesteven5117 Yes, it works perfectly. I made an application in VB that brings in the signal and displays it compared to 5 other sensors. I put that on a windows tablet. The sensors are battery powered and very small. I used an aluminum piece to mount the INMP441 and suspended it with a rubber diaphragm so that the aluminum is pressed against the surface to get good sound transmission. I can see which sensor is near the source of the sound by the SPL displayed in bar graph, one for each sensor, on the tablet.

[@sermadreda399]

Great video thank you for sharing ,useful for predictive maintenance

[@TradieTrev]

Interesting concept Eric! I understand this video was mostly about learning how DMA buffers work, always thought you needed a dedicated audio codec chip?! ESP-ADF would totally be worth looking into should you want to improve audio quality :)

[@ThatProject]

I can't find many references to ESP-ADF. Anyway, I'll try to do it.

[@szn00]

This video is really helpful!! I am trying to broadcast the audio through a Bluetooth speaker...is it possible to take the mic input and send it through a Bluetooth module to the BT speaker from the esp??

[@ThatProject]

@@szn00 I recommend you check out the Bluetooth examples provided by ESP-IDF. I'm unsure, but there's probably a way to do it.

[@wjn777]

Thanks for sharing, very interesting project

[@gauravtalekar7332]

Can we store the Data in the csv file or text file, and then do the audio processing further.

[@ThatProject]

@@gauravtalekar7332 Sorry I don’t get it. What data do you want to save as CSV or text?

[@gauravtalekar7332]

@ suppose i record a audio using the microphone . And if i use putty to log the detials.

[@gauravtalekar7332]

@@ThatProject I am new to this, can you help me . i wanted to achieve beamforming using microphones. Can you help me how can i achieve this. I have a buzzer as a source and a 4 INMP441 microphones . How can I use this setup to achieve beamforming that is detect the DOA of source .

[@ThatProject]

@ I don't know much about beamforming. I'll try it when I get a chance.

[@gauravtalekar7332]

@ oh okay, can you say how can we save .wav files from the microphones.

[@NITHIN-tu7qo]

hi great work!! i have couple of questions please answer them.....my sincere request 1.what is the use of worker.js file? 2. In which part of the code in audi0_client.html is receiving the audio chunks? 3.Instead of immp441 as audio input i want to give it by microphone and then to client page is there any way?

[@ThatProject]

1. You can think that worker.js is a background task. We need to convert data from ESP32 Mic to Int16Array to play. After converting, it returns to the main task. 2. Where the receiving the audio chunk is ws.on("message", (data) => in server.js 3. It depends. Maybe you need to get the PCM data or other data type from your MIC.

[@benjamincounter418]

Will this work with the esp32s3 non devboard, the esp32-s3with the coupled camera attachment.

[@mohammedbenhadine]

Great project ! , my question is related to the audio format broadcasted , the mic outputs PDM format , I dont see any format conversion on your code , how is't converted to PCM ?

[@ThatProject]

If you take a look at my code again, I'm using PCM player for a client Side. github.com/0015/ThatProject/blob/master/ESP32_MICROPHONE/Broadcasting_Your_Voice/wsAudioServer/audio_client.html player = new PCMPlayer({ inputCodec: 'Int16', channels: 1, //sampleRate: 16000, sampleRate: 44100, }); This makes it easy to output PCM data to speakers.

[@mohammedbenhadine]

@@ThatProject thanks for the quick response. Yeah I've seen that client inside js code , but from what i know PDM is different type of audio format than PCM , so I have two hypothesis, maybe the i2s peripheral is doing such conversion from PDM to PCM or maybe the PCM player is doing it.

[@mohammedbenhadine]

After some research it turns out that the I²S protocol outlines one specific type of PCM digital audio communication. And INMP441 is not PDM mic and has no PDM mode. It writes PCM format output signal right away.

[@ThatProject]

@@mohammedbenhadine That's right. That's why I can use it as it is.

[@arbo3688]

hello , great job , i want to ask you how to trasmetted both of video and audio in to local network in one code !

[@ThatProject]

That's super challenging. I've already tried a couple of "possible" ways but all failed. Haha. If I have a good solution for this, I'll share it with you.

[@tis4real]

@@ThatProjectI made a sketch to achieve this.

[@tis4real]

@@ThatProject I tried to share Github project link here but message get deleted.

[@noweare1]

trying to do this also using the esp32 for sending camera frames to clients ( esp32 is a the server)

[@isuruvithanage5850]

Can you please give us a tutorial to get the INMP441 audio sound from a speaker realtime using MAX98357 using ESP32

[@Dewal4]

How to expand this project to send data to not only local server?

[@李偉佛]

Sir, please tell me how to send real-time audio messages from mobile phones to ESP32 and make two-way voice calls with ESP32?

[@behzadazizian9405]

Hi. Is it possible tu run server in ESP32 instead of Windows or Linux?

[@ThatProject]

I've never heard of ESP32 working as a server. Why don't you go with SBC like RPI series?

[@noweare1]

@@ThatProject esp32cam uses the esp as a server. The video is sent to a client browser.

[@ThatProject]

@@noweare1 Yes, you can run a WebServer/Websocket Server etc on ESP32. But I thought the question here was about the possibility of running a Node.JS server on ESP32.

[@christian7229]

Hi, first of all thank you so much for such a great video, I'm an amateur speaking about I2S, so please if you could answer to me this question, Can I use the ESP32 for sampling my voice like in this case, but at the same time playing it with the same ESP32 using a DAC?, I already know ESP32 S3 doesn't have an internal DAC unlike the normal ESP32, so I'm planning using this model or maybe Can I use the PCM5102 which is a DAC decoder so that I can use the ESP32-S3 again? Thanks a lot in advance

[@ThatProject]

I've never connected a separate DAC to the S3, so I'm not sure. Check this out. github.com/volodya7292/portcard-esp

[@buimanh4971]

hi, a hava a question, i use ADC1 to read analog, it ok, but when i turn on wifi connect, my adc get noise, Can you give me some advices, thanks a lot

[@ThatProject]

This seems to require more low-level signal debugging. I recommend asking questions on the ESP32 forum.

[@bathinasumanth6708]

Hello ! That Project we had setup this project as it is . we had getting the samples and some what of noise but the voice is not getting in this. can you give some more explaination on my problem

[@ThatProject]

Same issue as others below. As I mentioned to him, Being able to hear noise clearly means that information is being received from i2s, but I'm still not sure what the exact issue is. How about checking your i2s settings again? Or, check your voice response with serial plotting before sending the data to the server.

[@bathinasumanth6708]

@@ThatProject we are also getting the wavefroms in the server also. the exact issue is it recogniging the voice and giving the samples in the server , but we are not getting the voice in the server . is there anything to add to get the voice

[@ThatProject]

@@bathinasumanth6708 Is the waveform of voice data transmitted from the server drawn? If you put a loud noise into the mic, can you see the waveform's pitch change noticeably?

[@rodneytapit5636]

Thank you for a really informative video. Can you offer guidance on how to use ESP-NOW to transmit voice quality to a second ESP32?

[@ThatProject]

Oh, that's an interesting idea. So far, in my project, I have only transmitted simple data through ESP-Now. If there is a good opportunity, I will try it. Thank you.

[@rodneytapit5636]

@@ThatProject It would have some practical usage(s).

[@MariánJandačka]

HI, maybe stupid question, because i am amateur. Could it be possible to create two ways flow between ESP32 and web, where both side can send voice and recieve voice...basically like a call with WebRTC between two web apps.

[@ThatProject]

As far as I know, it is still difficult to operate WebRTC in an MCU environment. It can send and receive voice data in half-duplex like a walkie-talkie. kzbin.info/www/bejne/oaiWYamCmdd6e5o

[@TecnologiaExtraterrestre]

wifi connected, but can't connect to the server. can you help me. what goes here: "const char* websocket_server_host =" ?

[@ThatProject]

This will be the IP address of the host running on your PC or server.

[@KyrstianK]

Best of the best 🥰

[@quangdungngo2836]

Hi sir, I can send file record audio to S3 aws iot core? Thankyou

[@ThatProject]

Amazon IoT Core is based on MQTT communication. The maximum payload you can have here is 128KB. It seems difficult to send voice data as a file with this.

[@quangdungngo2836]

@@ThatProject thankyou repply!!! I'm project send audio esp32 to s3 aws ,you have solution? Thankyou

[@kklol07]

ive been looking for a project like this but there is very less documentation online, how did you manage to make this? pls tell me the workflow. its amazing

[@ThatProject]

@@kklol07 Well, it's hard to answer. I'm the same as you. I googled the parts I didn't know and learned them one by one. Anyway, thank you.

[@MI-bm2yy]

Can you tell us the settings you did in the arduino tool menu so we can upload it accordingly, because i am confused with a bit of settings and configurations and can't risk the devkit as it takes days to reach to us. your help will be appreciated. thank you

[@ThatProject]

If you set your ESP32-S3 in your Arduino settings properly, I believe there aren't any issues. There is no risk if you select the wrong settings for this. Just try it.

[@MI-bm2yy]

Thanks for the help just one more question What should I use FATFS or LittleFS in partition scheme because if i use fatfs and i try to include spiffs it doesn't support and then i might have to use sd card or external storage process and if i use littlefs then it says spiffs partition is too large for spiffs_page_ix type. Please increase CONFIG_SPIFFS_PAGE_SIZE.@@ThatProject

[@abdelnacer7721]

This is awesome!, is there any way to connect it with apps like zoom ,meeting ect...thanks

[@ThatProject]

Unless we use a PC microphone driver, it may be difficult to use in PC applications such as Zoom. I'll take a look into it.

[@abdelnacer7721]

@@ThatProject Thank you, i appreciate it 🙏

[@EICS10]

We have done the code and html in visual studio code and the server works, but if we open the website and click connect. It doesn't work. The waves are not forming. can you help to resolve this issue?

[@ThatProject]

In this project, I'm using plotly to draw the graph. Are you sure that your internet connection is alive to use CDN?

[@josemiguelsucasasmejuto6246]

Great proyect!. I can hear the microphone from chrome on a PC, but is it possible to hear it from an iphone?

[@ThatProject]

Sure. You can access the same from Safari or any other web browser on your iPhone.

[@josemiguelsucasasmejuto6246]

In my case it cannot work. I have tried it on three different iphones. It seems that pcmplayer does not play any sound on the iphone (safari, chrome, opera navigators)

[@Steve-xb7dn]

Does this server allow for multiple microphone connections to stream to the main server

[@ThatProject]

This server only supports single input and multiple clients.

[@simon1904]

is it also possible to broadcast your voice from your latop or phone to the esp32 / max98357a?

[@ThatProject]

I think it's possible. After transferring the PCM data obtained from the PC's microphone to the ESP32, writing i2s to MAX98357A seems to play sound.

[@simon1904]

Can you maybe make a video about it?

[@hystorical5009]

if we are doing with normal esp32 will it work if we follow your schematics and code

[@ThatProject]

It is officially recommended to use the ESP32-S3. github.com/espressif/esp-skainet

[@hystorical5009]

​@@ThatProjectbut I only have esp32

[@arbo3688]

I get this error when I went to stream in the navigator : "Cannot GET /client". Please can you help me ? Thank you for your response !

[@ThatProject]

There is no path /client. It's /audio. e.g.) 192.168.86.42:8000/audio

[@Lisas4us]

I would like to implement this project. Unfortunately I am not understanding everything. 1. What server are you talking about, I believe I see Linux code, where do I put this code on my Windows Notebook? 2.

[@ThatProject]

1. I ran a local server on my Mac. This operates based on Node.js. 2. No. You can also create and run a local server based on Node.js on Windows. You don't need RPI or Linux for this project. 3. Please google “create node.js server on windows”.

[@hystorical5009]

Can we do this in dba

[@ThatProject]

What do you mean, dba?

[@hystorical5009]

@@ThatProject sorry decibel{dB}

[@hystorical5009]

pls tell fast thx

[@ThatProject]

@@hystorical5009 There are already projects that can get decibels. Check this out. kzbin.info/www/bejne/f6K7h3R9bdimoMk

[@NITHIN-tu7qo]

Hello please try to clarify my doubt!!! In the client side the delay in my voice is gradually increasing.....initially there will be no delay but as time progress the delay keeps incresing gradually Could you please analyze and tell me what could be the reason?

[@ThatProject]

Not sure but maybe it's Plotly issue. Have you tried without it?

[@NITHIN-tu7qo]

@@ThatProject yes....i have implemented it without plotly

[@NITHIN-tu7qo]

until now i used your client side code in some other project.... but today i have tried to just replicate your project but the output is white noise completely what could be the reason?

[@ThatProject]

@@NITHIN-tu7qo Sorry, I have no idea what's going on there. If you think this issue comes from my code, please try it again without it.

[@ErisonSilvaa]

If i understood correct it works from esp32 to browser, or is it worker in both sides? Browser to esp32 Esp32 to browser?

[@ThatProject]

It's ESP32 to the Local Server and Web browser. You can see the running node.js server. Users can connect to the local server through Web Browser and check live audio streaming.

[@behzadazizian9405]

Hi. thank you for very useful video. I use ESP-camera board. with sample rate 16000 or 40100 all thing is Ok. but when I change it to 8000 all data in buffer is zero. (I print value of bubber for ratr 16000 and 8000)

[@ThatProject]

I tested only 16k and 44.1K so not sure about the 8k bit sampling rate. But I believe it should be ok with 8K though.

[@ErisonSilvaa]

Could you tell me if to use MSM261S4030H0 mic , is it the same i2s config?

[@ThatProject]

Not sure but it's not that different if it has the same pinout as INMP441.

[@EEECHINJUMONCC]

Sir i have one doubt is any method to save that audio file...

[@ThatProject]

I think you can add code to save the audio data coming from the Node.js server as a file.

[@IWAYANRENTANU]

thak you for updated and simple code, i have seen your code in walky talky, my queestion, why in mic i2s inmp441 is too over if my voice too high, and this mic very sensitive respon, can add very simple gain for code. thak you, sorry my bad english.

[@ThatProject]

Have you tried this one? void i2s_adc_data_scale(uint8_t * d_buff, uint8_t* s_buff, uint32_t len) { uint32_t j = 0; uint32_t dac_value = 0; for (int i = 0; i < len; i += 2) { dac_value = ((((uint16_t) (s_buff[i + 1] & 0xf)

[@making-arte5710]

Is it possible to use a normal ESP32 ? ( No S3 Model ) Is it possible ?

[@ThatProject]

You can do this with your ESP32.

[@muhammadsaadyaseen4988]

can anyone help which code need to be upload in esp 32

[@ThatProject]

Have you tried this project source from GitHub? You can download and modify them and then try it.

[@muhammadsaadyaseen4988]

@@ThatProject thanks it is running but the audio volume is very low can u suggest anything.

[@ahmedsherif7732]

where are the header files in the Arduino code please?

[@ThatProject]

Currently, in order to run ESP-SR, you have to try in ESP-IDF environment.

[@leeji6418]

I love this project, just how to find ip address host for filling const char* websocket_server_host = "";. (windows user)

[@ThatProject]

@@leeji6418 You can get your local IP by running ipconfig in CMD.

[@brianchen1163]

This is awesome! Thanks for the video! I was able to get my project working with i2s, but when I switched to i2s TDM mode with 4 channels and it doesn’t work anymore 😔

[@ThatProject]

According to the official document, it says, I2S on ESP32-S3 supports TDM mode, up to 16 channels are available in TDM mode. But I have no idea how to set up for having separate settings for each channel. How did you set it up?

[@tensiondriven]

This is great, I’m looking for a multi mic streaming system like this - can’t wait to look at your code! Let me know if youd like to collaborate.

[@selimbesimzade6845]

Great content. thnx

[@Willpowerstudios]

very cool !

[@ingenia-tec5194]

Thanks a lot for your video :D Its really Impressive !!! :D Thanks a lot :3 I have a Question 🙏 Is it possible to Record the Audio from the client side ? How could it be possible ? to get as a result a .MP3 file maybe. What would be needed to have that fucnctionality ? Thans for your videos , Right Now Im suscribing :D

[@ThatProject]

www.npmjs.com/package/audiobuffer-to-wav It looks like you can save the audio data as a wav file on the client side using this, but I haven't implemented it yet.

[@ingenia-tec5194]

@@ThatProject Thanks :D I will check it :D I have a last question: In the video you use a ESP32-S3 ... Could it be used a normal ESP32 to integrate this project ? or what changes would be needed to use the code within a normal ESP32 ?? Thanks a lot in advance And Thank you for your video and your answer 🙏

[@ThatProject]

@@ingenia-tec5194 It seems that the same can be used on ESP32.

[@hystorical5009]

can you also upload schematics

[@ThatProject]

Are you looking for this? kzbin.info/www/bejne/p6KVd4WsZdF2htk

[@lavacat720]

Now i know why we use microphone modules for rgb lighting...

[@ThatProject]

😂

[@lezbriddon]

i've heard 8bit 8k and thats just what that sounds like. for a 16bit stream it sounds excessively harsh and metallic. reminds me of a mid 90's cheap bluetooth headset. I think its the hw to blame so 10/10 for getting it working, i'm sure custom hw would be a much bigger pcb with lots of filtersto tweak the audio

[@ThatProject]

Using a professional board with an audio codec will of course give you better audio. ESP32 is a very generic board. Many things are possible, but conversely, many things are lacking. It focuses on improving the understanding of overall system construction.

[@lezbriddon]

@@ThatProject I'm not knocking your achievement I couldn't even get NFC to work properly on the 8266 gave up and used a 328 .....

[@__________________________6910]

Nice

[@dr.aswinkumersv5803]

WiFi Connected, Web Socket connected, Web Socket opened, but the audio is not plotting and not audible at Web Browser

[@ThatProject]

Have you checked the data incoming from ESP32?

[@dr.aswinkumersv5803]

@@ThatProject yes verified, data coming from ESP32

[@ThatProject]

@@dr.aswinkumersv5803 The next checkpoint is to check whether any errors appear in the console through the web browser's inspector.

[@dr.aswinkumersv5803]

@@ThatProject Voice signal and its plot is not coming, audio and PCM plater errors appear in the console t

[@Biomedbites]

@@dr.aswinkumersv5803 im facing the same issue did you figure it out??

[@EICS10]

is kzbin.info/www/bejne/o2mvqIOEjdZ-b9E same as this videos setup?

[@ThatProject]

I don't think so. Please use this way.

[@EICS10]

@@ThatProject u r so cool no youtuber ever replies this much to comments thx a lot

[@HEMANTHNAGAKIRANKANTU]

every thing is perfect but voice was not coming only noice was coming

[@HEMANTHNAGAKIRANKANTU]

how can i resolve the problem??

[@ThatProject]

@@HEMANTHNAGAKIRANKANTU Being able to hear noise clearly means that information is being received from i2s, but I'm still not sure what the exact issue is. How about checking your i2s settings again? Also, check your voice response with serial plotting before sending the data to the server.

[@johnaweiss]

sounds terrible. Noisy and distorted.

[@ThatProject]

If you increase the bits per sampling rate, you can get a higher quality of audio input.

[@arbo3688]

I run the code with esp-eye and i had this error in the serial monitor : esp_core_dump_x+͡⸮ Incorrect size of core dump image: 65537

[@ThatProject]

The memory allocation in your code doesn't seem right for your esp-eye board. Check available SRAM.

[@arbo3688]

@@ThatProject Thanks man I will check

[@arbo3688]

Hello, Please I need your help. When I set with this sample_rate = 8000 and bit_per_sample = 8, I don't hear my voice just noise. Thanks for your help !

[@ThatProject]

Did you change the PCM player's sample rate to 8000?

[@arbo3688]

@@ThatProject I didn't. where in the code I need to do that?

[@ThatProject]

@@arbo3688 Check this one. kzbin.info/www/bejne/p6KVd4WsZdF2htk

[@arbo3688]

@@ThatProject I get it thanks !

[@carberrypd]

Would this project work on an ESP32-PICO?

[@ThatProject]

It seems possible because it does not specifically use ESP32-S3-specific components.

[@AmanSinghal-ny3ik]

can i use a esp32- s2 for this as well ?

[@ThatProject]

This project also works on s2.

[@AmanSinghal-ny3ik]

@@ThatProject Is the delay better or worse in this case ?

[@ThatProject]

@@AmanSinghal-ny3ik There is no difference in this project.