ESP32 DIY polyphonic MIDI synthesizer module for Arduino

[@tdhoward]

This is absolutely brilliant. I'm stunned by how powerful yet simple this project is! I may need to come back to this. Super inspiring.

[@marcellicence]

Thank you very much. The project may be changed a bit over time. There is also a multiplatform version which supports the ESP32 as-well =)

[@romanopekushin3099]

Just super.I was able to connect an external midi controller. By setting it up, I was able to achieve your result. It just shocked me. Magic sound. My homemade controller is glitching a bit, but it's a breeze. The main thing is the sound and capabilities of your synthesizer !!!Add manual control, not midi, and you get a miniature and very attractive synthesizer.

[@marcellicence]

Thanks for your feedback. I am really happy that you enjoy the synthesizer.

[@romanopekushin3099]

@@marcellicence I realized what your synthesizer is missing - the LFO. Filter automation will be just a gift.

[@marcellicence]

@@romanopekushin3099 I fully agree. But I think there is a lot more missing. But the next project is coming soon =)

[@romanopekushin3099]

@@marcellicence Super!!! =)

[@mikolasstrajt3874]

Very versatile sounds. I would definitely buy one of these if it was on reasonable price.

[@marcellicence]

Thank you, sounds great =) I could try to prepare something

[@romanopekushin3099]

Fine! There is one idea. Connect multiplexer to esp32 for control. I have a very simple midi keyboard. But an external midi controller solves this problem. True, its pass-through midi channel gives a small delay. I am very glad that you decided to share the code. I will definitely repeat and write about it.

[@marcellicence]

Yes a multplexer sounds good. I tried to keep the effort low as possible but feel free to add your own controls. I've seen that there are also some DIY midi controller projects

[@rockpopmando1]

@@marcellicence Hi Marcel, I believe, it is great to have a basic project like Yours which works for all. Adding a Multiplexer makes in more komplex for others to start with it. I have tested some analog multiplexers as the HP 6067 which brings other problems with it and would make projects more complex. A better approuch would be to decouple the CC-Controls with the changing of the values. I mostly tried to create a function which changes the values like void changeDecay( uint8_t inDecay ) and then call this from the MIDI-Dunction for a CC and to call it in the case if a "Input" of a physical UI is changed. That makes it more simple to let others adopt Your code. It is only a hint, not a recommendation. :)

[@gilleslacaud]

Hi Marcel thank you for sharing the code.

[@marcellicence]

You are welcome

[@atalibajunior5055]

You are a genius! Thank you so much for this.

[@marcellicence]

Thank you too for the nice feedback!

[@hansmeier1608]

Very good video, you're explaing optimal! Will take my ESP32S3 take out.

[@marcellicence]

Thank you very much for your feedback

[@LoFiWeekly]

This is so awesome my friend - I now have a nice weekend project coming up soon!

[@marcellicence]

Sounds great. Enjoy!

[@realSethMeyers]

This is so exciting! I can't wait to build my own. I've been looking for something like this for a long time now.

[@marcellicence]

Thank you =)

[@cbrpnk]

This is awesome makes me want to experiment with hardware synths.

[@marcellicence]

Thank you. I can highly recommend that. Working with low latency makes much more fun than using software only

[@feldon27]

What an awesome project and great music resulting from it!

[@marcellicence]

Thank you. I am very happy to hear that =)

[@rockpopmando1]

Hi Marcel, wirklich cooler Sound und sehr coole Lösung. Ich war eigentlich auf der Suche nach dem Code für Deinen Sampler aber der Synth ist eigentlich noch interessanter!

[@marcellicence]

Hi, danke für die Rückmeldung. Ja ich habe nicht alles veröffentlicht. Ich werde nach und nach schauen, was ich für eine Veröffentlichung nehmen kann. Zudem steckt ja leider dann doch immer etwas Arbeit dahinter (auch wenn etwas "eigentlich" schon fertig ist) ;-)

[@OMNI_INFINITY]

Thanks! Shopping for a DAC tonight.

[@marcellicence]

Good luck. You are welcome

[@Lubawaandme]

YEAH BABY. YOU FEEL IT

[@marcellicence]

Haha, thank you 😁

[@fenexj]

Awesome project, very inspiring.

[@marcellicence]

Thank you very much

[@robotmr]

Cool Project! Thank You for sharing the code

[@marcellicence]

Thank you =)

[@mamax73]

You are my inspiration. Thanks

[@marcellicence]

Thank you so much! 😄

[@marcellicence]

Unfortunately you second comment was filtered by KZbin. I could only see the notification but the comment is not visible. I could read the start regarding the sound fonts 2.0. I had this also in mind but the implementation is more complex requiring some additional filters etc. I guess. Hope I can extract the samples from .sf2 soon =)

[@sandrocavali9810]

fantastic job

[@a-z-n-b-a-i]

LOVE THIS ONE

[@marcellicence]

Perfect 😁

[@pantalalabs]

fantastic projects!! congrats!

[@marcellicence]

Thank you very much

[@valdeciribeiro7839]

Thank you for sharing your knowledge, you are a good person! Can I send midi messages directly to the module with another microcontroller as hardware?

[@marcellicence]

Thanks, you can attach another controller. You should just be aware of the voltage. The tx signal must not exceed 3.3V. I am not sure but I think the ESP32 is not tolerant to 5V.

[@valdeciribeiro7839]

@@marcellicence Thank you so much, it's hard to find people like you!

[@gorillaau]

@@valdeciribeiro7839 I'd use an optocoupler between the two devices, just in case you do something bad. It might just save the other device, either the ESP32 or your microcontroller.

[@valdeciribeiro7839]

@@gorillaau thank you my friend!

[@RoyalTech_2024]

Hi marcel, What is the parameter you are changing with the last slider? which showing only numbers from 1-8(Unlike other plaremeters like "voice volume attack") in the serial like display on the top corner?

[@marcellicence]

Hi, I've checked my code. It seems that it changes DETUNE_1 or WAVEFORM_1 depends on the current configuration. You are right there should be the definition in z_config.h

[@zo1dberg]

Very nice!

[@renaulth2009]

Good coder/engineer and keyboard player hehe

[@marcellicence]

Thanks a lot =D

[@ainu_channel]

When using esp a1s audio kit, which channel using for audio out? Or it stereo? In case if I want to connect output to audio interface combined IN jack

[@marcellicence]

I am using stereo output. Depending on the audio codec on the audio kit there is some mixing possible (LineIn to Out without processing on the ESP32). I've tested different audio kits and got a stereo output which can be connected to a speaker system, computer or other device with line-in. Hope this helps

[@MagnusWedmark]

Maybe the noise could be implemented using LFSR which with good resolution if similar to real noise and very cheap calculation .

[@marcellicence]

Thank you I could give it a try. In the meantime I've received a code snipped and merged into the project. So real noise is also available

[@OMNI_INFINITY]

Recommend any links or videos with thorough explanation of what a DAC does? (From for loop to output basically...) And is the DAC using a couple of capacitors to smooth the waves?

[@marcellicence]

Hm... I don't know have recommendation at the moment. In external circuit I used capacitors usually in series to remove the DC offset (it is a high-pass filter).

[@doonutaprivoise9917]

why don't you use the ESP32's ADC?

[@marcellicence]

Do you mean the DAC for audio output? In that case I had some problems using the ESP32 board not supporting driving the DAC via the I2S init structure. The support was missing in the SOCs

[@RoyalTech_2024]

Hi, marcellicense. Which variation of ESP32 is required for this project 4MB one or 8MB one? Aslo for your esp32 fm synth?

[@marcellicence]

Hi, the latest generated bin file has a size of 777 KB. I think the smallest one should be good to go. If you add some graphical stuff / display etc. it may require a lot more memory to store images etc. The bin file of the fm synth has a size of 806 KB.

[@RoyalTech_2024]

@@marcellicence Thanks. Is the ESP32 Audio kit 4mb or 8mb? On buying sites it shows 8mb, but when I search for ESP32 A1S Module(used in audio kit) I found it to be 4mb? Do you have any idea?

[@cuddleslapine]

amazing

[@marcellicence]

😄

[@davidkain9046]

wow ...esp32...wow . wizard

[@marcellicence]

Haha ... maybe xD

[@AWSXDER]

This is very cool. Have you tried to use the recorded samples of real instruments instead of synthetic waves?

[@marcellicence]

Thank you. Not yet but it would be possible. This feature is covered by the sampler project. The projects may be fused together =)

[@AWSXDER]

​@@marcellicence Yes thank you. I saw this project. Very well. For many people, this opens the way to a cheap, but very well-sounding midi synthesizer, which is not based on synthesizing waves, but on samples of real musical instruments. I follow the development of this project. Now I am just studying it, trying to understand whether it is possible to redo it under the TTGO TM (music albums) board.

[@silic1um127]

its unbelievable a single esp32 can do all this, great job!!! since the sound engine is customizable is it possible to modify the waves/filter to make it sound like a retro juno/jupiter/prophet sound?

[@marcellicence]

I was also suprised. The integration of new sounds and oscillators might work. Getting the filters into a digital form might be more challenging. There are some designs open source which I could integrate. I do not know how good they are for mimik the retro sounds. At the moment I have only the alpha juno inspired sound which can be modified. I can imagine to make some new sounds soon.

[@silic1um127]

@@marcellicence oh i see, is the alpha juno sound already implemented on the github code? and also do you use a wav file to generate the wavetables? (sorry if the question is stupid, i absolutely suck at coding)

[@marcellicence]

@@silic1um127 The Alpha Juno is available on GitHub. It is part of a library. It does not use any wavetables because of it more complex modulation. You can control the shape and modulation speed, depth via MIDI. The latest version is here: github.com/marcel-licence/ml_synth_pwm_osc_example - I am also working on a kind of better sounding "pro" version. You might recognize that the version on GitHub has some audible aliasing in the higher range.

[@cbrightly]

I’m curious what the CPU overhead is on the one core as currently implemented; could the second core be used for additional processing further down the signal chain?

[@marcellicence]

I think I've set the polyphony as high as possible in the project. I didn't use the second core for audio yet only for the other stuff (serial messages, adc etc.). I could imagine that it would be possible to do some audio processing as well.

[@cbrightly]

@@marcellicence Thanks again for posting your source! I’m particularly curious to see if it’s possible to get the second core to apply a basic effect or two onto the audio or if lowering polyphony to free up more resources is necessary…

[@alexrozkoff814]

Привет, скоро получу с Алиэкспресс uyz32 и pcm5102, тоже хочу сделать такой синтезатор. Есть одна проблема, хотелось бы сделать контроллер на Arduino Uno и управлять параметрами через него. Но я не могу найти ни кода, ни проекта такого миди-контроллера.

[@marcellicence]

I wonder that there is no existing project using the Arduino Uno to build a MIDI controller. I would expect that you can setup Serial with a baudrate of 31250 and sending the MIDI messages to control the ESP32. If the IOs work with 3.3V levels they can be directly connected otherwise you will need some circuit in between. Regarding the messages there are example messages for NoteOn, NoteOff and ControlChange online. Hope this helps to get a step further

[@alexrozkoff814]

СПАСИБО!@@marcellicence

[@alexrozkoff814]

kzbin.info/www/bejne/h6fbfJ-FitSrna8

[@deejayfree]

HI marcel, should i put a pullup for rx ? And the missing diode can be replaced by a led diode ? Many thx for your work 😉

[@marcellicence]

Hi, yes I think you need a pullup for rx because the opto coupler has an open collector/drain. There are some little schematics which can be found on google. Which you a lot of fun =)

[@deejayfree]

@@marcellicence thx, i've found. I've put a 1k on the 3v pin for the pullup. Sound work Nice, but the notes won't stop 😂

[@marcellicence]

Oh I guess the problem. I've updated the code of all projects on GitHub. There are some controllers using NoteOn with velocity set to 0 instead of the NoteOff message. This might be causing the stuck notes. The fix is now online. Please also feel free to use issue/discussion area for ideas/feedback etc. Hope it will now work ;-)

[@RoyalTech_2024]

Hi, Very good work. Is there any way to remove dac and use pwm signal instead or the internal dac. Because the dac you are using is a very expensive one. 😄

[@marcellicence]

Hi, thanks! It might work without DAC (just mono), more information on GitHub. Regarding the DAC .. I've payed less than 5€. There are some cheap based on the PCM5102 =)

[@RoyalTech_2024]

@@marcellicence Can you give me the purchase link.?

[@marcellicence]

I've found it on aliexpress. There may be also other places to purchase from. Hope this helps

[@RoyalTech_2024]

@@marcellicence Aliexpress is banned here in India. On other sites it costs around 19$. 😭

[@rockpopmando1]

Hi Marcel, I have tried to compile it for a ESP32 DEVboard but the ESP reboots every minute. Any hint? I used the optionen "large app, no OTA" . Any other suggestions? Second, - the VCC of the DAC is mostly connected to 5 Volts in other DIY-Projects but You connected it tp 3.3Volts. Is it really connected that way? 3.3 or 5Volts to VCC? I tried both. Any hint would be cool!

[@marcellicence]

Hi, you can check serial output in your console window. In arduino there is an ESP exception decoder which can show you where the ESP crashes. You can try to boot also the ESP without any additional hardware connected. If you like use the Q&A in GitHub

[@rockpopmando1]

@@marcellicence excuse my comment, - it was a Bug in my Arduino IDE. with a new installation, I was able to install Your synth without problems! - letting all settings on default, it works. As You are trying to create a sampler which takes more Flash, - have You seen these Waveforms from Adventure Kid: www.adventurekid.se/akrt/waveforms/adventure-kid-waveforms/ I stumbled across them in the context of a Teensy-based synth. Pre-Converted into C-Header: github.com/DatanoiseTV/AKWF_WaveForms

[@marcellicence]

@@rockpopmando1 no problem. Good to hear that it works. There are some tools for conversion. Maybe you can also use littleFS (SPIFFS) and use .wav files to copy them in advance. Maybe I will try to set up a simple project soon and publish it =) Instead of Teensy you could also take a look at the daisySeed. It also uses the STM32

[@OMNI_INFINITY]

Found this for BLE MIDI in today: kzbin.info/www/bejne/gHTReaJvZ7yIjrs&pp=ygUXVVNCIE1JREkgcmVjZWl2ZXIgRVNQMzI%3D Planning to make an onboard sequencer for the constant step duration slides though, so still some things to figure out on the MIDI transport input though.