Environmental Sound Classification on Microcontroller

[@maanavsingh1914]

Fantastic work!

[@YoavDvoishes]

Hi jon, does the code for this example exist in the repository? I'm trying to make a real-time audio classifier much like this.

[@Jononor]

The classifier is running on the microcontroller, the UI is just showing the predictions. But yes, both classifier code and UI code should be in the repo

[@vishalim866]

Can you please help me with the basic steps needed to classify dog sounds based on their mood, I have already seen your conference video it was amazing, but could not understand properly as I'm just a starter. Sorry for asking a lot. I know that I'm not requesting in a correct way😅. Thank you for taking your time.

[@Jononor]

Hi Vishali. The most important thing for such an application would be to have a sufficiently large dataset with labels for the dog emotion. If this does not exist already, it may be that you need to create it! Emotion detection is in general quite challenging, as it can be hard to define precise labels with high agreement among annotators, and the emotion can be hard to discern even for a human listener.

[@vishalim866]

@@Jononor thank you for the reply. I am currently focusing only on alert and pain, I have for now collected 160 datasets for both. Also is annotations necessary?? For now I have extracted mfcc, spectral centroid, spectral contrast of the frames into a python dataframe ( the frame were only 1 second length....need to decrease it)...and have defined a simple cnn model for it. The model gave 75.3 test accuracy. How can I improve it I have yet to try spectrogram image classification...

[@vishalim866]

And also time shifting data augmentation is yet to be applied

[@Jononor]

@@vishalim866 Sounds like you are well on the way! Smart to focus on alert and pain in the start. Make sure that you also include enough sounds that are neither alert nor pain, a "other" category of dog sounds.

[@Jononor]

@@vishalim866 Trying spectrogram image classification is a good idea. You can also try some audio embeddings such as OpenL3, for small datasets that might do better.