ESP32 Super Fast JPEG Decoder - 20ms!

This was really interesting - I got a random message on my ESP32-TV GitHub repo.
Is this the fastest ESP32 JPEG Decoder? What the video and find out!
Here's link to the sample code that I was provided with: github.com/esp...
Here's my test code: github.com/ato...
---
If you'd like to help support the channel and help me buy more nonsense - I have a Patreon page: / atomic14
We've also got a shop now! shop.atomic14.com
And if you're shopping on AliExpress or Amazon then if you click on these links I will get a little bit of affiliate money:
Aliexpress - s.click.aliexp...
Amazon - www.amazon.com...
You can also join us on Discord here: / discord
There's also the Atomic14 newsletter here: atomic14.subst...
And don't forget about the MakerNews newsletter - makernews.subs...

[@Pipe0481]

The new ESP-P4 will have a builtin JPEG and H264 decoder, so that will be very interesting!

[@atomic14]

Very interesting! I wasn't even aware of the P4... You've really got to be on your toes to keep on top of everything.

[@raymondmichael4987]

@@atomic14😂😂😂😂😂

[@xyzzy42]

I believe it will NOT have BLE or Wifi, so it won't for a lot of ESP32 projects as a replacement.

[@TT-it9gg]

Waiting and waiting.... When will P4 be available to public? Without WiFi, what would be the arena for P4? There are so many much more powerful MCUs out there already. BTW, the STM32H7 and AMB82 already have this in HW.....

[@piconano]

Waiting for ESP-3200 with it's 32 cores, 100 MB Flash and 100MB FAST RAM. Dreams do come true. ESP32 was thought to be impossible an dream 40 years ago. Would've beaten the IBM PC to the curb by clock speed alone. Just as the ESP8266 smashed Arduinos. Nice job Espressif and you of course.

[@atomic14]

Thanks! It is amazing what we can get our hands on nowadays. Completely bonkers.

[@Max24871]

At that point you should probably start thinking about moving to a raspberry pi compute module

[@GoatZilla]

and an errata list a mile long

[@TT-it9gg]

Thanks for the video. Watching this topic for a long long time. Tried to do the QVGA movie player by using JPEJDEC and TFT_eSPI on ESP32, ESP32 S3, RP2040, K210, Teensy 4, STM32F1 F4 H7, and AMB82. However, the final winner is the aged Pi Zero W.... for size, power, cost, and functions considerations... The Pi 0W can do 1024x600 MPEG4, H.264 decode in HW and output to HDMI, SPI, RGB, VGA, DSI with very low CPU usage and about same power consumption as MCUs. Further, the Pi 0W can do 384k 24 bit I2S out, but ESP32 S3 can not. The only difference is the longer boot time of Pi 0W. Looking forward to the updated pipeline on ESP32 S3!~

[@scienceoftheuniverse9155]

Great work Sir

[@atomic14]

Thank you!

[@MPElectronique]

Hello great job! Can you include in your sketch decode and display an image from SPIFFS? Thank You. Marc.

[@richgain]

Will any of this fast JPEG decoding / video playback work on an integrated board like the Lilygo ESP32 T-Display S3? I haven’t been able to find any examples yet.

[@atomic14]

I don't see why not. My ESP32 TV should work on pretty much any board. You just need to tell which pins to use.

[@massimomeli5960]

Many thanks for this and others videos. Please specify here in the description that it need PSRAM. I've converted your code to Arduino.

[@BrianLough]

Very interesting!

[@atomic14]

Cheers Brian! Tried getting it to work on the CYD - but I don't think it's got PSRAM?

[@BrianLough]

@@atomic14 no it doesnt, it can't be added though , there is a mod for it!

[@atomic14]

I'll have to take a look! I'm really hoping that there'll be better docs on the SIMD instruction set so the guy who did the JPEGDEC library can do some updates. That's pretty memory efficient so *should* work everywhere.

[@TheSlyMouse]

Ohh super cool. I enjoy the deep dive

[@atomic14]

Thanks - was fun to put it together.

[@jumpstar9000]

very nice. did you consider Theora instead of JPEG?

[@atomic14]

I went with JPEG as there's already some nice libraries for decoding JPEGs that run nicely on embedded MCUs.

[@TheEmbeddedHobbyist]

we will just have to wait until S32 gets a GPU in it. Then we'll just have to get used to bigger batteries and large heatsinks. 🙂 Wait a heatsink to Ken and Barbie that will look like a large electric BBQ, it will be party time 🙂

[@atomic14]

🥳

[@drfrancintosh]

I've been following this project for a while. Have you considered sending deltas between frames? This is what Netflix and others do to reduce bandwidth on broadcasts. You might also want to send "full" images every 1 second or so (key frames). It would require "baking" the raw video on the server side (which you're doing to convert to JPEG anyway). I continue to subscribe and am following with great interest!

[@atomic14]

Definitely possible. I've also seen some code for decoding MPEG1 video that looks like it will run on the ESP32. It's one of those interesting conundrums. We've got something that works pretty well with WiFi - it's not that efficient, but we've kind of got bandwidth to spare and higher quality/higher framerate are really limited by our hardware. Improving the compression is a really fun technical challenge, but may not provide any real visible improvements.

[@raymondmichael4987]

I love this series, for now I’m just following watching & enjoying the conversations. I have a project in mind, streaming video from esp32 camera to another esp via Lora (longer distance) or NRF24 (for under 1km distance); I might be dreaming huuh 😊

[@atomic14]

Lora is going to be pretty difficult - Lora is very very low bandwidth - 0.3 kbps to 5.5 kbps. So your frame rate would be very low - and there's lots of rules about not sending too many packets and saturating the network, it's really designed for very small packets with not much data. So probably not very suitable. NRF24 looks possible - it's certainly got enough bandwidth - really depends on what frame rate you are looking for. ESP-NOW actually has quite long range and you can get great performance with external antennas: kzbin.info/www/bejne/pauTkmqLp5yhodE

[@raymondmichael4987]

@@atomic14 , just wow 😯 didn’t expect such respective and such kind reply. Thanks buddy, so Lora is “No Go”, will check the link provided. I can’t stressing this enough, but thanks. Cheers from Tanzania 🇹🇿

[@raymondmichael4987]

@@atomic14 , thanks. Now I will be waiting persistently on your upcoming pcb design videos 😊

[@bobby9568]

Great video! Can you post the link please?

[@atomic14]

Here you go: github.com/atomic14/esp32-jpeg-perf-tests

[@bobby9568]

@@atomic14 thank you

[@stevegulick7656]

What choices are there for jpeg encoding?

[@ayavilevich]

If I remember correctly, PSRAM is slower and can't be used for DMA. So better try and fit that buffer in regular memory.

[@atomic14]

On the S3 you can now use PSRAM for DMA. Lots of details here (and I wouldn't even pretend to understand most of it!) github.com/espressif/esp-idf/issues/11269

[@atomic14]

One thing that did surprise me is that it used to be a real pain using PSRAM with the Arduino framework. You needed to use the special ps_malloc to get it. It now just seems to be pooled in with the other RAM which is much more sensible - new and malloc just work with it as normal.

[@TT-it9gg]

There is 8-lines 80Mhz PSRAM on ESP32 S3.

[@dimo3045]

Thank you for your video, the content is very easy to understand.👍

[@atomic14]

Thanks! Was fun to have a play with the decoders and get some measurements out.

[@johneygd]

Well there will be obviously always trade offs with such insane fast decoding,you cannot decode data faster then what’s coming in obviously.

[@xrete]

teach us more

[@dipanshudhote5671]

I am able to achieve 40+ fps using 128*128 display with esp32