ESP32 SD Card Speedup With a Couple of Lines of Code

No video

ESP32 SD Card Speedup With a Couple of Lines of Code

Рет қаралды 9,896

Күн бұрын

The ESP32-S3 can act as a USB Mass Storage Class device (USBMSC) - but how well does it work? Once again we're using my custom PCB from www.PCBWay.com...
It starts off pretty slow, but there's quite a few optimisations that we can make to get it quite a bit faster!
I'm using the USBMSC class to mount the SD Card connected to the ESP32 to my PC. It does work - but it's a bit slow - but we can make it faster!
The GitHub repo with my tests is here: github.com/ato...
---
If you'd like to help support the channel and help me buy more nonsense - I have a Patreon page: / atomic14
We've also got a shop now! shop.atomic14.com
And if you're shopping on AliExpress or Amazon then if you click on these links I will get a little bit of affiliate money:
Aliexpress - s.click.aliexp...
Amazon - www.amazon.com...
You can also join us on Discord here: / discord
There's also the Atomic14 newsletter here: atomic14.subst...
And don't forget about the MakerNews newsletter - makernews.subs...

Пікірлер: 50

@atomic14 10 ай бұрын

If you'd like to help support the channel and help me buy more nonsense - I have a Patreon page: www.patreon.com/atomic14 We've also got a shop now! shop.atomic14.com And if you're shopping on AliExpress or Amazon then if you click on these links I will get a little bit of affiliate money: Aliexpress - s.click.aliexpress.com/e/_Ac05mF Amazon - www.amazon.com/?tag=atomic14-20 You can also join us on Discord here: discord.gg/h2DDnDdjD2 There's also the Atomic14 newsletter here: atomic14.substack.com/ And don't forget about the MakerNews newsletter - makernews.substack.com/

@raymondmichael4987 10 ай бұрын

😮 This was the hard one; That’s me “😊😊😊” when I hear “we built a new pcb” Thanks buddy

@neilomalley9887 10 ай бұрын

Alot of this is over my head but I think I’m gaining an appreciation for possibilities and limitations of these MCs

@piconano 10 ай бұрын

I see you found the rabbit-hole!

@atomic14 10 ай бұрын

Yep, down we go...

@Dinkleberg96 10 ай бұрын

I'm loving the project! The improvements made with a few changes on the code are tremendous, and that could also work but by adding a USB controller and a SD card slot the writting to the PCB the speeds will increase by significant ammounts. I'm really keen to see how this project goes

@atomic14 10 ай бұрын

I’ve got the new schematic. Working on the PCB layout. It’s a lot simpler than the idea i Gad in the video as I’ve got an IC (thanks to a viewer comment!) that will put everything in Hugh impedance when’s it’s held in reset. But there’s now a lot more to layout and I’m trying to avoid going to a four layer board as the cost jumps up a lot..

@devcybiko 10 ай бұрын

I'm still following with interest. The serial I/O bottlenecks are news to me, so this episode is really paying off. I follow with rapt attention. Continued Success!

@nikthefix8918 10 ай бұрын

Using USBMSC on ESP32-S3 I've had good results writing and reading to/from Winbond flash chips instead of SD notwithstanding the USB1.1 limitations. I like the idea of a dedicated chip instead of a removable medium. I've also tried FRAM which is very fast - almost as fast as SRAM - but capacities at the time of writing are small. IDF and Adafruit libs are available to support serial Flash / Fram. Wear leveling for Flash is even supported.

@atomic14 10 ай бұрын

That's interesting, what kind of capacity do you get with the flash chips?

@nikthefix8918 10 ай бұрын

@@atomic14 128Mbit so 16MB. There are larger capacities but I believe the addressing scheme changes above 128. The chips are NOR rather than NAND flash so there are further advantages there. Look up Winbond W25Q128JV.

@nikthefix8918 10 ай бұрын

@@atomic14 There's also this chip: Adafruit SPI Flash SD Card - XTSD 512 MB I bought one but there were no real advantages over an SD card and the OEM silicon was still pretty expensive.

@nikthefix8918 10 ай бұрын

@@atomic14 FRAM is definitely the way to go for fast lower capscity MCU storage. You don't read much about it but I think it has potential. Great endurance too so you don't have to worry about wear or implementing wear levelling. You just treat it as non-volatile SRAM. Reliably clocked at 80MHz SPI in my experiments.

@JS7457 10 ай бұрын

What library did you use to simulate an USB Mass storage with a Winbond flash chip?

@user-op3fd2ie2y 10 ай бұрын

If ESP32 monitors the Presence of USB (5V + signal lines) then ESP32 can switch properly switch between computer reading/writing SD Card and ESP32 reading/writing SD card. It means finishing the ESP32 SC Card read/write cycle and enabling USB access to the SD Card. If ESP32 is not switched ON (by battery) then Logic is powered by USB 5V and access to SC Card via USB is enabled. If ESP32 is charged via charger (no USB signals are present) then Charging and reading/writing SC Card via ESP32 is enabled.

@user-op3fd2ie2y 10 ай бұрын

As well, USB switch IC is not needed. :-)

@donatasismanus6056 10 ай бұрын

werry interesting, was also looking for solution how to switch sd card from any device to usb, and of course best to usb3 :) thanks for sharing !!

@atomic14 10 ай бұрын

Fingers crossed it actually works...

@ArtiZirk 10 ай бұрын

Maybe instead of a USB mux, just use a hub ic and automatically mux the SD card to the GL823 card reader when you sense that USB is connected.

@atomic14 10 ай бұрын

Interesting idea - I'll take a look at options.

@nikthefix8918 10 ай бұрын

I think the upload speed via USB1.1 is acceptable but the SPI read speed of SD is not. A background copy of SD data to PSRAM might help in some circumstances - like a picture frame or gif player. At least then your display will populate quickly on demand 'cos the data are buffered in a much faster storage.

@Sogager 9 ай бұрын

Thanks for the helpful tips! Really appreciate the clear instructions, you made it so simple!

@atomic14 9 ай бұрын

Glad it was helpful!

@PsychogenicTechnologies 10 ай бұрын

Whelp, that GL823K datasheet ain't great... but the status LED and GPIO might be ways to give you status and a way to do the switching after it's done. If you're doing a PCB to play with these things, I'd add in a couple of DNP resistor/jumpers to experiment with these, in addition to just keeping an eye on vbus. Curious to see how it works out with this IC.

@monahanz 2 ай бұрын

I'm looking for code to read/write individual sectors on an SD card. Can somebody point me to a download/code. They seem to be mentioned here but no reference

@davidealessandrini2235 3 ай бұрын

Thank you for the very informative video. I have a problem with a software I'm developing: I read an Audio file from SD and play it via i2s. The reading is 3072 bytes at a time. It happens every now and then that the reading takes 500/600 ms instead of the 15ms it normally does. This behavior is not periodic but random. This ruins the audio streaming. Have you ever encountered this problem? I use esp32s3

@jespernoer5521 10 ай бұрын

Using the GL823 is an excellent solution. I have used a similar setup in the past with GL835. GL835 has a power-saving mode so you don't need the second switch (PI3A27518), just put the ESP32 pins into Hi-Z mode when connecting USB. GL835 can go into sleep/standby mode when USB is not connected. I'm not sure this will work with GL823 though.

@atomic14 10 ай бұрын

Interesting! Getting rid of that switch would be ideal. When it's in power saving mode can it still be connected to the USB data lines? Ideally I'd like to be able to use the USB for programming and debugging when not in mass storage mode.

@atomic14 10 ай бұрын

The GL835 seems to be very hard to find - I guess USB2.0 is old tech nowadays.

@jespernoer5521 10 ай бұрын

@@atomic14When the USB switch sends data to the ESP32, the lines are disconnected from the GL823 and you can use it for debug. I just don't know if GL823 will release the data lines to the SD card when the USB lines are disconnected.

@atomic14 10 ай бұрын

Gotcha - if it can be used like that it would be ideal. Or even if holding it reset will keep the GL823 pins in high z.

@atomic14 10 ай бұрын

@@jespernoer5521Looks like the USB224X will do what I need - it will definitely go into tristate when in reset mode. I think I might even be able to do away with the USB switch as when it's in reset mode the data pins are high Z as well.

@OLDNEWTECH 6 ай бұрын

how can i upload a code in esp32 dev board using Arduino IDE.... NEED HELP

@veheikko 8 ай бұрын

Thanks, this was very helpful

@atomic14 8 ай бұрын

Glad it helped!

@thomasbjrnsonlarsen7326 10 ай бұрын

How about using RTOS to run two threads, one reading data from the USB and buffering, and another to write from the buffer to the SD-card?

@atomic14 10 ай бұрын

That’s what we end up doing really, the writes are pushed onto a queue and then written by a background task. While that’s happening more data is read from the USB.