Web Control Panel - Micropython asyncio and dual core web server for Raspberry Pi Pico, ESP32

[@TehSpork1]

Great video. I was trying to figure this out by myself before finding your video showing how to do exactly what I wanted to do while much better than my cobbled together attempts.

[@BytesNBits]

Glad I could help!

[@robleepetty05]

Wow, what a great video and your web tutorial was was just as good. I have to say though, you mentioned this would work on the esp32 and since I had one already setup with all my sensors, I used it instead of my picow but just couldn’t get it to work. I’m certain it was something I did but I messed with it for several hours when I figured I’ll just try it with a picow, w/ nothing attached, and it worked right away. Your videos and web pages have saved me months of research. Thank you so much for the countless hours of research, coding, creating web pages, and making and editing these awesome videos. I look forward to more content.

[@BytesNBits]

Thanks for your comments. Yes. I was a bit to hopeful for the ESP32 being compatible. I think Micropython runs very differently on the ESP32, especially when it comes to multi core - basically I think the ESP restricts Micropython to a single core. If you just run in asyncio it should be OK with not too many tweaks!

[@robleepetty05]

Turns out the ESP32 was running circuitpython, and maybe that’s why it wouldn’t work🤔

[@dalesmith8666]

Nice work, what caught my eye was the displayed graphics. As for myself, I've decided to upgrade my PIC24 remote server (that controls antennas; transmitters, etc.) to using the Raspberry pi4. Since I've not done any webserver sites since the early 2000's, need to learn how to use PHP, JavaScript, CGI, etc. all over again. To bad this is all done in Python and not in C or C+. Not for me, but will watch the video's. Could be something interesting.

[@BytesNBits]

It will be a lot easier if you put in a Raspberry Pi. You can install a proper web server such as Apache to so a much better job than this server. Have a look at installing a LAMP setup on the Pi. That will gine you the Linux/Apache/Mysql/PHP server you need.

[@dalesmith8666]

"Have a look at installing a LAMP setup on the Pi. That will give you the Linux/Apache/Mysql/PHP server you need." Is this your work, if so you have a link? Regards Ya but unfortunately, I just simply do not understand Python, as it is to weird for me, not what I've been use to the last 40 years. I am familiar with Apache, and I did install it a couple of days ago. Along with Ajax, PHP, and CGI. My other option would be to use Qt Creator5/6 with the Pie, as I have experience with it, but just trying to figure out if I can run a web server and also Tcp/Ip Socket. My client uses Qt5 with the Tcp/ip Socket, and the Pic24 uses the latest Microchip Socket API. This is all new to me, using the Pie, I've done this kind of application years ago, with at the time modern commuters(X86). Back then used, Microslop "Front Page 2002". That really dates me! LOL! The last time I did this was, with my commercial website, still up, but not maintained since 2007! I retired and the old age of 48! LOL! www.northandradio.ca The nice thing about the Microchip TCP/IP Stack, with the imbedded processor, all of the Protocols are within the API, and the processor will execute any and all requests, since, of course it is interrupt driven. I am currently watching, "Build a web control panel for your project - HTML and JavaScript coding (1080p)" Man you know your stuff! But for now, for me it's just an "information gathering", getting some knowledge and experience. (Research)

[@keithlohmeyer]

Just what I was looking for and great timing. Thanks and you got my subscribe.

[@BytesNBits]

Great! Thanks.

[@qozia1370]

thank you! this topic has almost no tutorials and you made it easy. i love how you combined the two approaches in the end. i noticed there was a delay between the colored displayed on the web server and the neo pixels hardware, is this due to internet speed or does this mean asyncio introduces some delay? thanks a lot.

[@BytesNBits]

There is a slight delay over the network as the request is sent, received, decoded and actioned. Plus I was manually syncing up the video streams by eye so that could also be a factor :)

[@johncutler5995]

I hope you will be able to find the time to solve the threading/dual core issues and post the 3rd video in this series. It has taught me a lot. Thank you!

[@BytesNBits]

Thanks. Final part coming out next week.

[@6672378]

@@BytesNBits The threating seems to be fixed as of today. I am running two independent sockets, each on one core, and it seems to work without issues

[@BytesNBits]

@@6672378 That's great. Thanks for the info. I'll give it a try.

[@6672378]

@@BytesNBits it has crashed yesterday, seems to be a memory leak. I have put a garbage collector call after every http query. It looks like it helped. I am hitting now the receiver by 18 bytes, 20x per second, for 24 hours, without crash

[@6672378]

kzbin.info/www/bejne/fnu7i6imqdx-aa8

[@MarkPerry-h3t]

These tutorials are great, I would like more info on adding a file list or a tab for downlaoding a data file that has been generated from the control program. I am generating CSV text files on the pico and would need to transfer them to the client computer system. Thank you for the time and effort generating these tutorials.

[@BytesNBits]

You'll have to have a look at some html coding. To download a file create a get request that triggers the file contents to be returned as the http response. You'll need to set the header content type to something like text/CSV (please look up the correct header)

[@MarkPerry-h3t]

Thank you, I am not very familiar with HTML Coding so it is my next step in learning this process. Appreciate the point in the right direction.

[@yahmk3978]

Thank you!

[@cabasse_music]

34:35 - thank you very much for the sanity check. I was attempting to do just this and saw this exact behavior (first request processes, then that loop hangs without error)

[@BytesNBits]

No problem. Only one of us needs to go through the pain of debugging this.

[@stefanotuv]

Really great stuff here man. just a quick question i have tried with my own custom built webserver that also manages ap and wifi connections (will share soon) but i have encontered a problem. I use thonny to run the code and when stop the code it seems that the code actually stop but id restart and stop again the pico starts to be unreachable. Have you used thonny? Did you experience the same behaviour? Thanks a lot for all your hard work!

[@BytesNBits]

I think that's fairly common across all the development platforms. The REPL interface is not a proper debugging and control tool. I find powering off the Pico will give it a full reset and reconnect.

[@stefanotuv]

@@BytesNBits gotcha. I couldnt find any solution other than what you propose. I even tried with a global variable to stop the cores but it doesnt work. thanks a lot. you are the man. keep it up!!

[@tiagdvideo]

Brilliant!

[@BytesNBits]

No problem

[@6672378]

Thank you so much for this amazing tutorial. I am trying to achieve something similar, but by using W5500 from wiznet instead of wifi (wanna have wired connection and PoE). Unfortunately it looks like the uasyncio server_start cannot start the coroutine (looks like does not have access to the network). However standard socked does. Did you have similar issue? Do you have any sugestion? At the end, it looks like, I will end up with multithreating, looks like the mentioned bug is fixed. Thank you. Briliant work.

[@BytesNBits]

It's worth playing around with which tasks run in each core. I found that the WiFi hardware only runs on core 0.

[@pdenny1968]

Thx for this info. I’ve been pulling my hair out for 3 days trying to get multi core working on my project with a web server running on the second core. I get exactly as you describe, it seems to run for a few loops, maybe even get a web page to display remotely on another machine, but then it all hangs and even stops the main thread on the first core too. I searched and searched for a reason but came to the conclusion that _thread just isn’t ready yet and was going to give up. I’ll now try swapping the threads over and try running the web server on the first core. Hoefully this will resolve my issue.

[@BytesNBits]

I feel your pain. It seemed logical to me to run the web server on the second core at first as well. Oddly I got a comment not so long ago saying that on the ESP32 core 0 is hardwired to handle the WiFi and Bluetooth hardware. It might be the same on the Pico.

[@pdenny1968]

@@BytesNBits Yeah, I decided to add the web server on second core as an after thought to my project just to monitor my log files, so never thought to swap them around before. I tried thread locks/release etc and all that stuff but never got it working. My main thread still has MQTT posts so I hope this will still work. I’ll post back and update once I get round to testing it. Cheers.

[@pdenny1968]

Oh well, still looks like this isn't going to work. My code works fine IF i comment out all MQTT client connect and publish commands on the second core. Seems any call to use the network connectivity on both cores is the issue. Looks like I may have to go the uasyncio route at this rate :( I hope they can get this sorted out. Not really dual core if not. I don't see why an inbound socket accept() should prevent an outbound mqtt publish, but I'm no expert.

[@christopherlyons7613]

Overall how do you find uasyncio running on the Pico? I believe they still consider it experimental. Based on what you've experienced, do you think it's stable enough at this point? What are the main areas you need to be careful about when setting up uasyncio? Are there limits to the number of tasks you can setup and have successfully managed by the scheduler? How much bigger are your programs when uasyncio is included? Have you found it to be consistent across different uprocessors (not sure if you've used uasyncio with processors other than the Pico)? Getting rid of blocking code is so important and necessary to allow for more advanced projects. Would be great if you produced a video on more details about how to setup and use uasyncio. I was somewhat expecting a bit more info on the general setup of and do's & don'ts for structuring your tasks. Maybe something for the future. Thanks.

[@BytesNBits]

I'll be honest and say I'm not an expert with asyncio. There are some great tutorials online that will answer your questions better than I can.

[@christopherlyons7613]

@@BytesNBits K. Thanks. But based on your experimentation, what do you think? Can you provide any links to other tutorials that you think can better answer my questions? Appreciate any help. Thanks.

[@BytesNBits]

@@christopherlyons7613 multi tasking is quite a big topic and you can really go as deep as you want. Asyncio seems to work well on the rp2040 and esp32 devices but I haven't run any benchmarking tests to verify actual performance. There will be an overhead due to the scheduler and of course individual tasks will be affected by the other task loads. On the tutorial side I skimmed through a few articles to get the general overview of using asyncio. The basic ideas are fairly straight forward to get up and running. My tutorial shows the basic structure to get multiple tasks executing. Once you've got that sorted in your head I tend to find experimenting with little projects will let you delve into the more advanced use cases.

[@TOMTOM-nh3nl]

Thank You

[@BytesNBits]

You're welcome

[@shariltumin]

It's a pity that MicroPython refers to the multi-core library on the RP2040 as _thread; it should be called _core instead. This had caused a great deal of confusion. While the ESP32 has two cores, the first core, core0, is dedicated to system-specific tasks such as WIFI and Bluetooth. MicroPython operates on the second core, core1. On the ESP32, the _thread is preemptive multitasker, and we can have up to 25 threads running concurrently.

[@BytesNBits]

Thanks for the info. I think the ESP32 is a much more powerful and versatile processor but it does add in a bit of extra complexity is some areas. The Pico seems good at keeping things reasonably simple. I'll probably need to add some videos using the ESP32 to show the differences.

[@christopherlyons7613]

​@@BytesNBitsThanks for your videos. Would definitely be interested in seeing some videos covering the ESP32 and the differences between it and the Pico.

[@johncutler5995]

Thanks!

[@BytesNBits]

Thank you very much. Coding the web pages and JavaScript coming out in the next week.

[@umutkayacan7659]

Nice❤

[@BytesNBits]

Thanks 🔥