Cheap and convenient Computer Vision (Machine Learning). For RV1106/RV1103 (LuckFox Max Pro, etc.)

00:00:00 - Intro
00:00:58 - Quick spoiler
00:02:02 - How to install the system (Ubuntu, Buildroot)
00:04:29 - Ubuntu vs Buildroot. Python vs C++. Overview
00:05:21 - Cross-compilation (how to set up). How to build and run RKNN example.
00:06:57 - Inference. Int16? Int8!
00:08:09 - Yolov5 inference (yolov5s, yolov5n, etc.). Speed. Problems and solutions. 128 vs 256!
00:11:20 - Resent alternatives
00:12:20 - Energy consumption
My LinkedIn - / maltsevanton
My Telegram channel - t.me/CVML_team
e-mail: [e-mail]
Good video about the platform - • Luckfox Pico - Milk-V ...

[@wolpumba4099]

*Summary* *Introduction to Computer Vision on Budget Boards* - 0:01 Introduction by the host, discussing the capability of running computer vision on a cheap $10 board. - 0:17 Background on the host's channel and focus on Edge boards including Rock chips. - 0:33 Discovery of a new, inexpensive H and P chip that sparked interest. - 0:56 Discussion of Ubuntu compatibility and Python limitations. - 1:02 Quick spoilers about the Lock Fox Max Pro board, Ubuntu issues, Python limitations, and actual performance metrics. *Installing the Operating System* - 2:03 Introduction to the installation process for the system using sock toolkit on Windows. - 2:21 Mention of an alternate Linux installation process. - 2:36 Challenges faced with sock toolkit not recognizing the SD card. - 3:22 Ubuntu installation was successful but didn't work with the chip. - 3:33 Attempts to switch to Buildroot, a Linux-based system, which also faced issues. - 3:55 How the host solved SSH access problems by not following the guide. *Choosing the Right Operating System and Programming Language* - 4:30 Acknowledgment that Ubuntu is not suitable for using the rock chip NPU. - 4:51 Limitations of using pre-built Python wheels and the need for recompilation. - 5:14 The availability of OpenCV in the official documentation, avoiding the need for cross-compilation of OpenCV. *Cross-Compilation and Running RKNN Example* - 5:24 Explanation of the cross-compilation setup process. - 5:38 Clarification that full system recompilation is not necessary. - 5:45 Recommendation of using a specific version of Ubuntu on Windows through WSL. - 6:01 Alternative for Linux users to use Docker for the compilation process. - 6:22 Instructions on cloning necessary repositories and setting up the tool chain. - 6:38 Confirmation that after compiling, the results need to be copied to the board and run. *Understanding Inference and Quantization* - 6:58 Discussion on the possibility of running neural networks in different quantization formats. - 7:30 Explanation that only int8 quantization is possible with the rock chip RV1106. - 7:56 Consideration of creating a video about why int8 may not be the best format for neural networks. *YOLOv5 Inference: Speed, Problems, and Memory Constraints* - 8:09 Analysis of YOLOv5 inference, the official example, and associated speed. - 8:28 Challenges encountered while trying to replicate YOLOv5s and issues with the arcanon expert. - 9:01 Differences in model sizes and memory constraints. - 9:17 Realization that the board has less usable memory than advertised due to system builds. - 9:44 Potential solutions including rebuilding the system without the camera or buying a 256 MB board. - 10:17 Positive results with different YOLOv5 models and inference times for YOLOv5n. - 10:40 Postprocessing issues due to non-maximum suppression output discrepancies. *Considering Alternative Boards and Future Testing* - 11:23 Plan to order a larger board and share test results on social media. - 11:39 Observation that several new boards with NPUs have been released. - 11:51 Host's intention to review these new boards in the coming months. *Energy Consumption During Neural Network Inference* - 12:23 Idle (0.15W) and active energy consumption (1W) metrics for neural network inference. *Closing Remarks* - 12:41 Conclusion and invitation for the audience to follow for updates. Disclaimer: I used gpt4-1106 to summarize the video transcript. This method may make mistakes in recognizing words.

[@shakhizatnurgaliyev9355]

The truth is not bright😁 Самый честный обзорщик бордов!!!!

[@andreyl2705]

Thank you sir. Awesome. It’s very interesting to see your review of Milk-V Duo 256

[@nicolasfelipe1]

thanks for so much valuable information, there's so little in the internet, hope there is a second part like setting a dev environment let's say with visualstudio and Python.

[@LeonardoSapucaia]

Thank you for the great content, Anton!

[@sanyamarora4876]

I have followed the guide to install Ubuntu and it worked. Haven't used it much, but it worked 😂

[@powerHungryMOSFET]

How did you do the steup? have you followed their website? is it easy to set it up ?

[@amsoftgyexoa2681]

Yeah. excellent chip. I have reached 23 fps with 192x192 yolov5s images (detection + tracking + counting) on ​​my passenger counter with RV1103. with 640x640 images 7 fps is achieved. greetings.

[@AntonMaltsev]

Thank you for the numbers. Is it with the NMS? Did you use the onboard camera?

[@amsoftgyexoa2681]

@@AntonMaltsev Yes, the on board camera. There on the Luckfox pico wiki there is a link to a reduced version of opencv 4.8 that works very well, at least to open the camera, take frames, etc. Yes, it is with NMS. Based on yolov5s' example, I made the modifications to add the tracker (sort) and my counting algorithm.

[@AntonMaltsev]

Yes, I saw this OpenCV build. But I ordered a 128mb version, and it's not enough for a camera and a good network. Will probably re-order the 256 MB one.

[@amsoftgyexoa2681]

​@@AntonMaltsevMine is the basic Lucfox pico, without flash memory. I used a TF card. By the way, I recompiled the kernel to be able to use Wi-Fi via USB (mt7601). everything works very well.

[@amsoftgyexoa2681]

​@@AntonMaltsevMine is the basic Lucfox pico, without flash memory. I used a TF card. By the way, I recompiled the kernel to be able to use Wi-Fi via USB (mt7601). everything works very well.

[@hiteshrohila6236]

Have you tried using custom yolov5 model on this RV1106, i get segmentation fault when using yolov5 model dataset was downloaded from roboflow.

[@AntonMaltsev]

Yes, I used custom-exported models. I recommend using the official repository from Ultralitics for training and from RockChip to export.

[@mdnnjony4000]

Thank you, brother🎉

[@joaodragone2353]

Hey, love the video, i think you should try Milk-V Duo, it has 3 version and start at 5 dollars!!! Also, it is the same size of this one but the TPU has 1TOPS!!!

[@AntonMaltsev]

I'll check it out! :)

[@boumedinebilal7566]

Thank you so much, Anton, for your effort in exploring these slides! I'm truly grateful for the insights you've provided. I'm curious about the possibility of programming the Rv1103 NPU at a low level with C. Do you have any information or insights on this? Looking forward to your response!

[@AntonMaltsev]

RV1103 and RV1106 from LuckFox have similar pipelines. You need to prepare a cross-compilation C++ pipeline. I saw a lot of posts on the official forum about VScode integration but did not try it. I am more from the computer vision world and prefer not to go to c|c++ code if it's possible :D:D:D

[@samezarus]

Ландан из э кэпитал оф грейт бритэн =)

[@SK-bl1lp]

I want to speak from my heart!

[@unoarecare5728]

Thank you for the video. Is there any king of NPU board that could give 60fps or at least 30?

[@AntonMaltsev]

For which task? For the detection - RK3588-based boards from RockChip, nowadays Jetsons, Hailo-based boards, etc. For the Stable Diffusion - probably no boards can do this)

[@unoarecare5728]

@@AntonMaltsev I need 60fps for human detection, doesn’t matter what algorithm or kind of board

[@amsoftgyexoa2681]

the new radxa zero 3w is very promising with the rk3566.

[@AntonMaltsev]

Agree. I append it at the end of the video. Also, want to bye it. But I feel there will be some problems: 1) I saw the video that EMMC is not supported easily with existing images 2) I have many problems with RK drivers for RochPi 3A from them. So, it's interesting to test.

[@amsoftgyexoa2681]

@@AntonMaltsev but I have tested the orange pi 3b (rk3566) and it reaches 13 to 15 fps with yolov5s at 640x640. with the radxa zero 3w it's just waiting. I haven't received mine yet to try.

[@armenkalaidjian4494]

Антон, почему-то все твои видео расcчитаны на тех, кто и так хорошо во всем разбирается. Хотя бы раз сделай видео с конкретикой для остальных. Например в этом видео пройди от начала (установки ОС) и до запуска обнаружения объектов с YOLO с примерами на python. Посмотри как это делают многие на raspberry pi.

[@AntonMaltsev]

Но зачем? Ведь такие видео уже есть:)

[@armenkalaidjian4494]

@@AntonMaltsev я не смог найти ничего приличного на этих чипах. На малинках полно, а на этих нет ничего. Можешь сам поискать и убедиться.

[@AntonMaltsev]

У меня в канальчике недавно был пост о том зачем и с какими целями я делаю видео. Может будет понятнее почему мне не интересно делать видео рассказывающие о том что такое SSH - telegra.ph/Zametki-Computer-Vision-inzhenera-20232024-12-30 Про конкретно данную плату: 1) Есть документация, где все это есть. В видео я показываю те места которые в документации не однозначны. 2) Видео с установкой системы есть достаточно подробное, оно приведено в описании. Я тут показываю те проблемы которые при этом возникали у меня. 3) Я в этом видео достаточно подробно показываю как настроить кроскомпиляцию и сбилдить их пример. Так же есть отсылки к видео где я показываю как экспортировать модели для рокчипа. То видео на пол часа и оно достаточно подробно.

[@yuca133]

These cheap Chinese development boards are sometimes a nightmare. Instructions so limited and outdated. I spend many hours simply trying to connect my pico pro board 😤

[@AntonMaltsev]

I think it's much better than 5 years ago:)