Raspberry Pi 3 vs Raspberry Pi 4 Performance with TensorFlow, TF Lite, & Coral USB Accelerator

[@EdjeElectronics]

Does anyone else think it's crazy how fast the TensorFlow Lite model runs on the Pi 4 with the USB Accelerator? I was blown away when I saw it gets 34+ FPS on a live webcam feed. This is a game changer for real-time object detection! 👍👍👍

[@hussamsoufi1825]

YES!! please Edje make a video tutorial about how to build it with USB accelerator and the TFLite !

[@aaronfish2691]

I wish you could plug the USB accelerator into the USB-C port. Having to downgrade it to the slower USB is probably causing it to lose a lot of performance. I would what the FPS would be if this was running on the Coral BETA Dev board?

[@EdjeElectronics]

@@aaronfish2691 Here are some benchmarks showing how fast it runs on the dev board. It says MobileNet v1 SSD can process an image in 11ms, which means it could run at almost 100 FPS! If you were displaying the results to the screen, your FPS would probably be limited by how fast your CPU/GPU can draw images on your screen. coral.withgoogle.com/docs/edgetpu/benchmarks/

[@aidos9753]

With such a high speed on SSD-mobilenet, would it now be viable to deploy Faster-RCNN models on the Pi? What are your thoughts on what to expect in terms of speed, and if you have time are you able to demo that? Cheers!

[@EdjeElectronics]

Good to hear from you Aidos! Unfortunately, TensorFlow Lite doesn't support Faster-RCNN yet. It will only work with SSD-MobileNet models. Hopefully they will update it to also work with Faster-RCNN. Until then, we're stuck with the lower accuracy. (Source: second paragraph here github.com/tensorflow/models/blob/master/research/object_detection/g3doc/running_on_mobile_tensorflowlite.md )

[@barskorkmaz8323]

This is what I have been waiting thank you :D

[@arsenedelgado368]

I thank you very much for the video, and I look forward to part two

[@EdjeElectronics]

Thank you for the support! In case you want to get a head start before the next video, you can check out the written version that shows how to train and run TensorFlow Lite models here: github.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi

[@thomasoverly7802]

Wooo! Thank you so much, good sir! Looking forward to your upcoming vids.

[@mtheory1999]

Dude you’re badass! Thank you for the upload and look forward to your next video of the tf lite install on the rpi. Dope Blink-182 shirt by the way!

[@EdjeElectronics]

Thanks man! It will be out soon. Haha yeah go Blink!! 🤘

[@joels7605]

Great video. I love the layout of the performance comparison. Keep up the awesome work.

[@jimguffey7407]

I am looking forward to your upcoming videos.

[@NoName-ip4tt]

This is very helpful video, Thanks for sharing your knowledge!

[@sivaa6130]

Great work. Thanks for Detailed Steps. Thanks for sharing your learning's.

[@hussamsoufi1825]

amazing! you won yourself a subscriber Mate!

[@rakhit]

Conspiracy theorists - "The government is tracking everything you do! They can now identify objects through software!" Edje Electronics - "Oh yeah, I do that as well with a $99 Raspberry Pi" That being said I Subscribed. Got the Pi3 and this video convinced me to finally pony up and get the Pi4.

[@nicesuu95]

omg, finally video that i been waiting for

[@fitzmagic]

Another great video. Thanks for sharing!

[@southDutchMan]

I like the content of this video. I look forward to the next one! I just subscribed to your channel. Best regards from the Netherlands!

[@vikrambuilds]

The huge difference in FPS between pi 3 and pi 4 with coral was most probably because of the usb 3.0 support in pi4 which is needed for coral!

[@robotgrrl]

Amazing to see 30+ fps running on a Raspberry Pi! Great comparison.

[@markdanielbelarmino477]

Helpful information! Thanks

[@shafiemukhre]

Subscription! Thank you! This help us to choose the hardware!

[@Andres-is8zz]

People may be asking why Pi 4 + USB is WAY better than Pi 3 + USB. It's really simple, Pi 4 has 3.0 USB, yep, it's really that simple. Great video!

[@esjuve]

Thanks! This was really useful

[@markifi]

subscribed this is amazing

[@mohammedkamran8559]

Really helpful,thanks✨😁

[@nguyenluu3082]

Nice tutorial man!!!

[@javaTL]

Great demonstration!

[@gravitylink8604]

really impressive！ thanks for sharing，recently our team is working on Edge TPU projects too, your demo inspired us more with ideas~~yep, we are also the distributor of Google Edge TPU

[@rhyzo1]

Awesome. Great video

[@raguaviva]

very nice, I wish you had used the same video stream for all the setups though but it;s a great video, thanks!.

[@Thanushri09]

Excellent Work ...

[@anoniem012]

Thanks for testing out.

[@SidsEClassroom]

Great Video !

[@giridharnr6742]

An amazing review

[@niteshverma2310]

Grt comparison. I am making a model to Detect the fish inside the Fish Cages. Plan is to collect more data and process it for better production. CheerS!!!

[@jps200010]

Thank you.Very helpful

[@bernhardahorn-moser2431]

Well done! Thank you!

[@lenkapenka6976]

thanks, this is just the video I needed )))))))))))))))

[@zeus1082]

Awesome output. Can you say how we can use USB accelerator with tensorflow lite in raspberry 4 plzzzz. Can you make a tutorial on that?

[@ramanan4589]

Sir will you please make a video on python based plant leaf disease classification

[@itsme8442]

thank you!

[@pisoiorfan]

Hi can you please tell how much memory the tensorflow lite process uses in this test? The one with unaccelerated tensorflow lite on Pi3. I wonder if it is worth playing with it on my 512MB Pi3A+ Nice videos, and good music selection on your clips, thanks.

[@pisoiorfan]

NVM, I had the curiosity to run tf lite on pi zero. Memory isn't overkill, just a tad above 100MB, with 200MB still available in headless mode. The example cows single image detection takes 16 seconds out of overall 55sec, most of which is spent in loading libraries. I should try it on the Pi3A+ now. Thanks for the great tutorial!

[@EdjeElectronics]

Cool! Thanks for testing it out. I'm surprised that it works on a Pi Zero!

[@dhruvsheth7260]

Thanks a lot!!!! Would love with NCS2 as well

[@ferdicelayir5351]

Nice work. Thanks for nice effort. Is there a way to speed up the raspberry pi 4 without using the USB Accelerator?

[@EdjeElectronics]

Besides switching to TensorFlow Lite instead of regular TensorFlow, the main way to make it run faster is to use smaller resolutions for the webcam. It will run fastest for a 300x300 resolution. The SSD-MobileNet model already squishes all the images down to 300x300 before running it through the network, so using a 300x300 resolution from your webcam won't impact accuracy. (But it will be hard to see the results!)

[@vinodlahire5]

you can try opencv DNN ; along with suggestion from @Edje Electronics

[@hakimke2]

Okay, subscribed~~~~

[@spectralcodec]

Thanks for the great video! Do you know if it is possible to use more than one Coral USB Accelerator at a time?

[@EdjeElectronics]

I don't think it is possible, but see if you can find any information that says otherwise on Coral's official website: coral.withgoogle.com/products/accelerator/

[@Subvac]

Great tutorial! Would you know by chance how much power usage differences between the the three comparisons? Thanks

[@stuartslyper1479]

As soon as I saw the shirt your video got a like from me

[@akselylmaz2017]

First of all your project and video is great! I'm confused about something. Is it possible to run these tensorflow projects with new Raspberry Pi 4 8GB(64-bit operating system)? Can there a big performance(fps) difference between Raspberry Pi 8gb and 4gb observed? Can new Raspberry Pi will be able to work with Coral USB Accelator as well? I have limitid time so is it worth to wait new Raspberry Pi to be offered for sale(I'm just wondering if there could really great performance difference )? I really need advice. I would be very glad if you could answer!

[@pisoiorfan]

The video already shows it on Pi 4. 4 refers the pi model number not memory . There are Pi 4 variants with 1, 2, 4 and 8 GB memory. Unless you have other programs abusing memory, 4Gbyte or 8Gbyte memory doesn't speed or slow anything. 64bit vs 32bit OS might make a difference but less on Coral USB since model is not run by Pi's cpu

[@EdjeElectronics]

Hi Askel, I haven't tested it with an 8GB version yet. I'm a little worried about the 64-bit OS being able to work with all the Python packages required for TensorFlow Lite and OpenCV. I don't think there will be a significant difference in speed between the 4GB and 8GB version, so I recommend just getting a 4GB version for your project.

[@akselylmaz2017]

@@EdjeElectronics Thank you so much!

[@adisantoso82]

Hi, thanks for the video. Is using a usb webcam like yours increase the fps compare to pi camera?

[@pisoiorfan]

Pi camera should be faster as most of image capturing processing is handled by GPU. Being multi core probably doesn't matter

[@EdjeElectronics]

Hi Adi San, thanks for the question. The FPS is the same with a Picamera and a USB camera.

[@santiagoperez6769]

Cool video! Would you happen to know if the Coral USB Accelerator is compatible with the RPi CM4 module? I would like to mount both on a drone, but I hear that they don't play well together, which is kind of odd imo.

[@jumpingPeter]

man i love Bozeman. i came here for the raspberry tutorials but i am subscribing because of Montana :)

[@gje8930]

Are you going to make a tutorial on how to you traind the bird raccoon model. and show how you make it for tf lite and tflite coral?

[@EdjeElectronics]

Yes! I already have a written version of the tutorial, and I will be making a video version in the coming months. Check the written guide here: github.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi

[@thenoobgam3r190]

I have a question. I have raspberry pi 4. I closely followed your tutorials. I only get 0.9 fps tensorflow lite with out coral and 13 fps with. Why is it so much lower?

[@EdjeElectronics]

TheNoobGam3r are you running the Raspbian OS, or something else? What type of camera are you using? Also, what resolution are you running at?

[@thenoobgam3r190]

@@EdjeElectronics I am running at a default resolution set in your code. I am using the NOOBS installation of raspbian. I am using the picamera v2.

[@Airbag888]

I wonder if I could interface google photos and all the work it did already to identify people in my family to train a model...

[@Car_Ram_Rod]

Would you consider doing a comparison of the Neural Compute stick 2?

[@1raskumar]

Спасибо.thank you

[@alpyldrm8448]

Thank you for the video. What I am wondering is, is this performance of raspberry pi 4 (if I don't use any usb accelerator) good enough to use in a rc autonomus car project? I would be glad if you asnwer this.

[@EdjeElectronics]

Good question! I don't think it's fast enough for an autonomous car without the Coral USB Accelerator. If you want to make decisions based on detections in real time, 4 FPS isn't fast enough.

[@alpyldrm8448]

@@EdjeElectronics thank you for your answer. I think going on with an Jetson Nano 2gb would be better than rpi 4. Great channel btw please keep going!

[@arsenedelgado368]

I work with my accelerator but it heats too much. I saw that we can cool, I would like to know with what device. thanks

[@DaniloMajhenic]

What are the expected differences when using a Pi4 1G/2G/4G ram model?

[@kbuilds7287]

Is it weird that I was listening to I miss you by blink, just before discovering this video?

[@ngsteven97]

Can you share the different model files that you used? TensorFlow Lite is a massive pain in just accepting .tflite and labelmap files instead of the .pb and .pbtext so I can't find many models out there and I don't have the time or processing power to train my own!

[@smartassist9700]

What cameras did u connect to Pi(s)? Brand webcam etc.

[@EdjeElectronics]

Check the video description :)

[@falseee4445]

@@EdjeElectronics Should i buy the new one ? www.amazon.com/Logitech-C920S-Webcam-Privacy-Shutter/dp/B07K95WFWM/ref=dp_ob_title_ce (C920S instead of 920S)

[@abelinkin]

Bozeman looks like a beautiful place to live.

[@EdjeElectronics]

Yes, it definitely is! And we're only an hour from Yellowstone National Park :)

[@tahamansoor7396]

pretty impressive

[@dopo8333]

Is the execution of the TensorFlow code parallelized on the 4 cores of RPi3 and RPi4? I'm really puzzled by the performance gain when using the accelerator, given that there's the USB communication bottleneck between Pi and accelerator...

[@EdjeElectronics]

I don't know! I'll have to check next time I'm using it. USB 3.0 has very fast data transfer speeds, so there isn't a bottleneck there. Here's a great article that explains why the Coral is so much faster. cloud.google.com/blog/products/ai-machine-learning/what-makes-tpus-fine-tuned-for-deep-learning

[@foolarchetype]

Спасибо! Больше гайдов с Raspberry Py 4 and object detections

[@REGNiZZ]

That webcam looked like Logitech C920. So, 1080p resolution

[@maxrizzato]

The Coral USB Accelerator tutorial link is a deadlink... :(

[@EdjeElectronics]

Thanks for the heads up! I fixed it. github.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi/blob/master/deploy_guides/Raspberry_Pi_Guide.md#section-2---run-edge-tpu-object-detection-models-on-the-raspberry-pi-using-the-coral-usb-accelerator

[@kurjak021]

These videos are great! Did you try regular TensorFlow with USB Accelerator performance?

[@EdjeElectronics]

Good question! The USB Accelerator doesn't support regular TensorFlow models. It only works with TFLite models (for now, at least)!

[@aaronfish2691]

I am trying to run through the install of TFLite on a PI4 and am getting the following error: ImportError: /usr/local/lib/python3.7/dist-packages/tensorflow_core/lite/python/interpreter_wrapper/_tensorflow_wrap_interpreter_wrapper.so: undefined symbol: _ZN6tflite12tensor_utils24NeonVectorScalarMultiplyEPKaifPf I had previously installed TensorFlow with your older video if that makes a difference. Any ideas?

[@EdjeElectronics]

Yikes! Which command were you doing when you got this error? Were you following my guide on GitHub?

[@aaronfish2691]

@@EdjeElectronics Yes I am using your GitHub guide. I put the following command in to run your py: python3 TFLite_detection_picamera.py --modeldir=Sample_TFLite_model Your command said to put in "python3 TFLite_detection_webcam.py --modeldir=Sample_TFLite_model --picamera" When I did that, I got an error. I then noticed that you had a seperate .py file for the picamera so I tried it instead and got the original error that I posted. One thing else I noticed is that in your command "python3 TFLite_detection_webcam.py --modeldir=Sample_TFlite_model" the "l" in "TFlite" is lower case and I think it should be upper case.

[@EdjeElectronics]

@@aaronfish2691 Thanks for the info! I'm going to make a few updates to the guide this morning. Once it's done I'll comment here again and ask you to try out the new code.

[@EdjeElectronics]

@@aaronfish2691 Can you try using the TFLite_detection_picamera.py script now and let me know what happens? It will work with either a Picamera or a USB webcam, no additional command arguments needed.

[@aaronfish2691]

Edje Electronics yes I will try. Would like me to run through the github from the beginning?

[@max8586ahzvfeh]

Could you explain, what you mean by "Webcam used in this video (works better than the Picamera!)"? I have a RPi camera. Is it worth upgrading?

[@aidos9753]

The RPi camera is very good for this purpose. You could honestly use any camera so long as it has a reasonable resolution to work with.

[@sergej529]

Hi that is a very good video. I try to build the detect.tflite file but I get the following error: "The run command is only supported from within a workspace". How I can solve this error?

[@juniorhernandez3014]

Hey! This is a fantastic video! I retrain a pretained model to detect only 3 classes, with the TFOD api, the SSD MobileNet V2 FPNLite 320x320, but it turns out that I get 0.8 - 1.1 fps when I run it on the raspberry pi 3 model B+, I quantized the model when converting to tensorflow lite. The model is 3.5MB. I saw yours run it to an average of 2.19. Do you know why? Thank you for the video :)

[@wannazmi4951]

Hi Sir and love all your videos.. Thanks for sharing.. May i know how to program raspberry pi to display on the android player.. can you share how.. and is there any code you used to run these test.. Thank You very much

[@techwithsar3763]

Can you tell me how to install tf lite in raspberry Pi 3b

[@EdjeElectronics]

I got you fam! kzbin.info/www/bejne/l5rQhHqFdrp8brs

[@237311]

Is there a way to resize the window live image displayed on the screen?

[@LiangDeng-k8c]

hey, I want to know about Can Coral USB Accelerator run on win10？

[@EdjeElectronics]

Good question! Unfortunately, the USB Accelerator doesn't work on Windows. It uses a package library that is only installable on Debian-based Linux OSes.

[@LiangDeng-k8c]

@@EdjeElectronics oh, I see. thanks a lot. And i ask a issue about some things wrong when building the CPU-only version. I hope you can have a look. Again, thanks a lot.

[@weirjwerijrweurhuewhr588]

How do they compare against the Jetson Nano though?

[@weirjwerijrweurhuewhr588]

Also, what resolution are you running it at?

[@jimguffey7407]

I think I have everything installed correctly. But I keep getting this error. Is this an easy fix? usage: TFLite_detection_picamera.py [-h] --modeldir MODELDIR [--graph GRAPH] [--labels LABELS] [--threshold THRESHOLD] [--resolution RESOLUTION] TFLite_detection_picamera.py: error: the following arguments are required: --modeldir

[@EdjeElectronics]

Jim Guffey Use “python3 TFLite_detection_picamera.py --modeldir=Sample_TFLite_model” . Are you following my guide at github.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi/blob/master/Raspberry_Pi_Guide.md ? From a perspective of continuous improvement, is there any way that I could make it more clear in the guide?

[@jimguffey7407]

@@EdjeElectronics I think you instructions are good. I just went through them again and now everything is working fine. Thanks!!

[@ddomit]

what resolution did you use?

[@arganaztuga]

just see on bottom at 03:56 kzbin.info/www/bejne/ipqyfKmFp76Bf6s

[@timelapse8396]

I got that same blink 182 t shirt 🙌

[@tahamansoor7396]

does it work on Yolov5??? i have yolov5 own trained models.

[@woolfel]

Hopefully TF-lite will keep improving, since reguar TF models run horribly on raspberry pi 3B+. I've tried several different models from the model zoo and faster-rcnn-resnet runs really slow. I ended up getting a jetson nano last year for my hobby project and that runs considerably faster than 3B+.

[@psc19999]

If I change .pb model to .tflite model in order to deploy it on Rpi, do I need to make changes to the code like the camera feed given to the. pb model in python file does those commands change for tflite

[@drdiesel1]

Wow thanks for the great how-to! Does anyone have available an Edge TPU model set with a much larger object db than Google's canned set? How many objects before the Pi4 4gb with the USB acc is a limiting factor?

[@EdjeElectronics]

You're welcome! Let me know if you find a good model that can classify more objects than Google's sample model. I'm not sure what the upper limit on detectable objects is, either! The most I've seen is 200.

[@EdjeElectronics]

Also, the YOLO-9000 model (which is not from Google or TensorFlow) can detect 9000 classes! arxiv.org/abs/1612.08242

[@MrApplePieDude]

​@@EdjeElectronics Firstly thanks for your super useful vids! :) Do you think YOLO can run on the RP3 with fps above 4? I'm trying to make an animal detection robot, but it's seeming difficult!

[@psc19999]

How to configure rpi for .pb file and .pbtxt ? I have trained them on pc want to deploy them on Rpi

[@EdjeElectronics]

Here's a guide showing how to run .pb object detection models on the Pi! github.com/EdjeElectronics/TensorFlow-Object-Detection-on-the-Raspberry-Pi

[@psc19999]

@@EdjeElectronics thank you so much sir 😊

[@arsenedelgado368]

hi . i m working with the raspberry 3b+ and i try to install python 3.5. i try a lot of method but when i tipp $ cd python 3.5.0 , it s does not work. can u please help me??

[@adryncharn1910]

You probably need to get a distribution upgrade, the latest distribution has python 3.7.3. To get the upgrade just type "sudo apt-get dist-upgrade" in the terminal.

[@bachhuynh1562]

Could you answer me some points, since last time I tried to install tensorflow on pi 3, but too many errors Which version for: 1. Python 2. Tensorflow 2.0? And os for pi, do you use orginal one (raspbian) or ubuntu?

[@EdjeElectronics]

Hi, please watch my TensorFlow Lite on the Raspberry Pi video. It gives clear instructions on how to set it up! kzbin.info/www/bejne/l5rQhHqFdrp8brs

[@bachhuynh1562]

Edje Electronics ok sure, thanks

[@real-akmc]

Any chance you could share the code? Thanks!

[@zahrasaqib7739]

check out his github sites mentioned in the description. He provides all codes with explainations over there.

[@real-akmc]

@@zahrasaqib7739 totally missed it, thanks!

[@officialdjmichiel]

Can you check your email? I have sent a question regarding the card counting robot! :)

[@2023-c9p]

Your so handsome

[@voldemore6300]

A few day ago, I used raspberry-pi 4 model B /8gb + pi-camera v.2.1 so it generated FPS:13-16 , In order to increase FPS, I should change a new webcam which one that isn't pi-camera ? or any suggestion

[@lesscaland6719]

U should use an intel ncs 2. Because even u change ur camera device, it doesnt really have an impact for fps performance, because its hardware limitation on rasp

[@freakazoid115]

...lol...did u just dubbed yourself at the end?