AI Robot: Object Tracking | Object Following | TensorFlow Lite on Raspberry Pi

This AI Robot performs object tracking using a Machine Learning model 'MobileNet SSD v1 (COCO)'. Once the object is detected, it follows it and manoeuvres itself to get the object in the center of frame. While the robot is tracking / following the object, working of tracking algorithm and Robot's view can be accessed on a browser.
TensorFlow Lite Python APIs are used to implement Object Detection and Tracking. OpenCV is used for generating Robot's view with information overlay. FLASK is used for streaming the camera view (or Robot's view) over LAN.
#Robotics #ObjectTracking #RaspberryPi
Source Code:-
github.com/jit...
More Projects 👇👇
***** Robotics: Level 1 *****
👉 Making of Web-based Raspberry Pi Robot
• How to build a Raspber...
***** Robotics: Level 2 *****
👉 Interfacing Camera with Raspberry Pi robot and Creation of Web Control panel
• Raspberry Pi Robotics:...
👉 Interfacing Ultrasonic Range Sensor with Raspberry Pi robot | obstacle avoidance robot
• Robotics: Ultrasonic D...
👉 The speaking Raspberry Pi robot | Text to speech through audio port
• Raspberry Pi speaking ...
***** Robotics: Level 3 *****
👉 Mobile Phone Accelerometer controlled Raspberry Pi robot | Javascript hardware control
• Raspberry Pi Robot: Sm...
👉 Controlling Raspberry Pi robot through voice commands | Javascript Web Speech API | Speech Recognition
• Robotics: Controlling ...
***** Robotics: Level 4 *****
👉 Teachable Machine: Gesture controlled Raspberry Pi robot | Export Machine Learning model of Teachable machine
• Teachable Machine ML M...
👉 Image Classification using Pre-trained Machine Learning model MobileNet V1 | TensorFlow Lite on Raspberry Pi
• TensorFolw Lite on Ras...
👉 AI Robotics: Object Detection and alert generation on remote Web Panel | TensorFlow Lite & COCO image model
• AI Robot: Object detec...
👉 AI Robotics: Object Tracking | Object following robot | TensorFlow Lite
• AI Robot: Object Track...
***** Home Automation *****
👉 Smart Water Tank: Water Level measurement using Ultrasonic Distance Sensor and Real-time online monitoring
• IOT Project: Smart Wat...
👉 IOT Project: Raspberry Pi GPIO controls over wifi
• IOT Project: Raspberry...

[@jitesh_helloworld]

Replicate this Robot's software on your Raspberry Pi using the bash script provided here:- helloworld.co.in/earthrover/ Hardware Connections and Software Features:- helloworld.co.in/earthrover/readme/

[@cristianooliveiradesouza3539]

Excellent project, it's a good idea for experiments students of tech high school, I will to make with my students. Congratulations !

[@jitesh_helloworld]

Please do. Thankyou very much 😊

[@jitesh_helloworld]

Hi Cristiano, you may find it useful kzbin.info/www/bejne/ioSziKWrlr12oqc

[@giovannirota3850]

Spettacolare ...................

[@giovannirota3850]

Complimenti ottimo lavoro.

[@funnydhopadaho5394]

Fine robots You are great man

[@jitesh_helloworld]

Thanks.

[@anjithaj8581]

Respected Sir, Respected Sir, First of all your tutorial is awesome. It is working good. But when I try to replace mobilenet_ssd_v2_coco_quant_postprocess.tflite with my custom trained model for fire detection, it is not working. Could you kindly assist me in resolving this problem? Your guidance is crucial to me.

[@ssgoh4968]

Brilliant.

[@jitesh_helloworld]

Thanks

[@sanukk8082]

which motors ae used for moving the motor and what power source is given to the motor driver

[@ajayghodke1464]

Hello Jitesh. In the install script, a lot of the dependencies fail to be fetched, as a result the webserver in the end doesn't work. Can you please help me this urgently. i have dropped you a mail as well. please check.

[@riskyminor6775]

I have a XGO lite CM4 robot dog with raspberry pi, does this code work on legged robots too? I'm still a beginner at python

[@yogaaditya2797]

Hello man, i tried ur code only object tracking(i comment the move_robot) i am using tflite model efficientNet1, and it's got error "only size-1 arrays can be converted to Python scalars" do u know how to solve that? and im planning to use intel ncs 2 for accelerate the fps (since google coral is out of stock in my country), what do you think? Thank you

[@webdot564]

I already install everything.. previously i cam access earth rover control pannel..how can i go again this control pannel...i install everything last day

[@avadhutkasekar9541]

Nice 👍👍

[@jitesh_helloworld]

Thanks 👍

[@kzs985]

I'm curious what's the capacity of the battery, and how long can it run with the pi and coral?

[@jitesh_helloworld]

I have used a 10000 mAh Li-Polymer Power Bank to power up Raspberry Pi and Coral. It can conveniently last for 3-4 hours while running the ML model on Pi. I could be more but I didn't explore beyond 3-4 hours

[@hchnnyl6936]

briliantttttt. can i get the source code for arduino microcontroller pls?

[@marvelous0306]

Very nice. I want to do this project again, but I have not synthesized the components yet. Can you give me the parts list? Thank you very much

[@jitesh_helloworld]

Thankyou. Pl go through these two articles. These articles cover the components involved in making this robot helloworld.co.in/article/basic-robotics-make-robot-raspberry-pi-web-controls helloworld.co.in/article/camera-robot-using-raspberry-pi-web-controlled-surveillance-robot

[@marvelous0306]

thank you so much

[@miracletouch8214]

I'm trying to make a robot that can track my ball for football practice essentially I'm trying to make a robot defender but the problem is i barely know anything about robotics so for the object tracking is there a component that does the tracking

[@jitesh_helloworld]

Nice thought. In this project, the tracking is done by a custom python code that takes inputs from a Machine Learning Model. The model is responsible for identifying an object correctly and provide it's coordinates in a picture frame

[@hamtigerb7849]

Where can I buy the coral accelerator from?

[@kavitasbakery]

Nice Project👍💐. I need your help regarding the same. I have replicated your code. Everything works very well except Image Classification, Object Identification & Tracking, Human Following part of AI. I am using RPI 3b 1.2 without coral.

[@proengeleceng6861]

good project bro what microcontroller you used and can i use pic microcontroller with python code ? >>> thanks for sharing

[@jitesh_helloworld]

Raspberry Pi single board computer is used in this project. The python code created for this project won't work on microcontroller. Any project that involves computer vision, ML and wireless networking would require a processor with suitable OS to run respective Python libraries. Microcontrollers don't have that capability as of now.

[@ZuNunchaku]

if there is multiple ball/objects how to select from following 1 object to another?

[@jitesh_helloworld]

This code works with only one object. If more than one object is present then the one with higher confidence score will be selected to track.

[@eliezerrovajurado5328]

Can i put multiple things there? so that i said, follow the person, car or an bycicle?

[@jitesh_helloworld]

Yes, you can. You need to specify the object name in following array in code:- arr_valid_objects=['person', 'bicycle' ] also, you need to modify the code a bit to generate forward/backward motion commands differently for large objects like these. I will soon be making a video on Human following robot and explain these modifications

[@eliezerrovajurado5328]

@@jitesh_helloworld ok thanks. Yeah I only need to steer, forward and backward is doing a friend with arduino.

[@eliezerrovajurado5328]

@@jitesh_helloworld one more question, in the link there are 2 codes which one do I need? I want to put it on my raspberry pi 4

[@jitesh_helloworld]

'object_tracking.py' is the main file and it calls utility functions from 'common.py'.. You will need both. BTW here is the human following version.. Check it out kzbin.info/www/bejne/fYS8paFud9F6eKs

[@eliezerrovajurado5328]

@@jitesh_helloworld ok thanks than i will look how it works. Thanks

[@francopessina8193]

Hola señor, ¿es posible adaptar la etapa de movimiento de 4 ruedas a una de 2 ruedas con una tercera rueda libre? Excelente project. Congrats. thank you

[@francopessina8193]

Hello sir, is it possible to adapt the 4-wheel movement stage to a 2-wheel with a third free wheel?

[@jitesh_helloworld]

Yes. The robot you see in the video has only 2 wheels attached to the motors. The other 2 wheels are free..

[@monstert2368]

I got an error "AttributeError: module 'util' has no attribute 'gpio'" Please help me fix it Can you give me the components of this project and how to connect Thanks very much !

[@jitesh_helloworld]

Hardware and connections helloworld.co.in/article/basic-robotics-make-robot-raspberry-pi-web-controls

[@jitesh_helloworld]

Try this kzbin.info/www/bejne/ioSziKWrlr12oqc

[@rakshithreddy1092]

Hello sir, how do i use this code for YOLO? Can you please help me out. Thanks.

[@jitesh_helloworld]

This code makes use of a tensorflow lite object detection model. You need to see how can you convert your existing model to tflite version to use it with this code. Tensorflow official website can provide some lead

[@eliezerrovajurado5328]

Does any one know how i get this coordinate Systeme on my Raspberry pi? Is ist in the Code or do i habe to install something?

[@jitesh_helloworld]

It is in the code. the overlays on the camera frames are generated using OpenCV. See the function draw_overlays() in 'object_tracking.py' file

[@robomatics697]

can i make this project without coral??

[@jitesh_helloworld]

Yes you can, but for responsive tracking you need to process at least 10 frames per second. Which is only possible with coral

[@HL_SAIRA]

Does it follows only ball

[@jitesh_helloworld]

The ML model has 90 objects. Ideally this code can track any of those. But due to the robot's camera height and tilt, it can follow small objects only. You just need to include the name of object in array 'valid_objects' .

[@GuopyWorld]

Bro, I go through your your source code, but shows some attribute errors, what I have to do?? Please help me

[@jitesh_helloworld]

Hi. try to install the code using the method described at this link helloworld.co.in/earthrover/ It installs all the dependencies/packages & code automatically on your Raspberry Pi.

[@GuopyWorld]

@@jitesh_helloworld hello bro, I have installed as you said, but at last for web page of control panel it shows page not found, please help me

[@jitesh_helloworld]

It is a tested script and works as expected. Try to access the control panel locally on Raspberry Pi first. Open Chromium browser and type 127.0.0.1/earthrover. You should see the Panel.

[@GuopyWorld]

@@jitesh_helloworld, 127.0.0.1/earthrover also shows error 404 not found message

[@jitesh_helloworld]

Restart apache. See if it is installed or not. Otherwise run that bash script again.

[@Mystogun14]

May i have a look on your sources ?

[@jitesh_helloworld]

The link of source code is there in description above.

[@sumanthkota8267]

I see there's a camera in the bot. Which camera is it?

[@jitesh_helloworld]

The whole project is based on camera input.. Camera here is Pi Camera. It's a Raspberry Pi camera module 5 MP

[@sumanthkota8267]

@@jitesh_helloworld can we also run the code in pycharm on PC, there are few board specific libraries in the code, are there any methods to import them on PC?

[@jitesh_helloworld]

I haven't tried that and not sure if it will work because some imported modules will work on Raspberry Pi only. So you should run this code on a Raspberry Pi only for the correct results.

[@sumanthkota8267]

@@jitesh_helloworld Thank you for this!

[@krishnadathanph6127]

Hello will it work on raspberry pi pico

[@jitesh_helloworld]

No. Pico is a microcontroller. We need a processor with os to run the code for this

[@krishnadathanph6127]

@@jitesh_helloworld can I use blob tracking method so will it run on pico w

[@musicforsoul7441]

Hello.bro i need your help

[@jitesh_helloworld]

tell me. What help you require

[@אביבכהן-ו3ס]

@@jitesh_helloworld Traceback (most recent call last): File "/media/pi/FEF6-7799/human_following/human_follower2.py", line 183, in main() File "/media/pi/FEF6-7799/human_following/human_follower2.py", line 149, in main interpreter, labels =cm.load_model(model_dir,model_edgetpu,lbl,edgetpu) File "/media/pi/FEF6-7799/human_following/common.py", line 76, in load_model interpreter = make_interpreter_1(model_path) File "/media/pi/FEF6-7799/human_following/common.py", line 27, in make_interpreter_1 {'device': device[0]} if device else {}) File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 152, in load_delegate delegate = Delegate(library, options) File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 81, in __init__ self._library = ctypes.pydll.LoadLibrary(library) File "/usr/lib/python3.7/ctypes/__init__.py", line 434, in LoadLibrary return self._dlltype(name) File "/usr/lib/python3.7/ctypes/__init__.py", line 356, in __init__ self._handle = _dlopen(self._name, mode) OSError: libedgetpu.so.1: cannot open shared object file: No such file or directory