Face & Movement Tracking System Using a Raspberry Pi + OpenCV + Pan-Tilt HAT + Python

Utilise computer vision systems to always keep your face in the centre of the frame. Then add a movement detection and a patrol layer to make finding faces easier (just like a portal turret!)
Full Article - core-electronics.com.au/tutor...
Make sure to use the Previous Raspberry Pi 'Buster' OS with this Guide.
Related Information
Quick Start Guide for Pimoroni Picade Pan-Tilt Hat (Used Here) - core-electronics.com.au/tutor...
Flashing 'Buster' OS onto a Raspberry Pi - core-electronics.com.au/tutor...
Facial Recognition Raspberry Pi - core-electronics.com.au/tutor...
Speed Camera with Raspberry Pi - core-electronics.com.au/tutor...
Object and Animal Recognition With Raspberry Pi - core-electronics.com.au/tutor...
Claude Pageau Face-Track-Demo Github - github.com/pageauc/face-track...
How To Use Your Phone to Control Your Raspberry Pi - core-electronics.com.au/tutor...
Cameras in combination with machine learning create the most powerful sensor you can ever put on a Raspberry Pi Single Board Computer. And now we are making them smart and mobile with artificial intelligence! The intention here is to not only create an easy-to-use face-tracking system with a Pan-Tilt Hat but also do so in a way that can be readily expanded upon no matter what systems or code additions you choose to use. All code utilised is open-source and fully commented. We are going to control a Pan and Tilt system with a Raspberry Pi so that it keeps your face (and the action) in the centre of the frame. I also demonstrate in the Full Article how to control the speed of rotation either making it very smooth or very fast. After achieving the above the next step is to code a patrol phase for the Pan-Tilt system. So, when it does not see a face, the system moves around its degrees of freedom logically to search for one. Then to aid this patrol phase further we can add in the code the ability to automatically turn towards any moving objects that it identifies. Open-CV is a huge resource that we use here to help solve the real-time computer vision and image processing problems.
If you have any questions about this content or want to share a project you're working on head over to our maker forum, we are full time makers and here to help - coreelec.io/forum
Core Electronics is located in the heart of Newcastle, Australia. We're powered by makers, for makers. Drop by if you are looking for:
Pimironi Picade Pan-Tilt Hat - core-electronics.com.au/catal...
Raspberry Pi 4 Model B 8GB (Used Here): core-electronics.com.au/catal...
Raspberry Pi 4 Model B (4GB) Ultimate Kit Bundle (AVALIABLE!) - core-electronics.com.au/raspb...
Raspberry Pi Official Camera Module V2 (Used Here): core-electronics.com.au/catal...
Raspberry Pi 4 Power Supply: core-electronics.com.au/catal...
0:00 Intro
0:23 Tracking System Overview
0:55 Open-CV
1:01 What You Need and Terminal Commands
2:01 Simple Face Tracking Code
2:35 First Pay Off
3:10 Specifics on this Face Recognition
3:29 Patrol, Movement and Face Tracking Code
4:32 Second Pay Off
5:30 Code Adjustments
6:10 Acknowledgment
6:24 Where To Now
7:08 Outro

[@michaelmanoukian]

This channel has literally everything, thank you!

[@gaggablagblag9997]

This channel is amazing!

[@tariklassouaoui1635]

very inspiring , well explained, I love this channel

[@Core-Electronics]

Thanks to you 😊 glad you dig it

[@MrRubion]

Thank you for your video. Do you think the set-up also works with the Raspberry Pi High Quality Camera? Or would it be to heavy for the pan-tilt hat?

[@innuendoz]

This is great thanks

[@Fraunzi]

THANK YOU so much for this 🙏🙌

[@Core-Electronics]

No problem 😊

[@jsuck2]

Incredible project! would it work with Sunfounder Pan-Tilt Hat? or some code should be adapted? Many many thanks!

[@BABALOOEY46]

So good, thanks for sharing.

[@Core-Electronics]

Glad you enjoyed it!

[@abdulrhamanalkinani1955]

Thank you so much. Could you please do eyes tracking

[@anasamjad3019]

I'm planning to use the same concept but instead of tracking face, I want to track Sun. Is the execution is still the same? I have a pytorch pretrained model currently.

[@_bruh_420]

ik im a bit late but i want to implement a doppler radar (for increased range). and the radar detects movement somewhere, so it tells the pan tilt to point the camera to the area with detected movement. how would i modify this code?

[@Bhardwajkushagra]

I couldn’t get my hands on Pimoroni Pan tilt hat but I have a ordinary pan tilt bracket. Can you please suggest me how can I manage with that.

[@Mohammedalhalabi16]

Question: If you disconnect the internet from it, will it still work on tracking moving objects without its connection to the internet, or should it be connected to the internet in order to continue tracking the movement of objects, and if it works, can we make a project to track an object using Arduino Uno

[@thomasgf5948]

Hi I have the head (eyes) from inmoov robot but without pimoroni picade pan tilt Hat. I connected directly to Raspberry pi. So i need not download software for it? What i must change from your example. Can you help me?

[@Ipsissimus]

This is so amazingggggggggggggggggg

[@amreenmander3809]

Great video! Quick question: Are we able to use the HAT without directly mounting onto the Pi so that the other RPi pins aren't covered and may be connected to other outputs (e.x. to control a relay or LEDs). Meaning if we mount the hat to a breadboard instead and use jumper wires to connect the 5V pin and 3 GPIO pins (2,3,16) from the RPi, will this still work?

[@Core-Electronics]

This is the GPIO Pins utilised by the HAT - pinout.xyz/pinout/pan_tilt_hat So long as you connect the Pan-Tilt HAT to those required pins it will work. Whether it is installed as a HAT or on a Breadboard. You are then able to use the other free GPIO Pins to control whatever you desire 😊.

[@Bhardwajkushagra]

i dont have pimoroni pan tilt hat but another one. can you tell me how to directly connect servos to rPi

[@digitalbirdfilm]

Great work I have been looking for a way to add this functionality to the Digital Bird DIY camera control system and this might just be what i'm looking for. Do you see any problem using an HDMI to Pi4 camera board in place of the Pi camera?

[@onojioboardwalk9748]

Could you build one of these for me, Like a commission? But with some added fearues?

[@woolfy02]

I wonder if a stepper motor could be used instead? The only thing I could compare it to is how smooth a c-ram / CIWS operates. (Minus all the weapons). I think it would look amazing if that was possible to setup something like that, with a project like this. It seems like the OpenCV you use, works great.

[@Core-Electronics]

Absolutely mate 😊 the AI system can remain the same and it can totally be ported to control stepper motors.

[@emeobioh7950]

​@@Core-Electronicsis there a way to record and save the live feed video? I want to build a surveillance camera and need to be able to record the video and save so it can be accessed later

[@onojioboardwalk9748]

@@Core-Electronics .. Hi could you develop this into an even more advanced version, To be built similar to the predator's shoulder-visor, But attached to an eye-head-piece? To have nightvision/Infrared & Tracking at like in this video? Do you do commissions?

[@afifzuhairr]

to save electricity or lengthen the lifetime of the servo motor, is it possible to add motion sensor first before switch on the camera module? so, the motion sensor would be the first layer security, then camera will catch that motion in sentry mode -> face recognition ! just my two cents!

[@Core-Electronics]

Very rad idea!

[@nathanbowser609]

Any advice on using this base in a SAR have application. Particularly night time wild game census taking. I want to build a diy eo/it up to 4 sensors. LWIR thermal sensor; RGB spectral with adaptive zoom, and LRF laser range finder are the big three. I need the computer to run BG separation, temperature threshold, and object recognition. end result should be a dark granular background; only things above threshold have color, and camera and gimbal zoom and move to keep target in frame. The info is not readily available out there, I’m picking from 15 year old experience, and articles buried deep on the internet. This stuff isn’t restricted or illegal, but there’s definitely an unspoken agreement to not make it readily available

[@Core-Electronics]

Now that makes for an amazing project! DIY object detection and face tracking is in its infancy but I have no doubt people will keep pushing the thresholds on what is possible. If you do nail your plans please share 😊 it would make for an incredibly powerful tool for all kinds of applications.

[@KJL_offical]

Hey can you give me a link to the things i need to buy?

[@ani80322]

Sir can we use Raspberry Pi 3 in the place of Raspberry Pi 4

[@wkinne1]

Will this work on an Orange Pi 5?

[@balyazulfikar4155]

hi great video and thanks for the compelte explain im wondering have you ever create project with raspi+usb cam to counting people in a room? best regards from me

[@Core-Electronics]

I've done some work with the OAK-D Lite and one of the machine learnt systems that it runs counts people in a room/as they wall past. Once you locate the exact Python script you'll be able to base your code in your system on that 😊 - core-electronics.com.au/guides/raspberry-pi/oak-d-lite-raspberry-pi/

[@ZenHulk]

The problem i have with all these you tube videos is they're never real world. Been trying to do this for 6 months and move two 6 volt servos not a pan tilt well yes but 12 inches long which NOPE, never happens. So my robot neck sits their gathering dust.

[@jonathanpartidagomez6847]

Nice video bro, one question, could a USB camera be used instead of the one you used?

[@Core-Electronics]

The code will require some tinkering with but you absolutely can 😊 all the Scripts are open source so feel free to modify away.

[@geddajayaram6380]

will it be implemented using pi4 and 3

[@immrinzler7814]

Hii , does Open CV is to be need open on system for working of tracker , does it still work after just installing code on rasberry pi?

[@Core-Electronics]

You need the Open-CV packages to be installed so that it can do the face and movement detection aspects of the code

[@fricosu007]

thats a very nice code i see here ! nice work ... question, i want to install this system on a ceiling ... i would add an x y and z axes on the ceiling moving the camera and the rest of its components, how can i add some more code to control the movement of the xyz axes together with the face tracking movements. THANNKS !

[@Core-Electronics]

I have a simple Keyboard PAN-TILT Hat control system that you can see here - core-electronics.com.au/guides/pan-tilt-hat-raspberry-pi/ Combining that Script and the one talked about here should get you moving in the right direction. If you need a hand with the Python Language hit up our Workshop - core-electronics.com.au/courses/python-workshop/

[@lovetangTV]

Is it possible in Pi 3 as well? Please answer

[@LI3AXE]

Hey, followed all the instructions but this error keeps coming up. enabled i2c, but problem persists ? Any solutions? File "/home/pi/track/facetracker.py", line 37, in pan(cam_pan-90) File "/usr/lib/python3/dist-packages/pantilthat/pantilt.py", line 466, in servo_one self.setup() File "/usr/lib/python3/dist-packages/pantilthat/pantilt.py", line 80, in setup self._set_config() File "/usr/lib/python3/dist-packages/pantilthat/pantilt.py", line 118, in _set_config self._i2c_write_byte(self.REG_CONFIG, config) File "/usr/lib/python3/dist-packages/pantilthat/pantilt.py", line 209, in _i2c_write_byte raise IOError("Failed to write byte") OSError: Failed to write byte

[@Videogame9559]

H I would this work with the 16 servo control board and camera module or would I need to change the code. I have pan and tilt mount.

[@imenguechtouli1518]

i have the same question!

[@Videogame9559]

Sorry you answered my question at the beginning of the video

[@tech-business-world3721]

can we make a 4 dof(using servo motors) robotic arm for pick and place small objects using pi camera to detect object

[@umairgujjar4576]

Ni usama

[@juheonchu6322]

If installing the binaries with those sudo codes, may I ask you any alternatives?

[@Core-Electronics]

Not 100% sure what you mean, but if you go through all the steps your setup will be good to go!

[@ajaykushwah6793]

My pi camera is responding very slow.. I m using rpi 4 model b 4 gb...how can I reduce the response time? Like I m waving my hand in front of picamera and it shows after 3 to 4 seconds

[@Core-Electronics]

That is very slow. To start make sure your Raspberry Pi is getting enough cooling, active cooling would definitely help. As soon as the board gets Hot it'll start going slow. In software you can also lower the pixel count, that will drastically improve reaction time. Hopefully that helps. If hardware is holding you back take a look at the OAK-D Lite, it is quite a remarkable AI Camera compatable with Raspberry Pi Single Board Computers - kzbin.info/www/bejne/bXPOeZaAqppqeck

[@ajaykushwah6793]

@@Core-Electronics thanks ..I will do the same

[@berk1953]

Hİ, In the "make -j $(nproc)" command, the loading process may take a very long time, even the screen may freeze, you wrote on your page. It came up to seven percent, then the screen froze, I'm being patient as you said, but it's been 2 hours, is it normal? :) should ı wait?

[@Core-Electronics]

2 Hours is definitely tooo long. Give the system a hard power cycle then type that same "make -j $(nproc)" command into a new terminal with it focused on the right folder structure. So it will be the following. cd ~/opencv-4.4.0/build make -j $(nproc)

[@Jerseylance1]

Great video, but can I use the code with my dynamixel servos?

[@Core-Electronics]

Absolutely, you will just need to alter some specifics in Python so that way your servos gets the right PWM signals sent to them.

[@Core-Electronics]

Depending on the current draw you may want an extra power input. Check this guide for wiring tips with a servo and a Raspberry Pi 4 Model B - core-electronics.com.au/guides/control-servo-raspberry-pi/

[@user-ei9yz6vm4t]

руки індуса в нього це така фішка? типу якщо відео технічне і не від індуса дивитися не будуть?

[@claybrd]

Would this work with a raspberry pi 4B 4GB or does it need to have 8GB?

[@Core-Electronics]

It will definitely work with lower RAM Raspberry Pi 4 Model B 😊

[@pritams_vlog2332]

hello sir love from india i have following on youtube since last year very knowledgeable channel is running you come to the point . I need you help on one of my project I am trying to develop face recognition system that system can identify people either he/she in database or not but I didn't get any specific resources I hope you help me out thanks

[@geddajayaram6380]

when two are identified, which person is identified first?????

[@Core-Electronics]

Big Question! The AI system when looking for faces I'm pretty sure goes from top left to right then down one row and repeats. If I was to guess the person in the top left of the preview window would be targeted first. Only one way to find out for sure though, Test it!

[@geddajayaram6380]

can it be implemented by using mobile as camera module????

[@Core-Electronics]

Sounds advanced but you definitely could

[@nikachachua5712]

hello bro , im new on raspberry pi (still waiting till i get zero w ) , im using pycharm on windows 10 to code with python , i dont know how to set up everything on raspberry pi so can u help me or link me where i can learn everything step by step? need to use rpi zero w with my codes from pc

[@Core-Electronics]

Hey mate, an absolutely great starting point is our Raspberry Pi Workshop - core-electronics.com.au/tutorials/raspberry-pi-workshop-for-beginners.html

[@nikachachua5712]

@@Core-Electronics thx 🙏 sorry for another question but i have to , could u help me to find tutorial or library or software to control action of robotic , for example i made rpi dog and want to make easy virtual motion and want to use that on real live rpi dog? pls help one more time ❤️

[@Core-Electronics]

All depends on what hardware your using to control the movement of the Robot. If your using Servos this is a great starting point - core-electronics.com.au/tutorials/control-servo-raspberry-pi.html

[@fsddsdsd2840]

Hello what do I do with this? VIDIOC_S_PARM: failed: Inappropriate ioctl for device I am using picamera v2.1

[@Core-Electronics]

Sorry to hear you are having issues, I haven't seen that error message before, we have a forum for this guide that might have some helpful information, and if not feel free to chuck a post on their with your issue. Give us a bit of details about your setup and where it goes wrong, we have lots of maker-eyes over there that can help. core-electronics.com.au/guides/Face-Tracking-Raspberry-Pi/

[@user-iw7ii4lu6h]

did the code can tracking after full body not just face?

[@Core-Electronics]

You can definitely replace the face detection layer with a object detection layer that focuses just on human bodies 🙂It would take some tinkering with the Python code. Check out this guide on how to set up object detection - core-electronics.com.au/tutorials/object-identify-raspberry-pi.html

[@user-iw7ii4lu6h]

@@Core-Electronics i need a full body not faces is it possible?

[@Core-Electronics]

Come check this out mate - core-electronics.com.au/tutorials/pose-and-face-landmark-raspberry-pi.html

[@johnjayveec.alindayo9278]

Its possible to detect the faces with temperatures? Thanks

[@Core-Electronics]

I've seen thermal cameras run with face/human detection - kzbin.info/www/bejne/nqHciGeCrM1_apY For this Pan-Tilt system if you added a Laser Thermometer you could definitely achieve that goal - core-electronics.com.au/non-contact-ir-thermometer-sensor.html

[@johnjayveec.alindayo9278]

@@Core-Electronics Can i contact you on gmail?

[@johnjayveec.alindayo9278]

@@Core-Electronics I saw now the SENO206? Did you know how to do that with temperature sensor?

[@Core-Electronics]

If your running into issues or want help with your project the best place for us to help is on our forum and as we'll be better able to help 🙂 forum.core-electronics.com.au/

[@johnjayveec.alindayo9278]

@@Core-Electronics do you have any idea how to get face detection with temperatures using adafruit mlx90640 using raspberry pi?

[@berk1953]

can ı do this with raspberry pi 3 model b+ ?

[@Core-Electronics]

It definitely will, just at a slower FPS.

[@berk1953]

@@Core-Electronics that's fine, as long as there's no conflict in the code :) thank you..

[@kerron_]

I made similar

[@talismr01]

I can do this using a USB camera, instead of one (opencv ai kit)

[@macixmajsterkuje5799]

Can I use raspberry pi zero to do this?

[@Core-Electronics]

Because the computation is so high with computer vision project like this I would reccomend using the Raspberry Pi 4, the delay using a Pi Zero I reckon would be too great for functional use.

[@CreativeGraphicsGroup]

@@Core-Electronics Maybe a Pi Zero 2 W? (If you can get one...)

[@jonathanpartidagomez6847]

Can I use a Raspberry pi 3A+???

[@Core-Electronics]

It should work 😊 you may need to overclock that Pi a little bit to keep up with the computational demands.

[@yee2e]

Is it tracking face without computer?

[@Core-Electronics]

All the face figuring out/calculations are done on the Raspberry Pi Single Board Computer 😊

[@adamakmal9401]

Hi great video where can I get the coding for it

[@Core-Electronics]

Hey, its in our guide linked below. Also here: core-electronics.com.au/guides/Face-Tracking-Raspberry-Pi/

[@oscar-vf6vi]

Is the hat absolutely necessary?

[@Core-Electronics]

Is definitely the easiest way to set up a Pan-Tilt System with a Raspberry Pi. There are other Pan-Tilt Hats but you will need to adjust the code running the system to accommodate for them. If your only interested in Face Identification come check out this - core-electronics.com.au/guides/face-identify-raspberry-pi/

[@DarrylAdams]

When people talk with horror about facial recognition and mining done by the big data companies and government, I point to openCV and say this tech can be done by anyone, so evil is now dramatically easy for anyone to do. And with the Pu, cheap too!

[@epluribusunum5318]

@Darryl Adams True, but an individual can do little but dox with it. A corporation or government can do almost anything.

[@onojioboardwalk9748]

.. Actually this can be turned into a self-defense weapon with a multitude of functgions.. So its not 'Evil' in the hands of good people.

[@Jirayu.Kaewprateep]

📺💬 Face recognition by raspberry pi, face detection is paused when you create a new alignment from the center. 🥺💬 You are correct that is for speed, you need to read about machine learning for face recognition task it is multiple convolution layers as filters do and dense layers with learning weight which means you can make it smarter as Google neurons net but it will require a lot of processes. 🥺💬 I think it is by the program intention to be able to run on raspberry pi, can you try adjusting parameters or mode⁉ 📺💬 How about this ⁉ 🥺💬 You may using stack frames it will little bit slow but good with moving object. 📺💬 Threshold sensitivity 🥺💬 Same as filters but you use the standards training from factory or pre-trained weights. 📺💬 I know about that. 🥺💬 It can be contrast colours but it also can be object weights or other function as weights that we called pre-trained weights.

[@lobmati5998]

Ima use this shit to build an airsoft sentry