Unity AR Foundation Tutorial - Getting Started with Face Tracking

Join our in-person XR Hackathon in Berlin and win up to $25k in funding and prizes 👉🏽: www.xrcc.events/
Want to build a fulfilling and well paid career in AR/VR (Spatial Computing)? 👀
Our best tips, strategies, and secrets from teaching AR and VR since 2016 - everything distilled in one Free Course 👉🏽: bit.ly/48MBgT0
Our AR/VR Education Platform offers
✔️ AR/VR Courses
✔️ Private Discord community
✔️ Expert Q&A Hangouts
✔️ and much more
insiders.dineshpunni.com/
How to create Augmented Reality Apps using AR Foundation and Unity?
In this Beginner friendly Unity AR tutorial series you'll learn how to create amazing AR Apps for your iOS and Android phones. (ARKit + ARCore)
This video will cover image tracking (or image recognition) with AR Foundation.
You wanna have a glance at the future of spatial computing? Then definitely check out the HoloLens Development playlist here:
• Unity HoloLens Tutoria...
🔔Subscribe to our channel to get started in AR/VR for Development, Business, Design, and much, much more: 👉 / @dineshpunni
🔻Let's connect!
Website: www.dineshpunni.com/
LinkedIn: / dineshpunni
Instagram: @dinesh.punni
🔻Let's collaborate!
If you have any questions leave them in the comments below,
for business inquiries contact me via e-mail: [e-mail]

[@sabmory]

Thnx for content. I just wanted to add something to this tutorial, if your device starts video stream from back camera u just have to select camera of the AR Session Origin and then set the value of Facing Direction from World to User. It took me a day to find out, hope it saves someone's time.

[@DineshPunni]

Thanks for sharing! Highly appreciated

[@sabmory]

@@DineshPunni NP. Do u know how to change or add something to AR Default Face? For example add teeth to it. I have problems doing this, can u help me?

[@torus1183]

thank you so much

[@diarykyoto8894]

All your videos about AR are very very helpful!!! I really rally thank you!! I hope you will upload other AR tutorials!! Thank you very much!! (from Japan)

[@DineshPunni]

Thanks man! :) If you need me to cover something else let me know!

[@anshgaikwad4268]

Great tutorial series on ARFoundation. Eagerly waiting for the next video.

[@DineshPunni]

More to come! :)

[@stickrobo]

Thanks for making this video. Helped a lot.

[@anshulsingh8326]

Not working. Back camera turns on but does nothing when it see face

[@rashaan64]

I have the same issue did you fix it?

[@anshulsingh8326]

@@rashaan64 Sry, I fixed it, but I don't remember it now

[@sabaadeeb1291]

Hi Dinesh, thank you for a wonderful video. I followed every step but sadly my built shows blank screen on the mobile. What can be the reason.

[@purbaphalgunimishra9114]

Wow! Face tracking. Will try it now... I am also waiting for the AR foundation image tracking video for multiple images with their respective Models... I hope you will upload this tutorial too... Love...

[@adiwijayakusuma2288]

Yaa same here

[@christiangab1227]

Thank you for sharing such a simple guide in UnityAR foundation you are truly helpful and nice ^^

[@DineshPunni]

Glad it was helpful! Thanks for the kind words :)

[@annywang4791]

@@DineshPunni hello,why it can't show the mask on my face?i had already followed all steps from your video,but it didn't work.

[@divi1700]

Hi Dinesh, thanks for the tutorial!! How do you test the AR on your face? Can you record the video on iOS? Cheers!

[@user-ml3nf2xn6o]

I have a question. Can I plug my cell phone and connection into the cable when I build it?

[@gastongonzalez2699]

Hi Dinesh! We have applied the mask correctly on android but on ios it shows with some issues, it is not very accurate positioned in the face, how we could fix this? Thanks!

[@justnoone7505]

Thanks man...Its really helpful hopefully it helps more people....

[@skudgee2]

Right to the point, keep it up !!

[@DineshPunni]

Thanks bro!

[@rahulrokade2048]

How to run this on our mobile I follow the steps till "Build and run" but it is not showing in mobile

[@saurabhshah559]

What ndk you use for projects

[@akash_the_phenomenal942]

please help , how Do I add accessories like Hat , Big ears etc , currently it being Face masked and cutting other things , Please Help

[@kawerishirke5568]

thank you, very informative tutorial

[@DineshPunni]

You are welcome!

[@user-dx9pu5db8d]

Gather this doesn't work on a forward facing camera?

[@amankaramchandani6100]

Hey, Dinesh Loved the video! can you make a video or guide us for ar web development too it would be great

[@pemainsolo94]

how to create filters in hand ? pls give me tutor

[@guestgamez]

how do u change face model?

[@mauricioalbarracin]

Hi Dinesh, Thanks for the tutorial. I have seen how the texture adapts to the features of the face and I want to ask you, how would it be with a 3d object, such as a cube, glasses, ect? thanks, do you have any tutorial where it shows how? or some video reference where they explain how, thanks

[@vitoriapereira4734]

hi! have you found the answer for this?

[@DeepakKumar-ce3rx]

change camera faction direction to user if is not working

[@adiwijayakusuma2288]

How about covering multiple image target or selection model for AR Plane finder (like IKEA apps that customer can chose the furnitures and apply it in AR Btw, thanks for the lesson

[@pakizanaeem653]

thank you so much for such a helpful tutorial. Now I want to detect face landmarks so that I can set glasses modal on eyes. please upload a helpful tutorial about it.

[@monishmakkar8398]

were you able to do it? I want to do the same. please help

[@wiwitfarianto7161]

please make ar tutorial using FAST Corner Detection

[@ravinash218]

normal face tracking works. when added the video, the app crashes

[@nyilonelycompany]

thank you brother

[@DineshPunni]

You're welcome!

[@facemotion3d971]

thanks for the video. I am intersted in your AR course. will it teach the basics of artech so that I can build an ios app for tracking and streaming facial expressions live from iphone to unity?

[@DineshPunni]

Heyhey, the AR course will teach you all the essentials that you need for building iOS and Android applications. The course is not specifically for face tracking and streaming etc. but we also offer community support, so if you enroll and you have questions with your projects we are happy to help! Check out insiders.dineshpunni.com/course/ar-course-title for a full curriculum

[@dhaniyanuar5024]

this is a simple and clear tutorial... it is possible for adding some UI / object 2d /3d in around the face and following the direction of the face??

[@DineshPunni]

Thanks for the feedback. Yeah that's possible since you get all the face orientation data from AR Foundation

[@EvilPact]

@@DineshPunni Hello, I know this is an older response, but could you tell me where I can access these face orientation settings? I have been looking all around unity and couldn't find where to change those.

[@Zichoe]

Hi, is there any way I can change the camera to vr camera so that I can view AR content with a Google Cardboard in AR Foundation/ARCore, just like in this video (kzbin.info/www/bejne/hoW7d2ycbLVjjpo ), or does only Vuforia have that abilities? Thank you.

[@DineshPunni]

Phew, good question. I'm not really sure about the use case especially since cardboard has become deprecated now. As far as I know, you can split your view but the content will most likely look unsharp and probably jittery

[@kamal.sale7]

nothing appear on my face

[@ThePinnxx]

Thanks Dinesh, Im wondering where I change the setting to look through the front camera? when I export the build to my android it sends it to the wrong camera so I can't see if it is working

[@DineshPunni]

Actually it should be setting it automatically when you enable face tracking capabilities as far as i know

[@ronweasley5573]

did u fix it? mine doesnt work on rear camera also :/

[@nithilesh175]

can i make an ar jewellery with this????

[@hnsalexchandriyaa4900]

Great tutorial.....If i Do the same after app started playing my screen becomes black. using samsang s7..which android and ios device supports face tracking and body tracking in AR foundation??

[@DineshPunni]

Hey, check out developers.google.com/ar/discover/supported-devices to find out if your phone supports ARCore

[@workflowinmind]

Thanks my friend

[@workflowinmind]

But I have an issue... on IOS it is using the back camera and when I tried to record the screen at the same time and film with the back camera my face was not tracked, I'm not sure if the recording is the issue but anyway.. Do you know how to set the camera to the front one so that I can test it :-) ? Thanks my friend

[@DineshPunni]

So far it's only available in ARFoundation 4 preview. But check out the documentation for changing camera direction here: docs.unity3d.com/Packages/com.unity.xr.arfoundation@4.0/api/UnityEngine.XR.ARFoundation.ARCameraManager.html?_ga=2.92616285.1992905119.1591172151-1355718908.1581523894#UnityEngine_XR_ARFoundation_ARCameraManager_requestedFacingDirection

[@weikang9915]

Black screen appears :(( I'm using Huawei P10 Lite, android ver 8.0, AR foundation ver 4.0.2, AR core ver 4.0.4, Ar kit face tracking 4.0.2, and min api level 7.0

[@danylolahodniuk5084]

you must choose plug in provider in project settings. Just go to XR-plug in managment and choose ar-core

[@danylolahodniuk5084]

hope i helped

[@OrSNCucumber]

Hi Dinesh, thanks for the tutorial! Do you know how I can place a hat on my head rather than painting with a material/video? Thanks!

[@DineshPunni]

Unity's AR Foundation provides you with all the positional and rotational data from the facetracking. You could (for example) add the hat as child of an invisible mask and add an y/z offset to it. Unfortunately by the time of recording the tracking was quite inaccurate.

[@OrSNCucumber]

@@DineshPunni Thanks! Trying to do it, but the hat keeps "escaping" my head.

[@manishkumar4049]

Hi, Dinesh Thanks for tutorial! Can we track face part like nose, ear, neck ?

[@DineshPunni]

Yeah, that's possible

[@manishkumar4049]

Can you make a tutorial or any article links Thanks

[@wilvaldod7218]

Hi dinesh , i got prob i get black screen on my phone. is there any way to fix it ?

[@DineshPunni]

This could have many reasons, maybe this post can help you out github.com/Unity-Technologies/arfoundation-samples/issues/2

[@ttttkk2]

i did what you did, but why XR folder won't show in the Assets?

[@DineshPunni]

Make sure you have the *exact* same versions as in the video for Unity and AR Foundation

[@dineshbhathad8308]

hey, any idea for what do if the phone has a pop up camera(OP7Tpro), and im getting a black screen.

[@DineshPunni]

Does the camera pop up and then show black screen?

[@dineshbhathad8308]

@@DineshPunni hey man, thanks for the reply. However I figured it out, just had to make the camera user facing instead of world facing. Camera popped up on its own after that.

[@fahimshuvo8232]

does it support android? I am asking that question because we are using here ARKit face tracking package. Thank you !

[@DineshPunni]

Hey Fahin, yes it does! docs.unity3d.com/Packages/com.unity.xr.arfoundation@4.1/manual/

[@Mr.FarePlay]

Great video. For ARCore, I'm trying to use Unity Remote app to test my apps on my phone, but it looks like it's not supported for AR Foundation. Is there a way to play test ARCore apps directly from the editor to speed up the process?

[@DineshPunni]

There are some tools on the assetstore, but be aware that you don't have your camera feed streamed. Unity is working on that but I don't know when it will be officially launched

[@user-nu3zp1ho1u]

Hi Dinesh ,May I ask if the Android phone supports ar face tracking?? thanks

[@DineshPunni]

Check out developers.google.com/ar/discover/supported-devices to find out if your phone supports ARCore and face tracking

[@DiogoCavalcanteLima]

Can other parts of the body be tracked? arms and legs?

[@DineshPunni]

As far as I know this is not possible with AR Foundation at the moment

[@AkashKumar-kl9bt]

Error building Player: BuildFailedException: ARCore Required apps require a minimum SDK version of 24. Currently set to AndroidApiLevel19 This error pop up when I click on build button to make apk file

[@DineshPunni]

Check out my ARFoundation Setup Video for Android.

[@lavizpandey9957]

Check android 7 for minimun requirements. IT WORKED FOR ME

[@ramanandayadav7919]

Hi Dinesh In my iPhone face is not recognizing , watched video many times............ Is there any setting to do ? Please help

[@dopaminecat3867]

I'm player settings there is a option xr plugin management and go to arkit and enable face tracking

[@gonulsozu]

black screen appears and I have unity 2019 3.7 and I had to set package version 3.1.3 for everything

[@DineshPunni]

What phone did you use? Should be working with those versions

[@theramonjr]

@@DineshPunni I have the same problem with IphoneXR. Thank you for your videos they are awesome. Much appreciated.

[@shahrainjohan]

Help! When i launched the app it shows rear camera by default and doesn't overlay the material on my face

[@szadora]

Don't forget to insert the AR Default Face from the Prefabs folder into the AR Session Origin -> Inspector - AR Face Manager (Script) component Face Prefab section

[@ronweasley5573]

@@szadora i did all the steps but it still doesnt work n is in rear cam

[@ronweasley5573]

can anyone help me? I did the exact same thing but when i export it on my device it launches rear cam and doesnt even detect face :/ im crying its so simple yet complicated Android: redmi k20 unity: 2021, AR foundation: 4.x.x

[@DineshPunni]

Try using the same Unity version as the one in the video i.e Unity 2019.3.13f

[@aldinowildhankurniawan8175]

How to change to front camera ? I try it cant detect the face ineed solution? Device Samsung A51

[@ThePinnxx]

Did you work it out? I only get the outfacing camera like you

[@aldinowildhankurniawan8175]

@@ThePinnxx im still searching for fix it , i see tutorial in youtube the same tutor but still not work it

[@dhruvkhashu4389]

can you please help ive followed the whole procedure but after installing it on my phone it just shows a blank screen what should i do?

[@puneethsrisai]

same problem

[@puneethsrisai]

have you found any solution

[@dhruvkhashu4389]

@@puneethsrisai no sir, I haven't. it shows a black screen. I rewatched the whole video multiple times and even tried it on iOS using the other iOS tutorial eventually giving up on it.

[@dhruvkhashu4389]

@@puneethsrisai I was working on project last month when I needed to work check out my playlist if you're working on augmented reality project it has many videos might help you too

[@Harshkumar-ox6rx]

Hi Dinesh, I am using version 2020.3.16f1 , create a project android based face tracking, I flowed your video instructions after that build and run then no face tracking.

[@ronweasley5573]

same did u fix it?

[@DineshPunni]

Try using the same Unity version as the one in the video i.e Unity 2019.3.13f

[@gbengaoyewole7306]

Same here, any solutions yet?

[@ronweasley5573]

@@gbengaoyewole7306 Yes, I solved It by switching from Position and Rotation to Don't Care in the Ar Session Script.

[@ravieze]

Why is the video so fast. Why not in normal speed. Would it be possibke to record this in real HD resolution?

[@DineshPunni]

Not really sure what you're talking about lol

[@ekkapanrattanazoot6079]

Which course do enroll for face tracking ?

[@DineshPunni]

The closest one would be our AR Beginners course. insiders.dineshpunni.com/course/ar-course-title We don't specifically teach Face tracking but it has everything to get started with AR development

[@chetanamali956]

Can you make a series of how to make Car race game in AR Foundation?

[@DineshPunni]

Noted! :)

[@chetanamali956]

@@DineshPunni Thanks :)

[@PS-kn5lr]

Great explanation, U got one more subscriber.....Can you please make a video for feet tracking for android....I want to develop project for virtual footwear trial...there are many queries regarding this on unity forum....but no ones's answered any of it...we'll be really grateful if u do it

[@DineshPunni]

Interesting to see that there are a lot of people asking for this. Will keep it in my mind when thinking about new videos! And thanks for the feedback :)

[@jeffg4686]

Are they gonna be able to detect faces of people with headsets on? You know everybody's going to be wearing them. $500 is possible for most people, and when they see others out and about wearing them, you know they'll want one...

[@anjanaa0063]

Can we set this face as an image target?

[@DineshPunni]

Can you elaborate on what exactly you want to do?

[@Gajendra.Sukalkar]

HELP! in my case the camera just shows the normal faces

[@DineshPunni]

Make sure you did everything exactly like in the video. Alternatively, check out docs.unity3d.com/Manual/com.unity.xr.arkit-face-tracking.html

[@pranjalshikhar]

Was that SnowMan in the game dodging the Corona? XD

[@DineshPunni]

hahahaha good one 😂😂 Well I made the game 2016, so maybe I predicted the future 😎

[@vaibhuuanand]

Only black screen coming. Device Used: OnePlus 3T

[@vaibhuuanand]

Solved! Working Now! I had to set package version 3.1.3 for everything. Thanks!

[@DineshPunni]

Awesome! Glad to hear you could solve it :)

[@raffyvergara155]

can anyone teach me how to do this I will pay $200