How Face ID Works... Probably - Computerphile

Unlocking a phone with a face? Why doesn't the phone need thousands of example pictures before it works? Dr Mike Pound explains how it might work...
Face Detection: • Detecting Faces (Viola...
/ computerphile
/ computer_phile
This video was filmed and edited by Sean Riley.
Computer Science at the University of Nottingham: bit.ly/nottsco...
Computerphile is a sister project to Brady Haran's Numberphile. More at www.bradyharan.com

[@dermathze700]

I've never unlocked my face with my phone, but now I'll give it a try.

[@SpykerSpeed]

It hurts.

[@SuviTuuliAllan]

Gruesome!

[@gionio007]

Try rebooting if that doesn’t work

[@michaelpound9891]

If you listen carefully I also say you can unlock a face with your computer. The more you know!

[@masansr]

@@gionio007 Take off your boots and put them back on to unlock your face. That's a sentence I never thought I would write.

[@xaostek]

0:16 "If you want to unlock a face with your phone..." Alright, I'm interested.

[@ciarfah]

nitro You can do it, just do it forcefully

[@victorwesterlund4826]

there's an app for that

[@Freedom-js4th]

?

[@SinanAkkoyun]

@@victorwesterlund4826 it's called face book

[@anandsuralkar2947]

Lol same

[@necromancerpencil]

"Bring in Sean and everything's ruined." Classic Sean.

[@user-or7ji5hv8y]

This channel raises really interesting questions along with great explanations!

[@JJRicks]

Whilst talking about face ID, I find it fun that the video is exactly 9:41 long

[@hoagy_ytfc]

Kyles Isler FaceId only stores the profile on the phone, it’s not sent anywhere. And it’s also in the Secure Enclave so physically not accessible to untrusted software. I don’t see any privacy issue there.

[@lenn939]

JJRicks Studios What’s funny about 9:41?

[@adamzemek8337]

one of the questions is if in the future will kids insult each other saying: "you are so ugly that face recognition assigned you a negative value"?

[@LeMustache]

I like how only Max is a "negative value"

[@girl_frotter]

why are you the dude from bakugan

[@LeMustache]

@@girl_frotter Because he's a trap

[@adamzemek8337]

During deep learning software spend so much time on internet it learned how to insult people

[@DerUnbekannte]

mike consistently makes the best computerphile videos, great job once again!

[@MrDaanjanssen]

I see a "Computerphile the card game" coming out soon

[@TheNightquaker]

I do, indeed, like to unlock my face with my phone. To replace my internal batteries, mostly!

[@bill7445]

hehehehehehehehehe

[@CLHLC]

0:37 Shrug shoulders... 'Mike!?' | My new favorite answer to any question I don't have the answer to! Thanks Dr. Mike!

[@zss123456789]

7:44 best face I've seen Dr. Pound make!

[@moubhattacharya9370]

Yes, really

[@aamirbangash985]

This guy has some serious teaching skills

[@goranjosic]

Dr. Mike P. is so good at explaining stuff!! I have learn so much watching this channel in last few years!

[@Janokins]

You just have to watch out for your evil twin.

[@Noise-Bomb]

MichaelKingsfordGray nah that guy‘s a minion of 1/2 my scale... nothing to worry about :D

[@hoagy_ytfc]

Marc Grec Photo can’t break FaceID cos it builds a profile in 3D, using info from the depth camera. An identical twin can often unlock it though, and they advise sticking to passcodes if that’s a concern. I’m amazed someone would think FaceID could be fooled by a photo.

[@neumdeneuer1890]

David Stevenson well what about a 3d printed face of you with wig? ;)

[@xxportalxx.]

@@neumdeneuer1890 in fact I saw a news report on law enforcement using 3d printed faces to break into confiscated phones lol

[@martinlutherkingjr.5582]

David Stevenson The people who believe Face ID is likely possible to be compromise by using a photo probably didn’t know it uses a depth camera. Very plausible.

[@alterfun]

1:21 For all of the Shawns out there

[@animusfault]

Love the Mike Pound videos.

[@rich1051414]

What people fail to grasp is, the neural network is not told HOW to classify things, they are simply handed data and it figures out on its own the most optimal way to sort that data in order to produce the highest score, whatever that score is.

[@foolo1]

While "data" is the plural of "datum" in Latin and in some scientific English contexts, it is an uncountable mass noun in everyday English.

[@IceMetalPunk]

Well, yes. Although you might consider some of the simpler neural nets to be told, in a way, how to categorize based on the feature selection used. On the other hand, I think traditional neural nets have fallen into significant disuse lately, in favor of CNNs and RNNs, which do indeed discover their own feature sets and therefore aren't even philosophically told how to classify anything.

[@justawatchin2]

@K.D.P. Ross do you understand what is meant? If yes, this is fine.

[@justawatchin2]

@K.D.P. Ross aye that's fine

[@justawatchin2]

@K.D.P. Ross this is youtube, not an academic publication

[@TheDeIYIon]

These videos are great. They could be made even greater if you included references to the papers from which these ideas are taken.

[@himselfe]

Interestingly no discussion of how it differentiates between real people and images of those people. I assume it has something to do with the phone's ability to identify depth, but it would be interesting to see a discussion of photos/images as an attack vector for this kind of security.

[@Kas-tle]

I don't think it uses color values at all since it seems to work in the dark. I would guess that just the shape of an individual's face is unique enough when you think about it. Since I assume it's making a point cloud it could just store a point cloud of the phone owners face and compare any login attempts to that.

[@cauemorenokersuldecastroca2917]

@@Kas-tle I think he used depth as in physical depth, not data structure depth, which an image case is color. Indeed I had the same question, is it possible to take a picture of someone, which nowadays, with social media, is Xtremely easy, and use it to unlock the phone.

[@Kas-tle]

Cauê Moreno Kersul de Castro Carvalho well the actual hardware used for Face ID is a LiDAR scanner. Indeed there are a couple of apps you can get on iOS that let you use it as a 3D scanner.

[@apporvaarya]

Your accent,your knowledge and way to teach stuff is incredible.😍😍

[@dhvalden]

yey! Mike Pound!!!!!!

[@TheSam1902]

Daniel Valdenegro the best topics by the best presenter !

[@Bigjunior987]

Well, I could unlock my brother's phone in bad lighting for the first few weeks. Now it's not even possible for me. My guess is, that with each unlock using FaceID Apple allows less and less differences between the points in space you successfully unlocked the phone with.

[@firefoxmetzger9063]

"The chances of a person that looks similar to you stealing your phone are slim." 5 minutes later: twin steals phone .... mwahaha.

[@ptviwatcher]

What prevents someone to unlock a phone with the actual owner's picture printed in a piece of paper and taken from his Facebook or LinkedIn profile?

[@adamcetinkent]

It uses two infrared cameras and uses the difference between them due to parallax, so you would need the picture to be three dimensional.

[@sau002]

I like the question-answer approach

[@nicktheoregonian]

Would fingerprint sensors on phones or other security devices work in a similar manner to this? Those have been around for a while longer I think, so maybe not?

[@romakamal7564]

I got massive The Office vibe from this video

[@l0renzz0]

I could unlock my friend's phone with my face. Maybe that's why people think we are brothers...

[@NamFlow]

*I could unlock my friend''s face with my phone FTFY

[@hanniffydinn6019]

l0renzz0 do you look the same ? Twin like ??

[@btd5311]

are you brothers? maybe twins?

[@superscatboy]

I feel like there's a bit of confusion about cause and effect in that statement lol

[@shriyarahulmishra3559]

Thanks a lot I am forensics student this video helped me for my exams

[@AvetrusYT]

...and it will go: *shrug* - Mike? 0:34 Brilliant :D

[@citricdolphin]

I could be wrong, but I thought I remembered hearing somewhere that the estimated overlap is 150,000,000, meaning that for any one particular person's phone that uses Face I.D, there are 150,000,000 people in the world who could unlock that phone.

[@bakkasur9614]

so it cant find the difference between identical twins ... god i am doomed ... i have a twin sister ! ... lol

[@UnliVW]

There sre videos dedicated to this, it actually has a pretty decent success rate afaik differentiating twins

[@MisterMcHaos]

So, just use a finger-print scanner. :)

[@samuvisser]

Know that they have to look similar to a computer, not a human. This is actually a different thing. We don't notice subtile imperfections in the face or other very subtle changes. Computers do

[@martiddy]

I guess it combine both facial recognition and fingerprint scanning

[@Shinlung66]

I was thinking about this as well but i do not have a twin just curious :)

[@Rtong98]

Face ID uses a dot projector to see the 3D shape of the face. Thanks for the video.

[@thet0ast3r]

I have a slight problem with this video: It says "Face ID" in the title, but Face ID normally refers to Apple Face ID; which does not work with normal image recognition, but rather building a 3d model of your head, like microsofts kinect 1 did.

[@VanillaLibrarian]

It is mentioned in the video and doesn't really affect what is explained; the 3d model just feeds additional numbers into the system.

[@cauemorenokersuldecastroca2917]

Good video, I've only one question, how easy is it to unlock a phone with a photo of its owner? Is there a way for example to use the perspective change of the camera to diference between a 3D object, and a 2D reproduction of such an object? Otherwise it may be a huge security flaw.

[@tobiasschachner2434]

why does a Picture in front of the detecting camera not unlock the phone? does it recognize the shadows and somehow the third dimension out of a two dimensional picture?

[@Biela2008]

8:48 "that's a bit of a weird one...":D :D

[@Zi7ar21]

Inb4 it puts points on surface features and compares the relative positions

[@alwanosuarez9022]

Great content dan interesting topic! hopefully, someone is gonna make a subtitle as I was not a native English speaker, thanks!

[@pw7225]

Great explanation

[@Nagria2112]

favorite computerphile Host :D

[@Ayzklymbr]

"Maybe it's security through obscurity" SeemsGood

[@MJay_]

amazing video, but on a side note: Whats on the white-board? Please discuss!!

[@peterantonaros6461]

Steal a celebrities phone and bring it to a wax museum XD

[@n8style]

0:17 I love unlocking my face with a phone

[@tanzeel7ify]

Very very interesting ! great explanation!

[@dannyisrael]

But, does it work if you show an image of the person?

[@emperorcyber509]

Danny Israel Yes for optical facial recognition (just one camera). No, if you have a depth camera and additional sensors like the ones on the iPhone 11.

[@SamratDuttabdn]

Science is so advanced that now we can unlock our face with our phone!

[@big_matt3496]

Use say it is in 2D space. However with IR depth you can also place them in 3D space. This makes it MUCH less likely for a false positive and vice versa.

[@Zimpfnis]

Help! My face is locked and my phone has run out of battery. Please, somebody

[@Raffra]

"High value of 2 and high value of 1, whatever that means..." Oh, I think we all know what it means.

[@edwardchovaklinov180]

Using MTCNN you can just locate faces in whatever environment and then you can do recognition, nevermind the background.

[@avalacos]

Once youve solved the one shot problem, could you augment/combine this scoring with a classifier that learns to recognise your face? Also on the hair issue, would the algorithm be trained to use facial features that dont change, like your bone structure, rather than eye/hair features? Obviously then it could break if you got botox/surgery but its less likely than locking yourself out because you dyed your hair or are wearing coloured contact lenses?

[@Wargon2013]

I know someone who bought a new iPhone recently. I asked if I can have a look and tried to unlock it. It didn't work of course BUT I held the phone in front of its owner and it was unlocked. It did what it is supposed to do, unlock as quickly as possible if the right face is in front of it, but we were both surprised that it actually worked. So you don't have to be that one in a billion person that looks similar enough to steal and unlock an iPhone...

[@m10rober2011]

Wargon just out of curiosity, why were you surprised when it worked? I just got a new iPhone and would have been shocked if it hadn’t worked considering there are plenty of people out there who have sensitive data on their iPhones so a faulty security feature would have been catastrophic for Apple

[@Wargon2013]

@@m10rober2011 because the idea of face ID is to make the phone more secure. My first thought was that you could steal it out of of someones pocket, hold it in its owners face, it unlocks before the other person realizes what happend and then you run away. With a fingerprint or even just a password that wouldn't work.

[@GilesBathgate]

Assuming you can get hold of the algorithm, couldn't you just brute force it, and see what image creates what vector? Maybe the image possibility space and vector space are too large?

[@iliakorvigo7341]

The parameter space in any modern deep network is intractable for brute-force algorithms. In other words, without the training data and precise knowledge of the training procedure, you won't be able to reconstruct the parameters. And even if you had the dsta and the knowledge, you would still need to brute-force the rng seed, because deep network optimisers and the networks themselves are stochastic.

[@GilesBathgate]

@@iliakorvigo7341 so I was just thinking in terms of creating every possible image as input and enumerating all possible output vectors. Much like rainbow tables. But I guess thinking about it... A picture is way bigger than an 8-10 character password.

[@ZandarKoad]

Yes, what the "features" are near the end of the neural network are knowable. There are methods of extracting this feature data, manipulating and displaying it, so that we can better understand what is happening inside. But these neural network "features" don't necessarily correspond to anything we consider as human "facial features."

[@justpaulo]

1) I'm wondering if someone's phone will unlock with a picture of that person? 2) I'm wondering if #1 is true, even if you don't have a picture of the owner of the phone, if you still could unlock it by showing it say 1000 different faces that sort of cover a broad spectrum of face characteristics?

[@youssefelamrani7905]

which is better Viola-Jones or deep learning? because Viola-Jones i think needs face forward images

[@JadeNeoma]

Is this neural network feature sorting also done for fingerprints or are they looking for particular human-programmed features?

[@bluekeybo]

Gold, Mike is just gold!

[@mortisCZ]

Now I have this funny idea that there might be an ultimate artifact "the Wand of faces" that is able to instantly unlock any face ID in the world and there will be some caberpunk campaign about obtaining it or destroying it.

[@BenMakesGames]

can we do the google deep dream thing, and "reverse" these networks, to get it turn a scribble into a picture that looks like Mike?

[@ninocavalo]

How about just training the network on pairs of photos as input and same/different person as output?

[@hellfiresiayan]

Hi! I don't know if you all answer questions, but I was wondering if someone could help me understand why compilers are such a massive research area. Obviously I get that making them more efficient generates a boost in speed, but it seems to me that the best and brightest computer scientists work on compilers. Am I right? If so, why is this?

[@franchescoprotsh507]

I find the concept of using 256 dimensions to be most fascinating

[@JonahNelson7]

Yeah, I mean in these sorts of applications it’s really just a list of values, so a person’s 5-dimensional face measurement would just be like [3, 5, 0, 2, 5]. Visualizing it as a 3-dimensional box works, but it doesn’t have to be physical

[@1rikineverflinchclothing729]

I’m sure it’s correct that information is withheld for obvious reasons; however I’d like to point out that Apple did in fact release publicly a paper detailing Face ID and also how the Image signal processor works in relation to the CNN’s training implemented in Face ID. It goes into detail on how Probability Distribution is utilised to allow for temporary variance in facial appearance. It also discusses the method used for allowing changes in the tolerance for deviation from the current threshold for a positive result. If you really want to get into the deep dive on it I’d definitely recommend giving it a read; not to take away from this video I should note...just for further info :)

[@Noise-Bomb]

I can unlock my brothers phone :D Genetics is fun!

[@VanillaLibrarian]

But can you unlock his face with your phone?

[@adarshvemali2966]

What other options are there?

[@joethomas2478]

Love these types of vids from you guys

[@TheRealAndOnlyAndrewRen]

what about fingerprint scanners and other types of biometric authentication? (on-device, no fancy pantsy cloud database or anything)

[@Alexveliz0898]

You click on the ad and then you watch Computerphile :)

[@woodie07]

0:17 “If you want to unlock a face with your phone” 😂 never tried it before but I’ll give it a go

[@TorgieMadison]

First time I saw facial recognition security, I took a photo of the guy with my phone, brought it up on my screen, held my phone in front of his camera, and unlocked it.

[@m10rober2011]

Torgie Madison that works on systems with a single 2d camera. iPhones use an infrared projector to create a 3d model of the users face so a piece of paper won’t work

[@moccaloto]

Why is the number of dimensions a power of 2 ?

[@smirnovslava]

I dream part 2 with a talk on adversarial attack used for faceID

[@ypanso]

Can i ask what does it mean when you say it's done in 128 or 256 "dimensions"? What do those dimension stand for? Like each face has a little "box" that is its dime sion? Thanks

[@SalmanMKC]

Okay, from my understanding a 2 dimensional array is an array of an array, which can be visualised like a grid, then a 3 dimensional array is n array of a 2 dimensional array, which can be a visulised as array of grids, so that you have a cube, then a 4 dimensional array would be an array of a 3 dimensional array, an array of cubes, and so on, this continues all the way up to 128 or 256, the reason I assume those numbers are used is because of binary. So you have 2^7 which is 128 and 2^8 which is 256. Hope this helps!

[@veda-powered]

It’s basically a compression algorithm right?

[@Arkhanno]

If it's just checking for features then it raises the question of whether or not it can unlock from a photo of a person. If it can't then what about a video (assuming optimal circumstances) of a person?

[@HoldFastFilms]

No, it cannot. It does depth perception as well so a 2D image would not work.

[@hanniffydinn6019]

Well everyone has a different facial structure, so that's the metric of numbers of 3D facial points that it's using. Along with checking it's a live face not 2d or static face mask.

[@joshkindy4826]

thanks

[@musikdoktor]

and what about twin brothers?? can twin brothers can unlock your phone?? just a question..

[@windstorminator]

such an awesome episode

[@emperorcyber509]

Any code implementation of this?

[@shanematthews1985]

Unlock a face with your phone? Sounds like the dark arts to me

[@shineq_]

Is the 9:41 video length intentional, considering it's about Apple?

[@notreallyasloth]

Where's Brady?

[@hblaub]

What about twins? Can there be enough depth information to distinguish them?

[@masansr]

What about Deep Fake (or similar software)? If I fed that program, for example, this video, would it be able to unlock Dr. Mike Pound's phone? I bet it would, but how long it would take? Minutes? Days? Months?

[@satyavratwagle4336]

Depth perception takes care of that, otherwise just a photo of someone would be enough to unlock their phone, right?

[@masansr]

@@satyavratwagle4336 Do phones have two front cameras for depth perception or is it just simulated from parallactic movement? In the latter case a video feed would work. In the former - it isn't unlikely that it will be possible to do Deep Fake or similar on live video feed in near future (basically change your face with a program).

[@satyavratwagle4336]

@@masansr I can't speak for the Android devices, but Apple's Face ID system uses things called VCSELs, which act as the depth perception system. They throw out a mesh of thin 'cones' of light, and hence the further a point is, the larger the corresponding point on the VCSEL map is. A combination of both pixel maps from camera and VCSEL allow the phone to identify a face (and perform the 3D emoji gimmick). So in a sense, you're right. The sensor cluster on top of a phone contains more than 2 cameras, usually (for visibility in the dark, etc), but they aren't sensing images in the same sensing modality, ie not all of them generate RGB images. I see what you mean by the deep fake thing. The learning algorithm behind it is called a GAN (Generative Adversarial Network). It basically consists of 2 neural networks competing against each other; one to generate images, and the other to identify if those images were original or generated by the other network. On the face of it, it looks pretty simple, right? But to get a GAN to work requires HUGE amounts of parameter tweaking, architecture experimentation and training times. Not to mention an enormous training dataset which has images of you from every angle. And this is for 2 dimensions. In my head, if you had to do it to screw with face detection, you'll be adding another dimension, and later maybe 3D printing the head or something. So in theory, it's possible, but you'll need a whole lot of time, money and data to be able to unlock Dr. Mike Pound's phone :P

[@Yupppi]

Sounds like it's very clever until your brother opens your phone with no problem. And while you take one picture of face, you have to take like 8-12 records of your fingerprint for the lock. What makes counting on odds being low for a person with similar face stealing your phone concerning, is that while this technology is evolving, at the same time face generation technology has evolved. I wouldn't be at all surprised if you could generate a face with enough resemblance to the owner and either manage to trick the phone with just a print or if you're unlucky, with a mask. And I'll be honest about this technology, I tried it for a while with my new phone and quickly disabled it. Reason? Every time I grabbed my phone to check the time or if there's new message indicators, the phone unlocked me into main screen and I had to check the notifications separately.

[@psicommander]

Most likely it's a direct relative who is similar enough to unlock your phone rather than a hard to find stranger. This happened in China not long time ago

[@federicogiardini7436]

Is this the same way how the fingerprints are stored and handled?

[@musikSkool]

Or as Jeremy Clarkson pointed out when speeding in a country that needed to ID people in the driver seat to ticket them... Wear a printout of someone's face to use their phone.

[@lemonade2473]

He looks like Gabe from the Office, but facial recognition software would disagree with that.

[@kyoung21b]

So there must be some startup (not too visible, so to speak) that will guarantee that they can produce a “good enough” look alike for your powerful target of choice...

[@mrcake4525]

clustering?

[@thomaskaltfuss9024]

0:16 "If you want to unlock your face with you phone..."

[@prasadsawant1358]

can we unlock it using a photo of owner???

[@kopyiddish8576]

seems like word embedding ?

[@ChonGeeSan]

Does Apple use my picture to enhance it's database?

[@xja85mac]

Chongee they say they don’t do that

[@Einyen]

@@xja85mac so they probably do that

[@kdmjf12000]

@@Einyen Maybe you are being sarcastic, but if not it is actually not very doable with strong privacy law like GDPR in place (which has a specific section placing very strong restriction for biometric data). Unless they get user's consent before collecting they could be fined up to 4% of their global revenue which is too much for even Apple to ignore.

[@drew295]

This is not how this algorithm works

[@Peds013]

@@kdmjf12000 don't forget you sign up to letting them do all that in their terms, conditions and licence agreements. So yeah, they can.

[@colinmurray2976]

What if you have a twin they will always be able to unlock each others phones 😂

[@Nilguiri]

What if you show it a photo of the person?