Can you tell when someone is lying to you?

Someone is fidgeting in a long line at an airport security gate. Is that person simply nervous about the wait?
Or is this a passenger who has something sinister to hide?
Even highly trained TSA (Transportation Security Administration) airport security officers still have a hard time telling whether someone is lying or telling the truth - despite the billions of dollars and years of study that have been devoted to the subject.
Now, University of Rochester researchers are using data science and an online crowdsourcing to create a screening system that can more accurately detect deception based on facial and verbal cues.
They also hope to minimize instances of racial and ethnic profiling that TSA critics contend often occurs when passengers are pulled aside under the agency’s Screening of Passengers by Observation Techniques (SPOT) program.
“Basically, our system is like Skype on steroids,” says Tay Sen, a PhD student in the lab of Ehsan Hoque, an assistant professor of computer science. Sen is lead author of two new papers accepted for major computing conferences hosted by IEEE (Institute of Electrical and Electrionics Engineers) and ACM (Association for Computing Machinery). The papers describe the framework the lab has used to create the largest video deception dataset so far -- and why some smiles are more deceitful than others.
“A lot of times people tend to look a certain way or show some kind of facial expression when they’re remembering things,” Sen said. “And when they are given a computational question, they have another kind of facial expression.”
An advantage of the crowdsourcing approach is that allows researchers to tap into a far larger pool of research participants, and gather data far more quickly, than would occur if participants had to be brought into a lab, Hoque says. So far, the researchers have gathered 1.3 million frames of facial expressions from 151 pairs of individuals (dyads) playing the game.
Data science is enabling the researchers to quickly analyze all that data in novel ways. For example, they used facial feature analysis software to identify which of 43 facial muscles were being used in a given frame, and to assign a numerical weight to each.
The researchers then fed the results into the University’s supercomputer, using a machine learning technique called clustering, which looks for patterns without being assigned any predetermined labels or categories.
“It told us there were basically five kinds of smile-related ‘faces’ that people made when responding to questions,” Sen said. The one most frequently associated with lying was a high intensity version of the so-called Duchenne smile involving both cheek/eye and mouth muscles. This is consistent with the “Duping Delight” theory that “when you’re fooling someone, you tend to take delight in it,” Sen explained.
More puzzling was the discovery that honest witnesses would often contract their eyes, but not smile at all with their mouths. “When we went back and replayed the videos, we found that this often happened when people were trying to remember [something]” Sen said. “This showed they were concentrating and trying to recall honestly.”
So will these findings tip off liars to simply change their facial expressions?
Not likely. The tell-tale strong Duchenne smile associated with lying involves “a cheek muscle you cannot control,” Hoque says. “It is involuntary.”
“In the end, we still want humans to make the final decision,” Hoque says. “But as they are interrogating, it is important to provide them with some objective metrics that they could use to further inform their decisions.”
Md Kamrul Hasan, Zachary Teicher, Minh Tran, Matthew Levin, Yiming Yang, Kurt Haut, and Raiyan Baten - all students in the Hoque lab - also contributed to the research.
Students featured in the video (in order of appearance): Diana Maloku '21, Kurtis Haut '18, Shagun Bose '20, Manny Rodriguez '18, Ethan Wei '19, Melissa Kagaju '20.
Subscribe to the University of Rochester on KZbin: / @universityrochester
Follow the University of Rochester on Twitter: / uofr
Be sure to like the University of Rochester on our Facebook page: / university.of.rochester
Help us caption & translate this video!
amara.org/v/BB...

[@jm-holm]

Got it right, too exaggerated, too much "pretend confident", too much movement of eyebrows etc. Whether it actually means anything is a different matter but I guessed correctly.

[@Wiberg2000]

Same here!

[@sorryididntknowwhattochoos2678]

Me too but wasnt this one too easy?

[@collinpeel3438]

Same. His problem was that he tried to force the facial expressions. He should have relaxed or method acted.

[@chopitup9950]

I got it too!

[@wonderingobserver2231]

I seen his eyebrows flinched when he said never

[@sksigil]

Every single one of them is being deceptive. The blonde guy was obviously the liar - he was enjoying being deceptive and nodded his head yes while making a negative statement. The rest were trying to hide they were telling the truth. Lying is complex, for a number of reasons. If you tell someone to tell the truth while pretending to lie, they are still being deceptive.

[@DarkEagle01]

right on. when i saw the first 3 my brain started screaming deception, but it wasn't until i analyzed their fabricated facial movements that i realized they were intentionally attempting to appear untruthful.

[@truth-ep5dz]

Actually you could tell by his uneven eyebrow expression, very simple

[@stevensavoie856]

I loved the angle that they were all deceptive, but I do disagree a bit on the "nodding". It wasn't an affirmative nod, it was a batonning gesture he did with his head. Meaning it was an illustrator he used to add emphasis. It was well synchronized with his words and came across as truthful. What happened after was more telling. He went into a bobblehead motion as he was stuck between nodding and shaking his head no for the rest of the answer.

[@stevensavoie856]

@@truth-ep5dz Correct, and you might be aware, but for others: this only works in this experimental situation when we know there is a liar. Theres enough absolutists running around saying "You lied because you did . . ." :)

[@deandred712]

I got it right too I didn’t even realize the head nod though I just noticed he seemed to have a slightly contemptuous smile after so I assumed he was getting off on deceiving us

[@newcastlemusicstudios]

Dear Rochester, when doing a test, it doesn't hurt to allow the viewer to see their face for more than .5 seconds.

[@Nonamelol.]

Exactly

[@stevensavoie856]

Yup. No baseline and abrupt edits. This was a very unfair test, but thankfully the liar was leaking stress like a cloud leaks rain in a storm.

[@deandred712]

Well it real life micro expressions last 1 to .5 seconds so in this case while not essentially helpful is more practical making it more beneficial

[@akufan]

@@stevensavoie856 uh, no!

[@dcworld4349]

@@stevensavoie856 In a normal conversation you have more of a chance to get a baseline for sure, but I would also say you are not always going to get access to the best conditions of telling if someone is lying or not. I think they probably were told to stare directly into the camera and give their statement. The reason you can tell that he's lying is because he's the only one who's top half of his face does not match what the bottom half is saying. He's concentrating on looking directly into camera would not be suprised if he was told to not move his eyes. So with those directions in mind. His face does the only thing he can't control, the muscle's around his eyes and his eyebrows, which if you go back, the right goes higher when he lies. Everyone else was completely deadpan and natural, I'm not able to tell exactly which micro expression he's showing, the closest I'm able to read from his expression is suprise. Which I don't understand why he would show unless they are not saying a prepared statement and he's reading from a teleprompter.

[@Clinty727]

Got it right because of his duping delight. Others smiled after their statement, however his was uncontrolled.

[@2DarkHorizon]

Perfect phrase for it "duping delight"

[@slimewxlf70]

Also said I have never which would be purposely putting distance. And his face wasn’t symmetrical.

[@markrobertson2052]

How do you know it was uncontrolled

[@deandred712]

I too noticed tht slightly contemptuous smile after he lied

[@deandred712]

@@slimewxlf70 how do u see tht as putting distance

[@raketnight]

there was something off about the way he said "never"

[@asnider3155]

Yeah, that was MAD tell for sure

[@wonderingobserver2231]

He flinched when he said it

[@gabrielhenriquesilva2014]

Totally

[@LordSeethe]

Way too over pronounced

[@akufan]

That is correct! See my explination...

[@adamm5205]

Damn facial expressions and actions tell a lot about a person. The way he emphasized his phrase was the giveaway.

[@akufan]

Uh, NO!

[@leroybabcock6652]

Yes I caught him. I thought I would surely fail but it was actually very clear.

[@Random-JustAnother]

I feel like he made it Way too obvious it was him. Most liars aren't Trying so hard unless they're really bad at it, but either way, he had many tells. His eyebrows, his eyes widening, even the look in his eyes, his inflections in voice, and lastly, his 'Dupers Delight' smile.

[@s_cuzz]

i was thinking like " no ,he is just trolling us so we will not choosehim, its TOO OBVIOUSE' .This test is shit.

[@akufan]

Uh, NO!

[@RainTheUpriser]

So how are you supposed to study their faces when it switches within seconds to the next persons face, it’s too much information all at one time

[@stevensavoie856]

Yeah I agree. We definitely have to press pause to get anything.

[@deandred712]

I’d recommend using a video or book that actually go into detail about micro expressions and body language etc and use this more as a tool to refine the skill because in real life micro expressions typically last 1 to . 5 seconds so likely done on purpose out of practicality but not useful rlly to someone wanting to learn more

[@gigi9287]

So the ugliest smile wins. Got it.

[@hazel8882]

hahahaa, yeah he got a creepy smile

[@akufan]

Uh, NO!

[@AV-bu5rw]

I thought he smiled that way so we guess it was him, and I didn't pick him

[@LurkerPatrol5]

The fluent in 3 languages girl was so overconfident to me that I guessed her first. Then the girl with 5 different pets had the worst smile early on in the game because she looked like she was trying to laugh and I was creeped out by it. Then finally the guy claiming he hadnt been outside of the country. So I thought all three of them were lying and that the 5 pets girl was going to murder me in my sleep.

[@dulapeep8895]

💀💀😂😂

[@wolfie1027]

She’s knows three languages badly, she meant

[@jockous]

The answer is no. You can’t tell whether someone is lying or not by their expressions.

[@Dionne_Clelland]

Soo happy that I got it right! Seen the guys microexpressions with his wee up eyebrow movements when he told the lie and was like HIM!

[@akufan]

Uh, NO!

[@flatways8502]

I could tell he was the liar because I just got lied to by someone whose non verbal communication was exactly the same.

[@madraccoon5934]

I'd like to add that although I got it right, it was still pretty hard to tell... there were 3 I was sure were telling the truth. The dude from shanghai, the girl with the pets, the girl who never broke a bone were easy to spot as truth tellers because their delivery and nonchalant attitude. The guy who's author was the next off my list because he delivered in pondering who his favorite author was, which doesn't mean he is lying but rather thinking about it by his eye movement. The girl who could speak many languages I was a little unsure about, but reconciled with myself that her smile seemed more genuine and a little forced, but not nearly as much as the guy who said he never travelled outside of the United states. Along with the fact his eyebrow movement and his delivery was way too forced with a directness and cut words which gave me social queues he was the liar.

[@airwolfcentral169]

I didn't even consider the guy who took a second to think because an authors name requires a second of thought. If he was asked whether he committed the murder and that was the reaction then i would think differently of him :)

[@deandred712]

@@airwolfcentral169 good use of context in an interaction n of said expression that’s an important thing to keep in mind something I should work on 🤷‍♂️

[@airwolfcentral169]

@@deandred712 body language is interesting

[@deandred712]

@@airwolfcentral169 that it is

[@akufan]

Uh, NO!

[@fengxiong520]

It was the way he said "𝘕𝘦𝘷𝘦𝘳..."! It was super sus!!

[@Khiarika1]

They're all actors, it's impossible to tell with actors because on some level they're all lying. The only one i could tell was telling the truth was the sad looking guy in the gray because he was trying to hard to look like he was lying.

[@up626]

didnt even watch the video and i could tell who it was xD

[@Guywithair2]

I guessed correctly based off the way he delivered it. His vocal pattern seemed to change.

[@carlathompson9308]

He stressed the word never which tipped me off

[@ees3155]

the way he said it was so rehearsed, that was easyy

[@calipachanguero]

Smiling white dude looks like he is trying to sell you Daewoo Lanus and swears everything works.

[@shauna1147]

it was his inflections as well! i wasn't 100% sure, but the blonde guy was my first choice!

[@HankTheTank23]

I honestly didn’t even pay attention to what the actually said (I couldn’t remember what the lie was), but my gut told me it was blonde guy. He just seemed too enthusiastic about what he was saying. Upon rewatching it, you can tell how much emphasis he places on the word “never” whereas the girl immediately after him passes over the word “never” like it’s just another word in the sentence. He was overcompensating for the part that made it a lie, which ultimately gave it away.

[@-ThePREACHER-]

*I NAILED IT!!!* 11 years selling cars... 2 years doin repos...... I can spot a liar from a mile away...

[@AngelKhalidah]

I was correct. I understood he was lying because he was shaking his head while saying the fact which was lie.

[@akufan]

Uh, NO!

[@lilgit9969]

I guessed correctly because his face was more animated like he was trying to convince me of something.

[@akufan]

Uh, NO!

[@wonderingobserver2231]

The way told he was lying is his eyebrow twitches

[@ishogunronin6626]

I guessed correct. It was the smile...it was almost like he was trying to convince me he was in a happy mood and was not in feeling stress. The smiled just seemed over exagerated.

[@Deedee-jc1ch]

I usually go, " only if I have one " and then I go, " that was weird " and leave the building.

[@stevensavoie856]

Got it! I am fully convinced that any expert in non-verbal communication and deception detection would spot this easily 100% of the time. I got out a notepad and drew each person and made notes about what I saw. I took the approach that all, none, or some could be lying. I would use a lot less time on this if I felt guaranteed that there is only one. Spoilers: By the end, I told myself: "I would never try to guess whether or not one is lying based on so little information... except one." That person's page was filled with stress markers. His nod was congruent with what he said and it was on time. It suggests emphasis. That was the only thing not remarkably deceptive. The rest of his head movements were bobbling motions. He couldn't decide to nod or shake his head. This happening during a strong denial makes me take notice and look closer. His initial eyebrow flash was contrived, not like the first girl who did a genuine eyebrow raise. Raises one brow as he talks. It could be in his baseline, but the strong denial combined with everything else he does, signals a likelihood of deception. His strong denial was a 'non-contracted denial'. "I have never" vs the most commonly used honest denial "I've never". Finally, you can catch a lip compression just before the footage cuts. Lip compressions are normally interpreted as withheld information or withheld opinions. If he displayed only one or two of those stress markers like the first girl did-Her right shoulder was way high and her left one was very low and her arms were tight against her body-I would basically be forced to guess with little info. All of the stress markers combined (including the one verbal sign of deception) told me that regardless of the others, he is definitely lying. I was initially annoyed. Each clip is so isolated, the camera cuts immediately after they speak so I can't read their post-statement body language. Even if I did, I wouldn't have a baseline to compare it against. This was a very unfair test, but thankfully the liar was leaking stress like a cloud leaks rain in a storm.

[@akufan]

Uh, NO!

[@MortusSweet]

@@akufan….but they were right?

[@thomaseden-dy2kk]

One other giveaway for the guy is his pause in his answer

[@briannadboyd]

Not me clapping like I won an award because I got it right lmao

[@zemlya378]

a real liar would not act like that

[@Mr2blue2]

How do we know he is telling us the truth about lying?

[@Ruzakiii]

I will explain how I discovered the liar: The first girl clearly English is not her mother tongue, so she is actually fluent in more than one language. The second person, from his appearance and facial expressions, appears to be telling the truth. Third, if she wanted to lie, she would say it simple lie , and because she is confident, she can say something strange like that with ease Fourth, since he was gathered with people of different nationalities, it is clear that he did Fifth, he did not think much before answering, and it is clear that he is making us believe that he is a liar so that we doubt him

[@wonder7798]

Yahoo, I got it right. It wasn't his smile. It was the way he spoke, his tone, and his emphasis on the word choice

[@boris.teoharov]

I guessed him right. However the first girl is lying, too. She is not even confident in talking in English.

[@lan2039]

Boris D. Teoharov bro fucken same lmfao she was throwing me off lmfao

[@ninawillow2429]

In my opinion, the first girl enjoyed playing the game. She had a game face on. Challenge me if you dare. I think it would show more if they said more statements.

[@mohammadharouak7890]

She didn't specify which languages she's fluent in.

[@johnrainsman6650]

I want to get y'all's opinion about something. I think a manager at my job lied to me about not trying to scold me. The context is a long story, so I'm gonna start with this first. Okay. I made some mistakes, and she and my boss had me in the office, and the first manager lectured me. There was no dignity, respect, or patience; she made me feel like an idiot. She even said to my boss, "Is there anything you want to add." And then the next day, she said "How are you?" Patronizing, I know. Well, I gave her an apology letter for my mistakes, revealing I had ASD and wanted to "redeem" myself, and looking back, I realize that was unnecessary and self-degrading. We talked, and I think she sugarcoated her side. She said she wasn't trying "to give it to you" while wagging her finger and expressing an angry face. I don't know. The evidence in the office contradicts her claim. She also said she was just trying to help me be successful. Oh, so in other words, I'm a failure without her help? She claimed she was trying to make sure I understood...what was right? I can't finish her sentence, because I'm already forgetting her words. All I know is it included helping me understand. She sounded patient and understanding about my challenge in life, but where was this in the office? You can't just talk to someone firmly and without dignity and then act wary the next day. Now, despite not writing the context of my mistakes and the lecture, do you guys see lies and patronization on her side? I'll explain my secret side later, but let me say first, I own my mistakes. I don't want anyone thinking I'm making her the one and only bad guy.

[@ShelbyLimited2006]

He had humor it gave it way..... Shanghai guy thou was sus with his poker face

[@daniellamasimango7399]

Nah this test was a failure. The blond guy was too obvious and the smile was too fake. I lie with a straight face and a normal smile TF 👀

[@Vishu-w9e]

One who lied was so obviously..... Don't know why but some part of me got him

[@jenniferbillings9825]

"dupers delight"

[@anmol._.saggu007]

I thought it was either 1st or last person. Cause 1st one is giveing fake smile at last. And last one starts thinking about their favaurite writer.

[@samiratamma]

ok to be honest, I forgot their faces because of the montage after montage after montage. after montage. So it's like...i barely remember their faces, but i remember thinking the blonde is sketchy af....but like...damn rochester y'all need to chill it with the editing.

[@romulogrimaux]

You can tell if you're lying if someone is saying something i don't then it will be i did

[@jamster2403]

The faces did nothing for me, I guessed the blonde guy because of the way he said "NEVER". Almost like that was the most important part of what he was trying to tell me.

[@akufan]

You are the only one who got it right so far!

[@adean4146]

the liar sounded like a politician, so guessed him.

[@spacetiime4786]

I thought the fourth guy and the sixth guy were lying right off the bat but was surprised that my guess with the fourth guy being a liar was right. However, I didn't really notice his facial expressions/difference from the others until I replayed it again. I figured it was due to the inflection in his voice being "more" than the other 5...

[@akufan]

Uh, NO!

[@elinberglund9824]

I got him because of how he flinched with his eyes

[@itsMalma]

the eyebrow was almost challenging us imo

[@bettersteps]

This video is about a minute long, there are six people and no baseline behaviors have been observed. If you guessed correct, it was luck.

[@Burzumlegacy]

Lying ain't a Crime, Hiding The truth Is.

[@9MMina]

I mean sure, blondie boy's smile was weird as fuck and selaed the deal, but he already made it clear he was lying. The way he talked: putting extra strength on NEVER and UNITED STATES, when everyone else was able to say their truths comfortably and calmly. He also chose to look intensely into the camera while the other people were relaxed. It was obvious, but that smile just hammered it in.

[@akufan]

Sort of! But correct!

[@lemariescorner]

I got it right. His eyebrows gave it away for me and the way he was blinking his eyes

[@Wtahc]

blonde guy was too excited

[@badjemima]

Too easy - so many of these "tests" involve shit acting.

[@savyabanga9500]

I guessed it correctly because his body language was uneasy

[@asnider3155]

I got the right one! I don't know why that was so satisfying but it was.

[@akufan]

See my explination...

[@ruru19991]

I knew who is telling the lie 😂😂

[@akufan]

Uh, NO!

[@soreal4that]

the way he emphasized "never" was unnatural imo idk💀

[@shirin1642]

For a second I thought it was new How to get away with a murder promo

[@giovanny1062]

All of them we’re lying because this is just a script

[@Seraphim7]

I guess correctly! The smile just was insincere.

[@akufan]

Uh, NO!

[@King-dsb]

I got it right because his smile was hold out for to long

[@itzjustabby]

the eyebrows gave it away 🤦‍♀️

[@cherryasmr2779]

the blonde guy just showed too much expression for his statement

[@Elite3kampioen]

I was already right after seeing the thumbnail ... xD

[@aakashbhardwaj476]

I caught him because he raised his eye brows. On one side slightly higher

[@SweetTeaTvTeaSippers]

Please don't tell me the Black Girl was lying

[@youssefhossam3941]

Is it me or is that dude’s smile really creepy?

[@spunkyvlogs5365]

Got it. He was raising his eyebrows a lot

[@johannalopez9972]

Easy, he nodded.

[@tristan4175]

i was correct. he was exaggerating his facial expression which made them look unrealistic.

[@petergromlich6529]

All I needed to see was the forced smile.

[@artsHscience]

This is how I knew the girl from Majory Stoneman Douglas High that said there was a second shooter was lying. She did a broad, full, 'I'm getting away with it' smile while retelling her 'account'.

[@akufan]

Uh, NO!

[@jjrumia]

didnt guess right, checked again, the blonde guy was confident and his frequent eyebrow moving, he couldnt control himself thats for sure

[@odiagam5755]

In reality everyone could be deceptive, its not something hard to do ... the problem here is that the weight of the lies is not that important to us and there is no need to future investigation even though we know each other 😂

[@marcelluswallace4076]

For me the blink at "never" gave it away. Although I had to watch it multiple times.

[@akufan]

Uh, NO!

[@airwolfcentral169]

I guessed either prince charming blondie or the girl who said that she never broke a bone. After much scrutiny it was clear that the girl who said that she never broke a bone was moving in the seat shifting her anchor point which can be a sign of deception. I thought the blonde guy was too obvious and was just a trick.

[@akufan]

Uh, NO!

[@richardcranium6081]

what I am trying to find, is something about people who tell crazy stories about themselves that no one believes, but they do it to impress people. I once knew an out of work guy, who was by no means dumb, but claimed to #1 be a professional ninja #2 was a former FBI agent #3 personally knew Marty Robbins (singer/ musician) and oh yeah #4 he was a professional stunt man. I also knew a chief petty officer in the US navy who was about to retire that lived in base housing apartments with his family that #1 used to be a police detective in San Diego #2 owned 300 acres in central Utah where he always hunted and #3 was married to the woman who owned the famous Koa restaurant. The frequency at which these people tell these cockamamie stories is a pretty good sign to me that at best, they are HUGE exaggerations. Through my travels, I must have met 12 people or so who were directly related to Jesse James (the famous 19th century outlaw). I met lots of people who lived in haunted houses (I'd been there several times and saw more paranormal activity in the backseat of my car (none) than I ever experienced in their houses) and of course don't forget many people I met who were biologically related to famous celebrities and who's families were filthy rich, driving around in cars that were on their last wheel and on the brink of breaking down because they were very old rust buckets. I mean, cmon, why can't I find any reference to this behavior online? Does this not constitute the kind of lies that after a while, the tellers may actually start believing even if no one else does?

[@shaneip2747]

Bro im 13 and i can spot these things after minutes of psychology books That blond person was lying because of the over-exaggerated facial expression and unnatural movement

[@camihl3841]

Lmao ok

[@davadu2256]

Got it right the right eyebrow, head movement anf facial expression gave it away

[@kingrandal01]

I got it right wow. It was his eyes he looked left and the only one who did.

[@MinMin-nn5ex]

Give me the answer

[@lampogriz7993]

First i thought about the last guy, because he moved away his eyes. And then on second watch i've noticed that blonde guy contrasted with others with his strange smile and brow moves.

[@odesangel]

This experiment is fundamentally flawed. For the person who was lying, their intentions are to be deceptive for the sake of being deceptive with no actual motive for them to lie other than for the making of this video. The lie is trivial and arbitrary. The best way to explain how to spot lies would be police interrogations and news interviews with people who were later found guilty of committing some crime.

[@Bohemecrochet]

I actually guessed it. His body language didn't align with his words :)

[@Lyf-3276]

I only did this to check my intuition and I got it right! 😅

[@PalmPier]

I WAS CORRECT

[@melinakorovessi713]

he was so obvious bro

[@Renew55574]

People lies thats all i hear 24/7 its kind of sickening that they cant be there authentic selves that they need to mascrade as someonelse or another individual that is authentic.

[@gunclebenny]

This is extremely invasive.

[@Johnadams20760]

i thought it was the guy in the blue shirt. he seemt to sit there thinking too long and looked up then back.

[@bobomo9107]

dang i thought he was being too obvious and trying to trick me to thinking he was lying i need to stop overanalyzing stuff

[@okije]

i thought all of them were lying

[@yoyo12314]

This could have been good if they weren't acting....

[@lalizcrisostomo8905]

I got right ! I see his eyebrow raise a bit and his awkward smile. 🙂 and he blinks often than usual

[@akufan]

Uh, NO!

[@2DarkHorizon]

I got it right because the guys voice sounded to have spikes in change of tone.

[@laciehamblin8584]

His eyebrows moved