The Kuleshov Effect - Improving Character Interactions in Games - Extra Credits

The Kuleshov Effect first came from the world of film, but it has fascinating applications in video games when players interact with non-playable characters. Context, juxtaposition, and emotions can be used in gameplay to create psychological fidelity in addition to the technical fidelity of mocap, voice acting, and artistic design.
Hitchcock Clip:
• Hitchcock Explains the...
The Only Podcast About Movies:
/ onlymoviepod
Get info about all our shows at becausegamesmatter.com
Subscribe to Extra Credits for more episodes every Wednesday! bit.ly/SubToEC
___________
Get your Extra Credits gear at the store! bit.ly/ExtraStore
Thanks for participating in this week's discussion! We want you to be aware of our community posting guidelines so that we can have high-quality conversations: becausegamesmatter.com/extra-...
Contribute community subtitles to Extra Credits: kzbin.info_cs_p...
Talk to us on Twitter (@ExtraCreditz): bit.ly/ECTweet
Follow us on Facebook: bit.ly/ECFBPage
Watch us play games and have fun on bit.ly/ECtwitch
___________
Would you like James to speak at your school or organization? For info, contact us at: james-portnow.com
___________
♪ Intro Music: "Penguin Cap" by CarboHydroM
bit.ly/1eIHTDS
♪ Outro Music:
“Da Funk Do You Know ‘Bout Chip”
By: That Andy Guy
ocremix.org/remix/OCR03620

[@extrahistory]

Finding the mechanics and systems that provide believable emotional context is one of the challenges of the next generation of games. Because if the Kuleshov Effect teaches us anything, it’s that creating believable characters doesn’t solely come from the performance of the actor… part of it comes from the audience itself.

[@stretopovermind9680]

That's all well and good, but no amount of "audience belief" will help you when your climax is "choose the colour of final explosion; there are three options".

[@somethingtothinkabout2446]

Hey extra credits, I have a suggestion. So I have these two cats and when I look at them, I wonder how they got here. I don't mean the reproductive cycle of felines, I mean how did humans create meaningful relationships with different species of animals, from the birth of civilization onward? Simply put, would you consider doing a series about the history of domestication and how pets and livestock changed society forever?

[@neon6341]

LOL i thought the second time he was going to smile to a bruning hous like this one meme XD

[@Ani-vs8gm]

Make a new series Extra Movie tricks kind of like Every frame a painting youtube channel

[@kingofnova673]

the link dead

[@AlexReynard]

This actually goes a long way to explaining wrongful convictions. When you are primed to see guilt in a person's actions, anything they do in reaction appears to be guilt.

[@crackedemerald4930]

Or how if you are told someone is insane, everything they do will make them seem insane, even if they just do regular stuff

[@emptyother]

Newspaper article: "Man claims innocence." Illustrated ad to the right: "Buy quality knives for your kitchen today!" I leave you to draw your own conclusion to wether he is guilty or not. :P

[@barditheweird6062]

It's more like the confirmation bias

[@ggwp638BC]

Great example of this: Danganronpa THH. (SPOILERS) Danganronpa is essentially a game about a Battle Royal where one person needs to commit a murder and not get caught. At some point, you are lead to doubt one of the main characters, who you probably spent a lot of time talking to and who have helped you all the way. For me, this was quite the roller coaster of going "no fucking way it's her" to "wait, it's not her, right?" to "please don't be her". Truth is, her actions never implied she was the culprit, she was innocent from the get go and was likely the one character who would never betray you - but, once you have one piece of evidence that puts her as a possible culprit, your brain starts to add that context to everything she was doing. Suddenly, anything she did that would be considered neutral became highly suspicious. And it's even better when that ties to the over arching dichotomy of the game.

[@DaizoDeeVonOfficial]

And this is why "out of context" compilations are often really funny.

[@Grz349]

It’s like the sentence “I never said they stole my pencil”. The meaning of the sentence changes depending on which words we put an emphasis.

[@Bibli_Cal]

Care to explain? I'm curious

[@NoNameAtAll2]

@@Bibli_Cal I *never* said they stole my pencil I never said *they* stole my pencil

[@Bibli_Cal]

Ahh I see. Thanks mate!

[@thezdude8512]

*I* never said they stole my pencil. I never *said* they stole my pencil. I never said they stole my *pencil.*

[@partyjams]

This made go through each possible emphasis. It’s great example

[@jonathanfaber3291]

"Temmie will remember this" Is a complete power move and I am now intimadated

[@guwuzmania]

intemmiedated

[@jonathanfaber3291]

@@guwuzmania !!!

[@imveryangryitsnotbutter]

"You will regret this."

[@kinocomix]

*temmie intensifies*

[@takebacktheholyland9306]

Hi im bob

[@brettd2308]

The Kuleshov Effect was *the* most important thing I learned in film class. Blew my mind when I was just starting to study this stuff. We watched the Hitchcock clip linked here and a few other examples, then we had a reality show editor come in to demonstrate how they'd use this effect to edit together conversations and arguments, using completely unrelated reaction shots and snippets of dialogue in order to manufacture storylines for the "reality" show. I've never been able to look at edits the same.

[@j.87558]

Yeah, that blew my mind too. It's Fascinating! Another think was the power of montage, basically: 1+1=3. Images together create something more. Vertov's Man with a Movie Camera comes to mind. Hey, I would love to know more about how reality show editors think and create their work.

[@the8flo1]

Just noticed that this is the core concept behind memes: If you look at a meme it's text on an image and they often (especially the really good ones) completely change the meaning of the image with that text

[@nathan1027Y]

I believe The Kuleshov Effect also works at many memes lol

[@brettd2308]

I agree, I think it's what makes them function as well as they do.

[@TheLegonaut]

Drake meme

[@the8flo1]

The Kuleshov Effect is basically the concept behind memes.

[@Tesseradical17]

The Unsettled Tom meme is all about recontextualizing normal actions.

[@legendarytat8278]

And cats. The reason cats are so popular is that we can project emotions more easily on cats than other animals/pets.

[@andrewphilos]

"X will remember this" is the perfect example of that neutral face you mentioned with regards to the Kuleshov footage. Because, like, duh. Of course the character will remember something they just saw. But just that line changes drastically depending on the context: I did a bad thing. -> X will remember this. -> Oh no, X is judging me! I did a good thing. -> X will remember this. -> Yay, X thinks I'm cool! I made a hard choice. -> X will remember this. -> X understands my struggle. I made an easy choice. -> X will remember this. -> ...wait, was that an important choice? Crap! I help X. -> X will remember this. -> Aha, X is gonna pay me back later. I hurt X. ->X will remember this. -> Uh-oh, X is gonna pay me back later.

[@oscarvinicios]

Amazing example

[@DeusExRequiem]

This is the bread and butter of visual novels, whole games of the same images, where context alters the meaning of any given gesture.

[@Rolaran]

The Phoenix Wright series did a great job using this effect to get a lot of mileage out of a surprisingly limited set of character sprites. Based on dialogue context, variations in timing and pacing, and auditory cues, the same change in animations could be given a variety of meanings. Edgeworth switching from his "neutral" expression to his "confident" pose, for example, was used for him expressing pride in his work, satisfaction with a witness's statement, and amusement at Phoenix seemingly choking under pressure, with no variation in the sprite itself.

[@PabloTheDinamic]

Dude, the music and sfx on that series is masterclass

[@itsallinthesoup2543]

Thomas was alone is a great example of this. Each character is literally a rectangle or square, but they're brimming with personality

[@thegodofalldragons]

That Hitchcock thing should be an Internet meme.

[@tomjackal5708]

be the change you want to see in the world

[@CFilmer]

Don't let your memes be dreams.

[@markguyton2868]

The fact that games can make me concerned for the livelihood of squares proves this effect honestly...

[@alexanderstuart7801]

To be fair, the "X will remember this" completely loses it's power once you realize it's just thrown in there for effect.

[@shojodraws3399]

Now imagine a game where they ACTUALLY remember.

[@Izandaia]

4:18 That is an EXCELLENT visualization of the importance of context. Bravo.

[@pcachu]

Pizza on plate == nom nom happy dinner times Pizza on manhole cover == this is so sad alexa play despacito

[@AlexReynard]

And why it's so effective for assholes to attempt slandering political enemies through use of out-of-context quotes or images.

[@Atariese]

@@pcachu i was thinking TMNT for pizza on manhole... but that's yet again an example of context.

[@pcachu]

@@Atariese ...holy cowabunga, I had somehow managed to completely lose that association from my mental rolodex. [insert mind-blown gif here]

[@sharif47]

[Insert Name] will remember you. [Insert Name] dies moments later.

[@Lady_in_Yearning]

Ah, The Walking Dead... Good times.

[@kahlzun]

[Name] had your actions as their last thoughts.

[@PersianMapper]

[Insert Name] will remember you... Will you remember [Insert Name]?

[@TheMunchkin9]

[Insert Name] would have remembered this if you had bothered to keep them alive.

[@sharif47]

@@TheMunchkin9 or maybe [Insert Name] dies either way because... plot.

[@15098D]

Russia: *Having a Civil War* Kuleshov: It’s movie time

[@mixiekins]

Same reason "blank slate" player characters are so preferable to play as, they're easy to project one's self onto, so it's more universally applicable.

[@TonsOfHunStudios]

Good point! If you assume emotions/interpretations, it's hard to immerse yourself into the game.

[@reddir]

Nice. I've recently been thinking on how the reason I don't play many games anymore is the lack of emotional engagement - specifically emotional reciprocation, but any increase in the emotions a game can engage will be a big step forward.

[@Gaarafan007]

This came out on the 29th of May and here I am watching it on the 1st of November...The Hitchcock clip link no longer works as the uploader's account has been terminated...

[@shuraizo2877]

The music putting players in the right mood reminded me of Ocarina of Time randomizers. Going through Shadow Temple with Lon Lon Ranch's sound track is almost hilarious instead of creepy and frightening

[@thomasrosebrough9062]

"character will remember this" is one of the most clever little snipits of writing to enter the world of brancing-story video games.

[@ZetHololo]

As a russian I giggle every time you pronounce “Kuleshov” like “Cool-shov” Also, kudos for finally pronouncing names with “ov”, instead of “ev”! It’s a common mistake in english - so surnames like “Khruschov” become “Khruschev”

[@Yanramich]

COOL SHOVEL YOU GOT THERE

[@brandonmartin-moore5302]

How should the name be pronounced, and why is ov often produced as ev?

[@rateater1857]

@@brandonmartin-moore5302 cool-ye-shov

[@stretopovermind9680]

@@brandonmartin-moore5302 "-ov" is pronounced as "-ev" (or, rather, "-jov") after consonants that are always soft - "ch" and "-sch". P.S. Pff... there are no digraphs in English to properly transcribe those sounds.

[@sol2544]

WAIT ITS NOT KRUSCHEV?

[@Rehteal]

4:22 *Talking to Nick Valentine* *Car explodes 30ft away and a hunk of metal flies between us.*

[@callmefox630]

Nick: "Do you have a second?" *Explosions, Molotovs, Gunshots* Jeez Nick, I know you're a synth but jeez.

[@masterdemon21]

One of the moments in games that come to my mind when the video said "This is just a game" was the never changing blank and semi-happy face of the protagonist of pokemon sun & moon / ultra sun & ultra moon in the dramatic and disturbing cut scenes of the game.

[@catherinestickels2591]

I don't know if this is specifically Kuleshov Effect, but it's certainly a testament to the power of montage and context

Fun and informative, lots of food for thought. Thank you.

[@Cinderbox]

This is a big part of what broke the experience of Fallout 4 for me. I think these ideas apply well to voiced versus unvoiced characters.

[@SAMURIADI]

4:09 now thats america right there

[@CockatooDude]

Wow this is a really good idea for all sorts of games and cutscenes. Especially for a lot of sci-fi themed games as the characters are often wearing helmets or other facial obstructions.

[@rashan141]

Tales of Symphonia did this really well with it's relationship points

[@archsteel7]

A game I remember that does this extremely well was Fallout New Vegas. The specific scene I'm thinking of is Nipton, where you're introduced to the legion. "I just won the lottery" makes a lot more sense after you see the crucifixions.

[@theunslickbiscuit611]

1:33 *i love this frame so frickin’ much*

[@polifemo3967]

The drawings in this one are superb! Awesome visual metaphors for the script. My favorite one was the thought experiment as a brain cap XD

[@kellylogs2642]

Mentioning the music aspect made me realize one of the reasons I love Primrose's chapters in Octopath Traveler so much: her music is some of the best in a game overflowing with great scores and always comes in at just the right moment to accent Prim's emotional movements, especially her defiant stands against those that have wronged her.

[@juliahenriques210]

Those dramatic pixelated 16 bit moments will never be forgotten...

[@crazymcjesus]

We literally just went through this in my film history class yesterday 😂

[@1996Pinocchio]

What are the chances

[@1agentchase108]

I just realized the dots on his hat correspond to the buttons on a controller, god I'm slow

[@ninasfelicillow3228]

1agentchase oh shit.

[@darkmatter32x]

This video is incredible. I am enlightened.

[@arcanineryu]

I feel like a good example of this is actually the old homestar runner cartoons. Like if you actually look closely at much of the animation, it's often actually much more limited than it first appears. But they're so good at supplementing the limited animation with surrounding context, you often won't notice that the only character expression you're actually seeing a lot of the time is a mouth opening and closing.

[@brunofeitosafl]

This was amazing!

[@GuitarRocker2008]

YOU GUYS SHOULD TOTALLY DO AN EXTRA FILM SHOW

[@XiangXinWoMenHui]

Kuleshov effect is always rather interesting, cheers. By the way, can you do Extra credit about Holodomor?

[@gigaflynn_]

Nice Endgame reference there, plus mmmm, 'Murica's booty.

[@redsnake69]

Sounds like games are a great opportunity to explore the use of psychology in this new form of narrative. As Lindsey Ellis says, "framing supercedes text, always, always, always".

[@vazak11]

Very insightful!

[@jamal69jackson77]

Felt like I was watching a scientific psychology channel. Great information, I actually never learned about this effect in my psychology college classes. Learned something new today!

[@World_Theory]

This idea deserves more time to be chewed on.

[@tlcartist]

I should take the time to leave a thoughtful response but honestly kudos to whoever thought about the butt shot lol

[@YondaMoegi]

Recently I played a visual novel with origins system, and because of these origins, some scenes in the game had completely different meanings, even tho the text itself didn’t change. Cool effect.

[@mehmeh3894]

Music goes a long long way. Imagine fighting Sephiroth with the chocobo theme playing, or escaping in Alien: Isolation while benny hill plays

[@mausklick1635]

Excellent choice of topic.

[@yourethatmantis5178]

That Hitchcock thing could totally work as a meme format

[@thatguywiththekobolds563]

really expected you talk about early games, like ps1 games, they have great examples of contextual emotions.

[@BlueRage0]

I agree with you on this and thank you for covering this.

[@user-og9mi9vh4d]

Aw yiss, I was waiting for this :>

[@sjonnieplayfull5859]

In an old nintendo 16 bit classic 'secret of Evermore' there is a part where you and your loyal dog are seperated by a flash flood. You play some scenes as the dog, so you know he is ok, and then some as yourself, as you grind your way to paying way to much for a ticket to the city your dog is in. One button lets your dig sniff around for hidden objects, but when you are seperated, it still makes the sniffing sound. The surroundings is a beach with three enemies who respawn only when you leave and enter the location again so its mostly empty,, and the sound of the ocean, the seagulls and that sniffing got me SO many times. A lot of times i found myself just standing at the waters edge, doing nothing, holding that sniff button, like a lone soul looking at an old picture of a long gone loved one. It took me weeks to grind my way to the city, so i spend a lot of melancholick hours on that beach. So yeah, Kuleshov effect for real, just with some extra sounds.

[@Munchkin.Of.Pern09]

The Pokémon Mystery Dungeon games do this really well, actually

[@jamescook5783]

what do you mean?

[@99Kuromaru]

I'd like you to elaborate, too. Unfortunately, I haven't had a chance to play Mystery Dungeon games, so no idea what you mean.

[@alexhaupt2134]

I feel the Kuleshov effect is part of the reason NPCs in Zelda games feel so emotive.

[@chriskallen1]

I was wondering about this, too. Even the earliest 3-D ones like Ocarina of Time had no lip motion animation and yet did a great job characterizing the feeling of the situation. Gorons about to give you a crushing bro hug, Ganondorf seizing you with a paralyzing stare, the Great Fairy landing somewhere between wondrous and terrifying with every appearance.

[@RoyalFusilier]

Extra Credits: "But you're not here for movie talk." *David Cage will remember that. Extra Credits: *says 'emotion'* *David Cage will remember that-

[@derekw8039]

"Temmie will remember this" LOL!

[@TheCreepypro]

glad you guys brought this up especially in an age where triple A studios are going way out of hand with their budgets to improve graphical fidelity when they should be more focus on the things you were discussing in this video

[@timothymclean]

Graphical fidelity isn't just a matter of getting better performances; it's mostly there to give players something that catches their eye, something memorable that keeps the game in their mind. You can't stick the Kuleshov Effect on the back of the box.

[@SolarGranulation]

This has given me such an idea but now I need a good story.

[@Pawtacle]

3:56 that is an AMAZING example of this effect! :O

[@tzisorey]

The power of editing.

[@PigothePig]

great video

[@johnfrommanagement6037]

3:47 They told their barber "gimme the 3.14"

[@AtomBacon]

I experience a form of this in Smash Bros. Ultimate. My main is Wii FitnTrainer who basically looks like a kicking mannequin, thus has a very simple and neutral face. Yet I read the face differently depending on her shirt colour. I always pick red because she looks focused and serious and feels more aggressive. If I pick blue by accident I often struggle because I inherently see her as more calm and pleasant (not good for a fighting game) Green feels curious and yellow feels happy and excited.

[@TonsOfHunStudios]

Great point! I think there is some sort of study with boxers, where Red wins more than Blue.

[@dissonanceparadiddle]

This is great! I think that more stylized games have more power in selling Emotion, i.e wind waker, tf2 etc

[@jakkubus]

"there is still only a few companies that really nail emotive facial expressions" You forgot CDPR.

[@danilooliveira6580]

I don't think they nail expressions itself, they nail context and variation. the create meaning by giving you a wide array of animations and reactions, even if the expressions itself are not that good. and I really think more games should focus on that.

[@VenButtons]

I think that a great example of environment to achieve the Kuleshov Effect in a video game is the Ludwig fight in Bloodborne. You understand everything about his character, and the changes in his emotional and mental state throughout the fight, purely by the environment, and musical key change.

[@lolcool7429]

Your KZbin channel amazing

[@werdna1969]

Try Planescape Torment for some of the best emotional performances of your gaming lifetime

[@miniyondu317]

I was wondering if you would do a series on Egyptian history in the prehistory channel

[@SaccadicSableye]

This reminds me of the player character in Pokemon Sun/Moon. Every other character has a range of emotions, while the hero's face is stuck on generic happy smile... even when your friends and the entire world are being threatened. Eternal optimist or complete psycopath? You decide.

[@LiroRaeriyo]

i prefer "emotionally challenged"

[@Loalrikowki]

And on the other side of the coin, you have things like L.A. Noire. It felt like Rockstar was so in love with their facial capture technology that they encouraged their actors to exaggerate their performances in many cases, and in my case that led to a lot of frustration because even people who were telling you the truth came across as totally over the top.

[@pokvirus5705]

1:32 appreciate the team actually animate that laugh

[@chriskallen1]

Deus Ex (the original Unreal 1 game) did this fairly well given the depth of the dialogue and paucity of facial animation options at the time. When J.C. Denton gives the thug Johnny a few seconds to flee or be killed, neither of the characters move positions while the "countdown" occurs. Creative camera changes between each spoken line give the illusion of building intensity. Otherwise, not much happens during this scene.

[@catfishwithwhiskers]

Animating a 3D surprised pikachu is the height of mo-cap technology

[@shawnheatherly]

Applying lessons the film industry has learned to gaming seems like a smart idea to me.

[@raphuscucullatus7845]

*Extra Credits will remember that*

[@anakinskywalker9729]

Hey great video

[@Fyrgon]

that big but moment... so cheap... so good :D

[@BC1ZM3]

I swear I felt more emotion in the "just shapes and beats" game over retry scene than all of fallout 4

[@gothicdragonwarriorqueen5819]

Such an interesting animation!!!!😄❤👍

[@TheMikirog]

Reason why "I put spongebob music over [blank]" works.

[@danicdrako4788]

Incredible

[@Overhazard]

I'd be interested in playing a turn-based RPG adapted from the Pinocchio book.

[@333kenshin]

Q to everyone: if you were to use the hitchcock video to teach the kuleshov effect in a class setting, would you play the whole thing straight through or pause for discussion between the 2 passes?

[@malaizze]

_Man, I'd love to watch the link in the description... _*_if only it worked._*

[@rift1067]

I believe a prime example of the Kuleshov effect that utilises context and environmental juxtaposition is interaction with NPCs in Souls games.

[@Idontknow-iu3bg]

can you make some vids about the 30 years war?

[@KenMathis1]

The other side of using not Paragon and Renegade gameplay tags to conversation choices is the game not registering the same intention the player wanted with their dialog choice. That is a much bigger emersion breaker than having a gamified conversation. Until we have more interactive conversations where the player can see the effects of their choices and corrected them if the game's interpretation differs from their own, we have to ensure the players is given enough information so they can make the correct choice up front.

[@rodytalks3989]

That explains why Levi from attack on Titan seem so emotional to me

[@TheFrozenfish]

Sadly, the Hitchcock Clip is no longer available…

[@camfunme]

"*blank* will remember this.", has never had an impact on me. Because either: 1. the game doesn't follow through on those characters impacting the story and thus it's wasted thought. or 2. the game follows through on most/all of the characters impacting the story, and I feel like it's forced because realistically someone I met on some random planet has a snowballs chance of meeting me again.

[@kahlzun]

i guess we're at the point where we need to be consulting movie makers about game stories.

[@smt4090]

Basically, fill-in-the-blanks style films (known as meme templates now) can lead to a wide array of emotional responses when paired up with something in the blank Coolio

[@bryce4395]

I keep hearing that as "cool shot effect".