Recreating Animal Crossing's Dialogue | Mix and Jam

The villagers from Animal Crossing express a lot of their emotions through dialogue, and this is one of the main reasons why players create such big connections with them. I wanted to experiment and see if I could achieve a similar system, using Unity!
Support Mix and Jam on Patreon!
/ mixandjam
PROJECT REPOSITORY
-------------------------------------
github.com/mixandjam/AC-Dialogue
REFERENCES
-------------------------------
William Enders - Game Dev
williamenders.com/
EXTRA FOOTAGE
-------------------------------------
Animal Crossing Trailers (Nintendo)
• Animal Crossing: New H...
• Animal Crossing: New H...
MUSIC
-------------------------------------
Animal Crossing Theme
Composer: Kazumi Totaka
Follow me on twitter!
/ andre_mc
Watch some other of the Mix and Jam videos!
Superhot's Bullet Time • Recreating Superhot's ...
God of War's Axe Throw • Recreating God of War'...
Monument Valley's Level Design • Recreating Monument Va...
Metal Gear Rising's Blade Mode • Recreating Metal Gear ...
Metroid Prime's HUD Visual • Recreating Metroid Pri...

[@mixandjam]

Who's HYPED to play the new game?

[@Beengus]

DOOM ETERNAL BABY

[@noponn8007]

I am just waiting for my delivery. Just come already, deliveryman

[@diegodj66]

Super hyped, but sadly broke

[@Lukz243]

not hyped, but I could use some escapism

[@menatrs1]

Remake a Hat in time's movement plz!

[@lvbboi9]

Now recreate DOOM ETERNAL's glory Kill system Perfecly balanced, as ALL things should be

[@lvbboi9]

@@katowo6521 ...

[@conormorgan6256]

@@katowo6521 bro that really ep ic ehen happen be ucause he des tyhe swera and hhaaha hig hzahah

[@conormorgan6256]

YSe my f iner. did sliop dueinh tha sentndxce i wiil do ir berreer j ext tije

[@pauliesnuts8886]

@@conormorgan6256 sex machine awesome writing

[@Danidev]

just in time baby, very nice

[@Thonder]

Dani o hi

[@FirstLast-le2rf]

Dani watches Mix and Jam! Step aside Endgame, this is the most ambitious crossover in history!

[@serialchiller9183]

Is this a spoiler?

[@Nenad12345]

is this a spoiler? milk

[@legome3431]

Dani, you should make animal crossing in unity

[@MRBONES-fg6cs]

Pretty good! Something you missed in this, though, is that Animal Crossing voices actually read out the characters of whatever's in the text box as they appear- it's not just randomly selected soundbites.

[@Desi-qw9fc]

MR. BONES exactly this, it makes a huge difference because sometimes you can almost hear them saying the word.

[@NightChime]

Also, different characters have different voices.

[@weezards1014]

Oh nice I didn't know that good fact!

[@phoenixmorphix]

The developers did use this in their other game Splatoon! 😮

[@llenastro5986]

know this is an old comment but just wanted to say i'm a big fan of your wild ride

[@BlueberryMauro]

Cute, I feel like I played animal crossing like all my life, good to see that you recreate one of the things that probably is the most used mechanic as you talk with all the little animals, also Jammy is so cute, his voice is also cute, I like jammy.

[@BlueberryMauro]

Jammy appears in 5:04

[@Cyranek]

I've been researching how to generate that sound-effect mumble speak that all the characters have. Glad you made this video. I'm curious to see if there's a way to automatically generate those sounds rather than cutting out the clips from pre-existing ones. Like a character mumble speak synthesizer.

[@acdi33]

In Animal Crossing, they actually synthesize the dialogue (kinda like text-to-speech), then modify the resulting sound (pitching it up a bit and adding a few filters) to produce the speech in the game. If you slow down clips from some of the games, you can kinda make out the individual words they're saying.

[@Norivee]

@@acdi33 Huhum I think they've made a collection where every letter or couple of letters is associated with a sound. That would be why sometimes on some words it really sounds like the actual word, and sometimes it sounds like gibberish.

[@Hello-qg4yk]

@@Norivee yes but the gibberish is because of englishs phonetic inconsistency, "usually" you could understand what they are saying

[@GabrielAguiarProd]

Unexpected, but lovely! The dialogue tag system is an awesome concept.

[@GymCritical]

When you talked about offsetting the texture for the eyes, I thought you were a super genius.

[@DrRishabhGarg]

Isn't he?

[@GymCritical]

@@DrRishabhGarg it would appear that you have a point

[@MM102]

this is really good! I was hoping you'd also add the letter based sounds instead of random sounds but this is still a really good starting point, keep it up!

[@CharacterDesignForge]

I really love the solution you used for recreating the audio! We may have to borrow that...

[@LightVelox]

What about Dragon's Dogma/Shadow of the Colossus climbing?

[@fAKSDFKamsdf]

I like this

[@NeverduskX]

This... this would be amazing.

[@neverknowsbest1717]

is that a huzzy icon?

[@LightVelox]

@@neverknowsbest1717 yeah lol, i only know now, i literally just went on google and searched "gamer profile pic"

[@rootmew]

You're a fantastic developer, and a great teacher. It's been humbling to watch your channel grow after first seeing it pop up on r/gamedev. Keep on doing what you're doing - wishing you every success!

[@craigsmith9767]

Thank you so much for making these videos! This one in particular has been especially helpful. I'm swapping between cameras at the moment for normal 3rd person vs a targeted camera for locking on, and this video has been incredibly useful. I'm going to have character dialogue as well, and this has been extremely informative for guidance on how to create a more engaging conversation!

[@JelleVermandere]

Amazing work as always! You always go that extra mile with those subtle finishing touches that really improve the feel of the mechanic!

[@davidmills47]

Years into Unity development and I'm still learning new things -- the scaling the mouth trick was genius ( just recently learned I can apply scale options to different body parts in a character rig, I can use this to simulate body parts getting removed now ). But my best find here was the group tracking option for the camera using Cinemachine, definitely putting that to work, love this channel!

[@ast_rsk]

This came out so great! You guys came up with some really creative solutions.

[@cq5857]

Started adding your videos to an idea folder i have but I'm just gonna subscribe and look at all of them because you're taking over my ideas folder!

[@jacysplonk9134]

These videos are so so so so so good! Thanks so much for making them and keep up the incredible work!!!

[@Halloweenharen]

Aw hell ye new mix and jam video. Discovered your channel from the last vid and benched all of your uploads! Great stuff!

[@tomasb96]

WOW, this video is gonna be very helpful in a project i'm working on! Your videos are great and inspire me a lot, thank you!

[@liznt]

Amazing work again! Thank you for bringing inspirations to developers. I am excited to watch your videos and be inspired every time!

[@cooplucidtales]

Your channel is amazing as always ! Its been a great inspiration for us and thank you for the continuous stream of top quality videos

[@AaditDoshi]

I appericaite you making really good tutorials for non-flashy mechanics from games. These are as important as any other.

[@SnutiHQ]

Learned so much from this! 😍 Lots of little details about the TMP and Cinemachine we're still not familiar with it seems. Top quality! 🚀

[@calebcollins4037]

This is my favorite channel on all of KZbin. The production value is so high

[@imSpenc]

Amazing as always ! Keep up the good work and nice videos !

[@brannanvitek1035]

Andre this was CUTE!! I love this so much- definitely one of your best videos yet.

[@Bigaston]

Again a fantastic video. It's very nice to see how game dev are creating some games ! Thanks again ^^

[@bvrdsinthetrap3810]

I just discovered your channel, immediately subscribed. You're doing some great work, you're definitly blowing up sometimes soon. Keep on keeping on

[@BennyBennyBennett]

This is my favorite one so far! What a great recreation!

[@rmt3589]

This is amazing! Instant subscriber.

[@Pokopish]

I love seeing different dialogue systems in Unity and I really love how this one turned out!

[@xstrafez_c2017]

This looks awesome man! Great job o;

[@neverknowsbest1717]

This is genuinely amazing!

[@bootybjones9643]

You have some QUALITY content, my friend. Much love❤

[@minty.cuddle]

I've been really enjoying these type of vids I only been watching for a week but this is awsome

[@gamedevemon]

This channel is pure GOLD!

[@eftorq]

Great work as always! I feel like you could make anything if you wanted

[@ricksimonds2923]

I wonder: how was Revali's Gale made with the paraglider system. For me, Revali animation showing up is very beautiful in Zelda BoTW. Editing: I'm Brazilian and I don't understand much English, but I like your videos and I'm sorry if there is an error in the comment.

[@Hello-qg4yk]

Why do people always say this? I only see ONE MISTAKE per 3 comments and I can understand every one of them easily. You guys always write good english

[@ktmochiii]

The amount of useful tools I discover while watching your vids is amazing. 👺

[@HampusOlsson]

Wow, Amazing! Perfectly timed as well :)

[@Hunter725R]

Holy Cow, this is perfection!

[@Charziot]

This is wonderfully made.

[@BusyAnas]

Your videos are very inspiring and helpful Thank You I love it keep it up

[@thorn_lekoh]

Great as always! What could bring the final touch for mimicking the real game's dialog system would be to record voice samples for every letter in the alphabet, to then make the npc read out loud each character of the line it speaks at high speed, that's how the yogurt language of AC works in the official games

[@AlexandroAnaya]

Cool! Really liked this

[@matthewbrown995]

I needed this!

[@CrossImpactGame]

I was thinking about if you would create the fishing in Animal Crossing earlier today. Cool to see you made A video on Animal Crossing.

[@Jojo2]

These are so freaking cool! Always makes me want to get back into game development, alas I'm not able to right now.

[@jeroen3648]

This is sick. Thank you for this video

[@ska8terboy4]

I used the same process (custom parser with pausing, etc) for creating similar zelda botw dialogue for fun. I instead used alpha of the text for fading. You gave me some ideas about events. Awesome video!

[@IOSoraOI]

So subtle yet so beautiful.

[@heyclip]

Thank you! This is going to be so useful!

[@GamesBySaul]

Amazing!! Just playing New horizons now, so this was fantastic! Great job!

[@r0flcopter]

thank you for this!

[@3333218]

Would love to see you recreate some classic and historical important systems. Such as: - Complex RTS Pathfinding that works on dynamic terrain such as in Age of Empires/Command and Conquer/Warcraft 3 - Doom's BSP Trees - Shadow of the Colossus' Inverse Kinematics system - Half Life 2's Physics system (+ Gravity Gun) (including the water and fire) and Character Facial Expressions system - The Diablo series' effect affix, postfix and suffix system - The Thief series AI system and physics-based sound system - Tekken's fighting combo system (try to mimic 1 character's moveset) - A series on perfectly recreating all sorts of character movements (Like for example recreating every movement set of every Mario Game - one game per video) Would also love to see a video on recreating the most juicy games out there too, like Vlambeer games for example.

[@chuchu9649]

I’ve always wondered about this!

[@cihanemreozen3152]

Andre "The King Of Inspiration" you are so awesome :). "Mix and jam uploaded: a new video " notification excites me every time. I wasn't expecting a new video so seeing that surprised me and made me happy a lot. Thank you so much man i love you :) Btw are you planning to make a live video sometime that would be awesome :). Take care and stay safe my man.

[@nowhere.to_go7335]

one minor thing, the voices in animal crossing is basically kind of a sped up japanese text-to-speech program, other than that, this is a really awesome video!

[@appdata8476]

I made the same system as a challenge in 24 hours , that was so tedious, awesome work.

[@axcpt9028]

I’m busy binge watching all ur vids.

[@dacolib]

such deep dialogue! Im moved! But fo real tho, grat video as always! Really helps to make characters more than NPCs and have more personality! Great stuff!

[@MayasMotivation]

Still waiting on the Okami brush mechanic. Great video as usual. 👍

[@daslolo]

so cool and light

[@SharpDressedBear]

I know I'm a bit late to the party but this is an awesome video! Great idea using tags for different effects during speech, I wish I had thought to do that instead of having a truckload of bools ^^'

[@djblast101]

I wonder if you have tips about the parsing the dialogue part. How do handle options and nested options

[@Macionik]

I've tried to change the font in the TMP_Animated component, but I can't. The one that's fixed is the predefined LiberationSans, but I can't change it and can't find why. Any help? EDIT: I'VE FOUND IT. It looks that you have to convert the font usint TMPro's converter in Window -> TextMeshPro -> Font Asset Creator. After that I can change the font in the TMP_Animated component.

[@Capt_Streams]

The greatest thing about the dialogue in Animal Crossing is that the voices that the villagers make aren't random, they're based on the letter that they're saying. Each letter is a different sound that gets played as the text is being drawn. You can hear the sound that each letter will make when you type them on the in-game keyboard in ACNH.

[@WanderyenErin]

This is awesome!

[@noahr607]

Awesome work as per usual! Idk if you take requests/suggestions but have you considered looking into Lethal League Blaze's hit distortions? I'm really curious how they achieved impacts that feel so powerful during gameplay :o

[@i_am_phibian]

What if you wanted text that had certain words shaking or such? How would you go about doing something like that? Just wondering, any ideas are appreciated.

[@rainthstrive]

That's so cool, thank you!

[@marlonruvalcaba386]

The embedded actions inside of the text is really useful. We use emojis for example "[:D] A new video. [0_0] it is about dialog systems [stars] " it is easier to read and transmite feelings

[@jeancarrascoplane5579]

This is too awesome

[@Linkselott]

I did something like that a while ago in game maker studio but as in animal crossing the villagers say the phonemes of the letter that appears in the text, what I did was detect that if there was a letter "a" reproduce the sound of phoneme "a" like that with all the letters. Sorry for the bad english hope you understood

[@efrenarizmendez5578]

Wow that's really cool!

[@ecereto]

That was amazing

[@levi2408]

Nice, just in time for quarantine!

[@fedenadur]

Great job dude! Maybe do a video on Doom's glory kill?

[@jyckena]

so cool !

[@IyokuUnlimited]

Awesome video, thank you! Any chance you could dissect and explain the custom clothing system?

[@trs81234]

OMG!! It's so good !!

[@vettorazi]

Putz que orgulho meu. Acabei de descobrir que vc é brasileiro. Parabéns pelo canal, tá mandando muito!

[@ikaika2690]

serio que foda maluco .

[@cesarsfalcao]

So so great, again! Please, my dream request - paint guns from Splatoon!

[@jakecook697]

I've just developed them! Hit me up!

[@cesarsfalcao]

@@jakecook697 hi, nice ce to hear. Can you share?

[@jakecook697]

Hey mate! Sure can! Just a little bit busy at the moment today but later on I’ll package up the project and provide a link here! Alternatively if KZbin doesn’t let me upload such file do you have an email? The mechanic I’ve created uses get/set pixel function to dynamically change the colour of pixels in a texture applied to an object, creating an overlay “splat” texture that has an alpha blend with the original texture beneath it. I hope that makes sense, but in practice it allows you to paint objects using a paintball gun within Unity! You can set the colour of the paint and choose a random splatter texture each time a shot is fired so it’s pretty neat! I actually just went about creating a paintball gun, without reference to splatoon for reference - The idea of my creation was actually to recreate the “goop” mechanic from super Mario sunshine, so in a scene first I paint objects (using the paintball method) and then repaint the pixels of the overlaid texture with a transparent splatter texture, thus erasing the paint that was applied. Let me know if you still want the package! Cheers! @MixandJam - as always, your channel rocks!!

[@slyrytang]

although its not exactly the same type of game, im pretty sure this all could be used to make something just like the systems in the ace attorney games where characters emote and move around and even slow down their text speed when talking. really interesting stuff

[@Ramzo40]

Nice Job

[@SenshiSunPower]

It would be cool if you added a second part to this topic. Animal Crossing's voice system is able to recognize phonemes in words and play sounds that correlate to various phonemes. It's not full voice acting, but it adds more life to the game. The English and Japanese versions actually have different voice mumbles that match their language.

[@nekkowe]

This is really interesting! I wonder how easy it'd be to tie Yarn Spinner into this...

[@slimesmcdingle4169]

the sounds that the villagers make is actually different depending on each character, and if you listen to it closely, it sounds like recogniseable speech, just faster

[@tadhgivors]

That is awesome !! , How long have you been using unity for? I'm in my second year in a Game Design Course rn and videos like these make me wanna learn more 💪💪👌

[@lucaferro8881]

dá uma felicidade sempre que chega notificação de vídeo novo 😄

[@saxosipho]

Therapist: Jammo with a mouth isn't real, he can't hurt you. Jammo with a mouth: ...

[@jonathanputra6873]

i think making mgsv tpp quiet invicible animation would be soo interesting

[@tonywoods1226]

Just wondering, wouldn't using Animation Look At function for the head rotation be better as there is no need to import whole packages?

[@mixandjam]

LookAt wouldn’t work if the villager had an idle animation (which is the case). The package allows you to blend constraints with existing keyframe animation

[@tonywoods1226]

@@mixandjam I was talking about Animation IK system. You can have it influence only the head with animations playing along with it. I used to achieve the effect of player character looking where the camera is pointing, it only influenced the spine and other animations like idle and running played over it. It took no more than 2 lines of code, first setting the weight and second setting the target.

[@MrNotteNera]

Wow, there is much more work than I tought behind a dialouge system .__.

[@TheSeen720]

wow this is excellent and we can download it.

[@davidwong8675]

Amazing

[@serialchiller9183]

Nice one! Can you recreate Speedrunners' Grapple Mechanic too? It's projectile based, not raycast, so it can be blocked/cancelled by several items in the game. Also the character's vertical (down) speed should convert to horizontal speed when you grapple. That's not all, it drops your character downwards when you release immediately after the grapple connects to the ceiling. So you should implement a pendulum move thing to make it work properly. Thanks!

[@lib4861]

Wow. Amazaing, Thank you

[@naezan]

i want to retargeting animation. But i cant, because animation is .anim not .fbx. I cant change animatiom to humanoid. Any solution in here? Need help