NVIDIA ACE Brings Digital Characters to Life with Generative AI ft. Convai

NVIDIA Avatar Cloud Engine {ACE) is a suite of technologies that helps developers of games, tools and middleware to bring digital avatars to life using generative AI.
With our partner Convai, the NVIDIA Kairos demo evolves to take on a number of new features and incorporate the latest NVIDIA ACE microservices: Audio2Face and Riva ASR for AI powered animation and speech. Convai’s platform features a set of tools and APIs to create character personas and enable dynamic conversations. The latest features from Convai enable real time character to character interaction, scene perception, and actions.
Learn More: www.nvidia.com/en-us/geforce/...
Subscribe to @NVIDIAGeForce for more PC gaming content featuring advanced graphics, AI, and high-performance technology.
Subscribe to @NVIDIADeveloper to see more cutting-edge PC gaming development software and tools.
Subscribe to GeForce! / @nvidiageforce
GeForce Twitter: / nvidiageforce
GeForce Instagram: / nvidiageforce
GeForce TikTok: / nvidiageforce
GeForce Facebook: / nvidiageforce
GeForce Community Portal: www.nvidia.com/en-us/geforce/...
#NVIDIAAce
#ArtificialIntelligence

[@TheCrippledWerewolf]

What's really revolutionary is that the corporate-speak in this video sounds more unnatural than the AI.

[@charliewernette]

That is SUPER insightful

[@larsmuldjord9907]

The potential is great but it's hard to know what is actually possible when the video is scripted like this. It would be nice with an actual real-time demo instead of some overlaid, prepared sequences cut together to make it all seem supersmooth.

[@cpypcy]

There are already AI mods for Skyrim so you can try that.

[@larsmuldjord9907]

@@cpypcy This is true, thanks. However I am referring to nVIDIA's way of presenting the tech. I get some serious "Project Milo" vibes when watching this. And we all know what happened with that.

[@gazzy9136]

There's a matrix demo where you can talk to the AI using your own voice and they will respond. Forgotten what it was though

[@RuntimeRacer]

There's a software called Project Harmony AI, which is almost fully open source and allows you to have this kind of interaction with AI using a locally running Llama LLM for example. You can just search for it here on KZbin and see their tech demo. They're also working on adding AI-controlled movement and adding a lot more plugins right now.

[@dekumutant]

There is live demos of this with journos on youtube right now if anyone is interested

[@blueguy09]

"Break out the good stuff" he does proceeds to get up and walk away

[@iancalandro8180]

His good stuff wasn't good enough.

[@jonlava173]

The good stuff was hidden under a hot, smoking cooker, not even chilled.

[@paddington2063]

Obviously first gen and rapidly improving. Really excited to see where this will be in even 5 years. Saw an iteration of this in Skyrim a couple months back - some form of this will certainly be the norm for open world eventually.

[@SkeltherBot]

Skyrim AI companion mod is soooo good, when combining with modded VR playthrough - oh god😮 Ppl need to check on it

[@tuneboyz5634]

​@@SkeltherBotamazed baby 😮😊

[@MichaelGabriele]

I look forward to running out of conversation in video games just like in real life!

[@iancalandro8180]

Glad to see our AI ramen shop owner is doing better. Keep fighting off that depression, Gin!

[@user-fp3de1ez8m]

This is mind-blowing... We are in the Dystopian Balde Runner future ...

[@andresgallego5727]

I think this is great for unnimportant NPCs, it's clear it's not ready for a main character.

[@thiagovidal6137]

It already sounds more believable than any Bethesda RPG main character anyway. lol

[@gecko33yusuf60]

xDDDDDDDDDDDDDD@@thiagovidal6137

[@internetrules8522]

IMO some of the coolest parts of games are when unimportant NPCs have an interesting piece of dialogue, or have a really small interesting side quest that makes the smaller part of the games feel alive. IDK how much of that feeling you can capture from generative AI.

[@pepo_pipi]

@@thiagovidal6137 sure bud whatever you say

[@DimakSerpg]

​@@pepo_pipistarfield fan detected

[@Pheer777]

Wow, at this rate, gamers will become the most socially aware and capable people out there

[@thc4825]

We will look back at this in a couple years & be like "wow this OLD ai used to be rough"

[@fiery_transition]

Gotta admit, compared to modern games it's weird, but growning up in the whole era where 3D became a thing, this reminds me very much of the beginning of that era. A few years later and people will prefer their virtual games lol

[@123reivaj]

It's good to start but it undoubtedly needs to improve. This is going to be the future of video games with story mode

[@iloopcip494]

Not really, why I should be main character? In most games is better to hire a good actor and create interesting character

[@GerrusEdits]

i think you missed his point there, you are right with games who already have predetermined characters for example the witcher with geralt but in some RPG cases where you create your own main character that option can make a huge difference in immersion, take skyrim for example , if YOU are the one who talks and the NPC's react accordingly to that , i think that this could be a real game changer @@iloopcip494

[@sunnyjiang6245]

​@@iloopcip494true, but it's still good for RPG games.

[@Ciner18]

that noise when he moved to play really sold me on this

[@AironExTv]

Here's our start. You have to wonder how much actors could contribute to such characters. Perhaps a template could be recorded through performances to then be utilized for in-game characters. It'll all be a little rough until characters can be controlled in ways similar to the bot in Intestellar. Probably a lot of research to go.

[@imeverywhereandnowhere56]

Why bother with actors? The AI could do it all.

[@CrzBonKerz21]

I think there’s some really amazing opportunities (and scary) for AI avatars as NPCs in games.

[@commarmi]

If they manage to make this usable for studios next gen of games will be incredible, always thought this would be posible, but this came out much earlier than anticipated.

[@MadPotty01]

Looks a bit rough for main characters, but would be great for random NPC's instead of repeating one or two lines

[@danishbashir4184]

You have AI generated text, speech, video etc.... But you need AI generated emotion and expression to make it feel like the character actually means what it is saying. Create a Joker vs Batman conversation. Create Pagan Min or Vaas vs Ajay Ghale and Jason Brody. Create Lara Croft in a life or death situation. Create Lee Everett and Clementine death conversation. . You get the point. Acting/Emotions/Expressions/Tonality. That is where it is at

[@gradius12]

I see this as being used for street characters and other out of the way NPCs.

[@danishbashir4184]

@@gradius12 Then it is irrelevant. Every character should be extremely unique in terms of acting, emotions, backstory, missions etc. Because otherwise it is the same case as Watch Dogs Legion. People like 1 more than 2 and 2 more than Legion from what I have seen. So story and Gameplay are important, not scale. If it is only NPC thing then what's the point.

[@Harmony-AI]

That's exactly what we're trying to do with our project. There's still a lot to be done until it works seamlessly though.

[@danishbashir4184]

@@Harmony-AI I wish your team the best of luck. Go, create exciting things that I look forward to dear. A 16k VR headset with 360 degree treadmill in an AI generated futuristic living world with realistic graphics, story, and to top it all, reward based system for your virtual efforts, that have real money value in real world. That is what I look forward to.

[@Harmony-AI]

@@danishbashir4184 Unfortunately I can't make any predictions on VR Hardware; we're a small Community-Driven AI Software Project at this point; but full AI-driven worlds are definitely part of the long term vision.

[@TheLegendOfNiko]

Work on it a bit longer before releasing.

[@danishbashir4184]

Great stories are written, not generated.

[@monolithsoft_guy]

This is amazing. Imagine any open world-rpg with this. Every npc would be unique and you could hold endless conversations. Wow. Hope for implementation in games very soon.

[@nesseihtgnay9419]

This is revolutionary for gaming and the future

[@Fears_SpectreHW]

These software is so extremely sick. I wish this in Rockstar Games. I love it.

[@ac6339]

Maybe in gta 7, this tech isnt smooth yet.

[@phillaysheo8]

@@ac6339 ah well, that means waiting till 2030 then...

[@ac6339]

@@phillaysheo8 we would be lucky to get it by 2040😁, not even kidding

[@90kuky]

Listen, Nvidia developing game with its features and AI 👌

[@AL_EVOz]

Just thinkg about this in CP2077 would be insane.

[@PhilosophicalSock]

Panam gona talk to you like an actual person, not just a scripted waifu.. That is actually scary 😳but also mind-blowing for certain people who has no girlfriend

[@iWillMakeYouTea]

Speaking to NPCs by a Microphone, or more precisely, making a conversation with them, sounds like quite long future unfortunately, but i wouldn't mind if that would be sooner than later 🤭

[@vitmartobby5644]

Hopefully newer architectures can make it viable to have just-in-time generated conversations on a consumer-grade PC, hell, you wouldn't even need a story for a game like thiss, just put them player in a virtual world... Who knows how procedural generation could also bring us! The possibilities are endless

[@thanemarston4037]

It's not terrible but it is it not great. But this is the first generation look at dlss when it first released now look it. This would be great for random npcs, places like taverns or a busy street. Looking forward to the future.

[@trumplostlol3007]

How do you create "personality", "emotion", "feelings", .. for AI characters? Do they learn how to "react" to the environment differently because of their own unique experience in the game playing with other real players? Will each of them having their "personality", "emotion", "feelings" all converge, like AlphaZero in chess, so that every move has a best corresponding best response? Meaning that all NPCs will eventually have the same set of "personality", "emotion", "feelings", ...

[@brokenpadgaming2538]

It’s all about hype. Then we will see something like this but it will not be at its 100% potential. Script will be always a priority. And think about this: at Stanford they teach you using psychological principles in order to keep people sucked into apps. Imagine the time needed to complete a game. Now think how much time you will stay on the app with this kind of interaction… If they will add a way in order to advertise brands within the game companies can earn more money too

[@drx95117]

This is the right path

[@bestrelaxingmusic21]

Can i fine tune with my data? Which Nvidia Product i’ve must buy?

[@camdencantwell846]

rip to good dialogue

[@gecko33yusuf60]

With this were one step further to SAO. I hope you get what i mean

[@charliewernette]

Another step closer to Stepford Wives

[@endamcnabola]

So with this, every game can have a different story every time.

[@Atilolzz]

Where does the data come from?

[@brandonleemoss]

SpaceX 😅

[@thorstenh.7789]

Das ist genauso geil, wie es auch krank ist!

[@birdyrules]

This demo has a strong resemblance to Project Milo on the Xbox 360 with Kinect. Let's see if it actually happens this time instead of turning into a meme.

[@SoopaFlyism]

"Hey Nova, take off your clothes." 😛

[@domagojbelic5772]

This looks like a great tool for indie devs that work on a small budget. However, it's a bit concerning if AAA developers decide to use AI text to speech generation instead of hiring real voice actors.

[@WholesomePotato1]

Why would it be concerning? The whole world is changing, sooner or later. Every one will need to be able to adapt, including voice actors, artists, otherwise they will be left behind. The same way industrial line operators needed to adjust to automation in factories. Why would voice actors be any different? AI will displace almost all jobs eventually. I don't see why we would need to artificially preserve one profession over the other.

[@YouAreStillNotablaze]

@@WholesomePotato1 Yes media quality as well as quality of life for workers in this industry has _clearly_ changed for the better. This is sarcasm of course.

[@WholesomePotato1]

​@@YouAreStillNotablaze it's all about money.. You are correct about that. But that has nothing to do with technology. It's about the monetary system and labor-for-money paradigm. Instead of machines freeing people from hard toil, labor, repetitive, monotonous, dangerous, dirty jobs...they take their means of living. But that's not machines' fault. Entrepreneurs and businessmen downsize because it's cheaper to implement machines. They don't require health insurance, light, air conditioning, high ceilings, they don't go for smoke breaks, they don't fuck around or lie. So they implement it. You have to keep in mind that this model is on its last leg. It's unsustainable, because if they'll keep doing this, people will lose purchasing power and even though there will be goods and services, people will not have enough money to buy them. That's the end of the monetary system all over the world. Check out the book End of Work from J. Rifkin for more info.

[@Muckylittleme]

@@WholesomePotato1 You are one of the few who is looking forward to the anti human future so called the elite have planned for us, but then you clearly have no soul to lose so no worries.

[@Sengial]

@@WholesomePotato1 Its really shit compared to actual good voice acting and it will NEVER have actual good writing with proper character progressions, quirks, unique speech patterns etc

[@weill6872]

I know it's all set up but he could have moved the mouse when he turned his character to talk with Jin

[@Хакод]

Весьма круто.)

[@kabob4636]

The people complaining about the audio and lip sync desperately need to see the bigger picture here

[@johnboysautobody4214]

He didn't drink the good stuff he ordered

[@darthtorment]

With AI's like this, who needs Player Characters?

[@Roach22]

Why did I feel Peter Molyneux was going to jump out from the background in this video? 😝

[@realkingofantarctica]

You can write a video game story using AI, but can you _tell_ an AI story through a video game? I doubt it.

[@yashprasadh9492]

Damn it makes the gamer an npc damn

[@EnderCraftKid]

can anyone download this at all?

[@sambowman91]

Have they tried making the audio, visuals, and words not completely shitty?

[@TheErtop]

Can I play this demo myself?

[@AdithyaShakthiKumar]

this is sick

[@skilat193]

Is this only in English or can we make AI conversartion un other languages ?

[@Ongez]

The speak feels really robotic, no emotion, it`s like they are reading a script without even trying to convey. also the words, who talks like that?

[@ypichardie]

Even if today npc cannot answer in real time, I suppose devs can generate huge number of réactions, selected and finally scripted

[@misterinfinity4076]

Hm is this a demo or a hint/leak of an upcoming GPU ??!

[@elementalstudioyt]

Love this in fallout 4 or GTA

[@rhysh2900]

huge milo vibes from this lol hope this is actually good

[@klnmn3722]

this is not just something that can be thrown into some scenes and select interactions. if this is in your game, it has to be ever-present, and the whole game has to be built around it. this isn't just a toolkit; it's a genre in and of itself, for better or worse. personally, I'm not interested in it and don't see much coming out of it except some neat tech demos (like VR).

[@nintendians]

now that's insane, now do it in vr - it'll be god-like insane.

[@ThisIsHowYouCry]

I am fearing that we approaching or already approached "ready player one" kinda world

[@jonanddy]

does the bartender react to you leaving right after he hands you the good stuff? that would be cool

[@user9z]

Project Milo by Molyneux came out after all?

[@Tyuwhebsg]

nice!

[@Muckylittleme]

Every line delivered in the exact same amateur dramatic drawl, that's how we know.😉

[@capusta_uchiha]

Это будущее. Я в шоке.

[@DTunezOfficial]

There's a long way to to but the concept is cool

[@Vostadues]

Secret new GPU... I thought everyone knows that RTX-50 series will be launching around Sep 2024...

[@tuxuhds6955]

1:38 "Nova's success calls for the Top Shelf celebration..." Goes right to the lowest shelf possible. Evil AI is Evil.

[@SimianVirus]

Maybe a new sandboxes games like minecraft will be on another level

[@Bernardos2002]

new secret gpu huh

[@Wuking93]

4090 ti

[@sergeyt2947]

4065 ti super

[@davicoutinho7921]

skynet on ?

[@josuad6890]

jesus. last time I saw it, it was horrible but had some promise. Now? It's not quite there yet, but if a game launched with that right now, I'd be damn impressed.

[@pecora31]

So there will be no more fixed conversations, and we can talk freely with NPCs like normal people and still get clues of the game. Great, now I'll make it do some math.

[@justincase4812]

"Product Manager" A fancy job title for someone who specializes in gooble-dee-gewk they pretend to understand and no one else does.

[@relic6431]

is no one else caring they left the "good stuff" at the bar?

[@muresanandrei7565]

This is so bad what a joke 🤣

[@qownsqud3207]

Damn you really one of those people that when presented with something thats going to change the world, you just straight up dont know what youre looking at. Its so sad

[@Pishin1]

mas duros que una piedra

[@SoUnD_MAN]

Variable conversation is good. But it's time for the characters to look better than silicone dolls.

[@effeduevolte]

So... no more screenwriters and voice actors. stunning

[@gjermund8053]

That's not true. Just to a lesser required extent. They still have to be the shapers of the personalities and voices.

[@dicemoney2]

Npcs need this update bad

[@Drstrange3000]

Sorry to say, but I don't think this technology is anywhere close to where it needs to be. It comes off very soulless. I still think stylized, more focused, and handcrafted games are better suited for my tastes. This just looks very robotic and artificial. It doesn't help that only the NPC's mouths move with no facial expressions. It isn't convincing to me and actually takes me out of the immersion.

[@GaminylGames]

It's first generation AI tech and it's not even out yet. Give it 3 years and you'll have a different opinion, one can assume

[@JonoFunk]

5 years ago this would have just been a dream for gamers, 5 years from now it will be the norm!

[@Scorpwind]

Gee, I hope not. This feels so artificial. No pun intended.

[@JonoFunk]

@@Scorpwind That's why I said it will take some time. There's definitely work to be done such as smoothing out the emotional delivery of lines.

[@Scorpwind]

@@JonoFunk But even then; Do you not prefer something that was crafted by a human being?

[@forcews]

People want offline AI neural accelerator based not yet another cloud that needs stable internet to function.

[@kireeeels]

let's look what she can show us at the lab friend asks

[@TheMrWARLORD]

Just wow. Wanna see AAA games with this tech, especially VR games. Would be a day 1 purchase for me

[@cap7ainskull]

The guy playing the demo feels more scripted than the ai demo . It seems both the ai function on a goap like module so a plugin for this is going to be fun

[@Cosmic_Caravan.]

Now imagine this technology in 10, 20 or 100 years, you would have something indistinguishable from reality. Wait a minute, could that mean..

[@MrVohveli]

It most definitely does. Either we are the first or heaven starts with a score board.

[@hanbill]

When game is so boring that AI has to fix it edit: also I hate speeches that doesn't really bring anything to the story, just like talking to randos irl just typical daily small talk. So if these interractions with NPCs aren't sidequests, NOT INTERESTED

[@phillaysheo8]

I bet they edited this, to remove the latency

[@tushartanvir5672]

are the voices AI generated.

[@HasekuraIsuna]

Nah, the generative AI only produces the text and then there are voice actors om stand-by in each region 24/7 to read the line to you in real time. This unfortunately means this will be always online.

[@Cosmic_Caravan.]

Every player will have a team of dedicated voice actors that live in the bunkers beneath Nvidia HQ, they never sleep and reply instantly!

[@mantodeastudio]

This is going to be crazy once they can figure out the naturalness of the human voice.

[@TechTalkTobi]

maybe in 5-10y Todd is also able to have NPCs that are not from the past 😂😂😂

[@witcherpie]

i mean, this guy is just playing a recording on his monitor X'D camera moves and he's not even using his mouse

[@CVSiN]

Whats really amusing is they are already more literate and engaging that 3/4 of those commenting it's not good enough hah..

[@brokejaw]

This is cool, but you know for sure that ain't no one gonna stick to the game story.

[@MAJ0RTOM]

This will be awesome, but the tech is not ready yet, this whole interaction is still very robotic, and I'm not talking about the voice, imagine you want to go to the hood in GTA and exchange slurs with some NPC, instead, you get moral lectured by the AI, or worse ignored.

[@Danuxsy]

This is the future

[@Kaztrofy]

Nice facial animations, almost PS2 like.

[@gjermund8053]

!!!