My Playtime with Ubisoft NEO @ GDC 2024

[@AIandGames]

Here's a little bit of video goodness while I prepare this months extra special episode. Thanks again to the NEO team for inviting me to the event at GDC. It was great to be invited, and I had a fun time chatting with the devs both before and after the demo itself.

[@MelvinGundlach]

I'm getting really nostalgic seeing the Splinter Cell Blacklist drone in the background video. It's so disappointing that there hasn't been a Splinter Cell game in over 10 years.

[@AIandGames]

I know a bunch of folk working on the Splinter Cell remake right now. I've heard some interesting things... fingers crossed!

[@djbeema]

Bring Michael Ironside back or I'm out 😅 ​@@AIandGames

[@KenTWOu]

I wish they'll marry this AI tech with 'play as anyone' mechanic, NPCs' backgrounds, schedules, recruitment missions from Watch Dogs Legion, where it actually makes sense.

[@AreWeLearningYet77]

I think adding just the smallest hint at an attitude or personality will do a lot to cover for this technique's current shortcomings. Hide the pauses behind some canned reactions and train their dialog with professional voice actors so you'll actually WANT to talk to them more. If I'm going to have to listen to very vanilla insights and information that can be easily inferred, at least allow me as a player to generate an opinion about the personality of the NPC I'm talking to.

[@AIandGames]

Really good point. Giving them some actual flavour can really help develop a relationship with the player, be it positive or negative.

[@CBaggers]

Great insight and footage. Incredible to think how many additional people it's going to take to make such hollow, vacuous dialog. What a shame. Would love to companies put the same budget into people making the narrative experience deeper. The voice recognition seemed reasonable for the most part though.

[@djbeema]

My sentiments summed up more concisely. "We've spent billions of dollars to make the equivalent of an NPC from 1998" doesn't feel like progress to me but what do I know. I realize it's a work in progress, I just hope this is still very far from being considered for use. Then again we still have David Cage making scripted characters talk the same way 😂

[@joshmccaleb3540]

I think it helps to think of the AI as an actor rather than a character for this discussion. That feels like a better framing when talking about breaking immersion, and helps to understand the difficulties the "actor" has when dealing with a player who won't play along. Maybe it would be better for the "actor" to break character when the player does, and have a jokey little meta moment before returning to character. I think that would break immersion less than trying to pretend the player was just joking or mistaken.

[@Deliveredmean42]

That does sound like a concept for a unique game out itself.

[@Hey_IMBM]

Honestly I do not see the pipeline that would work well to use this in gamedev. With such an amount of afford it requires now it would be still quicker to just using ML to generate dialogue options, validate them and deploy manually.

[@AreWeLearningYet77]

Like all tech before this, pipelines have to be invented and methods made more efficient over time. This is like complaining that a cake is only half baked right after turning the oven on.

[@Hey_IMBM]

@@AreWeLearningYet77 Same story as with a VR. Technology requires increased development costs without a clear benefit for the end product. Even if successful with your implementation you would still have to fight with a gamer's habits and the psychological effects of creepiness when engaging in conversation with NPC. I don't think it is possible to use it commercially at least within the Target Audience of the current generation of gamers. And it's not because I'm a grumpy regressive dude but because of the reality of things. P.S. Please don't consider this as an offence or challenge call. I'm just sharing my opinion and I really don't want to unpack more of this topic.

[@maskoblackfyre]

Nice one. Sounds like a really interesting project and tool that management will find a way to monetize and ruin in their games. I can already see a "Vocabulary expansion" microtransaction in plans. You want an NPC to have a special type of personality, like "Cheerful" or "Deadpan"? That'll be $4.99 in the in-game store.

[@nickludlam]

One of the bigger problems to my mind is jail breaking. Is it more fun to try to convince the conversation system to tell you something unintended than to play the actual game? Also, the more guard rails are added, the less interesting this will be, IMO. Players want surprise, game designers do not.

[@AIandGames]

My favourite thing to do in Inworld's Origins was to talk about everything but the murder I was sent to solve 😅

[@Thegamingdino133]

I would love to see more videos looking at open world game ai cycles. Alot of games try fake it but it would be interesting to see how they do it

[@nilsmuller-cleve6769]

More need for narrative designers is a win in my book :D Tbh conceptualizing a character, defining a way they speak and setting the constraints for their narrative AI is way more appealing to me then writing every single line of dialogue.

[@TheOrian34]

It would feel more like making a DND character than filling spreadsheets.

[@LunaRoseManor]

This is an alarmingly convincing illusion. Truly impressive tech, but I'm not fully convinced it'll catch on. That short pause between inputting an answer and getting a response is significant enough that it'll get maddening after twenty hours with it. Not to mention that any work that goes into designing the narrative in English will explode exponentially for international players. What about the voice actors? Do they get to come into the booth and record a few phonemes or are they expected to stay home while the machine autocompletes their performance for them? Requiring the player to use their voice at all times is an accessibility nightmare and I doubt any of the computation for this is done on the user's computer. If it does ever catch on, it'll be another excuse to burn the history books once the cost of computing answers outweighs market relevance. Ubisoft already does this to its back catalogue seemingly at random and I don't want them to have any more excuses to keep up the habit. I'd much rather have more limited, hand authored content if it keeps game budgets down and work hours shorter. I can't imagine writing canned answers into a spreadsheet is nearly as fun, but at least then both the player and designer is immersed in the team's vision instead of feeding it into a meat grinder and hoping for the best.

[@AIandGames]

Agreed the delay in response is still too long, and that's given it's calling the LLM on Inworld's cloud infrastructure. I don't think running this stuff online works in the long run, but then they need to right given the LLMs are too big to host locally. And I agree also re: accessibility, while it sounds like a great opportunity on paper, in practice it can be very limiting. While I found the voice recognition had improved since last year, I got fed up of playing Origins because I had to repeat myself several times.

[@Sp0rksta]

will this be a game ubisoft expect you to pay an exorbitant price for only to prevent you from playing it in a few years time? (aka The Crew)

[@KenTWOu]

Yeah, it will be a game like that, cause cloud based AI will require an always online connection.

[@PlebNC]

Bloom: "-and maybe brainstorm some ideas for taking down those pesky mega corporations." There's an irony to a bot made by Ubisoft of all people saying this.

[@JockeSohls]

I see a lot of potential for this kind of tech in games, but I also think most people and companies are focusing their efforts on the wrong part of the game - I think this tech has a lot more use for gameplay rather than narrative purposes. When it comes to narrative, there is so much that can go wrong by "unleashing" a character to talk about whatever, and it can ruin the story, the theming and the mood very quickly if, for example, your evil necromancer dude starts answering questions on what ice cream flavors he prefers. On the other hand, if you shackle the NPC's responses too much, then this tech becomes fairly useless and, as you say in the video, requiring a lot of set up for very little gain over traditional captured VO and mocap performances. I would love to be proven wrong in the future though! :) As a game designer focusing a lot of my career working with enemies and game AI, that is where I see the huge potential for this; having enemies that can make accurate callouts on where the Player is and is currently doing, voicing their plans to each other, giving more flavor and personality to each individual enemy etc. This is where I think the next step should be as it could save the need to record thousands of lines of very similar VO with variations for each and every miniscule scenario you want an enemy to react to AND it would have the potential to be so much more granular as well. It is also far less of a problem if the voice aren't up to voice actor level quality as it will never be the entire focus of attention for the Player, but rather something that happens around them. Same type of thing could also be extended to friendly non-important NPC's so they can react to things like "Hey, why did you throw that paper cup at me?" or things like that, though that is a bit trickier for a variety of reasons, for example the fact that you might not want to invite Players to go out of their way to mess with NPC's to get a reaction out of them.

[@ruolbu]

There are usually two directions to test such a system. Can it recognize and accept your prompts? I think you gave a good example that it can comply and give the player the feeling of agency. Here it did that within the intended scope and story, with you playing along. The other option is opposition. Will it always comply and let the player steer in any direction even an absurd one? Or does it have some foundation that can not be altered? You say that the AI questioned you when you made reference to objects that don't exist. I must have missed that. Overall I'm still interested in seeing a more extreme test, having a player intentionally steer the conversation away from the goal and see how the system responds.

[@AIandGames]

Indeed I've had many conversations with developers about how most players first instinct is to try and push the boundaries of the system. Right now, I think this NEO demo is too strictly defined to let me deviate too far. I had a fun conversation with the NEO team afterwards about this exact topic. They recognise the value in letting players push its limits, and had some thoughts on how to support it in context. Like what if Bloom and I just party for an hour? He gets drunk and starts falling all over the place. There's a lot of interesting narrative design opportunities there that could be supported with sufficient framing. In essence anticipating many of the ways players will try to break it, and lean into them rather than block them off.

[@Kabodanki]

Always connected games otherwise your NPC will not work

[@mateit1479]

Hi, I would like to suggest a video idea. How does the AI work in the car racing game Flat Out 2? It is an old game, more than 15 years old. But I find it fascinating. In the game there are other 7 AI Bots Drivers. Each of them has his own driving style. Some are usually good drivers, others are usually bad drivers. Some are very aggressive, others drive clean. Some are agressive only if you hit them first. Some choose to take the risky road, other play it safe. Some are goo only in some specific tracks. Others became poor only if their car is damaged. The examples can go on and on. The fact that the contacts between cars are a majour part of the game complicates the things I guess. And it offers huge variety. But this is especially what makes the game great. How did the boys from Bugbear programmed the game? I think it would be an interesting video.

[@Maecmpo]

Was it running offline or in cloud?

[@AIandGames]

The conversation part was definitely in the cloud (it's Inworld's tech). But the other components I think are all offline.

[@djbeema]

Here's my first question: Are we meant to judge this solely on the conversational ability of the AI, or judge it as a total presentation. Because if it's the latter, I feel like it represents significant regression from our current video game NPCs. The animation and syncing is terrible (was a bit glarong how he repeated the same animations right before respondinf tonyou every time), the talking is incredibly awkward and stilted. At best it feels like you're talking to Siri or another helper AI. At worst it's severely offputting. I know you said they admit it's not ready for commercial use but I feel like that's an understatement. It also didn't help that Bloom looked like a total douche (honestly ubisoft needs to stop dressing people this way) and his expressions kept reminding me of Pete Davidson lol but that's probably a personal thing. Put it back in the oven for a while. If fully voiced and acted NPCs started getting replaced with this as it stands now, I'd be irate. Now, if we are solely judging it on its ability to respond accurately to you/unscripted speech, then it's pretty good.

[@AIandGames]

The conversation and animation part is Inworld tech which I've talked about before (and noted it has a lot of issues). What NEO is trying to do is the framing part.

[@Ajax-0137]

However interesting it looks, just remember that it's Ubisoft.

[@mjaada]

Impressive how fast also the facial animation is generated

[@KryyssTV]

Ubisoft executives are jumping on the AI bandwagon because they don't know that most of their customers click through narrative text and cut-scenes that are just lots of talking. This will be a marketting ploy that immediately loses its value once the novelty wears off.

[@KenTWOu]

Even if this tech will fail, their games could benefit from it, can't say the same about their ill-advised jump on NFT bandwagon.

[@KryyssTV]

@@KenTWOu I'm sure the tech will work but the issue is that once it's up and running the executives will force developers to implement it into their games and, regardless of how well it's made, time to have to be spent on getting it up and running instead of that effort being put into systems that offer genuine value to the player. What will happen is that players will get bored of it within an hour or so and then go back to ignoring the NPCs. And it's not as if this will replace essential narrative dialogue either because in those situations you have a fixed script.

[@KenTWOu]

@@KryyssTVI agree, my point was that since it's an AI research, even if it ultimately fails it will be valuable for something else: predicting players behavior during dialogues, figuring out optimal length of a conversation, recording of enemies contextual barks, etc.

[@KryyssTV]

@@KenTWOu Unfortunately AI isn't actually AI. It's very sophisticated mimicry and as such any development in one skill doesn't have much use outside of that context. The recent advances in robots demonstrated this as they use different AI for different features of the robot. ChatGPT for word recognition, a separate AI for visual recognition and yet others for managing the physical movement of the droid and voice synthesis.

[@CaptainJeoy]

Interesting...