Playing to Lose: AI and "Civilization" (Soren Johnson)

Google Tech Talk
August 26, 2010
ABSTRACT
Presented by Soren Johnson.
Artificial intelligence is crucial to any strategy game, providing a compelling opponent for solo play. While many of the challenges of AI development are technical, significant design challenges exist as well. Can the AI behave like a human? Should it? Should the game design be adjusted to accommodate the limitations of the AI? How do we make the AI fun? Should the AI cheat? If so, how much? Do we even want the AI to win? This session suggests some possible answers to these questions using the "Civilization" series as a case study. Ultimately, developers must choose between a "good" AI and a "fun" one, with an understanding of the trade-offs inherent when deciding between the two.
Soren Johnson was the lead designer and AI programmer for Sid Meier's Civilization IV. After working at Firaxis Games for seven years, Soren joined EA Maxis in 2007 to work on Spore as a lead designer/programmer. He is currently building web-based games with EA2D, such as the moddable strategystation.com and other unannounced projects. He also writes a design column for Game Developer Magazine and is on the GDC Advisory Board. His thoughts on game design can be found at www.designer-no....
Download slides to this presentation here: www.designer-no...

[@EoghanFallon]

I return to this every few years as Civ 4 is my fave game and this was a cool talk from that perspective but now I am studying to become a software developer it takes on a whole new dimension. Just awesome.

[@whyshouldwecare3267]

This was a very interesting talk, the guy was very open and explained concepts well. Thanks

[@sedghammer]

This talk is truly awesome and one of the best talks on game design I've seen. Thanks for posting.

[@kkanoee]

11:15 "Go AI is much much worse than Chess AI" yeah it was :D

[@rendros88]

It still is

[@Heliaced]

You cannot really compare them today. Deep Blue used brute-force AI to beat Kasparov, but that has severe limitations in the context of Go, because of the near-unlimited outcomes. Deepmind's AlphaGo uses deep neural networks, so it trains itself through repetition. Given enough time and a large enough network... if you were to train Deepmind's AlphaGo to play chess, it would eventually end up beating Deep Blue, or drawing every time. As we learned recently when Elon Musk's OpenAI beat a pro-gamer at Dota2: It was noticeable how it used faking tactics and "playing the player" to get desired outcomes. I imagine this is something Deep Blue would never consider as a good move, and this could be leveraged by a deep neural network, because it has tried and tested (and remembers!) the real outcomes of such scenarios.

[@reallyf8717]

@@Heliaced you cannot effectively compare them because both beat humans by significant margins, and beyond that you are comparing apples to oranges. you also cannot naively claim that deep neural networks as a an architecture is intrinsically stronger than "brute-force search" because, in the case of chess, the chess engine Leela (neural networks) is weaker than Stockfish NNUE (hybrid neural net and bruteforce search) i would also add that openai is not "owned" by elon, not anymore, and not before, as he only co-founded it

[@BuildinWings]

They don't mean you're not working at game design when they call your design choices "lazy." They're saying that your creative choices did not engage them, or your chosen means for the game to express those choices are poorly thought-out. Invisible walls are a great example of this in action.

[@Fluxquark]

Great talk but unfortunately it seems Firaxis has not heard it because the current Civ VI AI is neither fun nor challenging, mostly annoying lol

[@possumverde]

Why not have the AI code determine what it's "best" X choices are then include a modifier (based on difficulty setting) that determines which of these options it can choose from and roughly how often it chooses each. For example, settler mostly goes with it's "worst", occasionally with the next to worst and never with the top 2 or 3 then adjust that range all the way to deity which would be the opposite. That way you have an AI playing to win on harder settings but easy enough to be fun on lower

[@timothymclean]

That could theoretically work for a game like Chess. Civilization, though? It's a lot harder to define what a "good" board-state is, or to program the AI to understand how starting to build the Pyramids will influence the board-state a hundred turns down the line. And of course, the AI isn't the only factor in play. It needs to account for actions by the player (and other AIs), which makes the numerous decisions each civilization makes each turn add up quickly. For instance: Imagine that you have two warriors that can move one tile in any direction (or not move), a scout which can move to anywhere within two tiles, and a city that can build half a dozen different things. There are 9×9×25×6=12,150 possible turns you could take, even ignoring free diplomatic actions or what tiles your city works or other little things that players _can_ mess with between turns but usually _don't._ And that's per civ, per turn, _in the early game._ It would be pretty much impossible for the AI to evaluate board conditions more than a few turns in advance without spending multiple minutes on AI turns. A human player can eliminate most of those possibilities (they probably won't charge the scout at that barbarian camp or sit the warrior two tiles behind their city), but that requires intuition that the AI _does not have._ Unless we program the intuition, in which case we might as well just use that intuition to guide AI behavior. Which is basically what Civilization AI does. P.S. Also, if the AI blunders badly or weirdly enough, it becomes obvious that it's making random moves every so often.

[@Spoonwood]

I disagree with Mr. Johnson's comments about the AIs targeting behavior in civ III being bad. In Civ III the AI's targeting behavior makes it possible to spawn a military great leader on some archipelago maps with a small army. If you control an island all to yourself, you can empty out all of your cities before amphibious warfare. The AI will then land units next to your capital or as close as it can to the highest value city (which is also likely near your capital). One can keep a stack of artillery-type units (trebuchets/catapults/cannons/artillery proper) next to a unique landing spot (pick the weakest square on the coast near your capital) with offensive units also. Then after the combined arms attack all of the units there can get defeated quickly. Since the units in that stack do not have to move around, the first elite becomes more likely to engage in battles at a greater frequency and thus spawn a military greater. Such a tactic, I've found, has high value for 20k games, because the desideratum of spawning a military great leader becomes more complicated with other tactics. Such targeting behavior also enables that warring AI to seem like less of a pure enemy, but more like a frenemy, who can help you win your game more quickly. For non-20k games that warring AI could also help you win more effectively on some archipelago maps, since you can probably get more armies or armies more quickly.

[@qwertyb18]

I believe they should work on future games sort of like Bethesda does in there elder scrolls games, and that is to specialize in a specific game concept or factor while simply transferring over everything else from it's successors. I personally agree with him. The AI "personalities" should be improved on. (im my opinion it's the most important feature) How would that leader react to this circumstance? This should be the overall goal to AI development in my opinion.

[@JuraIbis]

Putting a fake 10 turn limit on breaking alliances is a dumb, easy way to approach AIs. I would rather go with a "human trustwortiness" meter, and the AI would share that rating with each other, making it so the AIs would treat your word accordingly to how much value you put to it. If you were at peace since 50 turns it would evaluate the chances are low you will break it, and if you've broken up 3 pacts before it would treat your word like garbage.

[@gammelgymsokk]

im guessing this guy didnt work on civ 5

[@x6xk1LLx9x]

using this video as a source for my psych paper, thank you so much

[@Hedning1390]

At the end: "Civ 5 will solve the amphibious assault problem by not requiring troop transports." lol. That only exposed AI stupidity even more. I'll just pick off their unprotected tanks with my caravels one by one.

[@BelligerentWombat]

I think "lazy" is a bit harsh. I'd say more "pragmatic" given the time and resources it takes to make an AI. The game itself takes long enough to make, to make a properly incredible AI would probably take an exponential amount of time to make in proportion to the complexity of the game..... and Civ IV is probably one of the most complex games out there.

[@mpanty]

Fascinating stuff

[@Buggaton]

I cite the AI in Warhammer: 40k "Dawn Of War". The AI in that game, in the original on the most difficult setting was fantastic. I would always win a 1v1 against them but only after huge amounts of practice. The AI is spectacular and DOESN'T CHEAT. Against 2 AIs they sync with each other and play brilliantly. They're even more difficult 2v2. And so on, 3v3 and 4v4 are just much harder. I once beat 3 AI with 1 human ally but we had to resort to cheese and luck. It *is* possible.

[@Buggaton]

I was talking about the original DoW without the terrible, terrible expansion packs. The increased types of units and the destruction of the interesting differences between the factions resulted in the AI for Soulstorm and the other expansions being massively inferior. I never played DoW after people stopped playing the original online. Did he make any mods for that?

[@MrScherbert]

This is a very enlightening video. Thanks.

[@IADaveMark]

@Luke1d20 This is a pointless comment unless you define "cheating". The AI doesn't have eyes, is it cheating for the AI to look up the precise location of the player?

[@BelligerentWombat]

After reading some of the source code is certainly doesn't cheat as much as you'd expect. Doesn't peek to see if you're making wonders, doesn't even peek to see what tech you're researching (although it does know, as the player also does if a tech has already been researched).

[@JimBob1937]

@frenzyfol And what logic and factors would you use to achieve this goal? You yourself use a multitude of factors and rather complex logic based on your own intelligence, environment, and past experience (aka you learned) for the most basic decision of which cities are vulnerable to attack. Try recreating this logic in a game AI and see how well you fare.

[@Magicpancake123]

@ORCA4312 Where does he do that? Part of the presentation is about it.

[@SindarinElealar]

In other words, we can agree that corporate games (or any other product) will never be perfected and thus we need to do something about the economics to fix this and make it profitable to release fully developed projects.

[@CravingForLiberty]

i think soren johnson establishes a false dichotomy here (roleplaying vs playing the same game). it all depends on game design: if a strategy breaks the roleplaying, eliminate it. design the rules so that roleplaying is the same as playing to win. have a look at the paradox titles (eu3 and vicky2): they do that pretty good by having a few simple mechanics (stability, infamy, religious turmoil) that move strategies that are playing to win and those that are roleplaying close together.

[@AnonSeacat]

But AI in Paradox games is usually terrible? So it doesn't help anything.

[@sooooooooDark]

so pretty much: good ai: ixtj fun ai: exfp (so its just like in real life)

[@MrTomcus1234]

I came here from Civ IV marathon, looking on how exactly AI cheats to be able to counter it XDDD

[@felipebrandao3996]

55:55 Old World!

[@jkp1187]

@KarlKFI Yes, damn those businessmen trying to put food on their tables...

[@GabiN64]

The last great civ designer.

[@CravingForLiberty]

his example of religion providing a plausible casus belli is actually not a very good one: yes, if you are owner of the holy city you want to spread your own religion in civ 4, but other than that you would have no gameplay-reason to dislike players with different religions - yet the ai is doing exactly that for roleplaying reasons in the paradox titles having other religions spread actually causes nasty uprisings plus you get valuable casus bellii on heathens, providing a real incentive for war

[@BelligerentWombat]

You're wrong the AI in Dow was shit. Very easy to defeat, HOWEVER there was this amazing modder called Thuddmesier (or something like that) that created DowSkirmish. His AI was absolutely incredible in contrast to the vanilla AI. Shame Relic never hired the chap. Its worth noting as this guy says a lot of the advantages that AIs get in multitasking in RTSes is lost in Civ IV as its turn based making it much more difficult to make a challenging AI.

[@kabloosh699]

Yeah, thats cause for some reason they never seem to build a Navy to protect their ships.

[@urmo345]

And yet there were (and are) trolls around who says that Civ 4 ai does not cheat. Well, you can always cheat back anyway, LOL

[@360tomahawkill]

Acraplast video 1 hour and now this one?! sorry Goog im gone :/

[@KarlKFI]

@Luke1d20 & @frenzyfol Game developers do not have infinite time nor infinite resources to devote to AI development. Expecting a perfect AI in a commercial game is unrealistic. Sure they could have fixed it after they realized it was a problem, but that means either longer development or more maintenance after release. Then it becomes simple economics: "Is it worth it in dollars, reputation and/or time for me or my company to solve?" Blame the Businessmen.

[@jgcooper]

the guy speaking at 53:17 sounds like Microsoft Sam

[@fascistphilosophy5649]

I'm wondering if letting AI do our work for us will lead to making us intellectually degenerate .. but then I think having the challenge before us might compensate. engaging in aggressive dialectic with the AI, I think, would be more fruitful than degenerative. ...anyways wtf is CIV AI getting so much worse over time. I remember losing alpha centauri games. ...those were the days...

[@radoslawmazurekwaw]

Cheating AI is only woraround where is no good AI.

[@TheWombatula]

Who the hell do those two guys at 4:37 think they are, eatin noodles in the middle of a tech talk.

[@altfuns]

Me gusta más ver que hablen de tanking, healing y DPS en un entorno académico, que ver periodistas y "expertos" hablando de #SocialMedia

[@bendover1854]

It's impossible to make AIs that can beat you without cheating (mostly). For example, in HOMM 6, easy AIs started with about 200% more resources and they had at least quadruple my amount of units.

[@swahiliranger1022]

Isn't this the same guy who admitted the A.I. programming was garbage in the game and was totally done on accident?

[@Nerdcrusher]

"Uhm."

[@timothymclean]

24:40: Judging by all the funny ChessGPT videos from last February/March, it's actually a good thing if a chess AI cheats badly enough. (Example: kzbin.info/www/bejne/fZ_Ig5h_icaFrNU )

[@frenzyfol]

27.00 Disagree. Write some better AI that can identify cities that are vulnerable to attack.

[@MrGoodinton]

I know...

[@2Cerealbox]

This is an interesting talk, but the AI has been the worst aspect of the recent Civ games. So it's kind of a talk about how smart people rationalize making poor design decisions in a massive title.

[@2Cerealbox]

***** Good to know. I'll give the talk a second watch with a more open mind.

[@jizzinmypants191919]

Civ 4 was the best Civ of all time. I'm still addicted to it to this day.

[@fraye]

same :))

[@sickre]

Soren said it himself. In Civ 1-4 the Game Designer was the AI Designer. This was not the case in Civ 5 and 6, AI was an afterthought. Plus, pathing units with 1UPT is basically impossible. They need to get rid of or substantially change their new 1 unit per tile system.

[@AnonSeacat]

To be fair, you say Civ IV had "decent" AI but even it had a "Better AI" mod. It's AI was mostly better by the virtue that stack-based warfare is easier for the AI to handle than the traffic jams of one-unit-per-tile. Not all, but most of the AI deficiencies in Civs V and VI can be laid at the feet of one unit per tile.

[@lmackenzie89]

Good talk - but it's pronounced amph-i-bious, not amph-e-bious.

[@anonymous243124]

Im sorry but the AI in the Civ series is subpar

[@AnonSeacat]

The same could be said for all large-scale strategy games that I've seen

[@Limosical]

, you know,

[@DXRemix11223]

thumbs up if wached the whole video lol

[@ORCA4312]

AI designer denies his AI cheats. BIG surprise.

[@TimJBenham]

The most galling aspect is his contempt for the players. They're too stupid to notice if the AI cheats, and if they do they're just being whiny little hypocrites because they all cheat anyway.

[@itspheno]

I realized the blatant cheating in Civ5 on prince mode! PRINCE! I won't ever play another Civ game ever again. What's the point?

[@petervansan1054]

***** Yeah AI can keep units while being negative which is ridiculous.

[@nvtschip]

I am REALLY interested in thsi topic but between the smacking of his lips and the fact that he is the WORST public speaker I have heard in months... I can't get through this.. I fall asleep after roughly 10 minutes.