Reverse Engineering Age Of Empires

[@nathanbaggs]

Want early access to new videos and some behind the scenes content? Consider becoming a channel member kzbin.info/door/QvW_89l7f-hCMP1pzGm4xwjoin Other videos you might enjoy: kzbin.info/www/bejne/eqetfXyNdqyhpZY kzbin.info/www/bejne/f328aKePma6GaLs kzbin.info/www/bejne/lWOqgJWZp9eKp5o

[@BANNED-GIRLS-AEGIS-AOE]

THEY need to hire you for aoe 5

[@Alt-gy7se]

It felt like you only just got started! As others have said it would be appreciated if you make these videos longer, and maybe even a part 2 of this video where you go further would be amazing!

[@nathanbaggs]

Responded in another comment but I basically don’t know how long a video will be till it’s done. Always looking to experiment with different video lengths though

[@bubbleboy821]

@@nathanbaggs Yes I would have liked to see a deeper dive

[@elbiggus]

@@nathanbaggs I'd vote for longer/more detail, or alternatively devote several episodes to a *much* deeper dive into a single game: off the top of my head, now you know how the AI gets its broader instructions what happens if you fiddle with them, how does it decide how to *implement* the instructions (what does it prioritise, how does it cope with resource shortages, etc.), what makes it decide when to go on the offensive, why did Geoff decide to go on an expedition, what strategy does it use to expand territory and explore the map, how does it select its targets in combat, is there a way of exploiting the AI, etc. Obviously a massive undertaking as it requires reverse engineering almost the entire game, but it would be fascinating to watch the process.

[@MobCat_]

@@elbiggus Yeah like 30 mins. You figured out how the file works, now you need to test your hypothesis by building new ai files with strange or fun behavior.

[@lewhughes3079]

Yep, what this guy said. :)

[@aruan7sp]

The AI in AOE1 is controlled by 3 files. The AI file lists the build order, PER their general behavior towards what they've built and towards other players, the PLY file defines how the AI attacks other players. The AOE Heaven website have guides for each file, but they're not all-encompassing since the AI wasn't publicly documented, it's sourced from modders and what they could find.

[@nathanbaggs]

Cool thanks for the insight. I try not to look too much up before I start as (for me) it ruins a lot of the fun in figuring it out

[@king_james_official]

@@nathanbaggsreal!!! but it also hurts when you re something and find out someone did the same thing 6 years ago lol

[@Loki-]

​@@king_james_official The story of life is people repeat a lot of what other's have already done, but the exciting part is giving it a try on one's own. ❤

[@AEGISAOE]

try the community made open ai for aoe

[@Mystixor]

While already discovered before, documenting the research process as if it had not been done before is still very educational and I'd love to see more!

[@Warbob11]

I came for the Wololo, stayed for the dissection of one of the games I played as a child.

[@nathanbaggs]

Wololo

[@SidTheGreat420]

Roggan?

[@ApocDevTeam]

Love it when people try to reverse engineer stuff in old games. There's a guy that spent years working on a 23-year-old WWII 2D real-time tactics game of which the source code was lost to time, and he managed to get modern screen resolutions working which nobody in the last two decades managed to do successfully (the game was hard coded with a crappy resolution with square aspect ratio). The difference it made was absolutely huge and it works flawlessly.

[@ErtugrulK]

Do you happen to remember the name?

[@ApocDevTeam]

@@ErtugrulK Sudden Strike (1999-2001) and its expansion pack "Forever". The author has videos on it on KZbin, HD mod it was called I think. Was a popular game back in the year 2000 in Europe, tons of mods and thousands of custom maps, but until 2022 nobody had ever managed to get higher resolutions working. Even the company which bought the rights to the IP years ago said they were unable to find the source code for the original games.

[@ThatNiceDutchGuy]

You mean drive.google.com/drive/folders/10PJJZEpNKPTI3qaGkq2riW-FhRaIDK4i made by Vladishav Ivanishin, and shared by Wilhelm Kop?

[@jamesdcuk]

I really appreciated this video Nathan. When I was about 10 years old and first getting into computers and I had access to this game. Curiosity drove me to try and unpick installation files in notepad but obviously didn't get anywhere. 20 years later I'm an IT professional and write software and for a living. The desire to learn how things work has not gone away. You've made the process of reverse engineering software as accessible as I think it can be, which is an real achievement, well done!

[@BrenIrwin]

This popped up in my feed and I audibly gasped. Eagerly and excitingly watching now. Keep this great content coming!

[@nathanbaggs]

Hope you enjoy!

[@BrenIrwin]

@@nathanbaggs I most certainly did. I'm needing to brush up on using Ghidra so why not also enjoy a video at the same time. Thanks!

[@patcoston]

I worked at Microsoft when AoE was being developed. My team would play the daily build at lunch time with 8 player games. We had access to the source code and build files. One of the employees created a custom version for us, making some improvements like unlimited population size. Once in a while we would crash the game since it was still in beta and we were pushing it in ways it wasn't designed. We had access to the daily builds of all software so we could try all the games being developed. We all got into Monster Truck Madness for a short time. I reported about 20 bugs that I found and got a thank you email when it released. We were playing truck soccer, and suddenly we each had our own version of the ball that the other couldn't see. A head on collision at the peak of a hill sent my truck into the sky only to get stuck there. Another time I reached the border of the map, and got stuck there too.

[@nathanbaggs]

What a wonderful story, thanks for sharing (: I bet you have some interesting war stories from MS back in the day

[@tommo4k]

One interesting AI quirk is in the original Dungeon Keeper. There's a map where you're pitted against an AI keeper, rather than heroes. If you spend too long building your dungeon, the AI keeper essentially just runs their own dungeon into the ground and all their monsters leave. You spend you're time building and training the perfect army to take them down, and when you're finally ready to go steamroll in and take them on all you find is a husk of a dungeon with a load of empty rooms and maybe two or three flies.

[@user85937]

Wow I remember that, I always took a long time to train most things to level 10.

[@theexplosionist2019]

The AI in Dungeon Keeper 1 and 2 is awful. Deeper Dungeons claims to have enhanced the AI but I gave up on the 2nd level.

[@VarionJimmy]

Really interesting reverse engineering! Especially the reveal of “this can’t happen before that”. It’s been 25 years since I did this kind of things and I’ve forgotten a lot of my tricks, so it’s really nice to follow along.

[@JayMaverick]

I've been learning the basics of C lately. I can't tell you how satisfying it is to finally be able to (somewhat) follow along with what you're doing!

[@koftabalady]

Ahhh a reference to the sausage variable from the horror game with spaghetti code written in Finnish 0:03 😂

[@markbevan5771]

I would love to see a in depth analysis on the grid path finding system it uses

[@sasquatchbgames]

Oh man the music throughout the video takes me back.. Really enjoyed this!

[@nathanbaggs]

Thanks! It was a real blast from the past for me as well

[@BrunoValads]

Would be amazing an analysis of Worms 2/Armageddon AI

[@Ciano56]

Yeeesss! This is the type of content I love from you Nathan. Take a game, and do a walkthrough analysis of exploiting it. Amazing, hope you can do more, and in way more detail!

[@WwortelHD]

This was a great watch, hope you can do more AOE debugging in the near future.

[@nathanbaggs]

It was great fun to do - I think I will probably look at something different next, but return to AoE in the future

[@1Bouncy1]

longer videos please!

[@nathanbaggs]

I am experimenting with video lengths. To give an insight into this particular video (and the challenges around it), I spent about two weeks doing all the reverse engineering at which point I looked at my notes and could see the story I wanted to tell. It felt like a lot of content but I don’t really know the video length till I spend another two weeks scripting, filming and editing. Basically I don’t know how long a video will be till it’s done

[@SirDragonClaw]

@@nathanbaggs This would have been better if it was about 50% longer (assuming you found the content), but every viewer is different. I know this felt a little short for me.

[@nathanbaggs]

Will certainly be playing around with different video lengths in future videos. Like I said I looked at my pages of notes and thought I had loads of content, but it all edited down to this

[@Exotic69420]

​@@nathanbaggsLonger videos also pay significantly more🤑

[@zgolkar]

12 minutes that was almost sure no less than 12 hours. No matter what, you have all my respect. That’s no easy feat.

[@nathanbaggs]

Yeah about two weeks of RE work (and another two to script, film and edit)

[@fuzzy-02]

This has the same vibe as watching National geographic on those investigation and historical shows. Had fun watching this video and hope you do a series of this. The end results does not matter, it was the vibe and the journey that was entertaining

[@nathanbaggs]

Thanks! That's kind of what I'm going for, just solving interesting problems using games people love

[@AEGISAOE]

are u familiar or heard of userpatch, capture age, openage, freeage, aegis aoe engine?

[@apresmidi153]

This is way above the level of programming I know how to do but it's very interesting to watch someone try to figure it out. I've played a lot of RTS games and used editors for those that had them and one of the things that really stood out for me about AOE was the fact that they had all of these descriptive names for the different AI scripts that specified civ and bulid order in the file name. I was confused when I started working with other games and was only some kind of "default" AI. Makes me wish more games would include some kind of more detailed AI script editor with their map editor.

[@HQbaracuda]

The level of programming isn't actually that high, when reverse engineering something like this. You do not have to come up with a solution to a problem you need to solve via coding and that usually is the hard part in programming. What he does here is just looking at someones code and try to make sense of what it does. The debugger is the main tool here, because it enables to stop code execution at any point, to look at the state of the variables used. The difficult aspect of this work is just renaming variables and functions to names that are more meaningful then the decompiler decided to use. Imagine this as some task you get from your boss to look at some code from another one of you coworkers and them being awful at naming their functions and variables and not using comments in the source code to describe anything.

[@batlin]

2:31 I listened to this bit 7 times to eventually get that it was "ProcMon" and not "ProcMutton". Also yay for Zerg rush; my uncle used to play AoE2 and was pretty good against the CPU but got trashed in online play. I was watching him do an online match once and he explained how everything would seem fine for about 5 minutes, then it would happen. Sure enough, a few minutes later his base was swarmed by an endless stream of cavalry units, followed rapidly by a loss. [edit] BTW I really enjoyed this. I might suggest one tip if you haven't tried it already: to make life more enjoyable when doing the "window dance", or for setting up a new game exactly the same way many times, it's worth automating with tools like AutoHotkey or AutoItScript.

[@1kreature]

Love the Star Trek reference! "I cannot do it cap'n" I don't have the power!!!

[@Beansswtf]

You have also got to think about things like player interaction. Like if you went over and destroyed their barracks, they would need to replace it to continue making troops. That 4th column definitely looks like it's where the action needs to be done. -1 just seems to be like universal, placing buildings etc which you wouldn't need to click on something to do. Whereas Man is 109, which looks to be the TC, and archer is 87 which looks to be the range.. Pretty cool tbf! Would be very easy to tailor the instruction set to difficulty by just making the build order absolutely perfect or making it terrible haha.

[@Wockes]

7:14 the AI got confused because it couldn't find any berry bushes or animals to hunt(except Elephants but it avoids those in the very early game).

[@u9vata]

I only reverse engineered the cracked version not being properly cracked back in the day. Fun-fact was that the RZR version was like 30 megs and contained the full game, while the full with CD was much bigger (also when installed I think) so I preferred that. However they only seem to have cracked for single player and multi did not work so I did it with softice, HIEW and some other similar tools. How great days they were - good to see people still do these kind of stuff 🙂

[@nathanbaggs]

Thanks for the story - it's great to know I'm just a small part of large community that loves playing (and breaking) these games

[@u9vata]

@@nathanbaggs I did reverse GT97 Racing though - because all version online are buggy and the time for checkpoints go too fast - but its a smaller game and dos game - likely my latests achievement in similar vein but smaller than looking around the AI of a game... Interesting finding there was that one can actually use dosbox as a debugger for smaller games and it eases a lot 🙂 I know there are people who reverse still - but always nice to see. There was this russian guy who created awful lot of DLLs for making games run over opengl instead of DX and often helped them work better on linux for example.... many great people - just never saw ai reversing before ;-)

[@raztaz826]

You make it look easy. I tried to reverse engineer a 1mb command line program using Ghidra and it was too hard (1 yr later the company open sourced it anyway).

[@nathanbaggs]

I cut out a lot of the misery and me banging my head against the wall

[@gristlelollygag]

@@nathanbaggs as a programmer very unfamiliar with reverse engineering, it would be cool to have a "behind the scenes" as a separate video to see how you tackle problems (all of the ways in which you've been banging your head against the wall lol)

[@CSDex]

This is great content. The format of walking through your thinking process and working through reverse engineering something is a great way to teach and get people excited about this subject. Thank you, keep it up!

[@LeftJoystick]

A comparison of sorts between Age of Empires and Age of Empires 2 in this style would be awesome too

[@spirittouchrecords2785]

would be great if you could find a way to make the game "TechnoMage" completely fullscreen, because it runs on modern computers only with a visible title bar on the top. That german game is very underrated and beautiful.

[@ysakhno]

It is easier to debug a game or app that uses DirectDraw to run in fullscreen mode on a 2-monitor setup. You use one monitor to run the game and the other one to debug. Alternatively, if you have 2 computers, you can use the remote debug feature (run the game on one computer, and debug on the other one).

[@thatLukeKneller]

mate, you're my hero. I loved doing this sort of thing as a kid but never mastered it like you. I want to get back into it. Thanks for this!

[@MrTalisauros]

Incredible video, as always. Keep it up!

[@nathanbaggs]

Thanks, will do!

[@saliherenyuzbasoglu5819]

This is the coolest thing I have ever seen! Immediately subscribed I need to learn how to do reverse engineering

[@nathanbaggs]

Welcome!

[@merseyviking]

The Spiffing Brit has shown that people do watch long-form videos. Something like this needs a good chunk of time devoted to it.

[@nathanbaggs]

This is a fair comment (and one others have also said). The issue I face is that there's no clear correlation between time spent doing reverse engineering and length of video. This was about two weeks of RE effort, which I thought was enough to tell a story. But by the time I scripted, filmed and edited this was the length of video that fell out. Just a behind the scenes insight - I'm always experimenting with content length and type

[@merseyviking]

@@nathanbaggs Yeah, that's fair enough. I guess you have just hit the right formula to get our interest and we're demanding more because it's easy for a viewer to assume you should be making content every waking hour!

[@HQbaracuda]

Also reverse engineering source code isn't actually entertaining for anyone else at all. There are probably hours and hours of just looking at values in variables. Good luck on commenting in an interesting way while doing this and keeping up on the mental concentration to not loose track of what you where about to track in the code snippet you are watching in the debugger.

[@serdaar]

This level of debugging!! It's amazing to watch. Thank you

[@praetorprime]

I'd be hyped to see more reverse engineering of this game. The pathfinding in particular always seemed a bit broken.

[@nathanbaggs]

There's certainly more I can go into in the future

[@ZColdrick]

If you would like a challenge you could attempt to get the players colours from game memory in Company of Heroes 1. I've been trying to do this for a while and its hard because the colours appear to be assigned by slot order when start positions are fixed but some kind of random algorithm when the player positions are set to random.

[@petemc4190]

turns out I most needed to hear the AoE menu music today then learn something cool!

[@iamtimsson]

2:55 ty internet dad this is funawesome

[@davidg421]

I've heard LLMs are good at deobfuscation code, what is your take on it?

[@GuildOfCalamity]

This just became my favorite channel

[@nathanbaggs]

Thanks!

[@treeoflifeenterprises]

the person who wrote userpatch for aoe must have done a fair bit of reverse engineering, because he managed to enable modern resolutions, increased population, update and improve ai capabilities, and fix many bugs in the map editor. interesting to see the tools used. can they generate c as well as asembler?.

[@AEGISAOE]

mostly cheat engine to get pointers. ik well the creator of up

[@JuanxDlol]

Now imagine what the guys behind Diablo 1 HD had to do with their reverse engineering, this is a very intresting insight.

[@YetiWizard]

I find that technique where you decompile the whole exe and then search for a particular offset in order to find where a member variable is used really interesting. A couple problems I found with that is that first, sometimes the decompiled code uses for the same member a different type cast, so 0x160 cast as an int becomes 0x580 cast as a byte (4 times that value). Second, how do you deal with low offsets, which are super common (e.g. 0x08)? I'd be awesome if you go more in depth with this topic in your livestreams. Thanks for the awesome vid!

[@nathanbaggs]

It’s not a perfect technique, but can be useful as a way to quickly search for offsets

[@lucasgoncalves8231]

I would literally pay to learn those skills

[@matthias916]

Id love to see your thought process while reverse engineering

[@nathanbaggs]

I try and capture as much as I can in the video, but a lot of the time it's just me looking confused

[@BlueFlash215]

Almost all screens were 4:3 back then. There were barely any customer models other than 4:3 hence why you only need one value for the resolution. The 800x600 automatically was chosen when you chose 800. Is this some very minor optimization?

[@samirbenabadji9311]

Is it still possible to create a map hack on age of empires 3 definitive edition still today? Even tho the game got a lot of patches?, and are you planning to get into age of empires 3 after you're done with aoe 1? Thank you for those videos, im just getting started

[@nixielee]

This was great and all, although very short

[@jameskobulyar786]

I stumbled on your channel by accident. How the hell did youtube know I wanted this?

[@saultube44]

Mr. Froddo Baggens of RE, why not you use your Ring on Starship Troopers, the 1st old RTS one, you command like 10 soldier with different abilities, in 3rd person; the thing is, the game will just bail, sometimes not even a error message; there's a specific mission I remember that will trigger this probably: is a mission where the Engineer, in a 'Mech Suit has the unique opportunity of all the missions, to deploy as many land-mines as you can set, you'd be alone, so be careful, and when deploying like 500 of them, the game crashes, other missions also have unknown limitations like that. Also DID F-22 ADF when you go at the right edge of the map, it'll exit without error, you don't actually get to the edge, but very close to trigger the exit. If you could fix these and increase the capacity of the buffers, or whatever gets overrun, and teach ud how to fix it, would be great and have our eternal gratitude

[@RootiferasRetroGameplay]

I've been playing the original AOE since its release in 1997 (last time I played was 2-3 days ago) and I never knew there are AI files in the game haha. I never checked. Interesting.

[@JamesTDG]

Hey Nathan, I would like to suggest another game to reverse engineer, and build a resolution patch for, Peggle. LOVE playing the game, HATE the resolution it runs at in windowed mode. It would also serve as a great tutorial for me to apply this info to develop personal patches to further games, such as the other versions of peggle, or sonic and sega all stars racing (While the resolution is nostalgaic (seriously IDFK how it invokes nostalgia for me lol), I want to use the full 1080p resolution of my monitor)

[@nikolainyegaard]

It’s fun seeing your channel grow! Keep at it :)

[@demipy]

Thank you for these videos! ❤

[@itzmorgoth]

Didn't know about windows key and tab. Thanks

[@oussama7132]

can you check if the remake uses the same system? maybe you can't debug it because of the drm (unless you get it from a questionable source)

[@doshamitv5020]

Are you open for jobs? How we can contact you ?

[@gaming.64bitdragon]

very happy I found your channel. Would love a collab one day

[@andlinux]

This is what the people at GOG do, a lot of respect :)

[@vast634]

So GOG is patching the old games to run on new hardware? I thought they only provide the platform.

[@andlinux]

@@vast634 That's what they do, there's a video of GOG where they explain all the things they do, it was very interesting watching that. I guess you can find it on youtube.

[@vast634]

@@andlinux Ok, will look for it

[@NowhereBeats]

I know absolutely nothing about computers. The whole video might as well have been in Arabic. But I did learn 'windows + tab' which I found helps me a lot. So thanks.

[@99seconds76]

I love your videos, they're very informative

[@RubenKelevra]

Okay, so the files are *not* the AI but just basic instructions what the AI should build in which order. But how does the AI work?

[@TheRickyJohn]

I wonder if the same type of practice / application could be used to disassemble car ECU Code?

[@grubzer1369]

Can you do worms armageddon? I have no idea how they did ai that is able to adapt to such dynamic environment

[@gergelykalman9822]

Great tip about dumping the sources, thanks. Also it seems that you cut it way too short, I could watch many more of these videos.

[@BreezeTalk]

Thanks for explaining so well.

[@ThePiko69]

I love this RE series

[@FridiNaTor]

I would love for you to check out the code for Full Spectrum Warrior, Idk just for fun, I like these videos so much for some reason. Oh and Ghost Recon as it's main menu resolution is "hard coded" which is so low it makes it unplayable on some monitors.

[@anon_y_mousse]

I've always wondered why Microsoft chose to name mangle basic functions by adding an underscore. I suspect it had something to do with not wanting name conflicts with kernel functions, most of which don't use an underscore, but I guess I'd have to ask someone from Microsoft that question. However, it is odd to me that Ghidra wouldn't automatically correct that name mangling.

[@ButtonBashOfficial]

Do Age of Empires 2, that’s the real gem in the crown!

[@yigityargic2814]

Can you please take a look at the unit pathing in Age of Empires 2 DE, and identify the pathing algorithm? It has been a hot topic in the game community for at least a year

[@bagofmanytricks]

I've been a programmer for over 30 years and I'm still not even near the debug-magician Nathan Baggs seem to be. I feel like it would really up my game, but there's so many new things to learn too. =)

[@nathanbaggs]

Always be learning

[@Dmitrii-q6p]

wierd way to play with AEGIS

[@stevobz]

When you want to patch out an instruction, what is the next step to run the patches version of the game? Does it need to be recompiled or…

[@nathanbaggs]

That’s a good point, sorry I glossed over it. Ghidra has an “export program” option that allows you to save a new copy of the binary with the modifications

[@stevobz]

@@nathanbaggs ahhh awesome, thanks for clarifying. In your defense, you’re the only one who answered this question which I’ve posted on several rev eng vids

[@elpinguinofrio4911]

would be interesting to see if sonnet/gpt can do this function guiessing to speed up overall process

[@dahahaka]

Where's the next video >:( 5 hours passed already

[@monad_tcp]

I hate that thing with debugging a fullscreen application in the same computer because of focus. I always use remote debugging for this reason, usually I run the debugee in a virtual machine.

[@nathanbaggs]

I don't think x64dbg has remove debugging (maybe as a plugin?)

[@shashanko]

I’m so glad (and a bit teary eyed) to see the O.G. AoE…started with this in 2000 maybe and then found AoE Rise of Rome Expansion Pack. Brings a lot of memories. I still play it with my brother on Steam. And no offence but I’ll never be a fan of another AoE variant. And btw, great work on the back tracking

[@GrandpaGameHacker]

Nice one, indeed RE takes a lot of time ey. AOE1 was also one of the first games I hacked/modded as a kid

[@Dragon20C]

Can you do a video on lego racers, love to see how everything works.

[@rodrigobogado653]

I finally came across this gem of a video!!! Thanks in advance bro, I subscribed immediately!!!!

[@BioHazarddasdadfasfsad]

Can you do Warzone Tower Defense? It was one of the first classic tower defense games but it is .swf file when Flash Player was alive

[@jeinnerabdel]

I see "Age of empires" in the title. I upvote immediately!

[@nathanbaggs]

Wololo

[@AEGISAOE]

@@nathanbaggs rogan?

[@tharakakarunarathne2420]

Can you add more range to British archers and play a game? That would be awesome

[@Malard779]

love the video but why is your monitor resolution so insanely small, run two monitors and you should have no issue with your debugging!

[@randomshs]

have you ever done a video on how to use Ghidra for absolute noobs? like explaining what everything is, useful hotkeys and configs, etc. I think it would be very cool to learn from you

[@mikeswheels]

i spent hours playing as an adult! still my favorite!

[@Vodouch1]

Hi, wondrous job. Just i think you partially reinvent the wheel. Check out "How the AoE2 AI Thinks (ft. Promi)" by Spirit of the Law. It breaks down AI concepts of AoE2, sure it will be different but some parts might be similar enough so you will not start from 0.

[@AEGISAOE]

are u ai scripter as well?

[@ownhaus]

Please do more age of empires reverse engineering

[@rogercruz1547]

9:40 I'm getting crazy ... when you finished this sentence I said "linked list?" and you confirmed it. I'm getting whatever that disease you have is called.

[@GoldBallTV]

I think this is similar to what the people who took over AOE had to do if I recall

[@Nunya58294]

Saw the title about shit my pants lol

[@nathanbaggs]

I’m pleased it had such an impact

[@Nunya58294]

@@nathanbaggs oh absolutely good sir

[@szilardtoth5642]

I know it wasn't the focus of this video, but how did your AI dude hunt elephants? I play exclusively on RoR expansion and the AI never hunts elephants there, unless they attack priests.

[@Ultimate-Gamer1983]

how to reverse engineer 'age of empires: just throw it in the recycle bin.. job done! XD

[@moonootoo]

If you could do Starcraft AI after this series that would be great!

[@RAMMY237]

That was so fun, I want more

[@JohnDoe-xv2sl]

Nice format. In a short video a lot information.