AI Learns To Play Lethal Company

I taught an AI to do the ship command role in Lethal Company. The AI mans the terminal to view the map, and communicate all the information to me. It can also counter outdoor enemies like dogs and giants. At the end, me and the AI play a full game to see how many quotas we can get.
Watch the final run here: • I Played Lethal Compan...
Source Code: github.com/Synt-Axe/lethal-co...
My Discord: / discord
Chapters:
00:00 Lethal Company AI
00:18 An Old Friend
00:49 Roberto is Born
01:05 The Map
02:41 An Upgrade
03:20 More Numbers
04:54 Hot & Cold
09:03 The Grid
10:54 Outdoor Enemies
11:21 The Giant
11:47 The Dogs
12:05 Commands System
12:47 The Final Run
Similar Videos:
Markiplier - WE LOVE THE COMPANY | Lethal Company - Part 1
• WE LOVE THE COMPANY | ...
Bread - What does v50 hold for us? Lethal Company
• What does v50 hold for...
b2studios - AI Invents New Bowling Techniques
• AI Invents New Bowling...
AI Warehouse - AI Learns to Walk (deep reinforcement learning)
• AI Learns to Walk (dee...
Sebastian Lague - Coding Adventure: Making a Better Chess Bot
• Coding Adventure: Maki...
AI Tango - AI Learns to DESTROY old CPUs | Mario Kart Wii
• AI Learns to DESTROY o...
Code Bullet - AI Learns to play JUMP KING
• AI Learns to play JUMP...
Primer - Simulating the Evolution of Aggression
• Simulating the Evoluti...
OpenAI - Multi-Agent Hide and Seek
• Multi-Agent Hide and Seek

[@princesscadance197]

'You're about to die.' in such a robotic tone had me in stitches

[@eggsYT]

Fr scared me for a second as well.

[@cursedhawkins1305]

Just needs to replace the voice with the quote from the Red Queen and it'll be perfect.

[@tpd1864blake]

I wish that was kept in the final product in some way. Maybe transmitting “UR DEAD” in the signal transmitter thing when it detects a lot of enemies around you

[@BrayDawg19]

F word

[@Arkmin]

reminds me of GLaDOS

[@calthitus3287]

11:33 I did NOT expect there to be a dog watching the giant too 😭

[@LilBuddyArmy]

"You seeing this shit?"

[@bigjezza]

dog was chill like that

[@spamtongreatsalesmanspamto4163]

​@@LilBuddyArmy *Yep*

[@eliasdex1236]

10:34 Roberto was haunted by the ghost girl and when he died to the dog, the ghost girl went on to haunt you instead

[@synt_axe]

Haven't thought of it like that. That makes a lot of sense.

[@spynix0718]

You can get haunted without entering the building?

[@eliasdex1236]

@@spynix0718 Yeah, she picks her targets by specific conditions even outside the building.

[@crem444]

I thought it was based on paranoia? Or is that only a small factor?

[@cmmpr111]

​@@crem444Smol.

[@LacourWave]

4:22 "The numbers Mason ? What do they means ! Mason "

[@user-wq6ow9om8q]

call of duty black ops be like

[@Lilacguitar]

almost sounds bingo related

[@paulmcmillan1831]

I just got flashbacks

[@JustHereForTheDialogue]

5:03 that bracken snap fucking killed me 😂💀

[@yyeetmax2849]

the one at 4:07 is straight up overwatch from half life 2

[@Silver-Arrow]

automatically the best variant

[@ERROR-yl3un]

Overwatch reports hostiles inbound. Keep sightlines open and sound on

[@yyeetmax2849]

@@ERROR-yl3un affirmative, scalpel-3 on standby, scalpel-4 reports possible biotics on site 3B, directions?

[@digi284]

Make this the new mimic AI

[@angelmatiastorres]

Oh god.

[@kylieeeeep]

No. NO😭

[@zedra4457]

this will be more fun

[@DrunkMoth]

I feel like the 1st method of loot finding has a lot of potential, you just have to present the information in a more human friendly form. Maybe using compass-like directions(front, right, left, back) and a scale from cold to hot to measure distance? That way you could process the information much easier. Numbering the items might also help(something like "items two, first left warm, second front cold"). Incredible idea, loved the video!

[@sagey08-backlash]

i don't think this is real

[@fandovec03]

I have similar idea in mind. Instead of using only numbers, AI could interpret the information in more understandable format like "Nearest item: 10 oclock" and when combined with hot-cold system, it would be very helpful. Additionaly player could use voice commands through radio. For example if player asks where nearest item is, AI would answer "Nearest item: 10 oclock, nearby". Last thing that could be added is whenever AI detects something, it would give a player an update "Enemy on radar, 6 Oclock", "Item on radar, 4 Oclock".

[@soupcangaming662]

LOOT, 6 O'CLOCK, COLD. TARGET, 3 O'CLOCK, COLD. TARGET, 4 O'CLOCK, HOT.

[@soupcangaming662]

@@fandovec03 You could also request radar booster flash/scan, or dropship with specific item/cheapest item (distract) i.e. [AI NAME], FLASH ALL. You could have the AI request confirmation for any recognition mishaps or use text commands like [AI NAME]_FLASH_ALL.

[@thema1622]

11:32 Hahaha I love this part

[@nilaier1430]

Despite how cold the name Roberto is, you can't deny how hot his voice is!

[@NubE-Co]

What

[@lordfrogkingofkings2533]

@@NubE-Coyou heard the man

[@randomisfoda]

his name is hot too. i want to date roberto

[@sqwimpy-4]

Why are there so many weirdos todays bruh.

[@CattusC]

4:22 you've fucking made an AWACS

[@hypeman958]

what's that?

[@CattusC]

​@@hypeman958big radar plane that tells their side where other stuff is

[@Artemie-np3qu]

⁠@@hypeman958 Airbone Warning and Control Systems. Big radar slapped to a plane (these aircraft are the ones with the big circle on top of them) that acts as a mobile radar for their forces.

[@foldervtolvr]

@@hypeman958A plane with a big radar that’s notorious for just throwing numbers at pilots and expecting them to know what it means. An average AWACS callout has I think 4 separate numbers, and that’s per aircraft or group

[@jikarin4575]

For the Company

[@davidsmith3382]

I would change the grid to a 3x3. It'll fit in one message, which will make it a tad bit easier to understand, even if it is slightly less precise

[@kylieeeeep]

Agreed

[@fluffydoggo]

Itd be cool if you also checked the type of enemy. You can do this because: 1.) The size of the dot changes depending on how big/small the enemy is 2.) How the dot moves. Examples: (keep in mind I havent played for a bit so the sizes may be off) - medium dot that stands still could be a headcrab. - small dot standing still around loot could be a lootbug - medium dot running towards the player, but when looked at moves away could be a bracken Some other suggestions: - switching from minimap mode to hot/cold - communicating if an enemy is approaching. - telling it to toggle doors/turrets - communicating about outside threats when you are looting in the facility

[@raysr6112]

If you guys could share the video, it's really well made and would probably be so interesting and cool to many people

[@BloxFruitsBloxTer]

[Him]: You Seeing This Shit? [Dog]: Yep- 11:35

[@StallionGD]

Your channel is gonna blow up soon man. The comedic timing and jokes in this video are so well done, the storytelling is great, especially how you gave some insight into why you decided to do this in the first place. Keep up the good work man

[@synt_axe]

Thanks bro, I appreciate you.

[@jayman1462]

woah this is neat. it would be cool if you added turning off mines and turrets and opening doors, too! surprised to see only 233 subs, so have another.

[@Grimm6583]

I would love if this was on thunderstore

[@bobthepickle3232]

Just found your channel, and I'm super excited to see it grow!!!

[@nameloc2035]

The first system with clock directions, but not stating distance could work pretty well. Sort the distances from nearest to farthest, and say just the clock directions, so you know what's nearest and which way it is.

[@draglorr5578]

PLEASE RELEASE THIS AS A MOD!! it would be great for people who have no friends

[@inx1819]

i know it's not the same but there's a minimap mod so you can see the map

[@ShockedCaucasian]

This was so cool to watch! I love the uniqueness of this haha

[@burakuzun1616]

Superb idea and a perfect execution edit-wise. Applause

[@The_Asasd]

i was expecting you to create an ai that grabbed scrap by itself, but this was good too

[@Emma-rt5ve]

This is incredible. I wish I could make my own Roberto/a, this is so cool. good job

[@APersonm]

This is insane!!! This guy needs more attention,

[@zylou8643]

such a great video, liking for the alogorithm

[@Marcella-mf6cy]

your creativity blows me away!

[@germojo]

0:41 hahahahaha

[@ry6554]

Instead of using a dropship to counter the dogs, you can actually ping the radar booster. This doesn't use any money. For example: ping Bob However, you do need to ping it continuously to keep the dogs there.

[@CChavez99]

Subbed, loved the video, keep it up

[@sorafu1746]

Great video, can't wait to see this take off

[@guts60]

Note that my comment below is incredibly presumptuous and hard to create IRL, and is more a fantasy than anything! I feel like the hot and cold system is better, and what you could have done is still do 5 second intervals, but the 5 seconds for enemies and 5 seconds for loot is spaced apart by 2.5 seconds. Personally, I’d make them both 6 seconds with a 3 second difference A new enemy enemy system that I believe is more ideal would be kinda similar to how the first detection method was described, though more efficient and easily understandable. If an enemy is getting closer, it says “enemy closer”. If an enemy is getting farther away, it says “enemy farther”. Now let’s add some chat capabilities on top of its power of speech. If a triangle is moving around, then the AI will finally use chat to tell you that there is “lootbug or nutcracker, closer/farther” and direct you with hot or cold in the chat. If there is an enemy that is stalking directly behind you and never loses sight of you, then the AI will tell you in chat “Bracken”. If you are outside, the AI will check for massive red dots and tell you “Worm” in the chat. Also, if there is no loot or no enemies, the AI doesn’t say anything or at least the option to not say anything when there is nothing should be toggle controlled. Now we can talk about various other jobs the “ship mom” role has that the AI needs to fill. Teleporting is a big one. The AI will just detect if you are rapidly spinning and if you’ve spun like this more than 3-4 times in a row, and then it will use the tp command on the terminal (overriding the flash command for a brief moment). If the AI sees the player approach a closed blast door and stand by it for a moment, it will open it. If the player is idle for more than 30 seconds with an enemy nearby (aka staring at a coilhead), it will automatically teleport you. If your player indicator on the terminal becomes a darker shade of blue (aka death), it will automatically teleport you. If you are nabbed by a forest giant, it will immediately begin teleporting you.

[@Aimyx]

Great idea (I didn’t read a single word of what you just said)

[@guts60]

@@Aimyx Hmm then maybe don’t comment?

[@Aimyx]

@@guts60 you got guts man

[@123890antonioj]

I can definitely see how the enemy recognition would work. Recognizing human player action and providing evac seems harder, but by the standards put out by this video, (hopefully) not impossible.

[@mckcormith3415]

I was almost expecting for your AI to say “keep summer safe”

[@Dwayne_LaFontant]

The “hot and cold” system sounds like it would be a item in game.

[@yamilmontes1365]

1:17 "You're about to die" Hearing an AI say that over the walkie is unprecedented.

[@recroomman4317]

11:32 blud got a buddy lol

[@TheTrueD6]

Will this be released as a mod on thunderstore, or something along those lines? Either way this is REALLY cool!

[@synt_axe]

The code is public now, so you can run it yourself. But it's not a mod. You'll have to run the code on a separate instance of the game if you want to play with it. I would also love to see a mod with a similar idea in the future.

[@phoenixrose1995]

the "hot cold" is like a todler complaining when they have the flue lol"

[@anggermiftahuddin1858]

Great video!

[@strangecolouredbird]

I just join random lobbies and either leave when they suck, leave when they spam music or skits through their mic, and stay for hours making friends and getting hilarious moments. I love the Company.

[@ashoraa4838]

الله يعزك يابو مصيرة ايش الشغل الفنان هذا يا ريس

[@remirichoux8424]

4:08 THE NUMBERS, MASON, WHAT DO THEY MEAN ??

[@tarantula-sb7og]

11:31 mate looks like he's about to cast some spells

[@The_pidey]

The grid technique is so cool

[@TheWorstestPro]

How do you only have 2.4k subscribers after this banger!?

[@readyRick]

Honestly i love the number system, it had a high skill ceiling and the chance to not understand everything with the robotic voice just adds to the atmosphere. I genuinely feel like it could be a kind of ship upgrade a player on the ship could manually activate.

[@KetsutuoRD94]

Take my sub and keep making great content like this please

[@germojo]

you are a mad genius

[@gnomestein12]

this is amazing

[@sileudies]

Couldn't you use the grid system to make the AI say something like "items straight ahead" rather than the confusing numbers? Like "Item up, enemy 3 item right, enemy down, 2 item left

[@sharkt9877]

Something that might be of use is depending on the size of the red dots you can see what enemy’s there are! Imagine the grid but say it puts a letter for what each enemy is. For example 5 hoarding bugs is just 5h A coil head and a nut cracker are cn Stuff like that

[@shrimpmanvangogh5877]

cold, hot, cold... very cold

[@holydoggo4822]

This is really cool!

[@mythicaltwinkie8216]

I would've increased the sample rate to as high as possible and made the AI play a beeping sound that gets faster based on distance like a metal detector for a item, and play a hum sound when staring at the item that gets quiter the further from the center of the cone to being quite if its greater than 45° I then would've made the AI verbally communicate the enemies, and verbal shout "Danger!" If someone starts moving really fast and "Caution" if a enemy is close to you.

[@Azertxyz]

You're the most masry person ever lmao, keep up the great work ysta

[@NotOneix]

يسطا الاكسنت بتاعك جاحد ❤‍🔥

[@synt_axe]

حبيبي, جزاك الله خير.

[@NotOneix]

@@synt_axe bro liked the content keep it upp

[@SoSkySaysHi]

this is hilarious and you should definitely explore more with using ai lmao

[@yourmomsboyfriend3337]

How does this not have 100k views?? Very good editing and cool concept

[@BlueFan99]

Agree!

[@finanzam1]

I find it missleading - there is no ai involved, just image recognision

[@stevejuststeve6275]

​Image recognition is technically a type of generative ai so it is artificial intelligence.

[@camel2096]

Now there is a 150k views

[@Destiny55687]

“Stop talking about me” kills

[@kylieeeeep]

I love the hot and cold straight into the Bracken

[@The_Fella.]

"Cold...Hot...Cold" *SNAP*

[@SnowflakeVRGTAG]

There are three enemies around you.... Your about to die.

[@lonelyPorterCH]

Not really an AI, more like an algoritm, but still pretty cool^^

[@KevinTheFluffyWolfy]

This is pretty cool!

[@LotsaDragonFly37041]

Fun Idea: You should use the sounds of the motion tracker from Aliens to detect the distance of enemies or loot.

[@sttjtrend]

Part of me is like: "Can you install this to play with yourself, and how"? The other part is: "Chaos with friends"!

[@victor6010]

amazing bro

[@alexrva433]

Makes me wonder if we can get (somewhat) authentic singleplayer bots at some point that run on similar navigation code to the monsters

[@catpokemon5]

11:33 HAD ME DIEING

[@kylieeeeep]

Same, I ran straight to the comments

[@bensvideos1882]

The eyeless fog do be chillin

[@THEanotheryoutuber]

you should make it so that it can tell you if your near the exit when shown

[@Nerozhen]

Underrated!

[@SFAMWASTAKEN]

the idea of just hearing "hot, cold, 1 enemy" is so terrifying, like knowing something is there and not knowing where is horrifying.

[@miawmiawgary2024]

11:34 well i didn't expect that

[@Orotmik]

chingon, introverts will love this lol

[@idkwhattonamemyself1841]

I feel honored that the ai has my name

[@IF27]

4:59 so out of pocket 😂

[@YamiDarkness64]

11:34 "⚡😲📳 " "You seeing this shit?🧍" "Yep 😎 🦯 👂" This is so far one of the greatest Lethal Company videos I've ever watched, the AI is so 🔥

[@SomeGuyIGuess-vu6sd]

Worlds most creative video. Does this use a neural network? How did you calculate the gradients for backpropagation? (If you used it)

[@synt_axe]

Thank you! It does not use a neural network. The script is completely reliant on Image Processing to get the data it needs from the map. I published the source code, you can find the link in the video description.

[@AtomicCrystal]

11:35 was really funny :)

[@connormckibben5986]

Now you just have to train it to spot the exact enemy by dot size, speed, and movement pattern so it can call out "Coilhead" .045 seconds before your demise.

[@connormckibben5986]

Additionally, it may be beneficial to mix these strategies. The grid is interesting- but I don't think as effective as a more advanced AI. I like the Hot/Cold method- but perhaps you could replace this with a tone. Every second or two Roberto would emit a tone- variable in frequency kind of like the anomalous detectors in Stalker, Anomaly. This would represent an item's location relative to you. The higher up in pitch (OR higher rate of beeping, alternatively), the closer item- additionally, you could train the AI to recognize different rooms- power core & loot rooms, walkways, broken walkways, stair rooms, entryways- and draw a quick line between your character and the closest (or highest density of items. I believe that a constant text update of your *"E1 I1"* would be great whenever the items and enemies on the Roberto screen change, regardless of location. Noise can still present as a hazard to the AI and to yourself, so you can train the AI to self-check at the ship every few seconds, and to automatically alert you its in danger, then switch to text. In text it would become more primitive- perhaps just an immediate switch to grid mode. I have a suggestion for more of a Radar-style of UI. Alternatively- you could have it set to a sort of compass/radar mode with *O* to represent the general direction of enemies, and the *Compass Arrow (-P>)* to represent the item / the densest amount of items if far away but still within reach. *P* represents your player- and E could represent the locations of exits. (Figures 1-4) Training the AI to recognize different enemy movements and identify them could be advantageous, with the robot being able to use any other letter to represent a specific enemy such as the letter *B* for Bracken or *C* for Coilhead. Unidentified enemies could remain as *O*, and counted as a number, with any ID-able entities to be labelled by their respective letter, but NOT added to the total # of enemies. (Figures 5 & 6) Examples: Figures 1-4: Figuress 5-6: Roberto Roberto Roberto Roberto Roberto Roberto E:0 I:0 E:1 I:3 E:1 I:1 E:2 I:0 E:B I:0 E:1BC I:2 Roberto Roberto Roberto Roberto Roberto Roberto xxx x^x xxx xxO xxx CO^ xPx OPx -P> xPx xPx xPx xxx xIx xxO Oxx Bxx /Bx Am I a programmer? No. I hate programming- but I think it would be a neat system!

[@Badcraft716]

Underrated

[@Regrozeer]

Tbh I'd prefer the numbers. seem pretty straightforward and you gets loads of information quickly

[@liambohn4692]

You forgot to make it so that he can tp you in emergencys

[@Poopdog312]

that was funny "I said stop talking about me"

[@grayanimate7166]

Don’t know how feasible it is, but would love to see this as a mod that people can download, would make solo play or small group play more fun without outright cheating via minimap mods

[@IsaiahIWC]

I always felt like Leathal would be even more fun if their computers to help assist you during solo gameplay.

[@tablebasse1299]

3mn in it's already great content lmao

[@devilroad_ttv]

''Tell me you have no friends to play Lethal Company, without telling me'' moment xD

[@gamertaggt4552]

Talking to the dog kinda scared me cuz it was right there but he just looked over like a pet lol

[@MrAndrew990]

super cool video

[@bandicamnick7903]

Would this work well with more players? Also, I have a hard time reading and was just wondering if you could make a video guide on how to set it up?

[@synt_axe]

Hope this helps: kzbin.info/www/bejne/nqnKeIZni5aDapI

[@oliver_afton]

i want this mop for playing alone

[@FennixGamingYT]

I wish Roberto still had the voice, that would be more hilarious lol

[@Arenoxitolikus]

0:38 The betrayal… and at 0:42 The revenge! 11:35 Me and my friend watching the professor trying to explain why they made a mistake 😂

[@matthewtimmermans2610]

interesting project, the clock one seemed like a good place to start, but I think the issue with the clock was more the voice and format, breaking the numbers up with an o'clock and also speeding up the voice and using a more natural one should make it a lot easier to understand, I feel like due to the limited vocabulary, using an Ai voice or text to speech voice is unnecessary, you could just record your own voice or use a better voice synthesis Ai. Then have it report the loot and enemies based on how far they are to the player. Another way to make it better to understand would be to replace the distance numbers with close/far categories. The hot cold seemed interesting and intuitive, but I agree there was not enough info but wondering why you didn't add more categories for distance, like having cold, cool, warm, hot. Another thought I have is instead of having the Ai periodically broadcast have only broadcast when you message it, if it gets a transmission of any sound it replies with the info. Something more advanced could be training it to recognize a couple commands, or using a speech to text, then you could send a message enemies and it will tell you if there are any enemies and where, and items and it will do the same with items. Another idea is that it can message when enemies within a certain range. You could probably even have it tell you what enemy it is based on the size and speed and maybe behavior.

[@piadas804]

11:51 You could also use the ping command

[@backgroundcharacter4376]

Me with no friends: "Finally... I can play this game"

[@ethoatom668]

The numbers honestly sounded like she was Overwatch from Half Life 2 giving codes to Combine officers.

[@randomperson9732]

This was a cool video, but Roberto isn't an IA at all. It's just a bot, an algorithm. I get that "AI" is juicier for the views, but it simply isn't one.

[@JoshrlNZ]

A video game AI doesn't always need to use the standard criteria of a "true AI". The Wikipedia page on "AI in video games" goes into this in the Overview section, although opinions around this can be controversial.

[@randomperson9732]

@@JoshrlNZ Sure, but we can agree that it's deceptive. Roberto isn't "intelligent". He was given instructions to follow in a certain order, and mindlessly executes them.