I programmed an A.I. for Need For Speed: Most Wanted

Source code: github.com/LupascuAndrei/nfs_...
This is a short video showcasing my simple Python A.I. algorithm for the self driving car. It's made with computer vision through OpenCV.
-----------------------------------
Script:
Hi guys
Today we’re going to build self-driving AI for Need For Speed Most Wanted using real-time image processing.
“But why Need For Speed most wanted? ” - nobody asked because i have no fanbase.
Mostly because my life is a pile of questionable life choices.
Our AI will be a background process which records the game screen, analyzes it, makes decisions and sends keyboard input back to the game. We will code everything using only Python and OpenCV. The code is publicly available on github and the link is in the video description, so feel free to play with it as you wish.
We will describe the whole process in 5 easy steps.
Recording the screen
Our first step is recording the screen. We do this by repeatedly capturing our monitor screen using a python module. Easy peasy.
screenshot cu parte din cod care copiaza ecranul
Detecting the road
Our second step is detecting the road, so we know when to steer left or right.
A precise approach is filtering the road pixels and calculating their trajectory.
The issue with this method is that, while it is accurate, it would take too much time to process, and by the time our AI would decide how to steer, it would be too late and the car would crash.
We need to compromise some accuracy in exchange for computing speed. So we need to find a more hack-ish solution.
We will, instead, decide how to steer based on the minimap.
By filtering only the white pixels, we can calculate their positions and approximate the direction of the road ahead. It is as simple as falling in love with the wrong person.
Detecting our current speed
Our third step is detecting our current speed in order to determine how much to steer when we are taking a corner. If we take a look at our car’s speedometer, we notice that it’s a number of 3 digits displayed in a digital alarm clock font. To read a digit, we’ll filter the black pixels, detect existing black lines and hardcode all possible line combinations into numbers. And we’ll repeat this algorithm for all the three digits.
Sending key input to our game
Our fourth step is sending keyboard input back to our game. Our AI will use some heuristics based on our current speed and the road ahead to keep the car from crashing. I made the heuristics myself so let’s see how this goes because I don’t deserve my driving license.
???.
The last required step is complimenting you. You are awesome and you deserve to be loved.
So let’s see how the AI behaves so far. I only finetuned the car up to a certain speed, so if you want to finetune it even further, feel free to play with the code, which is available on github.

[@simplesky4110]

Quite impressive, since all of this was done in python and plays better than me.

[@codeazy]

Thank you! It also crashes less often than me irl.

[@sergeibaskakov1014]

It's sad to realize, but this AI plays NFS much better than me... Good job, my friend! 👏

[@sanketkundu3657]

The effort you have given deserves recognition! God knows how much time you have given, hats off ,man,i loved ur commentary!

[@fauxhound5061]

Outstanding work, it drives like a real player

[@derekwest8636]

That's dope. Your videos deserve to be upvoted ❤

[@Soroosh.S83]

This is amazing bro, You Are AMAZING! I didn't see anyone coming up with such a simple but briliant idea, i wanted to implement somthing similar into my own racing game and thanks to you, now I can somewhat understand how to do it, thanks for your great help, I will defiantly Credit you in the game 💜

[@me-df9re]

Awesome. Please continue good work.

[@codeazy]

Thank you, that's the plan!

[@Outofthedust]

For a raw run based on the minimap that's pretty darn good; Better than I expected anyway. With some refinement, it could be taught a better race line to stay out off the dirt and on the road

[@janm8027]

Looks awesome!

[@osdever]

Whoa, this is actually impressive. I actually tried doing something similar about a year ago but with the actual AI path data and A* pathfinding, it worked pretty well lol

[@ralph8016]

How did u do it? :D I can't find the files for it.

[@teriyaki8643]

Very cool! I'd buy you an ice-cream for this well-done job.

[@codeazy]

Thank you! That is so cool and sweet. (hi hi hi)

[@alokkumar5123]

This is really cool actually

[@SarveshKumar-ml5jd]

good effort

[@montdedan]

Good Work Man

[@codeazy]

Thank you!

[@shankyoncloud7420]

The biggest problem with the algorithm is that you aren't accounting for fixed objects in the middle of the track like bridge columns or dividers. Pretty interesting idea to use the minimal tho. What happens if the car accidentally points the wrong way

[@codeazy]

You are right ! if it crashes and starts pointing the other way around, it has no way to know this and will keep running the wrong way ! It also is blind to objects in the middle of the road or enemy cars.

[@dhu3131]

@@codeazy which makes it even more impressive that it managed to finish the race

[@arpitshrimankar7787]

very nice sir keep it up :)

[@ragingfred]

Could you use this in combination with reading the road pixels at least to *roughly* detect what is road and what is grass/dirt or where the boundaries are?

[@codeazy]

Yes, you are correct, that could be a huge improvement! i.e. looking at the minimap to see the general direction, then use that hint to know where to check the road

[@hamsi1792]

This looks like if someone played Most Wanted like its a simulator game.

[@sabah8312]

Nice 👍🏻

[@codeazy]

Thank you !

[@antiw_]

can you do one for euro truck sim ?

[@doominator-117]

Oustanding work gentlemen.

[@codeazy]

Thank you, kind gentleman!

[@albertusaswin2012]

Does NOS work with your AI?

[@lindensd6273]

Would this work with modern need for speed as 2015?

[@deibgaming9363]

where to put the files? i don;t work

[@secretc.5214]

thx

[@its_ntm]

i lvoe this :D

[@codeazy]

Thank you !!

[@muhammadhanifkurnaen6689]

I just want that someone could end rubberbanding issue on nfs series

[@parthpatel9602]

Hey I'm not the greatest programmer but don't you think that putting some things like checkpoints for every track and then hardcoding the keyinputs should work better ?🤔

[@codeazy]

it might work better, but then you'd have to manually hardcode each track and it won't work on new tracks or roads. And it's a much uglier approach ! similarly we can also hardcode all the desire input for each track (i.e. speed up for the first 3 seconds, then turn left for two seconds etc.)

[@jacobpillai]

Can you make one for another nfs game?🤔 I would like to learn how to code this as well 😅😅since im still learning how 😅😅

[@deepbrar1]

wow , this is really awesome. Just how much time it took u to make this ...

[@codeazy]

Thank you! Around 60 hours I believe, I am a slow editor!

[@mrdlandrei]

Ez pez. When are we installing it in real cars?

[@user-dc9zo7ek5j]

I was impressed but when I saw the code was so small I was even more! Is it possible to keep the performance but still from time to time, lets say few seconds to actually look at the screen and see if it's actually moving, the strategy of sliding on the walls is not a viable strategy because there are maps where the corners stick out, not really problem of the algorithm but the game. Also is this the coding standart in python ? Why are there so many ifs and repeating conditions and why did you write 12/10 instead of 1.2 ? And why do you always calculate getImageDigitalNumber if the counter is always there and could be calculated once per race ? I've never programmed in python and I'm interested.

[@codeazy]

Thank you and i'm glad you liked it ! getImageDigitalNumber () should be called on each frame, because the speedometer might change on each separate frame depending on the car's speed; this is why i'm calling it in the while loop the 12/10 is the same as 1.2, it doesn't matter how you write it !

[@user-dc9zo7ek5j]

@@codeazy Why don't just write 1.2, it's much more to the point than 12/10. About the speeometer, I was talking about why always find the "spotsToCheck" if the speedometer is always going to have the same height and width and the same position, and why should a parameter "name" be passed if it's not used, couldn't it just be a comment ? Also saw that you've removed some romanian comments, so greetings from Bulgaria.

[@codeazy]

@@user-dc9zo7ek5j o ye that could be an improvement regarding the spots to check, but i dont think it would matter on the performance in a noticeable way; but nice catch ! also greetings, neighbour !

[@codeazy]

Also regarding 12/10, y i should change that to 1.2 for it to be more clear; sometimes i prefer writing numbers as fractions, dunno why (i.e. 3/4 instead of 0.75)

[@user-dc9zo7ek5j]

@@codeazy Well, it's not about the performance, it's about the ability to make it simple. I really like to remove all moving parts from my code and to make it as static as possible, it makes the code easy to read and highly predictable, because it also makes the functions do less, it's less noise overall and once set those variables you can forget about them. Performance wise, after all we're talking about python, there is no point in optimizing something that is meant to convey thoughts instead of bare instructions.