Capture Timberman Footage with Python and OpenCV | Gaming Computer Vision

Learn how to capture game footage for computer vision using OpenCV and Python.
In this tutorial, you'll learn how to:
1. Install dependencies required or computer vision
2. Capture game frames using PyAutogui
3. Render game footage using OpenCV
Get the code: github.com/nicknochnack/GameC...
Chapters
0:00 - Start
0:32 - Explainer
1:09 - Tutorial Kickoff
3:04 - Installing Dependencies
4:41 - Import Dependencies
5:28 - Building the Capture Loop
7:52 - Handling Frame Closing
9:24 - Test Run
10:17 - Cropping Regions
14:22 - Update Color Channel Order
17:59 - Wrap Up
Oh, and don't forget to connect with me!
LinkedIn: bit.ly/324Epgo
Facebook: bit.ly/3mB1sZD
GitHub: bit.ly/3mDJllD
Patreon: bit.ly/2OCn3UW
Join the Discussion on Discord: bit.ly/3dQiZsV
Happy coding!
Nick
P.s. Let me know how you go and drop a comment if you need a hand!
#opencv #python

[@jacobyoung6876]

So glad I found this channel! The topics you cover are incredibly interesting. Recommended to all my developer friends.

[@NicholasRenotte]

Thanks a million @Jacob!

[@TheEagleCeremony]

Hi, yeah, we all waiting for your next videos about interactions with games.

[@andresgarciaescalante8316]

Great tutorial 👌. I am waiting for the next ones :)

[@armaaanziyad2772]

i generally don't work on projects like this but it was fun and I brushed up my computer vision skills so it's a win win right

[@NoMercy8008]

This is pretty awesome! As always, thanks a ton fot the video, much much appreciated! i love it :) Something like this would be really awesome for giving players individual and relatively specific tips in an automated way. Like for example, you give the software a Rocket League replay and the software tells you Why you suck at Rocket League :D in the case of Rocket League specifically, there already are websites that you can upload your replays to and they show you some advanced statistics and stuff, but I think CV and ML could really take this to another level and help people find their mistakes and show them how to improve and have more fun. Obviously also applies to tons and tons of other games. I would definitely love to see more of this from you! As you suggested, using this footage to train some ML model which can then play the game itself would be fun, or analysing what's going on in the game and giving you some tips or info would also be really nice. Tons of possibilities here :) Another idea for a more complex project: I love playing a game called Cities:Skylines. It's a city builder. You build cities, manage traffic, etc etc. The game itself has a traffic overlay that shows you *where* traffic problems occur, but it doesn't necessarily show you *why* it happens or where the problematic traffic comes from. That's where it would be really interesting to capture the game footage, pass it through an ML model that does some analysis and then tells you more insight on why all the cars are backing up (for example maybe it notices that there is a choke point where all the traffic has to pass through one and only one road). Maybe it can even give suggestions on how to fix the issues it detects. This is obviously very advanced and complex, especially creating data for training would be pretty hard I imagine, but it has the added bonus of being potentially applicable in the real world, too. Though I'd imagine that tools like this for real-life traffic analysis are already out there. Yeah, I know my comment's getting long again. Just wanted to leave my ideas here in case anyone finds them inspiring somehow. I for sure find your video super inspiring, so once again thanks a lot! Wish you all the best, hope you are all happy and healthy :)

[@NicholasRenotte]

I for one, LOVE your comments! Your ideas are always brilliant. I'm definitely going to build out the ML and control for Timberman, I'm really enjoying it and it seems like you all are as well so I figured I'll just go for it. I love the idea of like an ML based coach for Rocket League, and traffic congestion would be awesome as well (I think i've seen a paper on this somewhere before 🤔). As always, love your feedback hoping you're doing great as well!

[@MyChris128]

Great video, thank you. It would be great if you could make another video to see how to play using an DL model or a genetic algorithm model.

[@guilhermetogniolo4221]

Man, this is CRAZY!! I love your tutorials!! Can you make a video showing how to train a model to play the game just by looking at the screen?

[@astronaut4229]

I now have the skills to capture Timberman Footage. I am all powerful.

[@NicholasRenotte]

Damn straight you are Yash!

[@TheCulturedSwine]

Keep this one going!

I would suggest adding the paid subscription to your channel, so we can ask and answer problems related to your videos, such as implementation and improvement. I'm more than willing to support your channel because it brings me so much value.

[@NicholasRenotte]

Thank you so much, I'm doing some reorganization atm, I might actually set it up!

@Somali Pyrates If you want to ask him, he may have to spend time doing reseach and give you advice. His time and experience is valuable, and I think it’s fair for us to support him somehow.

[@XChinaX00]

Thank you for the tutorial. It would be awesome, if you could make a video about objects detection with opencv in games which would lead to create AI model.

[@TripleCheeseExtraCheddar]

Hey man really appreciate these videos! Have you considered making a Patreon?

[@ClarityCoders]

Nice video 😉

[@NicholasRenotte]

DUDE! Your stuff is sick! You seen the follow up to this one: kzbin.info/www/bejne/ZpbJn4R5fcemr6c throw it at fall guys for obstacle avoidance?

[@samanthaahmed396]

Great 🥇 🧡🙏🏼

[@NicholasRenotte]

Thanks for checking it out!

[@nulifyer4460]

So how would you go and connect something like this to the open AI gym framework you were using in the last two videos, since it's hard to pull the game data when you aren't that integrated into the games system

[@NicholasRenotte]

Stay tuned coming soon!

[@mesutyilmaz8038]

Can I use the detection feature of yolov5 here? if i can i can also do it in transaction , also the fps of the yolo is more

[@thewdead3673]

great tutorial! is it possible for you to make a tutorial on how to level up a pokemon using this? to automate grass walking in the same square

[@emredemiriz7303]

Hey, Is it possible to take screenshot of a directx game because libs like pyautogui and pywin32 cannot do this.

[@mohamedhamdy8330]

Hello great can you make tutorials about Open-AI's Retro and NEAT for reinforcement

[@aliffnabil5542]

Great video man! Can you try using machine learning to beat any game maybe like flappy birds?

[@NicholasRenotte]

Yep! Doing timberman first then will probs double back and hit up flappy birds etc!

[@Unknown-cz1dt]

PART 2 how to detect a object inside the game ? :D pls

[@Jkauppa]

fps timing gap removed

[@Jkauppa]

ie, ai timing game fps limit removed

[@Jkauppa]

training as fast as possible, game clock cranked up

[@siput]

one step closer to creating fps game aimbot

[@NicholasRenotte]

👀 it's coming

[@officialonbekende9785]

why if i type in .imshow() / .DestroyAllWondow() / .waitKey() / .cvtColor() it says that the module isnt found