Using Image Recognition to play Mario Party Minigames

[@CodeNoodles]

To try everything Brilliant has to offer-free-for a full 30 days, visit brilliant.org/CodeNoodles. The first 200 of you will get 20% off Brilliant’s annual premium subscription.

[@lio1234234]

Fun fact, I learned on Brilliant that the polygon method you describe in the video to approximate a circle, is also how we calculate (or rather approximate) Pi!

[@lio1234234]

Given that the definition of Pi is the area of a unit circle that is

[@doggo7078]

how can I learn to program like you do? I'd make my hobby so much easier. I'm trying to find an rng seed with a certain pattern, but if I enter the minigame and reset for a new seed I take 15 seconds (with the emulator sped up), so math tells me I need to check 10k seeds if I want to find 2,5 suitable seeds (=5k for 1ish seed), which would take 4 hours and several PCs running 3 emulators at the same time. And I'd have to check 5 PCs to see if I found the seed during 4 hours, that's why it'd be useful if I was to learn how to write some code and recognize the rng pattern I'm looking for

[@PikTheRedPikAUTTP]

how tf does math have to do with mario

[@BarterChain65_Gaming]

what are you using to code with i'm trying to get into learning a bit of coding myself

[@SpaciousBoundary]

It’s really telling just how impossible it is to win against expert difficulty when even the automated code only just wins

[@KatherynneF]

To be fair, even as a kid I was faster than the code for the dominos game and humans can cut corners and won’t stop to turn on the drawing one. We can see multiple inputs ahead and play around that

[@mistorbear]

well near the end the ai accidentally grabbed a bob-omb so that could be the reason

[@jblen]

I mean it's automated code Vs automated code. Maybe they cheated a little because the game AI can see the code, but it's still not limited to human reaction times

[@AdaTheWatcher]

The AI is probably rubber banding.

[@SleepyBrady]

As a kid I always played with expert AI and usually was able to win. The only few mini games that were actually impossible were cherry go round and get the lead out. Actually impossible

[@ArisTheMage]

As a programmer myself, it’s greatly appreciated you show the code. I hate when the KZbinr at most shows a small snipit. Not to steal your code, but to see your logic :)

[@tornadowizard]

For the block one, you could have prewritten each shape, "square" "rect up" "rect right" etc so instead of writing out the coords for each map over and over you would just write which shapes get drawn in what order.

[@CodeNoodles]

That's a good point!

[@tornadowizard]

Thank you! I haven't used my coding skills in like 3 years unfortunately, glad to know I still have it!

[@KinuTheDragon]

For the first one, might I suggest checking if the goomba is at least a minimum distance from any bombs?

[@CodeNoodles]

I actually have that implemented already. It definitely helped!

[@renakunisaki]

And prioritize gold.

[@Laenthor]

​@@CodeNoodlesYou could also straight up draw squares instead of circles so it would be nearly instant instead of having to call the function for each circle point?

[@sarowie]

@@Laenthor did you watch the video? he has to use a polygon with a certain number of edges, as the game does otherwise ignore the shape. You can try to optimize the number of corners, but the advantage will be pretty small.

[@Laenthor]

@@sarowie Oh I must've missed that part, it just seems to look for a closed shape.

[@blackcat_064]

Some minigames I'd love to see you automate are Rail Riders, Cherry-go-round and Study Fall. These are some of my favourite minigames in Mario Party DS

[@woobgamer5210]

Rail Riders and Cherry go Round should be is, cause Rail Riders is just, slide up from the bottom as quick as possible, and Cherry go round is, just continously draw circles as fast as you can. tbh i'd love to see Sprinkler Scailers done using melonds or no$gba by being a python script and python script vs 2 expert cpus

[@GoldenAce17]

Getting a 00'00'00 on study fall is possible but SO hard! watching code do it every try would be awesome

[@thewiseoldfox]

​@@GoldenAce17I'm pretty sure everyone who grew up with Mario Party DS got 00'00 on Study Fall on accident, making the best unbeatable high score

[@tacothedank]

​@@thewiseoldfox I remember being able to do it intentionally without much difficulty. I'd be able to get it like 1 out of every, maybe, 5-8 tries?

[@disdonut]

Code Noodles’s taste in music is amazing. I’ve been listening to The Noble Demon for a while because their arrangements are so good. They deserve lots of love for their work

[@beeisland]

I'm glad my favorite mario party is getting some attention. Very fun seeing you destroy those infuriatingly perfect expert cpus. Good job and would LOVE to see more.

[@CodeNoodles]

I know! Nintendo made the expert cpus WAY too difficult!

[@carlyskolnik296]

Same 😊

[@raptore.x]

@@CodeNoodles glad im not the only one who thought those expert cpus were just wayy too good, feels like theyre harder than other mario games sometimes lmao

[@doctorgrubious7725]

Personally I would’ve included a counter-draw around bombs that basically detects if a bomb is in the radius of a goomba, and if the circle it would draw around the bomb overlaps the goomba circle, to subtract the bomb’s circle from the goomba’s, For the second game I feel it’s like a cop out due to the fact it’s not really playing, it’s just doing it from a list that you made, you did a good chunk of the work, I would’ve gone with the black and white image and than have it instead just find the vertices of every shape and than add it to the shape array, so it creates the list on its own before drawing

[@YeetoTheYeetafic]

CodeNoodles Friend: 'Yo I finally got 500 goombas without a bob-omb in goomba strangler from Mario Party DS!' CodeNoodles: 'I got more than humanly possible, I've already beat you'

[@KLGB76]

Mario Party DS is my favorite Mario Party ever.

[@Alex-gv9os]

Definitely want to see other automatization for Mario party minigame 🤩

[@x-x]

I enjoy your videos so much despite never playing these mini games and having no coding skills! You have a very calming and charming voice, so it's very interesting to hear and you don't try to downplay your skills at any point like many other KZbinrs nowadays. Keep going!

[@CodeNoodles]

Thanks, I'm glad you them!

[@andym5280]

@@CodeNoodles I really them too! Especially the NSM videos!

[@ermamu]

I’d love to see more! Do you think ‘Hammer Chime’ would work?

[@CodeNoodles]

Probably

[@nitro5247]

It’s really cool to see how people my own age are using these tools on our childhood games. I also missed out on MPDS, but I did play NSMB when I was younger. These videos make me want to take on a project of my own, haha.

[@TheLobsterCopter5000]

Try the stargazing one next! Give the automator a challenge by having to compete with 3 expert CPUs at that game...

[@canadianlucario2643]

Would you try any other Mario party games with this? I’d love to see more of these videos, they help me a lot with learning what I can do with programming

[@unfortunatedisgrace246]

you have the perfect way of explaining so that most people understand without it being too complicated or too long winded! it's actually really entertaining!

[@silverstorm323]

I would love to see more minigames coded. These were fun to watch. Maybe even other MP games in the future?

[@theAstarrr]

YO more nostalgia from probably my favorite videos of yours

[@Imaproshaman7]

This was really cool! I'd love to see more of these. The original Using Image Recognition To Find Mario video is how I found out about this channel in the first place. Oh yeah, I'd be interested to see if it would be possible to automate the minigames that require the mic.

[@velvetbutterfly]

I would love to see you do a video explaining each piece of software you use so that people can get educated on the details to better follow what you're doing

[@strawberrylemonadelioness]

I had this game as a kid and I still love it! It's still honestly my favorite Mario Party game.

[@leroyhak5081]

Oh man, I remember playing this on the ds. It was always a lot of fun to play against my siblings. And for everyone who thinks the expert ai is near impossible to win from, some games like the 2nd Trace Cadets and the 3rd Domino effect all depend on how fast and accurate you are.

[@louisnemzer6801]

Playing video games: Fun Automating a computer to play games for you: More fun

[@htspencer9084]

Awesome vid! Unsolicited suggestion for the first one: Maybe comparing the location of the chosen goomba over two frames to determine it's movement vector and use that plus knowing how long your circle takes to draw (assuming it's a constant time), means you can draw ahead of the goomba and get it perfectly centered. You could even then track the movement of bombs within a certain range of that goomba and know ahead of time if the circle would pick up any bombs so you can avoid that goomba. Is any of this multithreaded? Would be curious to see how much of this could be put on worker threads. The drawback to my suggestion is when they bounce off the wall, as you'll need to figure out their reflections which sounds like effort but you could probably just ignore it as its only an edge case (pun very much intended).

[@CodeNoodles]

None of this is multithreaded. All of your suggestions are fantastic, but they would be nearly impossible to implement.

[@BromeoWuggles]

Love the Image Recognition videos

[@DagoDuck]

If there‘s one thing I‘d really like to see, it‘s optimizing these scripts to near perfection. For example the first minigame if it wasn‘t for that bomb, it would‘ve had so many Goombas, the game had to squish them together.

[@johnsimpsen5]

YES!! I love Mario party ds!!! Definitely the best Mario party!!! Please do this game again!

[@Neoh53]

For the first mini game (circle goomba) i know from my childhood that you can draw pretty small circle on the goomba. Reduce the size of the circle to decrease the risk of catching a goomba

[@fernando98322]

For Trace Cadets, I'm sure you can automate further by tracing the blue lines (really, they're line segments with a set number of directions to take) for adjacent figures but that might take too much processing time that the Expert CPU might beat you at that point

[@TalkySteak]

I was stoked to see another minigame automation video hit my feed. Love this series! Thanks!

[@CodeNoodles]

Thanks, I'm glad you like it!

[@TalkySteak]

@@CodeNoodles Do you push your code from these videos to a Github repo by chance?

[@realElzie]

I’d love to see some more videos on automating Mario party games. I’ve got to design and make my own program for a Python final project for a college class and I might do something like this, not sure yet though…

[@akamai8097]

I'd like to see more mario party minigames, but I'd love it if you did an episode on a warioware game!

[@LevelUpGA]

Next time you can try all bloons TD with genetic algorithm, theoretically you need only string recognition and tower positions/hotkeys and clicking on the map and good luck with training, I try random clicking and I win BTD1 in 2/4 times with random clicking (generation 0) without upgrades but not with full hp

[@krankenwagen6042]

this is the sequel we didnt deserve but the sequel we got

[@Noah-ost]

This was such a good idea for a game to automate! i alwyas find it fascinating when we can make AI that is better than the in-game one. Would love to see more minigames automated. This was one of my favorite games as a kid ❤️

[@VolcanoGamingVR]

4:27 - bomb-omb lol I like it

[@ItsIdaho]

One of the best DS games ever. Miss our 2 vs 2 sessions😢

[@__--_--_-----]

I love automating, done some work with pyautogui myself to autowin the wordle battle royale game lol. great vid, would love to see more of these

[@TheMamaluigi300]

Maybe for Goomba Wrangler, the program could probably compare the latest screenshot with the previous one to gauge its target’s speed so it can predict where the goomba will be when the circle completes

[@ironici]

I swear this is the only way to beat Trace Cadets

[@b_megamaths]

It's good to know that, no matter how advanced code has become, I'm still able to beat it handily at Domino Effect.

[@tekayo63]

this is so cool! since these were all duel (or at least the duel versions of) minigames, i think in a potential next video it could be cool to see some ffa or 2v2 games like hot shots or sprinkler scalers

[@SinisterPixel]

For Domino Effect, I feel like it might be helpful to just read the upcoming input instead of the current one. The window to press the input comes up REALLY early so you could basically prebuffer the entire thing and smoke the CPU.

[@nathanlamberth7631]

Those second two were too close for comfort

[@rifatbobos]

"You know what? Fck you" **becomes an even more Expert CPU*

[@noahvasiliauskas233]

More mini games pls I love listening to these 😊

[@Puddincess]

I'd definitely like to see more of these. If I could choose minigames to try, I'd say Hedge Honcho (ladybugs and wasps can be predicted by the leaf before them, and Im curious to see if you can make it image recognize that or how well you can make it rub the ladybug), the skydiving minigame, and the boo-spotting 1v3 I forget the names of 😔

[@777omen]

I want to see Image Recognition dominate all of the Mario Party DS bosses.

[@bloodwolf2002]

That'd be so cool if possible

[@777omen]

For Hammer Bro it's easy but for the others, I don't know how you would find a way to implement the image recognition.

[@bloodwolf2002]

@777omen dry bones "might" be easy out of the remaining bosses but don't know

[@cravonomire9971]

It is funny how nostalgic I get for this game.

[@HPnodsu]

I love this! It's always amazing watching how all this works. I personally find it really entertaining and would like to see more.

[@__--_--_-----]

would be nice to see it taken to the next level to automate a more complex game, I think that is more impressive than doing a few simpler games

[@justarandomwoof2247]

Would be cool to see more!

[@batofdestiny7801]

I love videos like this. I’m not very smart with computers and technology but this shit is super cool

[@mememkyu]

I need more of this! This type of stuff is so cool to me

[@rainbowcat498]

The sweer nostalgia i got from watching this!!!

[@Vextrove]

I would like to see more minigames!

[@XenoPon2]

The best Mario party ever made.

[@Zyiah]

i love these sorts of things

[@pokeli2728]

I love these videos! Please make more like this!

[@CoffeeAddictEvan]

I think if you use what's called the midpoint circle algorithm to draw your circles, instead of generating a polygon and then drawing it, you should be able to grab goombas much faster

[@David_Rg]

dude, when I was a kid I hated soooo much the block one 😂 I still struggle with it, ngl

[@lightgodyt.]

It would honestly be amazing if you do more minigames!

[@p6t158]

One of my favorite games

[@bupp291]

These are so fun! I'm actually taking a lot of inspiration from these videos for my final project in my Master's degree for Biomedical Engineering. It involves automating a precise specimen sampling mechanism using image recognition, deep learning, and pyautogui. Thanks for the tips! 😃

[@CodeNoodles]

That sounds amazing! Good luck!

[@bupp291]

@@CodeNoodles Thank you! I'll let you know how it goes 😃

[@Kirby_Super]

Ok this sounds insane, and would require you to play through the story to unlock them, but what if you automated the Boss Mini games?

[@YoloYester94]

You should do coin minigames from Mario Party DS like Penny Pinchers.

[@EP1CxEMAN08]

Let's go! Impossible difficulty CPUs

[@agamerfan6647]

In case no one else has said it, pyautogui probably didn't work because it uses older methods of doing stuff that aren't always compatible with newer software, so it doesn't even see the key presses. This may have been the case with your "DS". Pydirectinput solves this problem by using newer methods. At least, that's my understanding.

[@oclone120]

Glad to know my Domino Effect PB is better than a robot

[@Haagimus]

Bro! One of the Warioware games!!

[@PokeyYapper1991]

new hardest!

[@kronkolweg5917]

Maybe you could try rail riders. I think that one might be interesting. Or maybe try a 1v3 or 2v2 like cube crushers.

[@frenchtheguy]

LET'S GO Mario party ds

[@pantheonmaker9437]

I’d be interested in seeing one of þese ais playing story mode

[@Leekodot15]

I think you were on to something with the scrapped solution for detecting the grid. But you needed to look for the shapes in a blue outline afterwards. THEN you could automate the shapes.

[@noyz-anything]

once you're done with mario party, maybe try automating pac-pix

[@zixea3318]

I think the goomba game could just be solved by drawing the circle even slightly smaller

[@CrabGuyy]

comment for the algorithm because i really like this video

[@SubroOrbus]

CodeNoodles' program vs Nintendo's program!

[@LetsPlayKeldeo]

I hope I see a codenudel and Codebullet crossover one day !

[@pablojp3498]

Loved the vid

[@Gettenhart]

This is codebullet but with actual coding! 😂 Cheers

[@Heeter]

More videos like this

[@steelplasma256]

5:57 I wouldn't call it finished yet. The circle is too large even the opponents circle is smaller than yours. Also it doesn't need to be a perfect circle, a 9 sided polygon would be much faster to draw and look good enough.

[@CodeNoodles]

For whatever reason, the game doesn't recognize polygons with a low number of points, so I increased the number of point one by one until it was always capable of being recognized. Also I forgot to mention it in the video, but the circle automatically changes size based on how close the bob-ombs are to it, but it still might mess up.

[@CommanderKappa]

​​@@CodeNoodlesou might as well add the feature that the progranm recognises the gombas mooving direction and adjust the coordinates accordingly

[@beesree39]

2 questions i have regarding goomber 1. will it be checking for golds? 2. could you make it pause if bobombs are too close? or maybe just make the circles smaller?

[@supremeruby2114]

Oh hey I use visual code edtior too!! Nice!

[@RooMan93]

Ahh! sudden TwoMinutePapers outro at 12:53

[@sahilhossian8212]

Lore of Using Image Recognition to play Mario Party Minigames momentum 100

[@Chackravartin]

This may aswell be magic as far as im consered

[@johnappleseed3428]

damn i love domino effect i got skilled at it as a kid that this play seemed so slow

[@Pheonix1328]

For the second one, is there a way you could have wrote a mini program that would have wrote out the coordinates and all you had to do would be to click on the corners?

[@Luigifan4ever11]

I'm curious - for Goomba Wrangler, did you try to make it prioritize Golden Goombas when they were far enough away from a Bob-Omb? Or is that a worse strategy?

[@Derrick_H]

Could you get the approximate vector of the goombas and basically lead them?

[@blanana_m]

PYDIRECTINPUT WOOO MY FAVOURITE LIBRARY

[@then00btitan96]

the goat is back

[@GarfieldWithAHat]

Domino effect made me think the computer itself is using python to play. You were literally neck and neck

[@Silv3rM4st3r]

I would love to see more.

[@bluegem8582]

so this is how i learn that domino effect has a fail state if you take to long, huh