Using Image Recognition to play Mario Party Minigames

[@CodeNoodles]

To try everything Brilliant has to offer-free-for a full 30 days, visit brilliant.org/CodeNoodles. The first 200 of you will get 20% off Brilliant’s annual premium subscription.

[@lio1234234]

Fun fact, I learned on Brilliant that the polygon method you describe in the video to approximate a circle, is also how we calculate (or rather approximate) Pi!

[@lio1234234]

Given that the definition of Pi is the area of a unit circle that is

[@doggo7078]

how can I learn to program like you do? I'd make my hobby so much easier. I'm trying to find an rng seed with a certain pattern, but if I enter the minigame and reset for a new seed I take 15 seconds (with the emulator sped up), so math tells me I need to check 10k seeds if I want to find 2,5 suitable seeds (=5k for 1ish seed), which would take 4 hours and several PCs running 3 emulators at the same time. And I'd have to check 5 PCs to see if I found the seed during 4 hours, that's why it'd be useful if I was to learn how to write some code and recognize the rng pattern I'm looking for

[@PikTheRedPikAUTTP]

how tf does math have to do with mario

[@BarterChain65_Gaming]

what are you using to code with i'm trying to get into learning a bit of coding myself

[@SpaciousBoundary]

It’s really telling just how impossible it is to win against expert difficulty when even the automated code only just wins

[@KatherynneF]

To be fair, even as a kid I was faster than the code for the dominos game and humans can cut corners and won’t stop to turn on the drawing one. We can see multiple inputs ahead and play around that

[@mistorbear]

well near the end the ai accidentally grabbed a bob-omb so that could be the reason

[@jblen]

I mean it's automated code Vs automated code. Maybe they cheated a little because the game AI can see the code, but it's still not limited to human reaction times

[@AdaTheWatcher]

The AI is probably rubber banding.

[@SleepyBrady]

As a kid I always played with expert AI and usually was able to win. The only few mini games that were actually impossible were cherry go round and get the lead out. Actually impossible

[@KinuTheDragon]

For the first one, might I suggest checking if the goomba is at least a minimum distance from any bombs?

[@CodeNoodles]

I actually have that implemented already. It definitely helped!

[@renakunisaki]

And prioritize gold.

[@Laenthor]

​@@CodeNoodlesYou could also straight up draw squares instead of circles so it would be nearly instant instead of having to call the function for each circle point?

[@sarowie]

@@Laenthor did you watch the video? he has to use a polygon with a certain number of edges, as the game does otherwise ignore the shape. You can try to optimize the number of corners, but the advantage will be pretty small.

[@Laenthor]

@@sarowie Oh I must've missed that part, it just seems to look for a closed shape.

[@ArisTheMage]

As a programmer myself, it’s greatly appreciated you show the code. I hate when the KZbinr at most shows a small snipit. Not to steal your code, but to see your logic :)

[@tornadowizard]

For the block one, you could have prewritten each shape, "square" "rect up" "rect right" etc so instead of writing out the coords for each map over and over you would just write which shapes get drawn in what order.

[@CodeNoodles]

That's a good point!

[@tornadowizard]

Thank you! I haven't used my coding skills in like 3 years unfortunately, glad to know I still have it!

[@MeneerEnMevrouwTrein]

05:30 I've never seen some collect so many goombas before! Too bad it picked up a bob-omb by accident because I would've loved to see how an even bigger goomba collection would've looked like.

[@sarowie]

well, that sounds like a second channel video: AI plays minigames for an hour! I am not ironic, I mean that seriously.

[@Makura6901]

Eyo trein kanaal nice

[@kierrific1357]

I've never actually seen the Goomba section condense like that to fit more, it's a little weird.

[@aaronconner1440]

This was AMAZING! Please automate more Mario Party minigames!

[@whee2390]

Seconded, this was really interesting to watch

[@carlyskolnik296]

Same as well I agree

[@blackcat_064]

Some minigames I'd love to see you automate are Rail Riders, Cherry-go-round and Study Fall. These are some of my favourite minigames in Mario Party DS

[@woobgamer5210]

Rail Riders and Cherry go Round should be is, cause Rail Riders is just, slide up from the bottom as quick as possible, and Cherry go round is, just continously draw circles as fast as you can. tbh i'd love to see Sprinkler Scailers done using melonds or no$gba by being a python script and python script vs 2 expert cpus

[@GoldenAce17]

Getting a 00'00'00 on study fall is possible but SO hard! watching code do it every try would be awesome

[@thewiseoldfox]

​@@GoldenAce17I'm pretty sure everyone who grew up with Mario Party DS got 00'00 on Study Fall on accident, making the best unbeatable high score

[@tacothedank]

​@@thewiseoldfox I remember being able to do it intentionally without much difficulty. I'd be able to get it like 1 out of every, maybe, 5-8 tries?

[@NoviceVideoGamer]

@@GoldenAce17 I think I got 00,00 on Study Fall. That's the chalkboard minigame, right?

[@YeetoTheYeetafic]

CodeNoodles Friend: 'Yo I finally got 500 goombas without a bob-omb in goomba strangler from Mario Party DS!' CodeNoodles: 'I got more than humanly possible, I've already beat you'

[@disdonut]

Code Noodles’s taste in music is amazing. I’ve been listening to The Noble Demon for a while because their arrangements are so good. They deserve lots of love for their work

[@chipsleftwing]

WOOOOOOO MORE MARIO PARTY THATS WHAT I'VE BEEN WAITING FOR THAT'S WHAT IT'S ALL ABOUT

[@0hellow797]

Yeah I’m with Chip on this one

[@totallyreyalfactsfsfs]

Hard to believe that your last "image recognition" video was nearly a year ago already! Time flies fast, doesn't it?

[@beeisland]

I'm glad my favorite mario party is getting some attention. Very fun seeing you destroy those infuriatingly perfect expert cpus. Good job and would LOVE to see more.

[@CodeNoodles]

I know! Nintendo made the expert cpus WAY too difficult!

[@carlyskolnik296]

Same 😊

[@raptore.x]

@@CodeNoodles glad im not the only one who thought those expert cpus were just wayy too good, feels like theyre harder than other mario games sometimes lmao

[@NekoEchoflower]

Man mario party ds was such a big part of my childhood. Only played with bots sadly, as none of my friends owned a ds, but it was still plenty fun Was glad to see a video about it, would love to see more automated games if possible :0

[@homerlol9058]

Loving these minigames automation

[@KLGB76]

Mario Party DS is my favorite Mario Party ever.

[@Freddie_06]

9:45 "The best solution to a problem is usually the easiest one." - GLaDOS

[@ironici]

I swear this is the only way to beat Trace Cadets

[@Alex-gv9os]

Definitely want to see other automatization for Mario party minigame 🤩

[@doctorgrubious7725]

Personally I would’ve included a counter-draw around bombs that basically detects if a bomb is in the radius of a goomba, and if the circle it would draw around the bomb overlaps the goomba circle, to subtract the bomb’s circle from the goomba’s, For the second game I feel it’s like a cop out due to the fact it’s not really playing, it’s just doing it from a list that you made, you did a good chunk of the work, I would’ve gone with the black and white image and than have it instead just find the vertices of every shape and than add it to the shape array, so it creates the list on its own before drawing

[@unfortunatedisgrace246]

you have the perfect way of explaining so that most people understand without it being too complicated or too long winded! it's actually really entertaining!

[@bongwater1856]

oh man, I loved playing this game as a kid. Loved watching it get destroyed, too! great work

[@MoundN]

That vid was probably as far away from "destruction" as it could be. Its worse than human skill which in your faulty logic would mean literally every person winning against expert bots "destroys" the game like they are playing it in a way that was unintended...... Watch tas of mario party minigames, there you learn destruction.

[@bongwater1856]

🤓

[@MoundN]

@@bongwater1856 I know your comment kinda made you look like that but that's okay!

[@nitro5247]

It’s really cool to see how people my own age are using these tools on our childhood games. I also missed out on MPDS, but I did play NSMB when I was younger. These videos make me want to take on a project of my own, haha.

[@x-x]

I enjoy your videos so much despite never playing these mini games and having no coding skills! You have a very calming and charming voice, so it's very interesting to hear and you don't try to downplay your skills at any point like many other KZbinrs nowadays. Keep going!

[@CodeNoodles]

Thanks, I'm glad you them!

[@andym5280]

@@CodeNoodles I really them too! Especially the NSM videos!

[@777omen]

I want to see Image Recognition dominate all of the Mario Party DS bosses.

[@bloodwolf2002]

That'd be so cool if possible

[@777omen]

For Hammer Bro it's easy but for the others, I don't know how you would find a way to implement the image recognition.

[@bloodwolf2002]

@777omen dry bones "might" be easy out of the remaining bosses but don't know

[@love_a_hater]

This game was my favorite, and seeing that beautifully written code is so satisfying.

[@TalkySteak]

I was stoked to see another minigame automation video hit my feed. Love this series! Thanks!

[@CodeNoodles]

Thanks, I'm glad you like it!

[@TalkySteak]

@@CodeNoodles Do you push your code from these videos to a Github repo by chance?

[@phantomgaming7544]

I played mario party ds a bit too much to know how long it would take just to unlock expert mode and then knowing how difficult it is to go against expert difficulty. Haven't played it in a while but I still have the game and haven't forgotten that I've spent way to much time on the game to have quite the collection and also every minigame unlocked naturally, plus actually being able to beat them quickly was another thing I've done, just never against expert. Appreciate seeing one of the most difficult ones get done faster than I could actively do myself

[@BromeoWuggles]

Love the Image Recognition videos

[@TheLobsterCopter5000]

Try the stargazing one next! Give the automator a challenge by having to compete with 3 expert CPUs at that game...

[@theAstarrr]

A suggestion to improve goomba wrangling - Check which way the goomba is going (and which way all bombs are going), based on the average time it takes to circle one, is it likely going to be near a bomb when circled? (maybe that would be easier if you had a program without delays) And I hope you automate all the skill based games in Marioparty DS, it's great fun watching your thinking and programs.

[@theAstarrr]

YO more nostalgia from probably my favorite videos of yours

[@piboiindahouse5169]

My guy's got a sponsor! Good job, you deserve it.

[@canadianlucario2643]

Would you try any other Mario party games with this? I’d love to see more of these videos, they help me a lot with learning what I can do with programming

[@ermamu]

I’d love to see more! Do you think ‘Hammer Chime’ would work?

[@CodeNoodles]

Probably

[@DagoDuck]

If there‘s one thing I‘d really like to see, it‘s optimizing these scripts to near perfection. For example the first minigame if it wasn‘t for that bomb, it would‘ve had so many Goombas, the game had to squish them together.

[@johnsimpsen5]

YES!! I love Mario party ds!!! Definitely the best Mario party!!! Please do this game again!

[@krankenwagen6042]

this is the sequel we didnt deserve but the sequel we got

[@Imaproshaman7]

This was really cool! I'd love to see more of these. The original Using Image Recognition To Find Mario video is how I found out about this channel in the first place. Oh yeah, I'd be interested to see if it would be possible to automate the minigames that require the mic.

[@leroyhak5081]

Oh man, I remember playing this on the ds. It was always a lot of fun to play against my siblings. And for everyone who thinks the expert ai is near impossible to win from, some games like the 2nd Trace Cadets and the 3rd Domino effect all depend on how fast and accurate you are.

[@MegaKBang]

Terrifying how close the expert cpu is to beating an almost perfect AI

[@velvetbutterfly]

I would love to see you do a video explaining each piece of software you use so that people can get educated on the details to better follow what you're doing

[@htspencer9084]

Awesome vid! Unsolicited suggestion for the first one: Maybe comparing the location of the chosen goomba over two frames to determine it's movement vector and use that plus knowing how long your circle takes to draw (assuming it's a constant time), means you can draw ahead of the goomba and get it perfectly centered. You could even then track the movement of bombs within a certain range of that goomba and know ahead of time if the circle would pick up any bombs so you can avoid that goomba. Is any of this multithreaded? Would be curious to see how much of this could be put on worker threads. The drawback to my suggestion is when they bounce off the wall, as you'll need to figure out their reflections which sounds like effort but you could probably just ignore it as its only an edge case (pun very much intended).

[@CodeNoodles]

None of this is multithreaded. All of your suggestions are fantastic, but they would be nearly impossible to implement.

[@memetech-]

1:51 bleh, upgrade to pynput or something, this library has aged like milk to me.

[@derpylemon852]

Automate Study Fall pls! I remember getting a high score of 0.00cm by pressing it exactly when I needed to!

[@louisnemzer6801]

Playing video games: Fun Automating a computer to play games for you: More fun

[@fernando98322]

For Trace Cadets, I'm sure you can automate further by tracing the blue lines (really, they're line segments with a set number of directions to take) for adjacent figures but that might take too much processing time that the Expert CPU might beat you at that point

[@strawberrylemonadelioness]

I had this game as a kid and I still love it! It's still honestly my favorite Mario Party game.

[@HPnodsu]

I love this! It's always amazing watching how all this works. I personally find it really entertaining and would like to see more.

[@ItsIdaho]

One of the best DS games ever. Miss our 2 vs 2 sessions😢

[@silverstorm323]

I would love to see more minigames coded. These were fun to watch. Maybe even other MP games in the future?

[@SubroOrbus]

CodeNoodles' program vs Nintendo's program!

[@Noah-ost]

This was such a good idea for a game to automate! i alwyas find it fascinating when we can make AI that is better than the in-game one. Would love to see more minigames automated. This was one of my favorite games as a kid ❤️

[@TheMamaluigi300]

Maybe for Goomba Wrangler, the program could probably compare the latest screenshot with the previous one to gauge its target’s speed so it can predict where the goomba will be when the circle completes

[@akamai8097]

I'd like to see more mario party minigames, but I'd love it if you did an episode on a warioware game!

[@Chackravartin]

This may aswell be magic as far as im consered

[@noyz-anything]

once you're done with mario party, maybe try automating pac-pix

[@bupp291]

These are so fun! I'm actually taking a lot of inspiration from these videos for my final project in my Master's degree for Biomedical Engineering. It involves automating a precise specimen sampling mechanism using image recognition, deep learning, and pyautogui. Thanks for the tips! 😃

[@CodeNoodles]

That sounds amazing! Good luck!

[@bupp291]

@@CodeNoodles Thank you! I'll let you know how it goes 😃

[@tekayo63]

this is so cool! since these were all duel (or at least the duel versions of) minigames, i think in a potential next video it could be cool to see some ffa or 2v2 games like hot shots or sprinkler scalers

[@vaporcranberries]

I want to see more of this. This was so entertaining

[@memetech-]

3:03 bleh, upgrade to PIL or something, this library has aged like milk to me. (I didn't copy the comment, I rewrote it, same joke, different recommendation. PyAutoGUI is bad imo)

[@SinisterPixel]

For Domino Effect, I feel like it might be helpful to just read the upcoming input instead of the current one. The window to press the input comes up REALLY early so you could basically prebuffer the entire thing and smoke the CPU.

[@Jagpagaco]

code noodles video lets gooo !

[@realElzie]

I’d love to see some more videos on automating Mario party games. I’ve got to design and make my own program for a Python final project for a college class and I might do something like this, not sure yet though…

[@Neoh53]

For the first mini game (circle goomba) i know from my childhood that you can draw pretty small circle on the goomba. Reduce the size of the circle to decrease the risk of catching a goomba

[@noahvasiliauskas233]

More mini games pls I love listening to these 😊

[@steelplasma256]

5:57 I wouldn't call it finished yet. The circle is too large even the opponents circle is smaller than yours. Also it doesn't need to be a perfect circle, a 9 sided polygon would be much faster to draw and look good enough.

[@CodeNoodles]

For whatever reason, the game doesn't recognize polygons with a low number of points, so I increased the number of point one by one until it was always capable of being recognized. Also I forgot to mention it in the video, but the circle automatically changes size based on how close the bob-ombs are to it, but it still might mess up.

[@CommanderKappa]

​​@@CodeNoodlesou might as well add the feature that the progranm recognises the gombas mooving direction and adjust the coordinates accordingly

[@kierrific1357]

Im glad some people remember this game at least.

[@cravonomire9971]

It is funny how nostalgic I get for this game.

[@b_megamaths]

It's good to know that, no matter how advanced code has become, I'm still able to beat it handily at Domino Effect.

[@bloctans_4]

Did you pirate it? DJ HallyBoo Is Upset!

[@itsybutsy]

When you realize he was literally just making another COM for these mario party minigames and then he pit them against the pre-existing COMs for these mario party minigames

[@rifatbobos]

"You know what? Fck you" **becomes an even more Expert CPU*

[@__--_--_-----]

I love automating, done some work with pyautogui myself to autowin the wordle battle royale game lol. great vid, would love to see more of these

[@LevelUpGA]

Next time you can try all bloons TD with genetic algorithm, theoretically you need only string recognition and tower positions/hotkeys and clicking on the map and good luck with training, I try random clicking and I win BTD1 in 2/4 times with random clicking (generation 0) without upgrades but not with full hp

[@justarandomwoof2247]

Would be cool to see more!

[@mememkyu]

I need more of this! This type of stuff is so cool to me

[@pokeli2728]

I love these videos! Please make more like this!

[@Mickelraven]

This was great! I would love to see you automate more Mario Party DS mini-games. Part 2 please?

[@Gettenhart]

This is codebullet but with actual coding! 😂 Cheers

[@rainbowcat498]

The sweer nostalgia i got from watching this!!!

[@Vextrove]

I would like to see more minigames!

[@pablojp3498]

Loved the vid

[@uberlord983]

Hear me out, I have an idea: Write automated code for every mini-game in Mario Party DS, as WELL as custom Party Board AI. Then, bundle all of it together as a custom mod pack that replaces the actual AI with your code, so its essentially like playing against a bunch of copies of you.

[@VolcanoGamingVR]

4:27 - bomb-omb lol I like it

[@__--_--_-----]

would be nice to see it taken to the next level to automate a more complex game, I think that is more impressive than doing a few simpler games

[@samuelthecamel]

For the first goomba minigame, why not just draw squares instead of circles? Squares are faster and can still enclose the goombas.

[@CodeNoodles]

The game doesn't recognize squares unfortunately.

[@samuelthecamel]

@@CodeNoodles ah, that's a shame

[@minustesty]

2:22

[@daskampffredchen]

I would love to see more

[@agamerfan6647]

In case no one else has said it, pyautogui probably didn't work because it uses older methods of doing stuff that aren't always compatible with newer software, so it doesn't even see the key presses. This may have been the case with your "DS". Pydirectinput solves this problem by using newer methods. At least, that's my understanding.

[@XenoPon2]

The best Mario party ever made.

[@batofdestiny]

I love videos like this. I’m not very smart with computers and technology but this shit is super cool

[@7dedlysins193]

high if you do not mind me asking where did you learn Python for these neural networks/imageReC?. any KZbin channel?

[@treehousegames7903]

When a highly optimized TAS only barely beats the expert level com

[@mariodevlog]

Actually, in Goomba Wrangler the area you draw can be another shape rather than a circle, meaning you could have just rendered a simple square ⬛️

[@Heeter]

More videos like this

[@Haagimus]

Bro! One of the Warioware games!!

[@jordancrouse714]

These are the best. Its so cool what you can do. I dont understand it but im going to watch it 🤣

[@nathanlamberth7631]

Those second two were too close for comfort

[@CoffeeAddictEvan]

I think if you use what's called the midpoint circle algorithm to draw your circles, instead of generating a polygon and then drawing it, you should be able to grab goombas much faster

[@benshattock4945]

For the second one you could have done some image processing on the shapes, using something like a Lapacian filter for edge detection, with the correct thresholding this would give you a black and white image (with the white part of the image being the shapes you need to draw). This can also be computed very quickly if you did the calculations in the Fourier space. This can be done with Modules like OpenCV-python. :)