Image Recognition DESTROYS Mario Minigames

In this video I write some more Python scripts to play minigames from NSMB and SM64DS using image recognition.
After my last video I decided I wanted to automate some more minigames in Python using image recognition. The scripts I write in this video can play 3 new games and it's impressive how well they do! I hope you enjoy the video and subscribe if you like this kind of content.
LINKS
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
Support the channel: ko-fi.com/codenoodles
Itch.io: codenoodles.itch.io/
GitHub: github.com/OfficialCodeNoodles
MUSIC
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
Thanks to @kingyoshi9019 and @ThePurpleAnon for allowing me to use their remixes in this video.
• New Super Mario Bros. ...
• Luigi's Casino - New S...
█▀ █ █ █▄▄ █▀ █▀▀ █▀█ █ █▄▄ █▀▀
▄█ █▄█ █▄█ ▄█ █▄▄ █▀▄ █ █▄█ ██▄

[@ThePurpleAnon]

very awesome video! watching the bob-omb game was more satisfying than i'd like to admit lol... (also thank you for using my remix in this video!)

[@CHILLIN2540]

Sort or 'splode was always really satisfying to me

[@rubiks_20]

oh hi I didn't see you here

[@SquirrelTheorist]

Ooo you made the remix?! Can you post the link pls??!

[@CHILLIN2540]

@@SquirrelTheorist it's in the description but here kzbin.info/www/bejne/q3ezdKSBmNmbnpY (unless thie isn't what you wanted

[@rubiks_20]

@@SquirrelTheorist kzbin.info/www/bejne/q3ezdKSBmNmbnpY

[@gigawooper2733]

Ah yes, my favorite Mario minigame gimmick... *s e g r e g a t i o n*

[@plaguedoctor3]

I laughed harder at this than I needed too.

[@trickygamer555]

That's actually my favourite mario minigame tho lol

[@thatsufferingguy1058]

I'm not so sure bowser's a good ruler anymore.

[@trashlambo]

That's my boy! 🧓

[@Spingerex]

Lmao

[@pumpkinzz5728]

For the trigonometry on the cannonball mini game, atan is an expensive algorithm. Your code would run faster if you simply multiplied your calculated vector by -1.

[@novarender_]

The time difference is negligible. But yes, negating the original vector is a good idea.

[@chwong9256]

Hey I have a project that needs to calculate the angle from a point to another point and I also used atan. Do you mind explaining a bit about how this simplified version works? Thanks!

[@pumpkinzz5728]

@@chwong9256 it only works if the angle is exactly 180 degrees (although 90 degrees is also straightforward). If you are trying to find an unknown angle between two vectors, I would recommend atan as in this video, or doing a bit of research into cosine similarity.

[@pumpkinzz5728]

@@novarender_ fair enough, I guess it’s not being computed every frame or smtg like that.

[@chwong9256]

@@pumpkinzz5728 oh ok thank you!

[@joekewl7539]

Meanwhile, I'm amazed Luigi never blinked that *entire* time.

[@aurafox1]

The funny thing is, the strat with dragging the bomb instantly to the base is possibly actually legitimately doable if you just play the game with two styluses.

[@iamboredfor2months]

That's what I was thinking!

[@ghastdude589]

I don't remember how DS touchscreens work; Does it update the "tapped" position instantly to the new location or is the distance between the two halved if two presses are done at once?

[@aurafox1]

@@ghastdude589 It averages the position. But you can press one, then the other, then lift the first for faster movement.

[@tonyhakston536]

@@ghastdude589it’s a 4-wire analog resistive touchscreen, so it only physically supports single-point touch. When multi-point touch is attempted, this results in the accepted input being the average of the attempted input. With a light enough grip on stylus A and a forceful enough tap with stylus B, you could theoretically bounce stylus A off the touchscreen. If you were to attempt this for real, though, I wouldn’t recommend using a stylus for the concussive touchpoint. You’d want to use your finger to spread out the force and prevent damage to the screen.

[@BigOlSmellyFlashlight]

or two finger

[@FfrankF]

I really like how you changed the background of the little CodeNoodles guy to a darker shade so all of us gremlins watching at 3 am don't get flashbanged

[@kaylasheblosky8454]

It’s a real help

[@TyroRNG]

The big problem with the bomb sling is that you don't account for the screen gap of the ds. On the physical hardware there is a bit of space between the top and bottom screen which is accounted for in the minigame itself. Aiming at a straight angle with both screens put together actually aims too low for the top screen.

[@pokepress]

Also, the bomb will fall slightly between the time the ball is released and when it reaches the position the bomb is at.

[@MrCheeze]

Yeah, you also WANT to be aiming slightly below the bomb, so in this case the two errors roughly cancelled out. But there's room for improvement.

[@Pixelhurricane]

also looks like there's no priority for bombs found near the bottom of the screen so it gets overwhelmed easily when a lot of parabob-ombs spawn at once

[@squidiskool]

Terry theres as Like!

[@devforfun5618]

@@Pixelhurricane he definitelly programed priority for the lower bombs

[@landon2plants]

that’s awesome how fast the bomb sorting minigame was so quickly solvable for the script

[@sluggknight7130]

12:05 toad needs a raise

[@spamton_g_spamtom474]

He giving bear to the entre muchroom kindoom exept 2 guys

[@BLANKdev]

I have no idea how you can do these amazing projects so quickly, but you have quickly become one of my most anticipated new devs to wait for videos from. This mini series is awesome!!

[@danielyuan9862]

Well, he only makes the algorithm work. He doesn't try to optimize it.

[@matthias916]

for the slingshot thing, you can just take the bomb position minus the slingshot position and multiply that by -1 instead of calculating the angle

[@jlco]

You would still need to normalize it to keep the position on the screen. I assume that's still way faster, though.

[@johanngambolputty5351]

or just do p2-p1 instead of p1-p2 in the first place... and then normalise as required

[@Howzieky]

@@jlco yeah normalizing is just dividing each component by the overall distance. Way faster. Also, you don't have to normalize it. You could just check if either component is a certain length. If it is, multiply it by a decimal that'll set it to the max value. Then multiply the other component by that same number. Basically, if the max component size is 2, and your vector is (2,4), the y component is too big. So you multiply it by a decimal that will set it to 2. That's .5. Then multiply the other component by that same decimal. Your new vector is (1,2). Same angle, smaller magnitude. Sure, this give vectors at a 45 degree angle a larger magnitude than those at 0 or 90 or 180 etc. but is that even really a problem?

[@jlco]

@@Howzieky That might be a better solution, assuming the projectiles don't move too slowly.

[@khatharrmalkavian3306]

You don't even have to multiply, just invert the subtraction.

[@FuzzyJeffTheory]

I’d like to see the ‘Puzzle Panic’ mini game (the one where you select a tile and its neighbors are flipped)

[@josephcole7399]

That game can actually be proccessed to show the exact tiles to solve it, using bit operations on a array of the lights state. Once they know the start and end points, it is a completely solved game.

[@PixieOfTheSewers]

The desert one? Is that the right one?

[@barttennekes544]

@@PixieOfTheSewers Yes

[@user-pr6ed3ri2k]

ddbtpdktktk

[@user-pr6ed3ri2k]

t

[@ResonanceHub]

I'd love a follow up of this video were you perfect the errors the AI's did so that they're absolutely perfect! And I want to see a livestream of that AI endlessly perfectly solving the minigames till infinity

[@Ohrami]

It's not an AI. It's just a script.

[@ResonanceHub]

@@Ohrami I know, but saying AI sounds cool 😂

[@TheBaxes]

@@ResonanceHub marketing department in a nutshell

[@insertgamertag5369]

@@TheBaxes 9 year old after learning how to make a script in counter strike

[@pokepress]

Seeing which cards were revealed at the start of the matching game where it got stuck might help determine what happened. From the video, it looks like the code thinks there is still a card there that needs to be clicked. Were either the Goombas or Boos in the cards revealed to the player? I looked at your code and it seems like if it reaches a case where it picks two unknown cards in a row that happen to match, the cards may disappear without the code realizing what the value of the second card was (due to 0.5 seconds of sleep), so it thinks it is unknown, and keeps clicking that space because there are no other matches for it to use (alternatively, it could have registered the value of the second card, but not realized it had been removed as a match and thinks those two cards still exist). You might not need every part of that to happen to get stuck, but I think the second unknown card being a match is the key.

[@blitz-o-byte]

literally just watched the "wanted" minigame video, and i love that theres another one of these! coding is super cool to watch be done

[@zdelrod829]

I would love to see these scripts be optimized to the point of being near endless.

[@happily_blue]

I think you hit the nail on the head with the first video. "Wanted" is probably the most interesting one of all the minigames. Next would probably be the Wario coins one, as that'd involve a lot more tracking than you currently do.

[@threemays]

I don't think I've EVER seen that animation where the bomb squares are emptied. Really cool and satisfying to see these programs at work.

[@PhantomAarantula]

Came for Sort or Splode, but stayed to have my mind blown at how the script could so well match all of those cards.

[@NewfieCatgirl]

Hello

[@i_forgor2813]

That bomb sorting game was always my worst enemy as a kid, because I lost the stylus and had to use my finger. The screen was incredibly finicky.

[@thewiseoldfox]

Same. And as a child I had to go through multiple DS because the touch screen ages like milk (mainly dust particles in the edges I assume?) to the point that reconfiguring the touch screen wasn't even possible

[@TearfulZorua]

my ds's touchscreen worked fine. I always lost the stylus, but it still worked fine with finger. Though, the top screen broke twice and I had to replace the ds (the part where it clicks into place broke off, and the screen was no longer attached to the controller base except for with wires). I also had only one 3ds and it too worked perfectly fine up until a few years ago when the SD card slot started to malfunction. Both I used constantly for years.

[@petoro9203]

Actually the ds and dsi screens work with other stuff besides finger/stylus so when i inevitably lost my stylus, i used a dull toothpick. It was great!!

[@tt_thoma]

That character somehow still terrifies me but it codes awesome things I wouldn't think of in the first place

[@breadeater1972]

10/10 video man! the part with the tan, sin, and cos with the paragliding goomba is the first time I've seen someone use it in "real life" XD

[@breadeater1972]

XD

[@renakunisaki]

If only they'd told me in high school that trigonometry is important for games.

[@breadeater1972]

@@renakunisaki XD so true though

[@ComplexOri]

This was very fun to watch! Thank you for spending time on these minigames!

[@jamesdoesthings1096]

I am an enjoyer of these automated script videos destroying my scores in these games. I’d love to see more!

[@SuperStarStudiosPlush]

I would love to see part 3! You’re actually really good at this, and I love your video style! :)

[@greebo4446]

The push and pull of encountering a problem, and then devising a solution is what makes these coding videos so fun and interesting to watch. That being said, I feel like each minigame falls one problem/solution cycle short, and it's a bit unsatisfying. IMHO, once you feel like it's "good enough", I think you should dig in and push for one more complex improvement to really bring it home. Regardless, this video was awesome and thanks for the memories of these gems, I can't wait to see you crush some more :)

[@SHADECIRCLE]

This is incredible, I'm really excited to see if you'll do the rest of them. Or even minigames from other games like Kirby.

[@maneruloquendero]

I admire whoever capable of programming such things. Because for the life of me, I will never be capable. Love this vid ♡ Also, that Toad from the 3rd minigame was hella busy

[@AbelShields]

Poor luigi going bankrupt at the end

[@batofdestiny7801]

This is really cool, even if I don’t understand computer/scripting stuff. Please, keep doing more of these types of things.

[@luma4682]

This is so cool, inspiring, thanks dude, you did great job with my childhood ❤️ I love it ^^

[@Imaproshaman7]

Wow, this was really cool. I'd love to see the other games done too, like the Pair-a-Gone and On would be interesting. Anyways, these are really awesome. Nice.

[@SinglExplosiv]

good video, much more calming version of a code-themed projectile that doesnt give me anxiety watching their process. can't wait for more!

[@memetech-]

The memory master one is clicking too early, note the goomba one where a match was found, but was ignored because the game hadn’t started yet

[@TyGuy2462]

That’s not the script doing that, that’s the game showing the player a few cards before the game starts

[@renakunisaki]

That might be why it got stuck though. It clicked before an animation had finished, and the game missed it. That, or it failed to recognize a card being removed. Perhaps it needs to recognize the "empty space" graphic too.

[@memetech-]

@@TyGuy2462 oh yeah, maybe if it waits a sec first that will fix it

[@Xander_X]

I love these sort of videos and want to practice programming soon

[@Mariomaster_rl]

I’ve been waiting for this episode not even knowing it was coming out😅

[@Kirby_Super]

I really hope this becomes a series because I would love if you did that one mini game where you have to draw lines to guide him towards the star and away from the piranha plants, Don't know what it's called.

[@Adrianucho_]

Just seeing that code being written down at full speed makes me think how many hours did it take to make this video... And why I prefer animating things instead of programming them... Good job, so interesting! Also love the Spaghetti.

[@melonypanda3571]

You just brought back suppressed memories of all these minigames

[@spadox3199]

Plot twist: The sort or 'splode game was played by a human and is absolutely insane.

[@tajini817]

Ummm. Acshuarry, it is being played by an algoritm and not a human! 🤓😂

[@boiifinnaroast5109]

@@tajini817"plot twist"

[@tajini817]

@@boiifinnaroast5109 r/whoosh

[@Panba97]

@@tajini817 r/itswooooshwith4os

[@Panba97]

@@tajini817 r/itswooooshwithouttheH

[@Yakoiisyaboi]

new super mario ds was my childhood, man. My sister and I LOVED(and still do love) that game. Glad to see it getting love :)

[@shoefight6302]

new to the channel and just wow! you do some really cool stuff! cant wait for you to go big noodle man

[@peodjdn]

Props to you man. I remember working with coding in high-school and that stuff really was difficult

[@jamesonshekmeister]

Watching the Memory Master game at 100x Speed and seeing Luigi bounce up and down at rapid speed was hilarious. It's like he was hacked up on caffeine, like he took a few Red Bulls and espresso shots before coming into the casino.

[@buhlsofst33l67]

I love the bye bye at the end nodding to the bye bye Mario makes when you close the ds on either of those games

[@rouge4736]

THIS IS AMAZING I CAN SORT OR SPLODE GET EVERY ONE IN AS I TRYED ALL THE TIME TO NEVER MISS ONE OR PUT ONE IN THE WRONG AREA YOU ARE A GOD

[@shadowzero3673]

I played the bomb op sort minigame SO MUCH as a kid, nice to see something so close to perfection Good job!^^

[@Arial55555]

39100 for Bob-omb squad isn't too bad, but it could be WAY Better. I at least expected the program to get higher than my high score of 87000, but it barely made it halfway! I recommend having the program prioritize hitting areas with larger clusters of bob-ombs to get more score per shot via combo-hits (and to more efficiently kill bob-ombs in general), and to prioritize shooting the Lakitu down at all costs later in the game, since even a computer would be incapable of keeping up with the number of bob-ombs on-screen due to how slow it takes the slingshot to reload. On a completely different note, you (as expected) easily beat my score of 330 on Sort or 'Splode- and could probably compete with the TAS, given some minor tweaks to the code. Nice work.

[@danielyuan9862]

I think the least thing an AI could do is to improve its accuracy.

[@Nyroque]

Interesting Video! One suggestion I have for the slingshot minigame would be to prioritize hitting the lowest bombs on the screen

[@PanKoperek]

bro making cheats for every super mario minigame, i love it

[@SirRebrl]

On Bob-omb Squad, if you take deltaX and deltaY from the bomb-omb to the sling, you can just add that to the sling position. No trig needed at all. Trig is good for modifying a vector by other angles than 180, but with a 180 you just have to pick the right direction and then translate it instead of rotating.

[@PsiPancake]

Just discovered a cool new channel to watch! Awesome!

[@TheLastArbiter]

Really fun. I loved those mini games! For the second one, maybe it would be good to add programming to prioritize targeting Lakitu if it is on screen, since killing lakitu will remove all bombs on screen. Seems like it was only able to hit Lakitu by chance

[@byrontheusurper6505]

MAN I love these minigames! Especially Bob omb squard, classic favourite!

[@CartiersRavioli]

This scratches an itch in my brain I didn't know I had.

[@glitchy_weasel]

Nicely done! This is a surprisingly fun introduction to AI.

[@Osai1234]

this is the content i signed up for. Subscribed.

[@maximilienantoine2718]

Your video is so awesome ! (For people who have some difficulties with maths, some graphics can help (particuliary with geometry 👀)

[@Pinkio]

For those lazy people -like me- here's the timestamps: 3:49 Sort or 'Splode 7:56 Bob-omb Squad 10:44 Memory Master Btw awesome job at doing this job, it's purely high quality content and a nice and kinda funny idea too lol

[@PedroJEgea]

I didn't expect to hear the Ride of the Valkyries over bob-ombs being automatically sorted but it was absolutely perfect in every way.

[@kooldogkid149]

My guy takes TAS to a whole different league. Nice work dude.

[@AnimaAim]

PLEASE do more I love this

[@antonioreyes421]

Reminds me when i Used to code. Wasn’t a pro like you. I started out in GML (GameMaker Studio). Didn’t move up from there because of life jejejej. Seeing these videos make my inner child leap of joy.

[@SuperShadow]

ayy fellow GML starter

[@antonioreyes421]

@@SuperShadow Hello!

[@Le_Dave]

The sound effects of the bob-omb mini games awoke memories from my childhood, and those SFX are still as satisfying as they were when I was a child.

[@isaacgreen9495]

What a cool project! Thanks for sharing

[@rinchanlife]

you chose all the ones that were my favorite as a kid god the nostalgia

[@saraysilveyra2841]

I don’t know a single thing this man is talking abt but I’m here for it

[@TheAdvertisement]

The Nintendo DS games are definitely way too underrated. I had so much fun with them as a kid! Also Luigi should still be teaching kids how to gamble.

[@brawlstar1748]

This is so fascinating, thx so much man

[@asoggyflipflop]

Thank you so much for making your background dark mode

[@pecemakor917]

I would love it if you did the rest of the minigames

[@jimbomillit2544]

Luigi after the bot wins at a gambling game for the 1,000,000,000th time: "I'm getting real sick of clapping"

[@laminateshark91]

Duuuuudddee I wanna try to do this so bad!!! It looks so fun and cool!!

[@Nintendude1357]

okay every bit of this video is very interesting to watch and all but Toad running around the casino at 100x speed is really hilarious to me

[@superbseth64]

Sort or splode was so satisfying too watch

[@Wompuskitteh]

This was deeply interesting, make this your channels thing! Break all the games

[@ItsTurboCharged]

Oh my goodness seeing all these games gives me so much nostalgia! I remember I would play on my cousins 3DS. I miss those old days 😢

[@AntB10.]

so many memories made with these minigames

[@sagacious03]

Neat video! Thanks for uploading!

[@sillyboy54]

finally SOMEONE recognises SM64 DS

[@FireDotExe]

Keep up the great work man 💪

[@maryfreegirl2029]

So satisfying to see that sorting black/pink minigame solved so quickly i was so bad at it ahah

[@afktuna1985]

Coming across this as a first time viewer, enjoying the content. sidenote: I wish there was a link to the original inspiring video somewhere alongside this video

[@Hlebuw3k]

ah yes, the good old "why play the game, when you can make a robot play it for you"

[@Johnny10-man]

12:03 Look at Toad go, serving drinks at lightspeed.

[@kanzaki7681]

I love your video, this is so cool to play automatically these minigames thx so much for sharing

[@Myrderbeans]

Got an ad with the exact same music you were using in your airship bob omb part and it lined up perfectly what the fucj

[@JarethR]

This gets me thinking of the wario coin mini game (I forgot the name) but I've always thought it was impossible to get a perfect game but I would love to see that mini game be done by code!

[@uhh9364]

Love these videos big man

[@mackado]

I would love a GDQ event (or similar marathon) with competing teams having built scripts independently for different mini games.

[@CatmunistCom]

luigis cold dead eyes as his entire gambling empire is brought to its knees by the god king "player" is the funniest shit

[@OblongOtter]

these are all so cool, thanks for sharing

[@slightlyistorical1776]

For the Bomb-omb Squad minigame, a tactic I discovered while playing way back when was whenever there were too many falling Bomb-ombs on top of the flowers, just aim the slingshot directly parallel to the ground or at a slight upwards angle and just slam-fire it like a M1897 trench shotgun. It actually worked really well so long as you can keep spamming it and your score can get ridiculously high for as long as you can keep up the pace and not let any Bomb-omb slip through.

[@iamagreekgod9811]

As a kid who played a lot of new super mario for the ds with my family this puts a smile on my face

[@TitanMC_]

12:45 Toad zipping around is so entertaining

[@wjamesmci]

Memory match and 100x speed: Toad gets a workout

[@gigiusack8521]

11:54 I love Luigi’s bobbling nose. Jiggle physics FTW

[@dreamtraveler6135]

out with another banger! nice!

[@fnafmasterof]

"m-m-m-mario, what are you doing?! mario you're bankrupting the company! mario please!"

[@bingobo2551]

I’d say the side content of both 64ds & nsmb are great their multiplayer modes add a lot of charm.