Creating An AI To DESTROY Fruit Ninja

[@TylerMommsen]

Sorry it's late guys, but here is the code if you're interested github.com/TylerMommsen/fruit-ninja-bot

[@justjuniorjaw]

Image Recognition? Dog. Pig. Dog. Pig. Dog. Pig. Loaf of Bread. SYSTEM ERROR!

[@goat_yllw]

@@justjuniorjaww mitchels vs the machines reference

[@rainbowman4723]

do you think it would be possible to make this for mobile?

[@CustomGameing]

​@@rainbowman4723nha i thik mobile software need different codes(I think)

[@AceOrS]

​@@rainbowman4723yes

[@DavidKassa]

I’m kinda disappointed that the AI didn’t absolutely obliterate the pomegranates

[@fizzlanderr]

yeah seeing it get like 100+ slices would have been cooler

[@Mikustan39]

Same here. Might just not be good enough unfortunately.

[@zincalloycopper2]

Sword draw, first form: death by a thousand cuts

[@KJ.420]

It would've been even cooler if it also went for combos as well, but it's just an algorithm that looks for given images and doesn't really know the game strategies, so maybe I'm expecting too much from it.

[@dadoul9905]

get a better cpu and you'll get better pomegranates

[@CuriosityIgnited]

Just spent 10 minutes convincing myself that watching an AI play Fruit Ninja is the productive break I needed. Can confirm: still procrastinating.

[@hamzamotara4304]

Psst. Stop procrastinating. Go do it.

[@miguelcardoso5084]

Not procrastinating, just doing side-quests

[@Vo_tex]

i am doing the same thing

[@whifle1445]

Doing the same

[@wesleyneff5510]

Same

[@GwnTim1]

I love how the AI was 95% certain that the bomb was a bomb, and as soon as it hit the bomb it plummited to 80% like the AI said "nuh-uh, that wasn't a bomb THAT WAS NOT A BOMB!"

[@morphe904]

I‘m guessing that the bomb as it was exploding, was looking less and less like the bombs the AI was trained to recognize

[@mingamerking]

I think allowing it to recognize how to combo fruits would make this even better, Ike have it wait before slicing and if multiple fruits are on the screen it does one big slice

[@gameplaysuffering1620]

yeah but that would add like 2 to 5 years to dev time

[@justascreamingcat2964]

​@@gameplaysuffering162099% of that time being procrastination

[@justenoughrandomness8989]

yeah

[@dillzilla4454]

i got one even better, make a simple bot slice where there is motion with the bombs being a no-go zone determined by image recognition. this means it is only looking for one item, allowing it to be quick and confident in its determination, allows it to pretty easily do combos, and makes it much more responsive when slicing the fruits

[@Ace-isket]

@@dillzilla4454it’ll trap itself on the movement of the slice it makes though

[@anonym5160]

No fruits are being harmed in the making of this video.

[@ika32]

more like no bombs

[@2haep]

he probably ate one during it

[@hamzamotara4304]

No, fruits were harmed in the making of this video.

[@mr_pigman1013]

Liar liar pants on fire

[@Randomname-yb7hs]

Except for the ones that were

[@theradomguy5581]

Big props for making an Ai on the orginal game! Happy to see that rather then a recreation of it

[@elliott6158]

I feel like the recreation is just as cool

[@elliott6158]

it shows off more fundamental ML concepts rather than "I imported an image recognition library :B"

[@slouch186]

​@@elliott6158agreed. i much prefer ML stuff when the AI has access to the data behind the game rather than just recognizing screenshots

[@ryforg]

1:43 that’s why captchas have you identify common things found on streets. You are training their self driving car ai.

[@nastykerb34]

Rhen why are the images fuzzy, would that result in a shotty AI?

[@ryforg]

@ they started with easier images but are slowly getting harder to closer represent the real world environments the ai would actually see. If you can understand the blurry images it teaches ai to understand blurry images.

[@reanimationxp]

@@nastykerb34 first of all this isn't true. the captcha images have already been recognized otherwise the captcha wouldn't work. however let's say it were true, training on fuzzy images would likely mean when it has more clarity in real life, it's just that much more accurate. however, items are often far away or in poor lighting conditions, so there's a good chance it could be for the purpose of better training on objects that are just far away.

[@johannesbohm6458]

@@reanimationxp I just saw an interview with one of the devs behind Captcha the other day and the images shown in the Captchas are actually not pre-recognized. They are shown to a bunch of people at the same time and the majority decides if something is a certain thing or not. Otherwise this would require a lot of work and would heavily limit the amount of available pictures, thereby reducing security...

[@HitSpaceGD]

If the AI sliced the menu fruit (🍉 = play again), it could keep training for hours.

[@jmvr]

Well, it's not training when it's running, so all you're doing is making it play forever, without it being trained on anything. Maybe a modification to it to make the current AI model train a new one while it plays would be cool, but wouldn't be necessary, because this isn't meant to be a perfect AI

[@gaymerjerry]

this ai is just the photo recognition if you want it to learn to play the game better youd need a neural network thats fed the image recognition ai data as an input

[@modzyy]

​@@jmvrit takes more pictures while its running which can be used for training later on, this is common sense.

[@jmvr]

@@modzyy key word: _can_ be. It's not currently made to train a new AI. All it does is take a screenshot, analyze it, and do some input. Then the screenshot may as well be discarded, because it does not use the image further. As well, AIs are trained at a specific point in the process, but aren't trained further. For example, ChatGPT will not get better when people use it, and neither will most AI models. Typically when an AI trains itself, it gets worse. By definition, the day it doesn't is the day of the AI singularity, where it can improve itself into infinity

[@modzyy]

@@jmvr yap yap yap

[@weetikveelpik-wq5bx]

0:39 amazing, well said

[@tuloxe]

totaly on your side 😂

[@evereq8970]

Language..

[@evereq8970]

​@@tuloxelanguage..

[@Sensei-Ollie]

@@evereq8970what?

[@weetikveelpik-wq5bx]

Fym language?​@@evereq8970

[@GemedetAdept]

If you think about it, YOLO (c. end of the world, 2012) and YOLO (object recognition) are really the same thing: You'll find out *very* quickly that You Only Live Once if You Only Look Once while crossing the road!

[@DanielLCarrier]

Unless anime was right and you find out that you live at least twice.

[@windy5405]

@@DanielLCarrieronly work if it’s a truck with its headlight open

[@locrianphantom3547]

@@windy5405Or a tractor going 2 mph.

[@lunademorgan]

​@locrianphantom3547 no not Kazuma.

[@iresineherb7]

Man that 2012 was awkward

[@gregoryk4365]

So painful seeing him do all this when he could've just had something search the specific pixel color of the fruit/bombs to slice them instead of training a whole ai to recognize fruit

[@blizyon30fps86]

Exactly

[@sakshambaranwal132]

I am guessing that if he did that then the ai would proceed to again cut the fruits which have already been sliced

[@Mostsanewriomain]

@@sakshambaranwal132I can totally imagine the AI doing that

[@Solrex_the_Sun_King]

Now teach an AI to play feed the deep.

[@clivah1499]

some time ago, they needed pigeons to do this.

[@HitSpaceGD]

I think England still does

[@ThisIsAHandle-xz5yo]

They needed pigeons to play fruit ninja for them?

[@HitSpaceGD]

@@ThisIsAHandle-xz5yo I guess

[@lukasschriefer799]

Bro, I agree

[@samstromberg5593]

This channel is the most perfect example I have ever encountered of a Blue Ocean Besides maybe the Wright Brothers

[@server642]

Trajectory prediction would be wicked to see! Especially in the case of avoiding a slice if a bomb will intercept. And couple that with what I’ve seen others say: getting combos by slicing multiple fruit in one motion. If it can predict the best time to get combos based on the trajectories, I’d love to see how high a score it can really get 😈

[@kabii]

i would love to see an updated version of the ai, doing fruit combos because atm it slices each fruit individually it would be so satisfying if it would do the best slicing combos possible

[@solodemon8465]

8:00 Mangoes have as much vitamin C as oranges

[@arthurkaramazov8932]

Amazing video, thanks! Can't believe you have only ~7k subscribers. 🔥

[@sunofabeach9424]

python is simple to created an image recognition AI with, but it is so slow that by the time it has finished processing the image the results are already outdated. this pretty much sums up this whole situation where the most performance dependent tasks are solved with one of the slowest languages out there

[@ahdog8]

Generally, doing image recognition in Python involves making calls to some library that is actually written in C (or a similar, actually fast language). As a result, relatively little time is spent in Python-land which mitigates the slowness of the language. I don't know what the code used in the video looks like. There could be a bunch of complicated Python code that slows it down. Or maybe it's not Python's fault at all.

[@sunofabeach9424]

@@ahdog8 I know that Python libraries utilize C, however, the overhead that Python introduces is still too much even for complied C code

[@mohammedalhammadi3547]

3:18 You know bro's been taken by the Terminator when he called the AI "His"

[@GameJam230]

I’m not sure how feasible it would have been to decompile the game and get the models for the fruit and bombs, but if those are obtainable then would it not have been easier to train the AI using screenshots of those models rotated programmatically? Then, instead of searching the entire screen space for fruit, you only need to look in areas with a significant amount of pixels changed between consecutive frames, as these locations have either a fruit or bomb in them. However, if it’s a fruit, then you don’t actually care WHICH fruit it is, you cut it regardless. You ONLY need to care about whether or not it’s a bomb, and so when screen pixels change in an area, check the surrounding region for a bomb. If a bomb is not found, cut, otherwise avoid. It should be noted too that this approach would likely have issues with areas where bombs and fruit overlap, but that can be dealt with. This approach could ALSO be done without the decompiled models at all, instead only providing the model with gameplay screenshots of bombs and fruit overlapping bombs. By providing the AI with these images, it alerts it NOT to cut them, but if it DOESN’T see matches for that data, it DOES cut whenever it sees changed pixels. The main efficiency of this comes from not needing to run image recognition for each individual fruit, and also not running image recognition checks over the whole screen constantly, but instead only in surrounding regions where changed pixels are when those changes occur.

[@Punch_Card]

wake up babe Tyler just uploaded a new video

[@musictomyshears]

It would be interesting to try a roguelike - pixel dungeon/shattered pixel dungeon is FOSS, and I haven't seen anyone do anything like it. You'd have a variety of skills the AI would have to learn: fight mechanics, resource management, item mechanics and selection, synergies etc. You'd probably choose just one class, and since it's FOSS, if it's too complex you could easily tone it down by reducing items, making it set seed, making item generation deterministic, just doing one floor etc (but it'd be a hell of a video to do the whole game haha but maybe there's a reason no one's done it before).

[@number1-willstetsonsimp]

that sounds actually sick

[@DANIDANI310]

In the same way you can teach AI to recognize an enemy soldier and an ally and make it shoot in less than a second. AI is becoming scary .

[@blubaylon]

Well yeah, that's how classification algorithms work lol

[@Jallahruud]

Your channel is gonna blow up dude this is really high quality and entertaining content. Keep it up!

[@Nobody-tw8wf]

Dude you are a genius🙏🙏🙏 Love the effort you put in each video ❤❤❤

[@XX-ot4pb]

Hi, I love your channel! I would say it is the best channel for game AIs, I love your videos. I am wondering what tool you use to train the YOLOS; thanks!

[@Label_sabke_niklenge]

Loved your hardwork ❤❤😂

[@kogames3191]

absolutley amazing video and also very educational, nice man

[@mayzar8646]

Cool it’s another KZbinr that is underrated and has good content (:

[@Me-0063]

Great video! I would suggest adding combos

[@mac2857]

2:12 You don't actually, you can just use a clustering algorithm that can group all the similar objects together which you can then label

[@Krissoh]

You could make a system of hierarchies, where the AI ​​cuts the least important fruits first and then the most important ones, this way you could avoid cases where it cuts the pomegranate fruit, losing the other fruits.

[@mathew6041]

1:58 lies, i have done this, but did absolutely nothing with it :)

[@jamieehoover3132]

omgg you’re only 9k subs ?! you deserve way more

[@martinkoblitz1285]

it would have been so much cooler if the ai would have learned to slice more then 1 fruit at a time but still a really cool vid man keep up the good work! : )

[@HSonyoutube]

Here's another challenge for you. Try hitting combos. Basically you've got to track the motion of all fruits by taking continuous screenshots. By that you can make predictions on when a bunch on fruits might be closest to one another. And make a hit. And hit individual fruits only when those are at the end of downward motion

[@HùngNguyễn-z8z7v]

May be another way of approaching this problem is using the AI to detech the colors instead of the fruit themselves? Because the background color is distinctive.

[@KoonKlausGamer]

2:15 respect this job

[@AxoTheAxolotl707]

I accidently scrolled too far7:50

[@Johnny31323]

A pixel search algorithm would've been faster, via color indexing. All fruits have one solid color, so having a simple hex variation of lets say (green) for an apple 🍏 would be way faster and probably even better to find all the fruits. Same goes for the bomb since it is a solid black color, so the AI never would try to attack it within a solid square hit box on screen.

[@HappyMajor]

that wouldnt work with bombs you need to use the red outline for them to get their actual hitbox the rest is fine as is tho if u wanna go fancy u can go with outline detection or just as u said a simple HSV range for each fruit #edit another problem would be the already sliced fruits since they have the same color

[@lekkobot]

I feel like that might cause issues with the ai going after splatter.

[@HitSpaceGD]

​@@lekkobotWhen the reply has more likes than the original comment:

[@-CENSORED0-]

@@HitSpaceGDthat’s called a ratio young one

[@HitSpaceGD]

@@-CENSORED0- thanks for enlightening me 👍

[@aleksanderwalenciej]

You could’ve tried first idea of image recognition but with checking only for range of colors. And the fruits would have a specific range of it. When you would make it small for each fruit it could’ve worked

[@ContentWrld1]

should’ve made it insane at the part where it combos

[@Poonda-ju8xe]

I love Yolo Ai. Amazing for digital surveillance and AI cheating in games like Counter Strike. What an amazingly versatile piece of software.

[@nostew1604]

throw the gameplay into after effect -> use motion tracking -> render sequence of image -> label into seperate folder and auto rename much more easy way to do the labeling task

[@plsdont7515]

I would’ve just “cut” the center of the area that popped up that didn’t match the backgrounds

[@castleofwonders1]

You're doing a great job in content creation.... 🎉

[@En3vity0]

3:24 the voices- they won’t stop…

[@Nobody_NPT]

FISH

[@whatsaname2usern]

grandma's gonna be happy with this one

[@T110-E5]

Way too underrated channel ❤

[@MusicLabMadScientist]

Here before this channel BLOWS UP!

[@Pavlov-shack]

It was pitch black in my room watching this a inch away from my face it felt like a real flash bang

[@josephholman2401]

If you wanted to improve the ai, you only need 3 frames and some calculus to predict the exact trajectory of the fruit meaning you'd be able to slice every fruit on the 4th frame that it's on screen

[@whatevenisazero]

We got fruit ninja aim bot before GTA VI

[@pawansapkota3970]

Yes using the predicitons from the previous models and reannotating is the best option for the faster finetunning of the model.

[@am_rzq6819]

Make it beat the world record next.

[@tonic-music]

Now you need a genetic algorithm model

[@IsaacLmao111]

This guy is godly

[@achilles7498]

Ai's worst enemy, pomegranates.

[@poohritkun5068]

Now this is a real AI. Finally!!

[@Kmbmz]

Bro you chose the most complex way, all what it needed was an image recognition only for the bomb and for other moving objects And an if statement If object != bomb Slice

[@blubaylon]

How would the AI be able to tell that fruits are on the screen?

[@DEVY69420]

now i want you to make an ai that can solve captcha

[@shaynegoldstein1329]

i think just using color to find everything that isnt a bomb would be faster. the red and black on the bomb look unique.

[@joselopes9061]

Idk if it is only me, but here: 5:40 you could have put only the AI Vision or 20 seconds one 20 seconds another. The both at the same time are weird.

[@SasamuelTheCool]

Vision issue

[@Not_A_Lazy_Dev]

@@SasamuelTheCool maybe brainrot isn't that bad, maybe the attention span of a goldfish isnt that bad ... MAYBE FAMILY GUY, SUBWAY SURFERS, MINECRAFT PARKOUR AND ROCKET LEAGUE CLIPS ALONG WITH THE ACTUAL VID AT THE SAME TIME ISN'T THAT BA-

[@BlankyB4]

Who gonna tell bro 😭🙏

[@call_me_ondreo]

You can easily increase your model speed if you use lower screenshot resolution (not a game resolution), then turn image black and white, then use color contrast to make objects looks clear. And also, check to click on fruit little bit away from bomb.

[@TheLazyCowboy1]

Now feed the recognized images (x,y,w,h,type,confidence) to a neural network and see how well it can learn to play. Maybe it would even learn combos!

[@unclesamsonite-y3k]

bros ai: if its green or blue or yellow then its a fruit but if its BLACK THEN ITS A BOMB

[@akio.69]

Me with yellow and green bomb.😈

[@mr.czesky]

TOP content bro

[@arkilagr5510]

Quick question, how did you get both visions side by side? I just wanna know for future use

[@iPxter]

What program did you use for the labelling of the objects at 2:16?

[@TylerMommsen]

Roboflow

[@wesleypang2158]

what is the program you use for the labeling?

[@DinosaurusTerbaik]

This guy worth more subscribers guys.

[@R2Bl3nd]

You could've scaled down the screenshots to a much smaller size before feeding that to the AI.

[@oldscooljoe6194]

Why arent you remaking the game

[@ninjaride1312]

Making it not hit the bomb should had been easy enough

[@mloopy461]

I feel like fruit ninja is on everything nowadays 💀

[@crimsonpheonix4609]

The ai isnt even getting combo slices either

[@gg-gn3re]

0:55 you use spiral detection from center and dynamic bitmap or just color tolerances. It'd take several thousand times less processing power and work about the same. Things like simba have made this very easy for decades

[@peterchristensen8843]

Color ID won't work well since the fruits splatter after being sliced, meaning your AI would continue slicing fruit juices long after the fruits were sliced.

[@gg-gn3re]

@@peterchristensen8843 that isn't ai

[@djatmikko9579]

Thank you

[@Croissan_]

i like this youtuber

[@Usual_turtle]

4:38 This is not a 77.78% difference. A 77.78% difference would be about 819118 pixels, not 2073600 pixels. It's actually closer to 43.75% difference.

[@cyn7261]

it's a 77.78% difference from 1080p to 1440p and 43.75% vice versa but he should've said there's about a 1.6 million pixel difference between them

[@Mazeop69]

0:39 That was a little to personal 💀

[@cloudyskies-exe]

Couldnt you have done some color detection? The fruits are all unique colors you could've taken a 500x500 image for example of where the color is and gotten the fruit labeling data that way? Just a thought idk.

[@R4in46]

He's him

[@GonFrecsses-kl8bx]

I like that he dosent respect the viewers when they say any bad comments and he doesn't ignore them

[@Kon6I9]

You could make the AI not chop the fruits when theres bombs overlapping.

[@scorpionhdkid8972]

The AI has not seen Arcade mode. Oh boy..

[@thegamerkitten2190]

is this guy just code bullet but not swearing

[@smartgamingpotatopotatoz4217]

Domain expansion: The fruit Ninja

[@noah2718]

This video is the exact amount of time it takes me to eat a Totino's pepperoni party pizza.

[@redhood-n2v]

Man I am so dumb I thought the normal version was the run where he reached 488

[@thunderd7904]

The fruits and bombs are seemingly predetermined and the same every time, so could you just use reinforcement learning until your AI could get to a score of like 100,000? Also could scanning the screen for non black or non brown pixels work?

[@hank4134]

I mean the vid is fine but without any bad/dumb/dark jokes of CodeBullet it just won't cut it

[@Herecules]

Now use it on a anti-aircraft cannon

[@henriks5008]

You should program an AI to label the fruits and bombs ;)

[@dudical111]

I would love to see what happens when you train the ai on as low a resolution as you can go on the golden fruit level. I wanna see it obliterate them, dude.

[@mr-nafari7355]

lets gooooo a new video

[@darkrathwheito9818]

My all time best was 11932 with the best ammount of pomegranate hits being 89, seeing an AI getting 600 with not perfect "technique" makes me cry imagining that in 1 year of development it could become so good that reaching 6 digits would be easy T_T

[@xx__miqu__xx3812]

its happening, rise of the machines and fall of the humankind. skynet starts here

[@blubaylon]

These image recognition models have been around for a while lol