Creating An AI To DESTROY Fruit Ninja

[@TylerMommsen]

Sorry it's late guys, but here is the code if you're interested github.com/TylerMommsen/fruit-ninja-bot

[@justjuniorjaw]

Image Recognition? Dog. Pig. Dog. Pig. Dog. Pig. Loaf of Bread. SYSTEM ERROR!

[@goat_yllw]

@@justjuniorjaww mitchels vs the machines reference

[@rainbowman4723]

do you think it would be possible to make this for mobile?

[@CustomGameing]

​@@rainbowman4723nha i thik mobile software need different codes(I think)

[@AceOrS]

​@@rainbowman4723yes

[@CuriosityIgnited]

Just spent 10 minutes convincing myself that watching an AI play Fruit Ninja is the productive break I needed. Can confirm: still procrastinating.

[@hamzamotara4304]

Psst. Stop procrastinating. Go do it.

[@miguelcardoso5084]

Not procrastinating, just doing side-quests

[@Vo_tex]

i am doing the same thing

[@whifle1445]

Doing the same

[@wesleyneff5510]

Same

[@DavidKassa]

I’m kinda disappointed that the AI didn’t absolutely obliterate the pomegranates

[@fizzlanderr]

yeah seeing it get like 100+ slices would have been cooler

[@Mikustan39]

Same here. Might just not be good enough unfortunately.

[@zincalloycopper2]

Sword draw, first form: death by a thousand cuts

[@KJ.420]

It would've been even cooler if it also went for combos as well, but it's just an algorithm that looks for given images and doesn't really know the game strategies, so maybe I'm expecting too much from it.

[@dadoul9905]

get a better cpu and you'll get better pomegranates

[@mingamerking]

I think allowing it to recognize how to combo fruits would make this even better, Ike have it wait before slicing and if multiple fruits are on the screen it does one big slice

[@gameplaysuffering1620]

yeah but that would add like 2 to 5 years to dev time

[@justascreamingcat2964]

​@@gameplaysuffering162099% of that time being procrastination

[@justenoughrandomness8989]

yeah

[@dillzilla4454]

i got one even better, make a simple bot slice where there is motion with the bombs being a no-go zone determined by image recognition. this means it is only looking for one item, allowing it to be quick and confident in its determination, allows it to pretty easily do combos, and makes it much more responsive when slicing the fruits

[@ryforg]

1:43 that’s why captchas have you identify common things found on streets. You are training their self driving car ai.

[@anonym5160]

No fruits are being harmed in the making of this video.

[@ika32]

more like no bombs

[@Mrshaep]

he probably ate one during it

[@hamzamotara4304]

No, fruits were harmed in the making of this video.

[@mr_pigman1013]

Liar liar pants on fire

[@Randomname-yb7hs]

Except for the ones that were

[@theradomguy5581]

Big props for making an Ai on the orginal game! Happy to see that rather then a recreation of it

[@elliott6158]

I feel like the recreation is just as cool

[@elliott6158]

it shows off more fundamental ML concepts rather than "I imported an image recognition library :B"

[@slouch186]

​@@elliott6158agreed. i much prefer ML stuff when the AI has access to the data behind the game rather than just recognizing screenshots

[@GwnTim1]

I love how the AI was 95% certain that the bomb was a bomb, and as soon as it hit the bomb it plummited to 80% like the AI said "nuh-uh, that wasn't a bomb THAT WAS NOT A BOMB!"

[@morphe904]

I‘m guessing that the bomb as it was exploding, was looking less and less like the bombs the AI was trained to recognize

[@HitSpaceGD]

If the AI sliced the menu fruit (🍉 = play again), it could keep training for hours.

[@jmvr]

Well, it's not training when it's running, so all you're doing is making it play forever, without it being trained on anything. Maybe a modification to it to make the current AI model train a new one while it plays would be cool, but wouldn't be necessary, because this isn't meant to be a perfect AI

[@gaymerjerry]

this ai is just the photo recognition if you want it to learn to play the game better youd need a neural network thats fed the image recognition ai data as an input

[@modzyy]

​@@jmvrit takes more pictures while its running which can be used for training later on, this is common sense.

[@jmvr]

@@modzyy key word: _can_ be. It's not currently made to train a new AI. All it does is take a screenshot, analyze it, and do some input. Then the screenshot may as well be discarded, because it does not use the image further. As well, AIs are trained at a specific point in the process, but aren't trained further. For example, ChatGPT will not get better when people use it, and neither will most AI models. Typically when an AI trains itself, it gets worse. By definition, the day it doesn't is the day of the AI singularity, where it can improve itself into infinity

[@modzyy]

@@jmvr yap yap yap

[@weetikveelpik-wq5bx]

0:39 amazing, well said

[@tuloxe]

totaly on your side 😂

[@evereq8970]

Language..

[@evereq8970]

​@@tuloxelanguage..

[@Sensei-Ollie]

@@evereq8970what?

[@weetikveelpik-wq5bx]

Fym language?​@@evereq8970

[@GemedetAdept]

If you think about it, YOLO (c. end of the world, 2012) and YOLO (object recognition) are really the same thing: You'll find out *very* quickly that You Only Live Once if You Only Look Once while crossing the road!

[@DanielLCarrier]

Unless anime was right and you find out that you live at least twice.

[@windy5405]

@@DanielLCarrieronly work if it’s a truck with its headlight open

[@locrianphantom3547]

@@windy5405Or a tractor going 2 mph.

[@lunademorgan]

​@locrianphantom3547 no not Kazuma.

[@iresineherb7]

Man that 2012 was awkward

[@gregoryk4365]

So painful seeing him do all this when he could've just had something search the specific pixel color of the fruit/bombs to slice them instead of training a whole ai to recognize fruit

[@blizyon30fps86]

Exactly

[@sakshambaranwal132]

I am guessing that if he did that then the ai would proceed to again cut the fruits which have already been sliced

[@Mostsanewriomain]

@@sakshambaranwal132I can totally imagine the AI doing that

[@kabii]

i would love to see an updated version of the ai, doing fruit combos because atm it slices each fruit individually it would be so satisfying if it would do the best slicing combos possible

[@Solrex_the_Sun_King]

Now teach an AI to play feed the deep.

[@samstromberg5593]

This channel is the most perfect example I have ever encountered of a Blue Ocean Besides maybe the Wright Brothers

[@arthurkaramazov8932]

Amazing video, thanks! Can't believe you have only ~7k subscribers. 🔥

[@Jallahruud]

Your channel is gonna blow up dude this is really high quality and entertaining content. Keep it up!

[@server642]

Trajectory prediction would be wicked to see! Especially in the case of avoiding a slice if a bomb will intercept. And couple that with what I’ve seen others say: getting combos by slicing multiple fruit in one motion. If it can predict the best time to get combos based on the trajectories, I’d love to see how high a score it can really get 😈

[@clivah1499]

some time ago, they needed pigeons to do this.

[@HitSpaceGD]

I think England still does

[@ThisIsAHandle-xz5yo]

They needed pigeons to play fruit ninja for them?

[@HitSpaceGD]

@@ThisIsAHandle-xz5yo I guess

[@lukasschriefer799]

Bro, I agree

[@musictomyshears]

It would be interesting to try a roguelike - pixel dungeon/shattered pixel dungeon is FOSS, and I haven't seen anyone do anything like it. You'd have a variety of skills the AI would have to learn: fight mechanics, resource management, item mechanics and selection, synergies etc. You'd probably choose just one class, and since it's FOSS, if it's too complex you could easily tone it down by reducing items, making it set seed, making item generation deterministic, just doing one floor etc (but it'd be a hell of a video to do the whole game haha but maybe there's a reason no one's done it before).

[@number1-willstetsonsimp]

that sounds actually sick

[@solodemon8465]

8:00 Mangoes have as much vitamin C as oranges

[@Label_sabke_niklenge]

Loved your hardwork ❤❤😂

[@kogames3191]

absolutley amazing video and also very educational, nice man

[@GameJam230]

I’m not sure how feasible it would have been to decompile the game and get the models for the fruit and bombs, but if those are obtainable then would it not have been easier to train the AI using screenshots of those models rotated programmatically? Then, instead of searching the entire screen space for fruit, you only need to look in areas with a significant amount of pixels changed between consecutive frames, as these locations have either a fruit or bomb in them. However, if it’s a fruit, then you don’t actually care WHICH fruit it is, you cut it regardless. You ONLY need to care about whether or not it’s a bomb, and so when screen pixels change in an area, check the surrounding region for a bomb. If a bomb is not found, cut, otherwise avoid. It should be noted too that this approach would likely have issues with areas where bombs and fruit overlap, but that can be dealt with. This approach could ALSO be done without the decompiled models at all, instead only providing the model with gameplay screenshots of bombs and fruit overlapping bombs. By providing the AI with these images, it alerts it NOT to cut them, but if it DOESN’T see matches for that data, it DOES cut whenever it sees changed pixels. The main efficiency of this comes from not needing to run image recognition for each individual fruit, and also not running image recognition checks over the whole screen constantly, but instead only in surrounding regions where changed pixels are when those changes occur.

[@mayzar8646]

Cool it’s another KZbinr that is underrated and has good content (:

[@XX-ot4pb]

Hi, I love your channel! I would say it is the best channel for game AIs, I love your videos. I am wondering what tool you use to train the YOLOS; thanks!

[@martinkoblitz1285]

it would have been so much cooler if the ai would have learned to slice more then 1 fruit at a time but still a really cool vid man keep up the good work! : )

[@jamieehoover3132]

omgg you’re only 9k subs ?! you deserve way more

[@sunofabeach9424]

python is simple to created an image recognition AI with, but it is so slow that by the time it has finished processing the image the results are already outdated. this pretty much sums up this whole situation where the most performance dependent tasks are solved with one of the slowest languages out there

[@ahdog8]

Generally, doing image recognition in Python involves making calls to some library that is actually written in C (or a similar, actually fast language). As a result, relatively little time is spent in Python-land which mitigates the slowness of the language. I don't know what the code used in the video looks like. There could be a bunch of complicated Python code that slows it down. Or maybe it's not Python's fault at all.

[@sunofabeach9424]

@@ahdog8 I know that Python libraries utilize C, however, the overhead that Python introduces is still too much even for complied C code

[@DANIDANI310]

In the same way you can teach AI to recognize an enemy soldier and an ally and make it shoot in less than a second. AI is becoming scary .

[@blubaylon]

Well yeah, that's how classification algorithms work lol

[@Me-0063]

Great video! I would suggest adding combos

[@HùngNguyễn-z8z7v]

May be another way of approaching this problem is using the AI to detech the colors instead of the fruit themselves? Because the background color is distinctive.

[@HSonyoutube]

Here's another challenge for you. Try hitting combos. Basically you've got to track the motion of all fruits by taking continuous screenshots. By that you can make predictions on when a bunch on fruits might be closest to one another. And make a hit. And hit individual fruits only when those are at the end of downward motion

[@nostew1604]

throw the gameplay into after effect -> use motion tracking -> render sequence of image -> label into seperate folder and auto rename much more easy way to do the labeling task

[@pawansapkota3970]

Yes using the predicitons from the previous models and reannotating is the best option for the faster finetunning of the model.

[@Johnny31323]

A pixel search algorithm would've been faster, via color indexing. All fruits have one solid color, so having a simple hex variation of lets say (green) for an apple 🍏 would be way faster and probably even better to find all the fruits. Same goes for the bomb since it is a solid black color, so the AI never would try to attack it within a solid square hit box on screen.

[@HappyMajor]

that wouldnt work with bombs you need to use the red outline for them to get their actual hitbox the rest is fine as is tho if u wanna go fancy u can go with outline detection or just as u said a simple HSV range for each fruit #edit another problem would be the already sliced fruits since they have the same color

[@lekkobot]

I feel like that might cause issues with the ai going after splatter.

[@HitSpaceGD]

​@@lekkobotWhen the reply has more likes than the original comment:

[@-CENSORED0-]

@@HitSpaceGDthat’s called a ratio young one

[@HitSpaceGD]

@@-CENSORED0- thanks for enlightening me 👍

[@aleksanderwalenciej]

You could’ve tried first idea of image recognition but with checking only for range of colors. And the fruits would have a specific range of it. When you would make it small for each fruit it could’ve worked

[@Krissoh]

You could make a system of hierarchies, where the AI ​​cuts the least important fruits first and then the most important ones, this way you could avoid cases where it cuts the pomegranate fruit, losing the other fruits.

[@shaynegoldstein1329]

i think just using color to find everything that isnt a bomb would be faster. the red and black on the bomb look unique.

[@call_me_ondreo]

You can easily increase your model speed if you use lower screenshot resolution (not a game resolution), then turn image black and white, then use color contrast to make objects looks clear. And also, check to click on fruit little bit away from bomb.

[@T110-E5]

Way too underrated channel ❤

[@joselopes9061]

Idk if it is only me, but here: 5:40 you could have put only the AI Vision or 20 seconds one 20 seconds another. The both at the same time are weird.

[@SasamuelTheCool]

Vision issue

[@Not_A_Lazy_Dev]

@@SasamuelTheCool maybe brainrot isn't that bad, maybe the attention span of a goldfish isnt that bad ... MAYBE FAMILY GUY, SUBWAY SURFERS, MINECRAFT PARKOUR AND ROCKET LEAGUE CLIPS ALONG WITH THE ACTUAL VID AT THE SAME TIME ISN'T THAT BA-

[@BlankyB4]

Who gonna tell bro 😭🙏

[@KoonKlausGamer]

2:15 respect this job

[@castleofwonders1]

You're doing a great job in content creation.... 🎉

[@Pavlov-shack]

It was pitch black in my room watching this a inch away from my face it felt like a real flash bang

[@Punch_Card]

wake up babe Tyler just uploaded a new video

[@Poonda-ju8xe]

I love Yolo Ai. Amazing for digital surveillance and AI cheating in games like Counter Strike. What an amazingly versatile piece of software.

[@josephholman2401]

If you wanted to improve the ai, you only need 3 frames and some calculus to predict the exact trajectory of the fruit meaning you'd be able to slice every fruit on the 4th frame that it's on screen

[@mathew6041]

1:58 lies, i have done this, but did absolutely nothing with it :)

[@En3vity0]

3:24 the voices- they won’t stop…

[@Nightly_MiniMilitia]

FISH

[@mohammedalhammadi3547]

3:18 You know bro's been taken by the Terminator when he called the AI "His"

[@iPxter]

What program did you use for the labelling of the objects at 2:16?

[@TylerMommsen]

Roboflow

[@gg-gn3re]

0:55 you use spiral detection from center and dynamic bitmap or just color tolerances. It'd take several thousand times less processing power and work about the same. Things like simba have made this very easy for decades

[@peterchristensen8843]

Color ID won't work well since the fruits splatter after being sliced, meaning your AI would continue slicing fruit juices long after the fruits were sliced.

[@gg-gn3re]

@@peterchristensen8843 that isn't ai

[@Guns4Peace]

Bro you chose the most complex way, all what it needed was an image recognition only for the bomb and for other moving objects And an if statement If object != bomb Slice

[@blubaylon]

How would the AI be able to tell that fruits are on the screen?

[@arkilagr5510]

Quick question, how did you get both visions side by side? I just wanna know for future use

[@Usual_turtle]

4:38 This is not a 77.78% difference. A 77.78% difference would be about 819118 pixels, not 2073600 pixels. It's actually closer to 43.75% difference.

[@cyn7261]

it's a 77.78% difference from 1080p to 1440p and 43.75% vice versa but he should've said there's about a 1.6 million pixel difference between them

[@MusicLabMadScientist]

Here before this channel BLOWS UP!

[@mr.czesky]

TOP content bro

[@DoobLand]

should’ve made it insane at the part where it combos

[@DEVY69420]

now i want you to make an ai that can solve captcha

[@IsaacLmao111]

This guy is godly

[@achilles7498]

Ai's worst enemy, pomegranates.

[@DinosaurusTerbaik]

This guy worth more subscribers guys.

[@Mazeop69]

0:39 That was a little to personal 💀

[@dillzilla4454]

there is a significantly faster way of doing this which is looking for clusters of pixels that have changed and only using the AI to detect the bombs to make sure those are in blocked off zones for the motion detection

[@blubaylon]

But then what would stop the AI from continuing to try to slice the fruits even after they've already been sliced

[@whatevenisazero]

We got fruit ninja aim bot before GTA VI

[@tonic-music]

Now you need a genetic algorithm model

[@unclesamsonite-y3k]

bros ai: if its green or blue or yellow then its a fruit but if its BLACK THEN ITS A BOMB

[@akio.69]

Me with yellow and green bomb.😈

[@thunderd7904]

The fruits and bombs are seemingly predetermined and the same every time, so could you just use reinforcement learning until your AI could get to a score of like 100,000? Also could scanning the screen for non black or non brown pixels work?

[@GonFrecsses-kl8bx]

I like that he dosent respect the viewers when they say any bad comments and he doesn't ignore them

[@noah2718]

This video is the exact amount of time it takes me to eat a Totino's pepperoni party pizza.

[@ninjaride1312]

Making it not hit the bomb should had been easy enough

[@whatsaname2usern]

grandma's gonna be happy with this one

[@nathanjohnson2066]

Now feed the recognized images (x,y,w,h,type,confidence) to a neural network and see how well it can learn to play. Maybe it would even learn combos!

[@poohritkun5068]

Now this is a real AI. Finally!!

[@jbsthegamer]

my personal best in that mode is closely around 800 is what happens when you have over 2 years of fruit ninja gameplay

[@darkrathwheito9818]

My all time best was 11932 with the best ammount of pomegranate hits being 89, seeing an AI getting 600 with not perfect "technique" makes me cry imagining that in 1 year of development it could become so good that reaching 6 digits would be easy T_T

[@fridolineckerd6135]

Can you show what kind of ai you used?

[@Xander4096]

I did this a while back but simpler, just looking for specific ranges of colors per pixel cause the lighting is pretty static, then made a modified TSP algo to be able to chain slice.

[@R2Bl3nd]

You could've scaled down the screenshots to a much smaller size before feeding that to the AI.

[@dudical111]

I would love to see what happens when you train the ai on as low a resolution as you can go on the golden fruit level. I wanna see it obliterate them, dude.

[@redhood-n2v]

Man I am so dumb I thought the normal version was the run where he reached 488

[@hank4134]

I mean the vid is fine but without any bad/dumb/dark jokes of CodeBullet it just won't cut it

[@GemsZone]

Great job, but watching the game play there was a lack of combos, if you ever want to revisit fruit ninja that's a thing you could look at.

[@DaviMineBoy-7862]

3:24, FISH🐟

[@steff3135]

What about Teachable Machine? because u could learn it with the 3d model right?

[@CheezeWreak]

You could of also made it so if the bomb is overlapping wait for it to un overlap

[@JulianGaming007]

Awesome video

[@joshuachan6317]

What is the software that is used for labeling?

[@Chezmanrealwow]

Could you have used pyautoguis image recognition? Because of the confidence parameter I feel like you would have only had to take 15 different images of the fruit

[@henriks5008]

You should program an AI to label the fruits and bombs ;)

[@GD_Abomination]

You could have used a second screenshot to predict the movement of the fruits allowing for a single slice per fruit rather than spamming for each one.

[@Yesbutnoimnot]

Overall that would make the ai slower because it has to take 1 SS, process it, take a second, process it and then calculate the exact velocity of the fruit then compare how the "gravity" affects the loss of momentum.

[@CoolDude-mq8dh]

@@Yesbutnoimnotso many idiots in the newest replies lol you are right

[@Yesbutnoimnot]

@@CoolDude-mq8dh lol the only reason I bothered replying to them was because I code and it's painful watching things like this

[@Andhira.Faldhi]

I think it's gonna sick if you train the ai to play FNAF, what it can be if you put the AI on custom night on 20/20 😂

[@psychtank8681]

What if you make it wait to slice perfect combo and account for bonus points by perfect rounded hundred score or triple number score? Like make it learn to leave some fruits at the end to gain bonus points from 700 or 777 score.

[@thegamerkitten2190]

is this guy just code bullet but not swearing

[@christophermoore6110]

The dev of fruit ninja made a new game called feed the deep, I suggest watching Aliensrock play it

[@BabySisZ_VR]

you can just hit any moving object that isn't a bomb, I think that'd be easier. but I don't know what you'd do for objects that need multiple slices

[@mirzu42]

You could have just found a hex color that only exists in the fruits (not the splashes behind) and looked for that color to slice.

[@blubaylon]

The AI would just keep trying to slice the fruit even after it's sliced

[@mirzu42]

@@blubaylon it wouldnt really be an issue though? Sure it would overkill the fruits but it would also make less mistakes because it computes faster.

[@xx__miqu__xx3812]

its happening, rise of the machines and fall of the humankind. skynet starts here

[@blubaylon]

These image recognition models have been around for a while lol

[@Moroha-m]

Wouldn't labeling the colour also work? Like make the ai slash the fruit colours and avoid the black or red colour found on the bomb Idk much about programming so idk if this would work

[@boxcar36]

coconut is too similar to the background so would be able to identify it as well