I Can't Tell it's AI! - AI Animator BLOWS my mind!

Toon Crafter is the beginning of AI based animation moving forward. These demos are AMAZING!
▼ Link(s) From Today’s Video:
Demo Page: doubiiu.github.io/projects/To...
Tooncrafter Github: github.com/ToonCrafter/ToonCr...
Try ToonCrafter: huggingface.co/spaces/Doubiiu...
Proper prompter's tweet: / 1797255628202918208
► MattVidPro Discord: / discord
► Follow Me on Twitter: / mattvidpro
► Buy me a Coffee! buymeacoffee.com/mattvidpro
-------------------------------------------------
▼ Extra Links of Interest:
AI LINKS MASTER LIST: www.futurepedia.io/
General AI Playlist: • General MattVidPro AI ...
AI I use to edit videos: www.descript.com/?lmref=nA4fDg
Instagram: mattvidpro
Tiktok: tiktok.com/@mattvidpro
Second Channel: / @matt_pie
Let's work together!
- For brand & sponsorship inquiries: tally.so/r/3xdz4E
- For all other business inquiries: [e-mail]
Thanks for watching Matt Video Productions! I make all sorts of videos here on KZbin! Technology, Tutorials, and Reviews! Enjoy Your stay here, and subscribe!
All Suggestions, Thoughts And Comments Are Greatly Appreciated… Because I Actually Read Them.

[@robertolanzone]

The example with paint is what sold me completely. This could be a renaissance of indie amateurish web animation ala "flash animations" and Newgrounds, where it's not about quality of drawings and animation but just the content, the ideas, people doing something creative and a lot of entertainment many people can feel encouraged to try their hands at

[@ecsrepair]

^^^You got it! A Golden Age is coming. We've been locked out from actual art by the corps, now we get exactly what you are describing I think...I just couldn't put my finger on it but you did pretty well with the comparison.

[@CPB4444]

Ideas are the plates for the spaghetti.

[@perfectlyroundcircle]

I am not going to watch them.

[@artyshreds6274]

"I like rusty spoons" IYKYK

[@reezlaw]

I feel bad for Japanese animators that earn a living interpolating anime. They were already underpaid and exploited, soon they're gonna be out of a job entirely

[@MilkGlue-xg5vj]

Well this tool will help them not kill them lmao

[@masakami5437]

@@MilkGlue-xg5vj no it wont it will make most of them not get hired, and human creativity will continue to decline more and more

[@mine.moment]

On one hand you people complain about those animators being exploited with low wage, now you complain about them being out of jobs. You keep blaming AI, instead of blaming the real cause here, the greedy corporations ? :D So blind and ignorant.

[@arcamira7918]

@@mine.moment no one thinks the concept and technology of ai is bad but the entire gen ai system is based off of an illegal data scraping of artists work, and it was all done by greedy corporations, and people are feeding money to these literal criminals by paying for AI. AI should assist artists not replace them, and so far, the industry is just laying off animators and artists. Its the fact that companies will not use AI to make life easier for artists, but instead just pay them even less, or lay off most of them, while keeping one or two to continue to work at an exploitative wage just so they can squeeze more money out in a shorter time.

[@masakami5437]

@@mine.moment well of course i would complain about the low wage and i would complain about the AI cuz their both two different problems and both bad. And the greedy corporations are bad, and so is AI.

[@THEnicholasvancosky]

I hope this AI could advance enough to be able to assist in finishing unfinished scenes from the thief and the cobbler

[@HidrHjjhh]

Or berserk

[@PixelFromDreams]

That'd be pretty cool actually

[@aresaurelian]

When high fidelity enough. Also, Watership down, as intended.

[@AZTECMAN]

Gotta throw in some cameos from Aladdin.

[@noop-chair]

I tried it ,tweaked the settings, sadly it doesn't understand the complexity (masterpiece) of the images. But it works great with coloring

[@SirusStarTV]

Miyazaki would die from heart attack from hearing about this tool

[@witext]

This is amazing for the indie animation industry Imagine how many fewer frames they’ll have to draw

[@ghost-user559]

Imagine how many artists they will no longer need

[@neomicryo]

@@ghost-user559 technology replaces jobs, this has been a thing for centuries

[@ghost-user559]

@@neomicryo Sure, that’s very true. But what do you think happens when we have passed the point where humans are needed any longer for anything? Do you think the people in charge, who say the world is over populated will need this many?

[@sunnymeee]

​@@ghost-user559dude why are thinking all this now? When it happen then you have to show concerns it's gonna take centuries to make all things automated, specially passionless job slavery

[@nigermant6347]

@@ghost-user559 you got it.

[@Brain4Brain]

This means that with a good enough workflow, an entire 20-minutes anime could be made in 11 hours and be fully automated, using GPT-4, Controlnet, and this tool

[@bigglyguy8429]

No, GPT is too nerfed for anime

[@AbyssalSoda]

As an animator myself, no this wouldn't actually speed up the workflow too much, research animation principles such as follow through, lines of action, squash and stretch. An AI can anticipate certain actions which is an animation principle in itself, but it's very rudimentary. So what this would be good for is speeding up the in-between process which will help key animators. Even SORA AI can't come close to actual animation at the moment despite creating fluid movement.

[@MarcoCholo-iz9js]

​@@AbyssalSodagive it a few more months

[@ScooterCat64]

@@MarcoCholo-iz9js I've been hearing "give it a few months" for over a year now. It's still going to be a while before it can do half decent animation

[@litjellyfish]

Not a chance. Unless you want to give away creative control in “make a cool 20 min animation “ People don’t get how much time gets into planning scripting etc. For more like family guy and such yes maybe it could be cut down a lot tho

[@ChandravijayAgrawal]

Just putting it here, doraemon had a gadget that turned comic book to anime episodes, so this is yet another 22nd century gadget coming to life, can't wait to meet real life doraemon

[@DisturbedNeo]

Couple more papers down the line and we'll be generating our own anime. The next few years are gonna be wild.

[@Chris_Hruska]

at first it'll shave 90% of the time off of creating an episode. then in a few years it'll just spit out an infinite amount of episodes of anything you want. hard to imagine. but its happening.

[@imanigordon6803]

@@Chris_Hruskabetter copyright if you have a good idea 😂

[@Madman-bi5bf]

@@Chris_Hruska What would you make?

[@larion2336]

@@imanigordon6803 You can't copyright an idea, only the execution of that idea. But I think copyright will eventually be kicked into the dustbin of history by AI anyway, and good riddance. Its primary outcome these days is maintaining corporate strangleholds over arrays of IPs they bought out, none of which have any relation to any of the people involved, which is a total perversion of its original purpose (to protect creators).

[@mycelia_ow]

The hardest part will still be writing good content, AI can help with that too but can't do it alone.

[@jujume1327]

The MAPPA employees will finally be able to go home and sleep!

[@martiddy]

Nah, they just fire the other employees and keep two guys to do all the extra work

[@asuka_the_void_witch]

they will out of work and no longer afford a home.

[@rocinantesh8391]

as if

[@alexmehler6765]

they will just make the animation even more incredible with all bells and whistles at all time - when this can be done in 1080p that is

[@ze1st_]

they will just make more anime but still work the same time

[@TheSyborgue]

The colorising animation is the most useful part of this at this point. I'd like to see it on random not anime start to finish tho

[@vi6ddarkking]

I have been predicting an Indie Animation Boom eversince SD 1.5 began to be Fine Tuned to hell and back. Now with SD3 Medium releasing to the public on the 12th and this. Let's just say the mad lads working with Confy UI are going to have one fun summer ahead of them. Especially once ControlNet gets updated for the new architecture.

[@davehugstrees]

This is legit amazing. Video title is definitely accurate for this one. Thanks for showing us these demos!

[@Caspian.]

All that money and time I invested into learning animation seems kinda worthless now. Its not at the point of it being a problem yet, but it looks like its getting close :(

[@saudiphilippines]

Don't worry. You will still be hired. Also, it will actually help ease your work with the sketch feature so that you don't need to color or shade all frames by hand anymore.

[@Caspian.]

@@saudiphilippines Hopefully that's the case 🤞

[@saudiphilippines]

@@Caspian. It depends on the circumstances and greed.

[@Dookiestain-jw8wy]

AI is supposed to be an assistant and not a replacement. It's the greedy corporates that are laying artists down girl 😭

[@saudiphilippines]

@@Dookiestain-jw8wy Exactly!

[@jbavar32]

This is incredible. I’d love to try this. The “Caustic” effects with the water is amazing

[@anthony_leckie]

This is pretty impressive. I hope AI video evolves quickly. I'm still frustrated that I can't even prompt SVD yet.

[@chickenfordays4543]

Cant wait till No Game No Life s2 is animated by AI

[@NoidoDev]

Haha, this!

[@AlphaProto]

I've tried this, and it works ok. I just don't know how to get the sketch in. What do you upload as the sketch? A video, or a animated gif?

[@FRareDom]

finally some advancements in ai video, moving at a similar pace to image generation at the start

[@seraphin01]

it's not there yet, but inbetweening might get processed by AI at some point.. it's already a terribly paid job anyway with terrible conditions.. Kinda feels like 2D animation will somehow be like 3D one if this keeps up, just drawing key poses and let AI do the rest.. interesting technology honestly, but don't expect to make a whole anime or something without skill or talent, this tool won't change that

[@ChandravijayAgrawal]

17:45 that is pippo from doraemon, i miss her to this day, she sactificed her race to save earth 😭😭😭

[@chineseducksauce9085]

Can't wait till I am able to remake one punch man season 2 in season 1 quality

[@thomblueart8448]

what was wrong with season 2? the animation wasn't as good but still pretty high quality

[@chineseducksauce9085]

@@thomblueart8448 Certain scenes were alright, but the majority was subpar. Lack of fluidity, incorrect proportions, still frames.

[@thomblueart8448]

@@chineseducksauce9085 I might be a bit biased just for Garou being in it I guess!

[@mikeparradise9333]

@@thomblueart8448 happens

[@OPM07]

​@@thomblueart8448Season 2 trash

[@scrutch666]

Lol what most people dont realize it how good your input has to be , means you better be an artist of some form. You can see even on the glowing piece flick how inbetween frames would be necessary. For all true artists and new artists this tool is dope. For the rest no its not sora

[@mooonatyeah5308]

just use ai to generate a good image

[@jaredf6205]

You just have to combine this with existing AIs

[@scrutch666]

@@jaredf6205 ok create me 2 coherent pics that would result in a clean animation like a character where the arm bends around an elbow in 90 degree , just wit ai gen tools

[@saudiphilippines]

I agree.

[@okolenmi7511]

The quality of animation is better than in some modern anime, so it should be useful.

[@brainwithani5693]

Looking good Matt!

[@bluesailormercury]

Finally someone can fix "when they fire all the animators" Twinkle Nora

[@contentfreeGPT5-py6uv]

Finally, but is basic

[@DomIstKrieg]

I imagine if you fed two frames of Twinkle Nora to this thing the entire server would burst into flames.

[@Wonders_of_Reality]

Well, if this AI can animate Sailor Mercury, then I’ll be worried for my studio. Until then, it’s just another semi-useful tool.

[@HumanGamer]

Someone needs to do that xD

[@harnageaa]

@@Wonders_of_Reality it seems it's only good at interpolation. That is just a part of animation... not the whole. Animators not getting fired yet 😂 . But on the bright side, just take 10 friends animators + ai and make your own company. It depends how you look at it... On one side you get fired and depressed, on the other side you become business owner and rich.

[@Ah__ah__ah__ah.]

I really dont want these creative ai stuff what I want is a medical renaissance thats what the world needs cures for cancer tinnitus foreskin regeneration all that stuff thats what I want so badly and pray to all the universe we see that soon

[@ArtificialExperience]

Thanks Matt,I was wondering what I was going to do this weekend.

[@h20dancing18]

Hey Matt, is there a good way to convert footage to sketches to use this tech effectively by starting with a green screen video filmed on anything? would love to see you try to connect these technologies end to end!

[@TheButchBerryCompany]

As an animation film producer, I am thrilled by this. Because of the hefty price of good animation, I keep waiting for do it yourself animation like this to get better. Its moving fast and getting closer to being able to make it cheaper to do a full animated film.

[@MrTk3435]

Nice one Matt! Thank you ✨🙏✨

[@kuromiLayfe]

this current version is absolutely amazing for animation clean up and pacing.. seen many high budget animations where they could have done certain scenes way better if they interpolated the frames better instead if just cutting away.

[@horrorislander]

If you have your old flipbooks, you might try putting sequences from them into it. Anyway, pretty interesting. Many years ago, maybe 1985 or 86, one of the guys on the Amiga Compuserve forum was working hard to build a tweener. Of course, it wasn't neural net based, just morphing by connecting points in one image to points in another and linearly pulling them together over time, but it stayed with me how very much that guy really wanted something like this. Don't know what happened to him, but he would certainly be blown away.

[@cyberworld9000]

Now I really want it to be the program creator. Haha

[@NirvanaFan5000]

really cool. could see this theoretically making animators' lives easier. (e.g. only animating 1/3rd as many frames.) but not sure it can work with their workflow (e.g. what if they need fine edits). But we're getting real close to useful stuff.

[@sdhpCH]

10:19 it is caustics, not causatics, Matt :) Keep up the great job, mate.

[@FriendlyVimanam]

All of my favourite miscellaneous models have always been covered 1st by you. This is why this is the best channel about ai.

[@Pepius_Julius_Magnus_Maximu...]

This along with the news that SD3 is to be released 12th of this month made my day! I hope this works well on 3D art too, I'd love trying it on mine, it's a lot of work animating all the pretty details like hair, clothing and vegetation.

[@GamersBlogX]

definitely cool stuff, can't wait for it to get better.

[@hardstylelife5749]

Most interesting. May I ask if this (or another tool) may be capable of creating and maintaining the very same set of character’s skills/proportions/look and use them in different environments? Meaning: can I use the characters I’ve created (both with drawing and digital art) for creating an entire illustrated/animated story? Thanks for sharing this video

[@ThoughtSignal]

I've been waiting for this, finally no more drawing for long durations of time

[@jakefelstet7116]

i have the feeling the training data they used is the exact animation that they used as examples. still impressive that it’s so much better than the others but i want to see how it does with frames it hasn’t been trained on.

[@Samuel-wl4fw]

Tested it on some kaiyu 8 frames, it was pretty good

[@jakefelstet7116]

@@Samuel-wl4fwoh that’s sick. i didn’t finish the whole video to see it was open source and he even did his own example which turned out pretty decent 😂

[@Zetarla]

I waited my entire life for this kind of technology and it's finally here!

[@moneyjuice]

What's amazing is the coherence through all the frames

[@norioku2020]

Okay, I'm sold totally installing this later tonight!

[@imeleventeen]

Holy shit. We are about to step into an age of cinema

[@paulsheldon8838]

Age of slop which nobody cared enough about to even make?

[@imeleventeen]

@@paulsheldon8838 nah, so many passion projects are ruined or made not as hood because the person can’t animate and has to come to a middle ground with the people who make it. Part of the reason people say mangas and books are better. We also will have way more of them made so by default there will be more good ones. And when the visuals are all really good for all the animes/shows people will have to really focus on the story and characters to make their show worthwhile

[@jackharrison4704]

now this is what I like to see

[@Mr_Wiley]

Incredible! but- The amount of work it would take to fix that many frames you might as well just do it yourself to begin with. Is it amazing? YES. but we're not 100% there yet. For animatic type workflow and prototyping scenes this is off the charts useful though; Testing emotional feedback to certain timings etc. without actually having to invest countless hours- crazy

[@lastchildz8567]

Why do all the example images come from an already existing anime?

[@Dave102693]

Ikr?

[@ackerwoman16]

because it's trained on existing data....no originality

[@ScooterCat64]

Because the people who want to use this tool don't draw

[@Ryuuko3]

Damn, i tried the demo and it worked great !

[@christianboi7690]

I love that small companies and individuals will be gaining tools to flesh out their projects and make awesome things. I don’t think more traditional animation will die, but I will lament its fading to a sub-sect of people and companies that appreciate legacy animation. If I want the little guy to have tools to flesh out awesome things, I have to accept the larger companies using them too when they could make something wholly unique and impressive without it.

[@jessicas-discoveries-age-6-12]

Is this really that much better than animatediff in stable diffusion and morphing between a number of frames using ComfyUI?

[@Terny228]

You would think that animation would be the most successful as it can absorb the uncanny valley of A.I. video generations far easier and the bonus of being able to digitally correct the animations generated. Getting closer.

[@fynnjackson2298]

When done with real video (in a year or so) then one can take video scene beginning and end and just regenerate badly recorded moments. Or do every even and have a new movie. Incredible stuff being released.

[@virgilxavier1]

I'd like to see somebody this with consistent characters and controlnet to try and do the whole animation process

[@Sharkcapp]

So much excitement for a potential threat to people's dream jobs. It's unreal.

[@chanpasadopolska]

Colorisarion looks really great

[@BestCartoonKahaniTV]

The coloring frames option is not out there..

[@TeddyLeppard]

The guys over at Corridor Crew are going to plotz when they see this.

[@MisterB123]

Hah! So I'm not the only one who thought this, huh? 😄

[@mycelia_ow]

Idk, last time I saw them talking about AI it was very, very negative.

[@cesar4729]

​@@mycelia_owIn what sense?

[@chariots8x230]

I like that it’s open source.

[@martinakula8421]

I don't think this AI will be able to replace animators.Interpolation is already done in the 3d software.

[@tomzer1]

Definitely a tool that will help save a lot of time and money for in-betweening work in anime studios. It could definitely help cut cost, and perhaps unfortunately, a few in between animators might lose their jobs.

[@andersonomori]

I couldn't use the rough drawings to guide the animation, did anyone figure out how to do this?

[@RecursiveTriforce]

4:12 The person actually always keeps their left foot in front. That's an issue that also happens in Sora.

[@wamteng3790]

Also, how will this work if you are using to speed things up on the workload but the end product cannot be licenced and distributed internationally due to indivdiual copyright laws - AI generated content cannot be copyright for example?

[@NoidoDev]

I think this is only the case if it's only generated using AI.

[@Retly_Ai]

I actually see this as a good thing for anime companies as it could help artists be more efficient once this technology becomes better and better.

[@Silks-]

Being more efficient means they need less animators…

[@andra9694]

@@Silks-no, same number of animators but ton lot of project handled by each of them 💀

[@Silks-]

@@andra9694 wat

[@xenn2996]

@@Silks-they can use this as a tool

[@imanigordon6803]

@@Silks-It means that the businesses will either replace them or expect more of them and overwork them twice as much 😂

[@adampenbrook5751]

I’ll be able to make my own Tokyo Ghoul remake if we don’t get one from a big studio. 🥳

[@what-uc]

It's exactly what I would expect - mind not blown

[@sirhammon]

It's something I seriously want to get into but it needs a huge system though. maybe 64gb of ram minimum and maybe 24gb video card minimum. It's seriously system resource-heavy.

[@MovieFanAi]

nice animation tutorial 😊

[@leekotvfilms5236]

😂bruh every time I come to ur page…I leave feeling like I got new super powers 😂

[@hakarthemage]

I knew this was coming but damn. P.S. since midjourney has character stuff now. you could concievably use MJ to generate frames, and this to generate animation. If need be you could maybe repose with SD with openpose.

[@povang]

Finally One punch man seasons can come out once a year instead of once every 5 years...

[@NoahtheGameplayer]

Lol

[@OPM07]

​@@NoahtheGameplayerSeason 2 is a non canon event

[@TeddyLeppard]

Amazing.

[@ChandravijayAgrawal]

just a few years and then I will animate one punch crossover with every piece of fiction ever existed and defeat every villain with one punch

[@chariots8x230]

16:04 I agree with you that this tool would be so much better if we could input more images. Just using 2 images is not enough, because the animated clip turns out too short.

[@ceejay1353]

I can't wait to see something like this for Unreal Engine.

[@consciouscode8150]

I see a lot of potential with this not for finished products, but rather auto-generated in-betweens and rough drafts to speed up the animation process. Weirdness doesn't matter then because you can just erase it or draw over it like they were guidelines.

[@ThoughtSignal]

Music for animation: Udio beta Sound effects: Audiogen V2 Video for animation: Tooncrafter 😅😊

[@Som3D]

2D animators getting replaced for the second time in history is crazy

[@nigermant6347]

Almost nobody is studying the consequences of all this in the med/long term.

[@NoidoDev]

​@@nigermant6347 Consequences for whom? People are not one group.

[@alexmehler6765]

i find the lineart interpolation and autocoloring the most impressive and usefull

[@ProzacgodAI]

I want to use this to go and make my manga have "still animations" like in a harry potter photo... I think that would be cool. While also colorizing the pages.

[@sxndypzofficial]

i never expected to see zunda horizon here (17:47)

[@LjaDj5XQKey9mSDxh4]

Completely crazy!!

[@IceMetalPunk]

Having more input keyframes should be pretty straightforward: just run it multiple times, pairwise. So run it on frames 0 and 1, then on frames 1 and 2, then 2 and 3, etc.

[@RandomGuyIsBack99]

The future of animation is uncertain.... but cool!

[@Scott-Zakarin]

I'm going to try it with midjourney photo realistic images and see what I get.

[@Anti-HyperLink]

Headphones make volume misleading because you're loud when my volume is at 80, and the Spyro videos of mine I'm looking through to see which dragons I saved is quiet. I don't have headphones on at the moment, but when I edit, I keep the volume at around 85 because I read somewhere that that's ideal for sound editing. I hope my videos aren't all super quiet. Regarding the video: hopefully animators and visual artists that create still image art choose to use this stuff to boost their workflow and gain inspiration rather than the typical whining and lies we've gotten from artists.

[@yannsalmon2988]

It’s about the fifth video I see showing tooncrafter and what bugs me is that so far only the samples provided on the project website are actually impressive. I tried it, and I’m also underwhelmed by the results. They’re not bad and much better than any other kind of in betweening frame interpolation I’ve seen, but still they have the same flaw that every AI tool I know have : giving you ALMOST what you want, but not quite, which makes you have to do a lot of post processing for being really usable. Sometimes it’s not that much and all in all it’s faster, but sometimes the time you gain is not significant. And the problem is that there’s no way to tell in advance if it’s going to do a good job or totally mess up, it’s all trial and error, which is not a really sustainable workflow.

[@Yipper64]

The main issue im noticing with this is the timing of things, it seems very robotic. Dont get me wrong, extremely impressive stuff, but once again, the computer isnt accounting for the 12 principles of animation.

[@jesse8068]

It would be great if it allowed more than one keyframe, but it's still cool!

[@MrDonCoyote]

I can see this being used for especially simple animations like Family Guy, American Dad and South Park. Dude! What a time saver. Because most of the time is spend on animation movement. This is so cool.

[@ilanlee3025]

Good one

[@Deljron777]

So exciting

[@DihelsonMendonca]

💥 The argument: "It's very weird and bad, but it's AI" 😅😅😅

[@FacundoComix]

3d practically killed hand drawn animation and most artists are against AI but I think this may be the tool to bring back "traditional" animation back into the spotlight, so disney can go make shitty hand drawn movies instead of another generic pixar flic and 2d animators will be once again employed to do what they spent all those years learning to do.

[@NoCodeFilmmaker]

Bro, this is awesome. You can simply create key frames and have this do the heavy lifting in-between. As an artist I'm not butthurt at all. I'm loving how these AI tools speed up pipelines and allow smaller companies to compete soon with the big studios.

[@taco1010]

Fucking finally😂 the world is mine. I cant waot until this is legit

[@Austinn72]

instant manga to anime mayhaps

[@Kopie0830]

Mappa animators: were saved! 😭😭😭

[@VanGourmet]

Don't worry. I am going to find a solution to not let old guards in the industry face severe suffering.

[@jawbee7]

Pls try Abstract motion graphics.