Real-Time VFX Demo | Rotoscoping 3 Shots in 10 Minutes Using Machine Learning

[@ActionVFXofficial]

Make sure to subscribe and leave a like on the video! 🔥⁣⁣

[@NOTMYSCENE]

So thanks to youuuuuu

[@alexandermehler5912]

i guess you guys get the clickbait award this month ...

[@EmbodiedNonDuality]

It's impressive in one sense but only for rough roto. You would still have to go back and roto it fully. the edges are super crunchy. It doesn;t cater for moblur and the mattes have that gloopy feel when elements cross over one another

[@m5d3e]

I agree Honestly I can't believe it. In the tutorial, all the programs are fantastic. This is thought to be because it tests the video that has already been experimented several times. Do you feel that this thought is too negative? However, in an environment where new and good features continue to emerge, I see it as a reasonable doubt. If this program becomes more popular and is heard by my colleagues, I think I'll experiment. But I can't easily believe it yet.

[@ActionVFXofficial]

In the end, it can simply be a time-saving tool to do quick/easy rotos!

[@animatz]

Could be nice for a rough core, and then you supplement with some softer keys for the edges. Could save a lot of time!

[@someonewithsomename]

yeah, I hope we could use such tools in 3-5 years. That's a great demo of future possibilities. But not usable as of now.

[@marcelovera8004]

Agree, now they added a feather for the mask, and you know, I exported with a green screen on background, and added in AE a chromakey and the managed how to face the edges, worked perfectly

[@Soulindex]

This seems extremely useful for previs. Even if it's not 100% accurate, I can totally see myself using this for rough edits to at least convey what the shots will look like

[@ActionVFXofficial]

Exactly! Good thought.

[@subhajitdas7595]

Oh my lord, is this any kind of a magic with rotoscoping?😱 Fabulous,no words to explain. It will be part of rotoscoping future. 🔥

[@ActionVFXofficial]

We're excited to see how it improves moving forward!

[@armondtanz]

not sure why adobe havent jumped on this?

[@BlackStarInitiative]

Wow ok. I need to try this out.

[@KingE6075]

This saved me sooo much time and Ram because my 16GBs couldnt handle After Effects Rotobrush. Thank you so much 😂

[@markdennison8326]

Looks promising. Not there yet for serious comps. BUT, tremendous potential for color-grading, and motion graphics.

[@ActionVFXofficial]

Speedy, effective workflows is the name of the game!

[@theActionMovieKid]

Ummmmmmmm.... yes please.

[@HarriRobins]

This paired with CamTrackAR will definitely make your life much easier.

[@FlyMusicMan]

Nicely done! Very cool to meet this tool. THANKS MAN.

[@ActionVFXofficial]

You're welcome!

[@LunarlightCinema]

This is definitely a game changer!

[@ActionVFXofficial]

I think so! :) -Luke

[@PhilipWesson]

Great demo! Thanks for sharing this info! I'm going to give this a shot this weekend!

[@ActionVFXofficial]

Thanks so much, Philip! Let us know how you like the tool once you're able to get hands-on with it. -Luke

[@sudhababu9742]

Many employees will lose their jobs due to this app. Stop ai

[@TimvanHelsdingen]

Wow damn that looks insanely useful. Real cheap as well

[@ActionVFXofficial]

Affordable and well worth it in regards to saving time!

[@markhudson5825]

I'm speechless. Thanks guys.

[@ActionVFXofficial]

Glad we could help!

[@sudhababu9742]

Many employees will lose their jobs due to this app. Stop ai

[@ilusionstudioz3617]

Nice, thanks to Action vfx

[@lazadafanboyz7970]

This is amazing

[@techmusic7009]

Amazing!I will subscribe now

[@ActionVFXofficial]

Awesome!

[@___x__x_r___xa__x_____f______]

Is there a way to refine hair further when you have loose hairs and gaps? Thanks

[@CompositingAcademy]

Awesome stuff! Hopefully we can get something like this in Nuke soon enough.

[@cobblesticks]

It's coming....

[@sikamoreify]

LOL. Don't hold your breath. The Foundry does nothing but band-aid something that needs to be rebuilt from the ground up to embrace modern technology and hardware.

[@syy1777]

It`s good. I'd like check a composite from another background.(Not Green)

[@yokanshree4621]

nice one

[@anaabines4105]

what are you gonna do next? after rotoscoping them?

[@bsrobinson]

I'd like to test this out for footage shot on a green screen originally. Alot of the keying I do still requires roto. This could REALLY help a lot!!

[@ActionVFXofficial]

This could be a very helpful tool for you!

[@fatjay9402]

holyshit .. i was thinking this is rotoscop 2... but that thing looks so much better .. wow

[@GarethOwenFilmGOwen]

Its cheaper as well lol

[@rogersinger1843]

I'd like to see the further end product.. any links?

[@ActionVFXofficial]

Hit the link in the description!

[@meikmeiker2615]

Are there some Tipps to better prepare at shooting? Maybe clothes to separate from Background better

[@michaelgardner2175]

Absolutely. Bananas. Love. It.

[@ActionVFXofficial]

Bananas, indeed! Glad you liked it!

[@dandon1968]

hey, i know this is an old video. but do we know if they used footages user uploaded to train their ai?

[@OllisFilmkueche]

Awesome! I'm speechless!

[@ActionVFXofficial]

Right? Cool stuff!

[@etiennetheriault7418]

Hi! Is there a way to know how our clips are kept secured after the upload? Where do they end up? I'd be considering using the tool in ''confidential before release'' projects, for exemple.

[@ActionVFXofficial]

That's a great question! I'm sure as this becomes more mainstream there will be tighter security precautions. But I do know that some of the ML models are available to run locally on your machine. Meaning, your assets wouldn't be stored on a 3rd party server, which could help on the security side of things. -Luke

[@ScribblebytesWorldwide]

It ends up in the CIA. 😉

[@mattynu1]

how is this? is this an app? where can find that rotoscoping apps? thanks

[@richalexander1138]

Wish it had more motion blur control and spline options for roto

[@TheMobCultVideo]

only HD export?

[@ActionVFXofficial]

There's SD as well. You can find out about different exports here: runwayml.com/pricing/.

[@TheArtofme]

Loved this

[@MurrayFrost]

SICK!

[@imrenagy8122]

Very cool.

[@CanalBigMaqui]

Awsome!! I'd like to see how far can you push this thing!. All the examples are high key shots, but what about low key shots?

[@ActionVFXofficial]

In my other testing, low key shots have also done the trick! :) -Luke

[@youngTAGZ]

how do i match it in after effects

[@ActionVFXofficial]

RotoBrush would be the closest thing. Or you could use the technique in the video and export a matte to then use in AE.

[@NOTMYSCENE]

So thanks

[@uknownsubject]

Cool but not professional , I have worked as a lead for a couple of companies here in Canada and we tried to implement this technology for a while but the final quality is far away from industry standards and it gives you many problems like motion blur, overlaping, glitches.. I wish it could work fine!. The owner of the company paid me a license to test it and came from LA to se me do a demo and the biggest feedback that I have after the demo in general was that theres not much difference between use this and a keylight keyer that comes by default with nuke. Developers have been trying to fix this problems for a while but no success so far so we will probably need to wait a few years to see a professional version of that and then to be implemented in the companies pipelines.

[@ActionVFXofficial]

@Nacho Castrillo Thanks for sharing! I think one good thing to note that this type of technology will only improve and become more intelligent over time. So, even if the current applications don't work inside of your environment, as you said, within a few years, it could be the primary thing that is doing the bulk of the heavy lifting to save time. Which would be awesome!

[@someonewithsomename]

@@ActionVFXofficial rip jobs in India

[@aaquibdanyal5139]

Really cool

[@jmcreativos]

am waiting 🙌🏾

[@Bathoggywinger]

Looks solid for creating fast core mattes, edge matte maybe not so much? Thank you regardless, I'm gonna take a look at that software.

[@ActionVFXofficial]

Have fun!

[@SeanFoyMakesMovies]

Really amazing

[@TokenCast]

how much IS runway ML?

[@ActionVFXofficial]

There's a free version! You can find the complete pricing info right here: runwayml.com/pricing/

[@anixcool]

Thank You 😀

[@ActionVFXofficial]

Happy to help!

[@EzDyt]

I prefer runway over rotobrush (yes, even 2.0!) because since it's a web app, it goes much faster because it doesn't matter what PC specs you have, it just depends on if you have internet access or not. sadly, it does need to be paid for if you want to render in up to 4k

[@ActionVFXofficial]

The time saved in post are well worth the paid options in our experience! :)

[@CineFlavor]

Is there a way to feather/soften the edges of the roto in RunwayML? It seems fast but AE has a lot more micro adjustments it lets you do to the roto

[@xanderross366]

I imagine that you could probably use a blur effect in AE and then use it as a luma matte

[@ActionVFXofficial]

Something to also keep in mind is this is model is technically still in a beta stage. So it will only grow and progress with features from here. But if I were to use this for a real-world shot today, I would get the matte as clean as I could inside of RunwayML, then I would most likely just put a RSMB on it inside of AE to clean up the edges enough for the type of work I would be doing. If I had to guess, I would imagine all of those other settings, like motion blur, would be added directly inside of the app before too terribly long. -Luke

[@Nick-yo1te]

Would you say this is better than after effects rotobrush 2?

[@ActionVFXofficial]

It all depends on personal preference. Certain people prefer staying inside AE since this is a separate AI tool. Use what is best for your workflow and the results you're seeking. :)

[@PierreDaGriot]

tnx

[@LunaKissBand]

This might seem like a dumb question: Once you've aquired the alpha is it really easy to turn that into a track matte in After Effects?

[@ActionVFXofficial]

This tutorial will be beneficial! kzbin.info/www/bejne/oXKcg4yXa655bZo

[@ActionVFXofficial]

The link above is a timestamp of the AE section. :)

[@midphase]

What I kinda wanted to see was how the mattes would composite in the final shot. I mean it looks ok on the screen, but ultimately it's impossible to tell just how good the quality of the mattes is without actually seeing it in the context of the compositor.

[@ActionVFXofficial]

I'm considering doing more of a deep-dive video that actually would showcase using this tool for a workflow. I.E. more in-depth of checking the generated mattes, following it through to an actual comp, etc. Is that a video you'd like to see us make? -Luke

[@midphase]

@@ActionVFXofficial Yeah. Mostly I am interested in just how detailed the edges are, particularly when it comes to hair and small details. I am planning on testing it out with some footage and seeing it for myself, but I'd love to get your take on it. I mean it almost seems a bit too good to be true, right?

[@bigpapi38]

Damn Luke. What a nice setup! What lens are you using?

[@ActionVFXofficial]

Thanks, Brian! I appreciate that. I shot the on-camera section using my GH5 + 12mm Rokinon, so nothing too terribly fancy. :)

[@naveenprabhu_Vlogs]

Waiting, i have used after effects for rotoscopy,it automatically creates alpha, is it something like that!!!!!

[@2minutsvideo]

What is the software?

[@ActionVFXofficial]

It's called RunwayML! Link in the description.

[@moorthyvenkatesh4519]

hi what sofatware? software name plz

[@ActionVFXofficial]

It's called RunwayML! Link in the description.

[@shekiba.646]

I dont understand this? background green difference picture//video ?

[@ActionVFXofficial]

Essentially this AI tool will speed up the process of rotoscoping, and let you create mattes in a matter of minutes! That process could take hours or days, but now is just several minutes!

[@shekiba.646]

@@ActionVFXofficial Oh I see. understand. This interest. Thanks.

[@SebastianNiight]

when i'm trying to include or exclude points, the points are never placed where i click. they are way off beneath. any reason why or how can i fix it?

[@ActionVFXofficial]

That is interesting! I haven't encountered that issue myself. If it's happening on a specific clip, maybe you could try to re-upload that one clip and try again? If it's happening on all clips you try, then perhaps you could contact their support team to see if there is any documentation on what might be causing that? I hope this info is helpful! -Luke

[@SebastianNiight]

@@ActionVFXofficial i only tried on a clip, i rage quited after effects yesterday cuz of rotoburshing out of memory :)) this dropped at the perfect time, and i only needed something to make a mask out of. Next time i need it and this occurs again, i will contact them. Thank you, keep up the good work

[@xevious2501]

This all looks ok from the standpoint of what you see on your monitors. and Yes recent times has brought forth some amazing automated AI learning auto rotoscoping. But the fact remains, most of the results are barely ok. Its not simply about cutting out a subject, but rather extracting the finer details. such as dealing with wisps of hair. this is were the majority of these automated tools fall far short of. The sort of stuff that becomes quite visible when dealing in films, and seeking approval from QC.

[@ActionVFXofficial]

I agree! I think it's also helpful to note that these machines will only continue to grow their intelligence and become better tools for editors as time continues to go on. I think if these tools are able to keep developing, ML will become common workflow practice within the next 3-5 years. -Luke

[@mariobalooci]

One quick question: since is web based, if I need to roto a 4gb clip, I have to upload it right?

[@GarethOwenFilmGOwen]

4gig whhaaat! lol

[@mariobalooci]

@@GarethOwenFilmGOwen I work in the movie industry and is not uncommon to have such files. Last movie I worked on was shot in 8K.

[@GarethOwenFilmGOwen]

@@mariobalooci cool what company? :) Are you working with Raw clips? Or pro res 444?

[@mariobalooci]

@@GarethOwenFilmGOwen I am a freelance, so I work with many production companies. Last one was a tv show for the italian television. I usually receive 10bit dpx sequences or Prores4444. No raw stuff.

[@GarethOwenFilmGOwen]

@@mariobalooci Ah cool cool. Fair dos. Do they ask for impossible work or what is possible for timeframes

[@petersolomon5227]

Such uniform viewer enthusiasm. I generally stay away from promising motion graphic novelties, and wait until they have been refined to the point of being robust, and made available as plugins within familiar applications such as After Effects. My concern of this online subscription service is that currently users have little control over finessing mattes with this app; and I'm not comfortable plonking unreleased media on third party servers.

[@ActionVFXofficial]

@Peter Solomon thanks for sharing your thoughts with us! One thing to keep in mind is that this specific model is still in beta. So I'm sure once they add some additional features for a full version, it will be quite powerful. And also: if you're still not comfortable using the platform from a security standpoint, then don't! This is simply an overview of a new tool that could potentially save you time, depending on the type of work you do. So using it now or waiting is entirely up to you! :) -Luke

[@petersolomon5227]

ActionVFX / Luke, I appreciate what you say above. Indeed, AI technology would seem a natural fit for rotoscoping, & it interests me. Moreover, I've purchased many Action VFX products over the years (and would still like to see blown autumn leaves on alpha channel made available). The thing is, once Runway ML begin to expand the capacity of their product other vendors will of course follow suit. As an example, aescripts + aeplugins sell a product called "Goodbye Greenscreen" by Blace Plugins. It appears to do much the same as the oddly named Runway ML rotoscope tool. Sadly, Goodbye Greenscreen is short on examples, & only available for Windows. As for Runway ML's online roto of video, this is a clumsy business. Topaz Labs trialled this for about a year with their celebrated "AI Video Enhance", & then under customer insistence released the product as standalone app for both platforms. In any event, I can see a huge market in this form of rotoscoping, as the older Digital Film Tools' variant of the tri-colour still photographic mask only seemed to work on a second or two of video. That said, any video semi-to-automated roto product would be especially welcome if it sensed moving objects, whilst not being dependent on colour. Until then, I guess many film artists will persevere with Adobe's Roto Brush?

[@artvisual-1244]

Why am I unable to access your site? It seems to me that there is a glitch.

[@ActionVFXofficial]

Is this issue still happening?

[@TheFreakymac]

This is too good yet scary for roto artist..

[@ActionVFXofficial]

I think there will most always be a need for roto artists, even if their tasks might look a little different! :) -Luke

[@TheFreakymac]

@@ActionVFXofficial Hey Luke, I was shocked to see how the software had done roto of yours... I think so manually it would had taken nearly more than 20 man days.. I think so for low budget projects this one is too good, but for heavy projects need manuals one to tweak.. And thnx for sharing..👍👍

[@xdreme]

Runway ml?

[@ActionVFXofficial]

Yes!

[@ATLAProduction]

What happens to the files after you upload them and finish rotoscope? Are they deleted from the server, or do you lose ownership of your own clips?

[@ActionVFXofficial]

Hey Tom! That's a great question. As far as I know, there aren't any hard storage limits as I simply download whatever mattes I would be needing and can then delete them from the web app. I'd imagine that more specific info like that would make its way into their FAQ once this tool exits the beta. -Luke

[@akhilshekar8427]

Waiting

[@HugosDesk]

What is this sorcery!

[@GarethOwenFilmGOwen]

Its like subject section in Photoshop but x100

[@chipsunlimited]

incredible lo rápido !

[@SplitLensProductions]

This is cool but I feel like Rotobrush 2.0 in AE is better since I don't have to jump to a different program to do it.

[@ActionVFXofficial]

It's all based on what's best for you! If that's what enhances your workflow, do it. :)

[@SplitLensProductions]

@@ActionVFXofficial for sure! I imagine this technology will also be refined with time as machine learning tools get more sophisticated. Maybe this company will invest more time into it and pull ahead of adobe before long.

[@valcriston]

Whoaaa

[@rainman2543]

Could be useful to Guerrilla Filmmakers with limited budget

[@ActionVFXofficial]

Exactly!

[@RohitSingh-xp1ie]

I want to work with you.can we do?

[@ActionVFXofficial]

We don't do any custom work, only stock footage products. Thanks for your interest though!

[@michaelgardner2175]

I just heard software companies all go silent, like the Death Star just destroyed a planet. 😳

[@ukmonk]

So it’s the same as rotobrush 2 basically.

[@GarethOwenFilmGOwen]

Yea and "a lot less cheaper" to have

[@ActionVFXofficial]

This is slightly different, in the sense, it isn't dependent on your machine because it is cloud-based. For example, I could have an artist creating mattes for me on a Chromebook laptop without having to get them a $5,000 computer (plus licenses) that will be outdated within a few years. I would say this route isn't for everyone, but is just an additional tool to have inside an artists toolkit. :) -Luke

[@starwarsfananimations]

"Runway ML Killed the Roto Star"

[@HankW]

Cool stuff... except the alpha is never applied! Not very useful if you don't show the extracted matte applied to the plate :P

[@ActionVFXofficial]

@ Henry Wilkinson Thanks for sharing! I am considering doing more of a deep-dive into pixel-peeping the mattes that are generated from this. This initial video is mainly just overviewing the process and tech as a whole. Is a more detailed tutorial + full comp a video you'd like to see us make? -Luke

[@HankW]

@@ActionVFXofficial Yeah don't get me wrong, it's a nice overview of the workflow. To check roto I like slapping the rotoed element over a 50% grey background or Nuke's checkerboard sometimes. This lets you see where your edges are going wrong much better than just looking at the alpha or even a slap comp over the plate! That's all, minor detail that would help assess the quality of this technique!

[@THE_ZONE_FILMS]

Yeah nah, tried it, faster just doing the roto in AE and getting it done. Interesting concept though!

[@ActionVFXofficial]

It's all personal preference!

[@Hindurashtra936]

goodbye manual roto🤣😂....welcome 1 min roto👍

[@dinoschachten]

Pretty awesome! A shame that it doesn't do non-binary alpha channels though, so that you can properly work with hair etc. or just have more reasonable edges - not the hardest thing to add manually though...

[@ActionVFXofficial]

This specific model is still in beta. Once they add some additional features for a full version, hopefully it'll be able to tackle that!

[@dinoschachten]

@@ActionVFXofficial Can't wait to see all of these technologies evolve (have already set up a Linux machine and played with FGVC)! But will still try this out this week as I do have some footage that will profit greatly from it either way.

[@Quaylin]

This is awesome stuff. I just need to find the ones who made the "soldier" video and smack them on the heads. *United States Marine Corps Uniforms so NOT SOLDIERS (Soldier = Army). *You wouldn't have 2 marines wearing two different MARPAT combat utility uniforms, especially if they're in a warzone they would be wearing what matches the environment (in this case woodlands). If not, then they would be wearing identical uniforms due to the season. *Hair is out of regulation (unless they're part of MARSOC) but they definitely don't look like it. Yes, this video is about VFX stuff, but I know a bunch of film makers are gonna be on here and if you don't get your military information right you're gonna get burned. Also, yes, this is important and pisses us Marines off.

[@ActionVFXofficial]

Thanks for sharing! :)

[@adobox4365]

This kind of rough roto has already been implemented in other software like After Effect, Flame and Nuke...it is nothing new

[@collectoroffrance]

I like but for a private person, it's quite expensive.

[@reed4109]

A promising start but nowhere near what would be needed for advertising or films.

[@m5d3e]

It is common in Korea. I worked on it a few days ago. I worked on a key with a mask at AE a few days ago. They always say they don't have a budget and hope the post-production will fix it. Even if that is not a reason to neglect shooting. And if the outcome is unsatisfactory, we are at fault. I am so sick of this.

[@ActionVFXofficial]

@Reed Thanks for sharing! another thing to keep in mind is that this tech will only continue to progress as time goes on. This model is still considered to be in a beta stage, so I'm sure that some of their future improvements will really help dial things in to where this will progress towards higher-level applications. -Luke

[@reed4109]

@@ActionVFXofficial Anything that saves compositors having to roto is a good thing

[@fusoedit]

Bye bye rotoscoping in Aftereffect ahahah

[@sudhababu9742]

Many employees will lose their jobs due to this app. Stop ai

[@seeithappen1]

LoL ..... in 5 minutes and still haven't seen anything essential except bla and commercials