PERFECT ROTO with After Effects & RunwayML

For amazing rotoscoping in less time you can combine the strengths of RunwayML and After Effects.
Rotoscoping can be a time consuming task especially if you want to get really nice looking edges and hair. Luckily there are some great tools out there and some are better than others. RunwayML is an AI based solution that can do really fast rotoscoping but only for rough roto. Meanwhile After Effects has great edge refinement tools, which are also AI assisted. So why not mix the two?
🔥BECOME A PATRON - for exclusive content and credit on new videos
🔗bit.ly/3pcBuBn
---------------------------------------------------------------
😁 Subscribe to Joshua M Kerr
🔗bit.ly/3l4U9wo
---------------------------------------------------------------
🎦 JOIN THE NEXT GEN FILMMAKERS DISCORD
🔗bit.ly/3kUzqvj
---------------------------------------------------------------
👀WHAT TO WATCH NEXT..
📺 An Unreal Engine Short Film In Three Days
bit.ly/3DwvPtI
📺 The Smallest Virtual Production Studio
bit.ly/3WW3KCY
📺 Virtual Production Playlist
bit.ly/3HnUeDe
---------------------------------------------------------------

[@JoshuaMKerr]

Let me know your preferred rotoscoping tools in the comments

[@DouglasDoNascimento]

I love these tutorials. Would love to see them for Davinci Resolve.

[@bca7839]

Green Screen is not equal to ROTO !!!!!!!!!!!!!!!!!!!!!!!

[@JoshuaMKerr]

I don't belive that I said it was. That's a lot of exclamation marks.

[@drsjamesserra]

Hi Joshua, I exported the pre-comp from the original and the Runwayml mask, however how do I proceed, I can’t overlay this precomp on my background.

[@MaskedShevchenko]

Which version of After Effects are you using on this? The one i have is different to the one in the video as its not the same. Thanks.

[@davidnappyhoose204]

As it happens, I am working on a project that I need to rotoscope and I was thinking about doing exactly what you did, using Runway for the rotoscoping and after effects for the edges. Thanks for affirming my workflow.

[@JoshuaMKerr]

Hey no worries. What I would definitely suggest is using the chatter reduction too. Very useful.

[@halitsanr4439]

Thank you for your efforts !!

[@imiy]

We need AI doing perfect rotoscoping and boring technical stuff like this, but they create AI doing creative tasks to replace humans...

[@JoshuaMKerr]

I hear you man.

[@marcfruchtman9473]

That looked really amazing in the final result. I only saw a few places where there was some odd behavior, for example at 6:44 when the cat's paw moves near your shoulder, it somehow causes your shirt to melt away. But this was barely noticeable.

[@oscaroneill8222]

Nice one! I just use the 3D keyer in DaVinci Resolve Studio and it works great, good edge detection and definition. It's also less work than this workflow. But whatever works for you 🙂 love seeing everything you put out 😁👍

[@JoshuaMKerr]

Exactly, whatever tool you're used to can work. I'm always looking toward new methods and the best possible result (which is rarely the quickest). I always enjoy your comments.

[@xXWillyxWonkaXx]

Amazing dude. I love your workflow. Such an inspiration and a plethora of knowledge. Absolutely brilliant mate.

[@JoshuaMKerr]

Hey thanks, Sometimes I wonder whether my videos are useful at all... so it's good to know you liked it.

[@wearetrackclub]

Looks great! Thanks for sharing such valuable tips 👍

[@JoshuaMKerr]

No problem

[@baikkuma]

Wow! What a life saver!! In my case the footage was filmed so badly with the green screen and any other keying effects or plugin would ruin the person in the footage. That's when I thought of using a roto brush but as its time consumes, I was almost gonna give up. But your process made my process way more easier and made worth trying to fix the errors made when filming. Thank you for your tutorial and have a good one!

[@JoshuaMKerr]

Rotobrush 3 is a good way to go too

[@dongraparesh]

Thanks to share great knowledge with us.

[@JoshuaMKerr]

My pleasure

[@cschbck]

so good!

[@KrunoslavStifter]

As a Davinci Resolve user I was curious to try RunwayML masking, but from what you have shown, it seems to be pretty similar to Magic Mask feature in Resolve, Almost identical to Runway, with the built in tools for refining that you did in After Effect. One thing that AF I think does a bit better is the refine hair brush for rotoscoping, For fine hair detials, but I'm less impressed by the Runway masking than I originally imagined it to be. Thanks for the tutorials.

[@JoshuaMKerr]

I've been fiddling with magic mask in Davinci 18.5. There's some impressive stuff going on there. But once ive mastered it I'll probably do a comparison.

[@KrunoslavStifter]

@@JoshuaMKerr Yeah. I think there is some user experience involved in all these tools, since AI can only take us so far, or at least give a decent starting point. For example, Magic Mask, was originally not supposed to be a rotoscoping tool, but just a mask for some effects or to help in color grading. Which is still what is best to use it. However rotoscopoing is what gets most attention for obvious reasons. And in my experience it works somewhat similar how you show Runway works in video, plus some fine tuning options. Where I think Magic Mask and Runway roto struggle is when there are some unexpected gaps between limbs or staff behind the person, it can get confused. That needs to be manually adjusted. But on the flip side, Magic Mask does offer lot of fine tuning and selection of certain body parts, so its impossible to do more than one pass with different settings and combine masks later. This appraoch off course works with all of the methods, Runway, Magic Mask and RotoBrush2 or any number of other selection methods, like keying or plain old school roto wish splines. One can use various Magic Mask modes, like person or object, and track only face, legs or body or arms etc. Sometimes gives better track and one can later combine the masks.

[@JoshuaMKerr]

@@KrunoslavStifter Some really great detail here. Thanks for this comment, I think you have hit the nail on the head.

[@KrunoslavStifter]

@@JoshuaMKerr No problem. Looking forward to see what you do in your videos. Also here are some good stuff from another guy, might serve as inspiration for new videos ideas. Davinci Resolve relight and depth map kzbin.info/www/bejne/qKjKdGiJq8mnY9k Magic Mask (roto) + CG room. kzbin.info/www/bejne/f3yqial3lr2Netk

[@jacquesbroquard]

How are you liking this workflow now that Rotobrush 3.0 is now in Beta?

[@JoshuaMKerr]

Exploring this now

[@jacquesbroquard]

Can't wait for the vid~!@@JoshuaMKerr

[@VFXHut]

What a cool video ❤. We didn’t even know about the web app, but will implement it for sure. Love that you download the mask so you don’t have to worry about duplicate HQ video files

[@JoshuaMKerr]

Hey! Glad you found it useful. Its a super quick workflow. I'm going to be using it for Unreal Engine Filmmaking for sure.

[@stucooke3983]

Great tutorial. Thank you. Subbed.

[@Cjw.4]

wow, end result was a flex

[@JoshuaMKerr]

Haha thanks

[@MemoryOfTributes]

Hey man I’m a big fan of your work and this is the first video I’ve watched of yours!😵😅 I have some crazy big buck ideas for shorts if anyone wants a buddy to roto with and make some banger clips, I have a KZbin channel (not this one) and trying to do some big things (with such easy work of roto and after effects😅)

[@JoshuaMKerr]

What's your channel?

[@conorsvfx]

That’s pretty flawless! Now I just need a few hundred dollars for an after effects license haha unfortunately I’m still doing everything by hand

[@Wazulon]

I'm going to give this a crack. Nice tips dude. Thankee :)

[@JoshuaMKerr]

Let me know how it goes

[@captainnatethegreat]

thank you very much

[@drsjamesserra]

Pretty awesome!

[@JoshuaMKerr]

Thanks!

[@motiondesignermx2847]

nice, i was very lazy for rotoscoping, but i believe this tec is faster.

[@JoshuaMKerr]

Fun too

[@redsun0102]

감사합니다.!

[@JoshuaMKerr]

You're welcome

[@ProvVFX]

I'm trying to avoid tracking a video clip of a room where there are screens where I need to put custom made videos in each display. Can masks replace the need for tracking in this situation if I have the green screen+masks from runway?

[@JoshuaMKerr]

doing it with masks sounds like an awful lot more work to me

[@ProvVFX]

Gotcha, appreciate the tip! @@JoshuaMKerr

[@raulcardenas58]

Thank you very much, can you create a video with this procedure exported to Unreal Engine 5? For example, to make videos for news or event presentations?

[@JoshuaMKerr]

How about this... kzbin.info/www/bejne/q4i6npd8hJxsnLM

[@sabahbubbler]

I'm on after effects 2020, I used refine matte but the pre comp becomes black. It kinda works... If i turn off decontaminate edge color but the whole video becomes a darker but i do see the edges lessen

[@JoshuaMKerr]

It's probably not a one size fits all solution but it works for me most of the time.

[@seanvideoediting]

can the mask be used in davinci resolve ?

[@JoshuaMKerr]

Probably, but I don't have enough experience with Davinci to know what result you can get

[@THENICOLUCAS]

Hey looks amazing, I would like to know how this works if I want to roto and export sparetad objects

[@JoshuaMKerr]

Give it a try. What sort of objects?

[@THENICOLUCAS]

@@JoshuaMKerr like for instance, separate the cat and you in different masks

[@JoshuaMKerr]

Oh I think so. But probably not seperate parts of the same subject... like cats body and it's ears

[@Steco__]

You have a greenscreen in your background. Is this process also more efficient if the object is not good seperated from the background?

[@JoshuaMKerr]

Good separation is always advised. But I do know that Runway is quite good at isolating objects in non greenscreen scenes. It always depends on the scene and you never really know what will confuse the AI.

[@ViralKiller]

thats not enough. you need to generate a spline path that can be cleaned up by hand. In essence let the AI do 99% of the roto and then you just step in to adjust a few areas.

[@JoshuaMKerr]

I'm just used to doing the cleanup with rotobrush. Results look good to me.

[@ianfml]

I feel like now with rotoscope 3.0 runway isn't needed anymore, right?

[@JoshuaMKerr]

Experimenting with this now

[@HadjFilmz]

Hi how can I add my rotoscope into unreal engine from run way

[@JoshuaMKerr]

I have a couple of videos on that. kzbin.info/www/bejne/q4i6npd8hJxsnLM kzbin.info/www/bejne/jYjNnaKPeaiLmpI

[@luisgonzalez8415]

Hey... I just tried this out for roto. Its pretty slow on my machine. I didn't know this was all web based. Its possible I need to be on a faster M1/2 system for web, currently on my imac pro. Also I'm using the sample footage and its not very quick at all with a lot of extra paint work. I will try another file thats 4K with more pixel info. And also do it on a faster machine. Report back later.

[@JoshuaMKerr]

How did it go?

[@luisgonzalez8415]

@@JoshuaMKerr I found a way to download Runway via Webcatalog, this is all on iMac Pro btw. I did this thinking I can get it all working on my machine, but I still think it's working as a web app. Either way working with it as an "desktop app" is much faster. Also, I worked with my own 1080x1920 clip with a simple person taking up most of the frame with fairly decent separation from the background. It went through the clip like butter. The export from Runway is only 720p (free version) and the black and white matte worked very well. I will try this out again with a tougher shot and report back. One thing...I wish this had a stand-alone desktop app. I would love to try this with footage from work but since we are uploading clips online I really can't use this. We have highly sensitive materials and it can't be floating around on the internet.

[@JEBLego]

is there any way to make the brush have a rougher outline on purpose? I want to make a stop motion video where it looks like the characters have been cut out of paper, but the super defined rotobrush wouldn't give that "rough edge" look.

[@JoshuaMKerr]

I would imagine you would do your roto first and then add an effect to roughen the alpha mask

[@JEBLego]

@@JoshuaMKerr do you know of any specific effects I could try? That's what I was thinking to do but I'm not to AE and I'm not sure exactly how to apply the necessary effects

[@EzDyt]

@@JEBLego have you tried the roughen edges effect?

[@JEBLego]

@@EzDyt no wait i wasnt able to find an effect like that. do you know where it is/how to apply it to a roto?

[@auxvainqueurs]

RUNWAY ML is a good step in the right direction but the control in the UI aren't very good. I still get a better roto in AE unfortunately.

[@JoshuaMKerr]

100 percent. Have you tried rotobrush 3

[@TheMredanderson22]

Cool work, but I’m a little confused why you didn’t actually composite it over an actual background. You could’ve just desaturated the green. It would’ve been nice to see an actual composite. so you didn’t really show any real artwork.

[@JoshuaMKerr]

The point is to show people a method of rotoscoping. Just that

[@dpredie]

your cat is named brisket?

[@JoshuaMKerr]

It is! That was the name he had in the shelter but I stuck with it

[@LiveUPMedia]

You are a genius, still looking through these, could you send me the invite to your discord

[@JoshuaMKerr]

You're too kind. Here's an invite link... discord.gg/5aFQGGBS