Using CopyCat for Roto with

[@FoundryTeam]

For more Nuke tutorials, check out: learn.foundry.com/nuke For Nuke product information, go to: www.foundry.com/products/nuke-family

[@DKGCustom]

When is Nuke going to run natively on Apple Silicon, when so many other software developers kicked in so quickly. Seem to be missing the game changer

[@Blank201]

Thanks for adding this tutorial, really glad to see an example of the usefulness of this tool. After following the directions I wasn't getting any useful data when my Copycat was done processing. So I went and listened once more and noticed something that you said that you actually didn't do in your script. In the first Remove Node (connected upstream from Copycat 'Input') you say to Keep the RGBA, but in your Node Graph you actually only kept the RGB. I think that's why I wasn't getting any results the first time through.

[@ContentMonsters]

SO DOPE! ⭐

[@doginconfusion]

While the idea of AI assisted roto is impressive, you could have picked a trickier shot for demonstration. The guy nearly moves, mocha would take seconds to track.

[@doginconfusion]

@Behram Patel I agree with you. He does say though that this approach pretty much saved the entire project and at some point there is a tricky shot with Gozer that he says he used copycat to extract a matte for. Cheers!

[@AlfieVaughan]

It definitely does work! I used it for the whole entire project. Some shots the actors were flying around on harnesses, landing and doing flips then taking off again... It was crazy. And copycat managed to more or less nail the entire 5 minute sequence. Seriously impressive!

[@doginconfusion]

@@AlfieVaughan That's great to hear! We are still stuck with Nuke 12.7 where I work but an upgrade is on the way. Looking forward giving it a spin. Cheers!

[@AiLife115]

@@doginconfusion me too, there is many plugins still not up-to-date that the company usually used..I have to stick with nuke 12

[@angosalvo5734]

Yeah I was disappointed also with the example used. Not challenging at all.

[@darviniusb]

Wow this is awesome. I had a very complicated project almost 3 years ago that required roto for 2 actors and each of theyr body parts including many objects for 2000+ frames nd a very short deadline. Was a cartoon conversion from real footage to 2D. I had to write a python AI program to run on 3 high end nvidia gpu's of that time and even after letting it run 20+ hours over a weekend on 4k frames it had a lot of trash in it and was never so clean like this, but was usable and better then nothing, was 70% there, so i had to literally paint each frame to get the trash out. Was crazy work and i wish i had this tool. Is incredible how the technology evolves so fast. Resolve 18 just came out and the Ai roto in it is also extremly impressive.

[@KedarKhot]

Thanks for sharing your knowledge...

[@omw9965]

Somethings in Nuke are just easy while other simple things are nearly impossible but with Mocha in After Effects this can be done in seconds. Mind boggling.

[@AlfieVaughan]

Sure, this specific one is easy. But I just picked this as the example. A lot of the shots were hundreds if not thousands of frames and much more complicated. The reason I used copycat for this is because once you train it on one shot it can do all of them as long as they're similar.

[@nixipanda798]

Mocha works for Nuke as well

[@FireballVFX]

Perfect, quick and precise explanation. Thanx!

[@Rubika-e8e]

thank's for making this video !!

[@ReinOwader]

To me this looks like perfect task for Mocha.

[@cupcakenest]

have you tried white balance the footage before keying,might have helped

[@maxie6990]

It will be great to have the footage to practice copy cat

[@ruta24studio71]

Where can I find the ColourEdge gizmo? Please, it will come very handy.

[@ivaylo.paskalev]

Did you find colouredge gizmo ?

[@TheNjordy]

@@ivaylo.paskalev he didn't find the colouredge gizmo

[@ghostinthemachine563]

@@TheNjordy is there anybody who found the colouredge gizmo?

[@elijahmurphy4980]

Thank you the good explanation.

[@gravitymg5315]

yo. tnx!

[@Bbentley81]

Is there a big improvment from copycat in 13.1? I have tried it on a pretty simple shot and it produced totally unusable results.

[@darrellclaunch1518]

Came here strictly because I saw sylvester mccoy in the thumbnail and ended up learning something.

[@353productions]

@AlfieVaughan, Hey, I'm prepping for some post work and want to use a similar approach. In my case it's a rigid bodied object in a roughly similar position in varying lighting conditions. Do you need to do per shot roto + training? Or can I do 2-3 frames on let's say 10 shots and use those as my ground truths and use one inference for 60+ shots for example?

[@madlookzvfx]

is there any difference if we do key for a single frame and use that to generate matte and do roto for the single frame? I appreciate if you would have used copycat to generate matte for the Screenleft character as that's the tougher task in the shot

[@stephen12holbrook]

I would think it wouldn't make a difference, as long as the key is clamped to min 0 and max 1 values, the end result should be the same.

[@AlfieVaughan]

You can do that too. When I show the bigger shot at 2:50 you can see I'm using a mix of keying and roto to generate the mattes

[@madlookzvfx]

@@AlfieVaughan Did you tried copycat with the other character in the FG?

[@mikakettunen7939]

We are still on the stoneage, speaking as as Vfx artist here

[@themysteriousunknownrevealed]

What Ghostbusters project is this? It looks great!

[@reed4109]

Does the roto you create for each reference require the same amount of anchor points?

[@jMcWill781]

I don't think so - as he mentions the copycat node is interpreting the input as Images rather than roto shapes, so I believe you could even have different numbers of roto shapes in your reference frames and it will be fine.

[@AlfieVaughan]

Nope it just uses the alpha! :)

[@stephen12holbrook]

If it took 40 min for copycat to produce an alpha for a 66 frame shot, how long did it take for that 2300 frame shot?? I can definitely imagine this being very useful, thanks for the tips!

[@AlfieVaughan]

For the really long shots I left it training overnight for about 9 hours. I don't think it needed all that time it was just a good opportunity to leave it running for ages. Most the time it was useable within a few hours :)

[@littlecurrybread]

@@AlfieVaughan how do you feel copycat compares to runway's tools? I'm not sure if you've talked about it online somewhere.

[@mylopintorizvi4349]

whats the seventh doctor doing here?

[@BrimoStudios]

Brilliant! I'll try this tomorrow on a green screen plate that I've been having issues with and was dreading doing a heap of roto on

[@Nibot2023]

where do I find colouredge?

[@LucasNoce]

Can we address the fact that this is a production level with actors, a studio BUT just one artist in post production !? Can we replace producers with AI already ?

[@zurasaur]

Can I try this in the non-commercial version? This is insane!

[@FoundryTeam]

Hi! Of course you can! Feel free to share your thoughts in our community once you give it a try too community.foundry.com/discuss/forum/197/nuke-non-commercial-users

[@rvllctt871]

I have been practising CopyCat on the non-commercial version ... so yes.

[@rvllctt871]

@@FoundryTeam Thank you for supplying the non-commercial version. I have large time gaps in working on Nuke and although I have been using it for 12+ years I find that downloading and practicing for a few days before I go back into a studio brings back all that muscle memory. At 68 the little grey cells need a bit of a kick occasionally.

[@alexjohnson2599]

Fantastic explanation, thankyou

[@AlfieVaughan]

Thanks!

[@TriniFlix]

Someone need to fire the VFX supervisors on that set lol...

[@pamgonzales9030]

how good does this work on shots that have motion blur? All the examples I have seen of it always has fairly static subjects.

[@AlfieVaughan]

It does a pretty good job. You can either roto the motion blur in the reference frames by blurring and feathering the edges of the mattes. Or what I sometimes did is run the hard roto through a 2D motion blur node to fake it a bit. It depended on the shot which one I chose to do. Hope that helps!

[@MrGnagnagnou]

Is the production video somewhere we can see ? This looks great !

[@AlfieVaughan]

Unfortunately not. It was for a live experience project so I don't think there are any videos online

[@macprofire]

At 5:26 you say to keep RGBA though I‘m sure you only mean RGB

[@juanmartinvillafane426]

Can I use the reference frames from one shot and use copycat with those reference frames in another shot? How would be the process?

[@AlfieVaughan]

You can yes! As long as they're the same sort of shot then you can use an inference node from the training on other shots and it will do the same thing.

[@rizwanahmad1130]

Mocha can do that in seconds.

[@rythmncolors]

dont you think it can be done with a planar tracker . Cheers!

[@bandersnatch9469]

"Unimaginable power, unlimited rice pudding!"

[@joesherman5209]

Hello Alfie how do I contact you to ask question on application using copycat for project, best joe

[@AlfieVaughan]

Hey Joe. I guess here! What would you like to know?

[@sara9373]

can copy cat node be used for paint-prep shots ?

[@FoundryTeam]

Hi! yes, copycat can be used for any image to image process. So if doing paint cleanup of markers, or things like dust, scratches etc, you could give copycat a few frames of the cleaned-up version and train it on that to have this applied to the rest of the frames in the shot. Like with any machine learning the quality of the data you put in will help determine the quality of the result you get out.

[@AndrewJarvis]

Is that shot you were working on at 2:57 from Ghostbusters Afterlife!?

[@alexjohnson2599]

Has anyone had issues with it saying "Empty frame range" when trying to restart it? Can't seem to crack it

[@ИльяЕрмолаев-ю7б]

I`ve got the same problem

[@heeeeeresrossy]

Is that Sylvester McCoy?

[@AlfieVaughan]

It is!

[@AKAHatter]

I'm a year late, but finally someone pointed that out. Like was I the only one who noticed the 7th Doctor??

[@Agniii]

This would be like 5 minutes of work with AE roto brush or Davinci Resolve's Magic Mask. This is too much work for automation..

[@Mega-Tales]

Roto is another useless job AI will obliterate...easiest thing it can do

[@pranitgadankush2296]

Will it end roto jobs? Plz answer

[@igormajdandzic7587]

no

[@pranitgadankush2296]

@@igormajdandzic7587thnku

[@TheCookingPixel]

Not now, but may be in future 🤔

[@Kuk0san]

If anything, it will create even more roto/comp work because you can now crank out these shots in a fraction of the time. So you'll be able to get even more shots done with less work.

[@angosalvo5734]

Adobe roto brush is killing them already.

[@maddiehad]

boring 1 min done in after effects.

[@AlfieVaughan]

I'd be interested in how long it would take you to do one of the 2500 frame shots of a woman flailing around in a harness in After Effects

[@Agniii]

​@@AlfieVaughan not more than 20 minutes if it is similarly shot.