Generative AI Part 3 - Hand-Held VFX Shots

[@mdturnerinoz]

We lived in SFBA (San Jose, Sunnyvale, Cupertino) from 1993-2006; we loved going through Mill Valley on our way to Muir Woods and onto Highway 1 north to the likes of Bodega Bay. Still miss Muir Woods' magic! Oh, and being near Mount Tam, you are lucky! :0)))

[@GameSack]

It's so weird that FCP softens the image with a little bilinear softness when playback stops. That's what's causing the line. I wish it wouldn't do that.

[@RS-Amsterdam]

Much better than the helicopter video. You earned that cigar !

[@PatrickvanderStoop]

Thanks Mark again for making this great video. Its always so understanding to watch.

[@bustaubie]

Dang, that looks great! you're a wizard.

[@ImaginetMedia]

This is starting to get freaky now!

[@TheUnMarketingGroup]

Cool. Early days.. replacing elements in a clip with motion must be around the corner. 24 still frames per second automatically replacing matching elements can't be far away.

[@AIAyAyAyAy]

Awesome... Now add a wipe mask to make it look like the car is "beaming" or "warping" in! If that's possible. :)

[@cefagro]

Very cool and thank you for showing it

[@simonjacobsphotography]

Amazing . Thank you..

[@JBManos-gg2zm]

Good stuff!

[@alankelly8245]

Another good video thanks Mark. It did make me wonder if you could motion track the still to the hand held footage, rather than locking off the shot in tripod mode? Its still mind boggling what AI does though :)

[@markspencer4557]

You really can't since the still has perspective and different parts move at different rates.

[@mppfilms]

Have you tried this with Runaway ML?

[@MiniModelPaint]

What about putting the handheld motion back after comping the generated image?

[@CraigBickerstaff]

Strictly speaking with an example such as this one, you could probably do the replacement without stablizing the footage first. It's a bit more work to get everything lined up but I've using the built in tracking with generative fill and it works just as well if you are only replacing a part of the frame.

[@markspencer4557]

Sort of - you could put everything into a compound clip and then do a faux camera move on the composite image. Or, instead of stabilizing, you can try tracking the still from PS to the scene but I have found that doesn't work most of the time because FCP's tracker is a point tracker but in a scene with depth different points on the still would need to move at different rates - I've never gotten it to look correct.

[@ahadsurooprajally4780]

Shame we haven't got that feature built-in to FCPX - the results are good but the round trip is a pain in the rectal passage

[@JRossHunter]

Magic

[@AzovPort]

Like 👍

[@offcenterconcepthaus]

OMG

[@AdelmanLucas]

mill valley!!

[@theatheobhv]

Now, to make the shot really „natural“ add some fake movement to your composition.

[@KrunoslavStifter]

Sorry, but SF is anything but fantastic.

[@markspencer4557]

I was referring to the weather, not the town - but I love this town. To each his own.

[@corbie8]

Thanks for the video. But I dont think this one delivered on the "hand-held shot" aspect. By doing what you did you essentially turned it into a locked off shot and then the process is the exact same as any other process in this AI replacement pipeline

[@markspencer4557]

Yes but what it allows for is this process to be used even when you don't have a tripod so it opens it up to many more use cases. That was the point.

[@Al-ImprovEd2022]

I assume you decided to remove car after finishing the shoot. It looks like if you just waited for the car to pass you would of gotten what you wanted or this is just a sample scenario for education?

[@markspencer4557]

Yes it's a sample shot

[@ezelinkmedia7228]

what if you wanted the car and changed your mind later

[@markspencer4557]

@@ezelinkmedia7228 Just go back to the original shot?