Virtual Production with an iPhone

►Get CamTrackAR for FREE: apps.apple.com/us/app/camtrac...
►CamTrackAR Breakdown's
• CamTrackAR Guide #1: G...
• CamTrackAR Guide #2: C...
• CamTrackAR Guide #3: H...
►Our Unreal Engine Episode:
• Is Unreal Engine the F...
►Astonishing Times:
Get the first issue: bit.ly/AstonishingTimes
Comic Breakdown: • A New Superhero Univer...
#FilmRiot #VirtualProduction #iPhone
-----------------------------------------------------------------
*GEAR WE USE*
COLOR GRADING LUTs:
bit.ly/buyFRluts
SOUND FX:
bit.ly/buyFRsfx
MUSIC:
bit.ly/buyFRmusic
VFX ASSETS:
bit.ly/buyFRvfx
Envato - 50 Million Resources for $10:
geni.us/8Yd7
CAMERAS:
Blackmagic Ursa Mini 4.6K G2: geni.us/xJ8yIm
BMPCC 6K Pro: geni.us/SoxUs9p
BMPCC 4K: geni.us/OXwec
A7s II: geni.us/GpLqxEU
LENSES:
Rokinon: geni.us/exWno
PLUG-INS:
15% off Mocha Pro. goo.gl/MZKwCv Use coupon code: filmriot
AUDIO:
NTG3: geni.us/a4ww1
H4n Zoom: geni.us/aBJ1fnj
Zoom F8: geni.us/hbAze
TRIPOD:
BV-10: geni.us/EdMy8
-----------------------------------------------------------------
Connect with us:
TWITTER:
FilmRiot - / filmriot
Ryan - / ryan_connolly
Josh - / josh_connolly
Ryan Thompson - / 1vfxproductions
Emily - / emily_connolly
Justin - / jrobproductions
FACEBOOK:
Film Riot - / filmriot
Ryan - / theryanconnolly
Josh - / thejoshconnolly
Ryan Thompson - / vfxproductionsrt
INSTAGRAM
Film Riot - / thefilmriot
Ryan - / ryan_connolly
Josh - / josh_connolly
Justin - / jrobproductions
Ryan Thompson - / rjt_vfx
Emily - / theemilyconnolly
WEBSITES:
Ryan - www.ryanconnollyfilm.com
Ryan Thompson - www.vfxproductions.com
-----------------------------------------------------------------
Theme Song by Hello Control: bit.ly/hellocontrol

[@TheDavan619]

Hey Ryan, I never did anything 3D related before, but after your UE5 episode I got interested and started diving into it... fast forward a few weeks and now I'm working on an (amateur) 3D animated series of my favorite book. So thanks for bringing my attention to this kind of "new" stuff! :)

[@IgnaceAleya1]

Great video Ryan! I am really stunned with how easy it's becoming to integrate VFX. All the hassle and headaches are going away, allowing us to focus on the most important thing! The creative process

[@sjmdigital5799]

Been waiting for this. Can always count on you guys. Keep it up!

[@AnantaVrindavan]

Thank you for always been at the top of new technologies coming

[@SmartphoneFilmmaker]

This. Blows. My. Mind. Well done guys!

[@cadenandkylan]

dude let’s go! film riot has just made friday better! luv y’all’s vids

[@AndresMarrero]

Damn. I love their content. I have learn a lot of things here. Your the master Ryan . THANKS

[@josephjebarajs3550]

Hi friends... Can you create a seperate playlist within the channel for virtual production stuff ? That would be super useful. Also thank you so much for everything you're teaching to us. They inspire us a lot brother ❤

[@aaronbazil]

Damn, someone beat me to this! I downloaded Spark AR for this specific reason, and hadn't seen anyone do this but looks like you guys already got it going! Congrats! What I'm gonna do is still different to what you guys did, but in terms of being the first ones I saw, you guys got it!

[@iamcued]

Great video! I hope Apple will take advantage of the M1 so we can get smaller versions of Blender/Unreal on iPad Pro! Between this, and all the new 3D/VFX tools on iOS...mobile filmmaking is in such an exciting place, and it’ll be fascinating to see how it continues to overlap with virtual production.

[@SeyaDiakite7]

Thanks for this video you're uploading today. I appreciate your existence :)

[@andyabajo]

This is mind-blowing! This technology is what made Avatar. Knowing that you can create the same level of production by just using your phone is just phenomenal.

[@alex0589]

Crazy, there’s an app to transfer your consciousness into a giant 9 foot tall cat man. Try it, it’s wild.

[@ALYTV13]

I’ve been searching for so long, finally 🙌

[@kylehessling2679]

Your rant on why understanding tech is important made me so happy, I literally know so many "filmmakers" who don't even know which lens mounts they can use with their camera... it's nauseating to me.

[@theopolisIII]

omg thank you so much for sharing this!

[@LincolnHilton]

I’ve been waiting for this!

[@karliemorris7318]

put the phone on top of your cinema cam and track the data with it, while recording your raw file to you cinema cam, thanks for this vid ryan

[@OTHERZTUFF]

You are true King, Ryan! Thank you!

[@techtim123]

🙏🏾My goodness thanks Ryan excellent video 💪🏾

[@orlandomartinez9119]

Muy buena herramienta, gracias por compartirla

[@j_akae_6982]

For the 24 fps problem you can use time interpolation in premier to set it to 24

[@LostUtopiaFilms]

Oh man this is going to be so fun to play around with!!

[@drne6244]

this tip is gold, thank you

[@toapyandfriends]

BEATIFUL PRODUCTION QUALITY

[@nitentv5142]

As always, thank you.

[@Yo-Tech]

This is very very amazing!

[@tbip2001]

Will film riot be producing more 'practical' based videos in the future such as on location lighting setups, or bts footage from projects? These are good videos but I'm really missing some of the more practical videos.

[@NaruphonPunphairoj]

Thank you!!

[@aronseptianto8142]

ngl just for the lighting reference purposes, it's quite useful

[@tagcrea]

Have you tried using a mount for the phone on one of your other cameras to track the motion but use camera footage? You would have to parent/offset the virtual camera later, but it would allow more options.

[@capturedbyfabian]

I was going to ask about this too. Hope he replies!

[@TheScrubsfanforever]

Great Video!

[@richardglady3009]

Thank you.

[@rawshootersclub5935]

Slowly learning unreal. Can’t wait to start creating worlds

[@emodate]

I feel like you could use this app side by side with a more pro level camera. Use the iPhone for the reference and tracking and then the footage can sync up the tracking points and blend the unreal with the pro cam footage. (Hope that makes sense)

[@TPFXTD]

You can also create usdz files with windows, using houdini 19. As far as I know, you'll have to create and USD asset first and then use the USD ZIP rop to convert the usd+materials to usdz file

[@OfficiallyOToole]

What I'm waiting for, is basically camtrackAR for a camera.

[@Badstyles]

Quick question, would you be able to tether the phone to the cage of an actual camera, while using the app to capture two individual shots? Like say the iPhone is capturing the 3D rendered background while the camera is capturing the subject? Then in post you could splice them both together. Or would there be some slight issues with that?

[@clarajosephine3295]

This is incredible i have to try this😍 unfortunaly i only have an ipad no iphone

[@jameslabs1]

Thanks

[@phill.19]

Could it be used to check if the lighting of the chroma key is good? So we wouldn't have problems while keying in post-production.

[@scdobserver835]

Very impressive!

[@msnspy]

Screw the apps ! We need to buy your hoody 😍🔥🔥🔥🔥

[@kasperstenbom]

Bless be the best youtube channel ❤️

[@Best14Creations]

Tnx dude

[@brettcameratraveler]

This is amazing! Whats the Android equivalent of this?

[@AndresArosemena]

that app looks great! I hate that all those super cool filmmaking apps are limited to iOS and are not made for Android too

[@tafpictur6206]

It’s different apple is there futur u must change your phone 😂

[@johnaart]

The problem is only iPhones and ipads have Lidar tech.. Which makes this possible

[@tafpictur6206]

@@johnaart yea take the 12 pro max

[@AndresArosemena]

@@johnaart that's a great explanation, for this case, but it happens a lot with filmmaking apps

[@AndresArosemena]

@@tafpictur6206 haha, I already live in the google ecosystem and like it, what I don't like is how a lot of cool apps are only developed for iOS. Like Adobe for example.

[@N8Dawgg94]

This is sick. Could you theoretically mount an iPhone to your camera rig and use the tracking data in post? And would the quality of tracking be high enough to justify adding it to your camera rig?

[@blevenduskiii]

It would be slightly different since they’re different angles with the subject, but it can be pulled off

[@Vassay]

There is one crucial setback - you cannot syncronize the iphone capture to cinema camera frames. For tracking to be usable, you need perfect sync. It's way better to do traditional matchmove using your plates, when you can.

[@stevegreen9474]

​@@Vassay Yeah, this has come up a few times, the devs say that the lag between the sensors of the iphone and the other camera is the problem. It would be great if they could calculate the lag and use it as some sort of assist to combine with traditional matchmove though.

[@Vassay]

@@stevegreen9474 Beter just to get an accelerometer+gyroscope mounted on the camera, and feed that directly into the metadata. There were several attempts of doing that, but I don't know if that ever took off. In any case, traditional matchmove is not that hard in most cases, and can be used with just the original footage.

[@stevegreen9474]

@@Vassay I've not really done a lot and most of mine tends to be the planar track distant stuff. If it was a rock solid automatic track, I'd love it, to me it's like roto, a boring necessity.

[@MiDaY_i]

This was insane!!!

[@Urbanmediashowcase]

Can you mount this on a video/mirrorless camera and use the tracking data with the footage to get better quality? Like a node for tracking?

[@hekutokku]

assuming that you could adjust in post for the position above the camera then change the virtual camera settings to match your dslr (lens size, aperture ect.) planning on doing this myself

[@SeekingtheTruthofGod]

This is a really cool app! I hope they make an Android version soon!

[@dm.b7560]

I think you can attaché the phone to your camera and use the giro data with the camera footage.

[@Jaggerrrr]

00:56 mm yes the *blender* logo

[@narkhiahbr5213]

Um dia vou ser bom como vocês!!

[@swiftdetecting]

will this work with the new unreal ? also did u film apple pro res for this ? and did u film everything on the phone and not with DSLR and what app would u use to film on the phone also do u need to use abobe effect ? and can u also use divinci resolve ?

[@AndyGrabo]

I wonder whether there is an App to use live tracking from the iPhone to control UE in real time. This way you could mount the iPhone on a camera and use virtual production… any ideas?

[@MrGarryRollo]

I love how Sam from Corridor Digital has worked so hard to make 3D space work and has messed with unreal etc for so long…and now there’s a app for it 😂😂

[@Cjw.4]

This is very awesome, is there a way to use all of this tracking data captured and apply it to another camera such as a Cinema camera. Say you rigged the phone on top of the camera and even possibly our previewing what the composite will look like through the monitor. But rather than having to use the iPhone camera the data from the track can be applied and used for cinema camera giving the ability to shoot at a desired frame rate, codec & With a lens of your preference

[@Cjw.4]

Follow up: if there is not an integrated way to do this already could you in theory film two separate takes one on the iPhone one on the cinema camera mounted together on a rig so the perspective is similar. Then in Post roto scope out The subject in the iPhone footage and then rotoscope out the subject from the cinema camera footage, then by compositing the footage of the subject from the cinema camera footage into the now clean plate from the iPhone take. In theory that should work, not sure if it would cause a problem such as having to use the footage of the background captured by the iPhone mixed with the footage of the subject from the cinema camera and if that would be problematic, would love to hear feedback on if this would be a suitable option for capturing high-quality footage with quality tracking data that can be used to track the camera in 3-D space

[@quantumcored]

Awesome!

[@videopro__]

you should do a video on using unreal with the macbook m1 chip

[@bantertv9332]

To get over the HD recording limitation, could you mount the iOS device to your normal camera and shoot with that but take the tracking data from AR App to use it with the video from your normal camera to chroma key and composite in unreal?

[@BenjaminLandauMusic]

I was thinking the same.

[@JMsoo]

I wanted to know the same, any news?

[@kalharmon]

Thinking the same thing! The most immediate problem I can foresee is that your 4k camera would have a different pivot point than your iphone, which would be increasingly obvious the closer you are to your subject.

[@uehreka]

The new version of ARKit in iOS 16 (currently beta) supports 4K and HDR capture specifically to aid with VProd use cases, so this should be resolved in the fall (or possibly sooner if you can get a beta from the developers)

[@bantertv9332]

@@uehreka thanks for that, will keep an eye open! Cheers

[@RoomTwentyNine]

This is obviously a game changer 🤯

[@bySterling]

Insane!

[@JulianAbreu]

Could we record with a camera also in order to use the camera footage and the tracking data from the iPhone?

[@Soluchi-InfiniteCoCreatorGod]

Nice.

[@robbiesands166]

I'm overwhelmed. there is too much cool shit out there and not enough time to do it all -__-

[@EricLefebvrePhotography]

I wish this was available on Android.

[@ShaggyMummy]

I would love to see what you guys could do, by just mounting the phone on top of a larger camera rig and using it like a tracking system.

[@N8Dawgg94]

I read this right after writing a similar comment. I wonder if the tracking would be good enough to use in lieu of tracking in post.

[@paul42scott]

I was thinking the same thing, but they're not actually shooting the same thing, so the perspective would be slightly off, so I don't think the tracking data would be useable at all unfortunately

[@Jsfilmz]

yes it works

[@leestripp7387]

Sounds like a good idea, but the track would be offset. Not sure it'd be great. Plus I've seen lots of iPhone stuff slip'n and slid'n. Is it accurate enough?

[@leestripp7387]

Oh and the frame rate issue as he pointed out.

[@SR-iu4zr]

I just finished a video that I made with Xiaomi mi 10 t. Every thing else is made in unreal engine. And I really like it.

[@lightkeeper6768]

I'm enjoying the camtrackAR.. but what if there's a MIRRORLESS camera or CineCamera that has built-in software that that will do the same as CamtrackAR that would really be so sick.I hope that would happened in the future for sure VFX artist would be happy and hobbyist small budget VFX will enjoy it.

[@NightWLKR1]

This is insane

[@NASA2008]

Hello Film Riot which iPhone type were used?

[@DethronerX]

Damn thats badass!!

[@GiancarloBombardieri]

CamTrackAR works with any IPhone or IPad? or it´s necesary the camera depth?

[@VideoPine]

Can you mount a smartphone to a film camera and track the production camera that way?

[@HelloGlamorous]

okay wow I didn't think I wanted to upgrade my iphone but that lidar..... 😍

[@HelloGlamorous]

also the amount of years I've been following and that is my first comment 😂 sorry guys

[@CaliHomie]

what are your setting for UE camera to match CamTrackAR footage with footage from app?

[@Hamzakhan-dt3gv]

Interesting video

[@Jsfilmz]

so glad they added 30fps. that 60fps last year was a big no no lol

[@Vassay]

Seeing as many peopel are asking about using this app with a regular camera - yes, you can do that, but the result will be quite poor. Not counting all the extra steps to get the track, it will not bounce around like crazy, unless you have extremely smooth movements. Even AE camera tracker, which is quite poor, will give you a better, more stable result, than trying to get the track from the app and Frankenstein it to big camera's movement.

[@jamesphilip6737]

So, I guess all we need is an iPhone now?

[@thaolinh2637]

Wow !! 👍👍

[@bkunal88]

Can't you use your camera to do actual shoot, and iphone just as a tracker?

[@maddiehad]

this video will get billions of views.

[@H4WKB13]

Would it be technically possible to just attach the phone to a camera and then later use the tracking data with the camera footage?

[@emielzier]

Why not stick the iphone to the big camera and use the tracking data. Is it posible to do?

[@BillStreeter]

So … I feel like it should be possible to use the phone to just capture the tracking data to be used for footage captured by another camera by mounting the phone on a different camera. Also newer Sony cameras have accelerometers and gyros in them that allow the footage to be stabilized in post. I feel like this should make it possible to capture data like this for post tracking. But I doubt Sony will ever open up like this.

[@VvipMunda]

Cool

[@SukhpreetSingh-el4dg]

Hlo, I am a beginner and I don't know how to write a movie script but the whole movie is in my mind... And it is based on si- fic.. and dinosaurs....so, what should I do because I don't have a laptop... Please help me....to solve that problem or suggest me something.🙏

[@stevegreen9474]

Has anyone managed to get animated USDZ files to work? I just get static meshes, but no idea if that's just my exporter, or if CamtrackAR doesn't support it?

[@YTFPV]

I have issue with the frame rate for tracking any idea why ? i try both 30 and 60

[@nicklausinthehouse8214]

I usually just do editing on my phone. I should use my Macbook...

[@lando_visions3275]

Could you use this with the iPhone 12

[@mutantrental]

Where's the fx home link he told?

[@frankyfantasticmedia9718]

Wait, can’t you attach the phone on a camera and use the camera footage with the phone tracking data? Please let me know if that’s possible 🙏🙏🙏 I’ll run some tests tomorrow 😊

[@LawderOfficial]

yes there's a tutorial somewhere on youtube that explains this, did you manage to do your tests? (make sure you have the two cameras as close as possible so your offset is easy to manage)

[@fraseralexandertoms]

I wonder if this could be strictly the data? I wonder if you rigged the phone to a cine cam how effective that would be?

[@BAYqg]

Could someone say, why CamTrackAR shots the video with so poor quality? It feels like 480p. Need help!

[@ooshaakao2006]

I want to know which laptop is best for after effects And you help me a little. Give me a laptop. I'm a big fan of yours but I'm very poor and I've learned a lot from you. Lots of love for all of you.

[@timothydrewfilms5272]

Y'all know we're in the future, right?

[@nikolesfrances1532]

Did anyone find an app that works instead of reality converter for windows?

[@L4ZUR3]

Guys, can u make a video using a Samsung Galaxy S21 Ultra?