This is my last depth rant

Head to squarespace.co... to save 10% off your first purchase of a website or domain using code CGMATTER
💝 ➟ www.cgmatter.c...
post also available on patreon, but i prefer the website anyhow:
www.patreon.co...

[@JDLeeArt]

We need to try this on some 360 video mapped to sphere...

[@wozniakowski1217]

I had the same idea! Though I think those ai depthmap algorithms could have a hard time decifering depth from a 360 video since it's a very wacky projection and they weren't trained on that. But I need to see it nonetheless

[@harry1010]

@@wozniakowski1217because I cbf looking through the literature, I remember seeing a couple of approaches. One includes figuring out what kind of 360 camera people use to correct the distortion, the other tries to convert the segmenting model to handle a curved plane instead of a flat one (which passes over the image and is used to guess depth). So yeah, definitely an area of research!

[@AirNeat]

Okay I will train one on 3d video sometime

[@harry1010]

@@AirNeat random question. Any good panoptic datasets, outside of cityscapes? Like, with people and stuff in it?

[@AirNeat]

@@harry1010 Maybe SUN360, Pano3D, or 3D60

[@lex_darlog_fun]

A couple of small, but VERY important additions: 1. Any photo is a perspective projection, not an isometric one. So, instead of extruding along axis, you should extrude towards a single point (camera), which is at some height right above the center of frame. The height itself could be found by eye looking at the etruded mesh. 2. You should also undistort the image before it and re-apply distoryion after render. These two things together can make the 2.5D mesh MUCH more representative to the geometry of real scene (and thus, more correct normals, shadows, VFX integration etc).

[@noyza2132]

yep i was just about to comment the projection thing. change the camera height by moving it until an object appears to have the same width at different depth levels

[@luipaardprint]

There’s actually a very nice piece of free software for that that will tell you camera position and lens by defining two or three lines in your image. It’s called fspy

[@khaim0919]

How would I extrude towards a point instead of a axis?

[@erti3d354]

@@khaim0919 did you find a solution?

[@DQBlizzard_]

Very in-depth video. I'll see myself out.

[@ZapPack]

👏👏👏👏👏👏👏🎬

[@florentraffray1073]

well done

[@necrorivas]

nono, that was good come back

[@Stonehawk]

now you can exit *3-dimensionally*

[@bre4k00]

It depthends, I think it was good too, hold the door.

[@LongPeter]

This is like a Corridor Crew video that gets straight to the point. 🎉 I actually really like just the depth map look without the original video tied in. I can see this being a nice, easy way to do a Kitty Pride effect with little to no masking. Just have your actor run through a door and hide the doorway with a plane that blends into or replaces the wall.

[@theexchipmunk]

I had a similar thought. Also: "Oh, in some weeks/months we are going to get a Corridor video on this." Current developments in computer graphics and AI are insane. I am currently generating photorealistic images on my Macbook Air M1 using Stable diffusion to make a dumb Meme. Hell, inpainting using AI has replaced photoshop (gimp really, I don´t have actual the PS) in a lot of aplications for me.

[@khalatelomara]

The deal breaker for this method if we can actually take a non shrinking depth for a room , as if you tried it on a room perspective it kinda go in a curved trapezoid shape

[@xabblll]

I’m not sure, but it should be possible to apply some exponential correction on depth. Usually in CG graphics then we want to save linear depth, we compress it to smaller range with logarithmic conversion, so objects closer to camera have more depth information compared to objects further away. So to get linear depth we need to apply reverse conversion

[@khalatelomara]

@@xabblll exactly, Most of monocular depth techniques are pretty good for close objects , but when when it comes to perspective it breaks down quite quick

[@merseyviking]

@@khalatelomara So you mean make the doorframe the same relative size as the objects near the camera? Well if you know the parameters of the capturing camera, you could easily apply the inverse perspective transform on the depth map, and retrieve the original scale. You would need to know the aspect ratio and FOV, and if you wanted real-world scale you'd have to measure something in the real world scene and scale the resulting geometry appropriately. It might take some faff because the near and far clip planes generated by the ML algorithm are arbitrary (well I presume they are, or at least are rough guesses).

[@somdudewillson]

@@merseyviking Most monocular depth estimators are trained to output relative depth, but a few can do 'absolute' metric depth estimations. Also, the most bleeding-edge model I know of simultaneously calculates camera intrinsics.

[@zackmercurys]

in the end, you could de-light your scene using the Ian Hubert's trick, to make it absolutely de-lightful!

[@Dude_Blender]

What trick?

[@omgbutterbee7978]

@@Dude_Blender Dividing projected textures by the light values of an hdri that was taken at the same place let's you flatten the image and remove shadows and highlights. InLightVFX had a good video called "How Ian Hubert Hacked VFX (and you can too!)" that goes over the whole process. It's REALLY cool.

[@Dude_Blender]

@@omgbutterbee7978 thanks mate!

[@MasqueArt]

@@omgbutterbee7978 You can fake the surrounding lights as well, if you do not have HDRI, but HDRI is simpler. If you can make them.

[@grinningtiki220]

Somehow Ian was doing that 18 years ago before hdri was even a thing. The man is a wizard.

[@Solanaar]

combine this with delighting and you have crazy possibilities for dynamically lighting a scene

[@stoef]

FYI you can open a command prompt for a specific folder by clicking into the path at the top of the explorer and then just typing cmd and pressing enter.

[@ClaytonOrgles]

Yep! Also if you have Windows Terminal installed, you can right click on the window and select "open in terminal"

[@VeOuria]

How did I go decades of using windows without knowing this. Thank you!

[@shydun]

brilliant! thanks :)

[@jajoothecoolman]

THANK YOU I'VE BEEN USING LINUX SO LONG THAT IT FEELS WRONG TO NOT HAVE A "open in terminal" BUTTON I DIDNT KBNOW LOL

[@HamStar_]

Shift + right click in explorer also gives an option to "Open PowerShell window here" (or Command Prompt in earlier versions)

[@sugar_ltd]

Dude that's wild. You're right that the applications are plenty.

[@jayfiled]

I'm the kind of person that can't think of any when people say that. can you give us some examples?

[@slightlyokvideos]

I don't think this really has any applications

[@Queen_Coda]

this video should have MILLIONS of views!!! This could CHANGE youtube VFX forever, and it's so accesible that ANYONE with a decent PC rig can take advantage of it. I'll definitely be looking into how to use this for my own projects going forward

[@4nto418]

My dude I've been looking for how to make detailed 3D objects from depth maps like this for a project totally unrelated to yours, you just saved my blender-inept ass, thanks a lot.

[@keshetallekaridi708]

If you're already using Resolve to deflicker, you can also generate your depth map and relight within resolve while you're at it

[@AMTunLimited]

IIRC, this is a somewhat common thing to do in Davinci Resolve so you can use it as a mask for various adjustments

[@this_is_mac]

Is it possible in premiere pro? I spent a good few hours in last few days for masking tediously and often getting it wrong

[@AMTunLimited]

@@this_is_mac I would imagine you could use a very similar technique to this and import the video to premiere pro to use as a mask

[@YAXHY]

Anyone else remember photoshop having depth map filter since the extinction of dinosaurs, but got it removed with the introduction of AI?🤔🤔🤔🤔

[@InterPixelYoutube]

Ian Hubert will have fun with these tools 100%

[@RetroPlus]

Exactly what i'm thinking

[@rustymetaII]

Awesome thanks

[@thejaredwilcurt]

0:45 You can just click into the address bar in Explorer and type cmd and press enter and it will open the command prompt from that folder.

[@undefined06855]

or even just right click -> open terminal

[@danfg7215]

or use Linux

[@1800HUNKNA]

Anyone else remembering the Doctor Who episode where they showed off Time Lord artwork which was a single moment of time captured into art. A full 3D model of that moment in time and space. This feels like a step towards that.

[@luipaardprint]

3blue1brown recently made an amazing video about holograms that’s exactly what you’re describing.

[@gordonbrinkmann]

"If this can be done with a photo, why not do depth on a video?" As far as I know DaVinci Resolve can do this quite easily (at least in the Studio version), so I would assume some other video editing software can do this as well...?

[@tomcattermole1844]

Would you be surprised if Adobe didn't? 😢 Only workaround I've found is exporting footage as a PNG sequence and then running a batch in Photoshop using the blur neural filter with depth output checked. Is it janky? Yes. Is it accurate? No. Does it work? Barely. Am I an idiot for still using Adobe? Absolutely.

[@gordonbrinkmann]

@@tomcattermole1844 No, why should I be surprised...? Never thought Adobe was the ne plus ultra. Maybe there is other software out there than just Adobe? I don't know, I'm not using all video software that exists... I just said DaVinci Resolve can do depth on a video and it probably isn't the only software.

[@tomcattermole1844]

@@gordonbrinkmann unfortunately Blackmagic knows exactly what their customers want and puts in the effort to implement it. Most other softwares don't have customers that want the features or don't want to put the effort in to implement it.

[@DailyCakeSlice]

This is SO sick!! I really wish you showed the that last clip’s full render in the video

[@nicolacornolti]

Davinci also has another useful feature which is color stabilizer, I don't know if it is in the free version or not, but it can fix exposure shifts like this, I've used it many times when asked to use videos shot without professional equipment

[@Fireheart318]

A horror/exploration game with a mechanic where your vision only works well at a certain distance would go hard!

[@GoodBaleadaMusic]

1:59 he started talking like that and i subscribed.

[@desvelame]

I think I just saw my favorite video of .. the month... the semester...

[@oguz-kagan]

dude. this was the maybe second or third best tutorial video I watch. I don't remember first two so this is first one now.

[@nullcircuit]

If only we pressed on with light field cameras, could seriously elevate things

[@Roebuck_]

Dude just dropped a nuclear bomb💀

[@TrekkingTabris]

its like every time you reappear there's some fun shit on blender to do

[@aedanp07]

YESS!! This will be so useful!

[@MichaelProstka]

You meshed yourself! That is so cool!

[@danser_theplayer01]

Movies are gonna be crazy with this one.

[@ArdaHamamcoglu]

Isn’t the top edge caused by the image being wrapped so its actually interpolating the bottom row of pixels.

[@one_stz]

huh??? THE LAST??? I NEED MOREE

[@toastbrot97]

I was using a similar method to this for image to video for a while now. Essentialy doing small camera pans and dollies into the scene to make it look a little fancier than just a scrolling 2D image. I was always wondering how stable the technique would be when done on a video instead and i have to say, it doesn't look too shabby. I think if you're trying to relight the scene the artiffacts will definitely be the biggest problem, but other than that it could be quite handy for some quick and dirty vfx, or enhancing a video shot on a tripod with some subtle realistic 3D camera shake.

[@AndrewMorris-wz1vq]

Shows like 3 mind blowing things. Says: "That's all I got for yah" Crazy good video.

[@clonkex]

Just FYI, you can type "cmd" into the address bar in explorer and it will open a command prompt at that location

[@FirstName-zt2my]

I cant wait till computers are good enough to do this live

[@3D-rg6kx]

this guy’s brain is not normal

[@DaveDFX]

Distort/project the z extrusion to the camera frustum would be a good addition to this workflow. I used this back in the days to convert 2d to 3d stereoscopic

[@zankfrappa93]

i like the bit where you did the thing with the thing

[@ChrisKGallon]

I love that you're on the Davinci Resolve/Studio Train

[@Omlet221]

3:50 Well the reason I guess it does this is because the AI just makes the closest depth pure white and the furthest depth pure black. So if you want to make the scale consistent maybe you could pick two stationary points and scale the color so that they stay the same.

[@blainecodes]

I was just experimenting with marigold and was thinking about trying out other models to see if they work better with video! Great video!

[@RyanGatts]

this is neat, but it looks like the depth texture you get out is not linear. It may need a logarithmic or some other remap curve to make it so that things like your office wall appear flat when displaced

[@NeatWolf]

Gotta say - as a non native speaker I was surprised I managed to get 100% of what you said despite the blazing fast speed, and I believe it's largely due to the fact you're pretty darn clear with your pronunciation so kudos for that ;)

[@space_tim9382]

Woah, i finally understood what depht is and how i can use it

[@ArielTavori]

Segment anything gives you high resolution outlines. You could dice based on camera distance with adaptive subdivision... I actually 'started' a plugin about 2 years ago to do all this...

[@olvrs]

now let's see it rendered

[@gabrielmoro3d]

This is awesome. Thank you!

[@geoinmot]

This guy is like the Vincent Van Gogh of 3d art. He's way ahead of other Blender tutors. No one is noticing. Many years from now they'll get it and sing his praises.

[@michaelchen2821]

You can use face tracking and a application to create simulated 3d videos, so you can look around the computer screen and the 3d model can tilt. It's weird but also kinda cool

[@BoyceBailey]

And that's how Minority Report videos got started.

[@uriinbar6046]

my thought exactly. amazing that we are witnessing that future materialize

[@SuperIceteapeach]

So much input my head is dizzy lol Crazy Video man!

[@TheAtomicSoul]

Can you separate moving objects from static ones? As is your body creates a constant shadow due to your body blocking all light to the right of the cube until you "walk" past it.

[@ralphmoreau2768]

Very cool experiment, would love to see it, in it's true glorious fidelity

[@batleram2946]

Fun fact, if you want to open command prompt in a specific folder, you can type cmd into the file explorer path, and it'll open cmd in the current folder

[@InterPixelYoutube]

Your profile picture invokes so many great childhood memories (:

[@slavsit7600]

@@InterPixelKZbin same, do you by any chance remember the name of that game

[@uusfiyeyh]

@@slavsit7600 Is called "Cut the rope".

[@slavsit7600]

@@uusfiyeyh thx

[@legendaryswordsman2279]

Not sure I understood a whole lot of that but god damn was it fascinating to watch.

[@bernatrosello4375]

Shame you missed the opportunity to show the power of this tool in comination with Ian Hubert's de-light. Would love to see it on a future video!

[@Neutral4096]

imagine using this with asynchronous timewarp and motion tracking to make videos feel smooth, but interpolation will be still smoother i think but they dont scale with monitors refreshrate

[@LifeFromAbove.]

Thank you so much this is Epic, I’ve been looking for a way to make 3D titles in Videos more realistic. I will give this a try!

[@Daikuro1]

Confirmed: video was not flat.

[@Inferryu]

2:10 Hobbyist tip for you, you can just set your texture to "extend" instead of repeat in the image texture node.

[@RetroPlus]

Ian Hubert is going to get some great use out of this trick no doubt

[@tristan6052]

can you consider camera coordinates and do perspective warping?

[@JakeBowenTV]

🐎

[@janienwright3895]

Can you do a video on how to do a parallax effect on like art pieces and old photographs and stuff using this technique? I think normally people cut out parts of the images one by one and then have the camera kind of move though the scene. I think that might sometimes work better than this specific technique but it would probably involve seperating and kind of clipping certain regions to a single plane and also have them be seperate objects or something instead of the single plane you have here.. No idea how to do it..

[@mven]

Lots of people mentioning the cmd in the address bar trick, but did you know you can open a Powershell terminal via shift + right click context menu on any folder? Cmd is old and busted, Powershell is the new hotness.

[@Kozlov_Production]

Can you make a stereo video from usual video?

[@DiThi]

I think that the depth is being interpreted as linear while it's being saved as sRGB. Or something like that. It would explain why the background is so flat.

[@josh.8104]

Wait, is this how they were post-converting 2D movies into 3D?

[@staticdrama]

you open new things! thanks

[@keeb__]

Results might be better if you project the depth towards the camera, rather than on a plane? I would be curious how that would affect the lighting and reflections.

[@tiagotiagot]

Sync multiple videos from different angles, solidify, and boolean-insersect to let you move the camera behind stuff?

[@mikakettunen7939]

YO! Finland hailing you - thanks bro!

[@wiktorp.9259]

finally some in-depth video

[@indoorraccoon4450]

didnt think of these applications. Omega cool brah

[@markobozic-c46]

You sir are brilliant. Thank you for your brain.

[@jamus1217]

That's pretty dang cool

[@speedy1000ism]

Please don't let this be your last depth rant

[@MattWyndham]

Is this how they did the blue cube in the minecraft movie trailer?

[@le0t0rr3z]

quite a deep dive into the topic

[@MECKENICALROBOT]

Subbed. I love the way you describe

[@Ricoxemani]

this is cool, but the problem with the geonodes setup is that it does not account for the perspective of the video. The geometry should get larger as it gets further away.

[@br11bronc34]

I have a question: why don't you use directly the depth map generator included in davinci? is there any practical reason? very good video, thanks and best regards

[@Thinking001]

How did you download "Depth anything v2"? am currently struggling to download it correctly, because i dont know what programm i am supposed to usse to run it!!

[@Eroktic]

Since the texture of z-depth is greyscale, you don't need 2k resolution of it. it can be 2-4 times smaller than original resolution and you wont notice.

[@juanperezvalladares]

Totally, I think the best would be to match the Depth resolution to the mesh resolution

[@ufffd]

have you tried apple depth pro model? been happy with some of the results

[@ICXCONE]

this is sooo fucking cool, for games you can use a post-processing injector call Reshade that uses a depth buffer aswell, this is so cool. y'all are going to get close to real time a.i. game filter. that would be awesome to see.

[@danielitoespiral]

this is so niche and genius

[@montikorbelle]

I like your channel a lot

[@ghostcontrollingameatsuit5119]

I had wondered if this would work but didnt know how to do video depth. cool

[@_Jude-St.-Francis_]

Omg you're literally my crush you're so cool❤

[@AndrewMorris-wz1vq]

How difficult would multiple image sources be (i.e. another camera in the hallway)?

[@ilanlee3025]

Great video, appreciated

[@FloKorp86]

what the hell dude you're a literal wizard!

[@agyab3d]

this can be a great weapon for 3d artist

[@owencmyk]

I can hear the adobe podcast

[@mysteriouslyseeing]

Are depth maps logarithmic? It seems like the closer something is to the camera, the bigger the difference in z-coordinate for an equal distance change.

[@ZapPack]

That's so crazy

[@jajoothecoolman]

That was siiiick...