This AI Creates Beautiful 3D Photographs!

[@thomassynths]

This is THE killer application for history documentaries ...

[@falcongamer58]

Finally someone said it

[@oolmfoxz8170]

or not...

[@THEMATT222]

I agree, pair that with colourizing black and white photo and it's hard to beat :)

[@aitor.online]

@@THEMATT222 that and i recently saw a REALLY good image resizer that can take low res low quality pics and turn them higher quality. the only thing that was kind of bad about it is that it it removed some textures but it could help a digital artist more easily retouch old pictures

[@THEMATT222]

@@aitor.online I agree, I also use one of those ai image upscalers. Some results are impressive while others introduces fake texture artifacts. Still better than not using it at all most of the time!

[@BrightAfternoonProductionsPlus]

Now...for the After Effects plugin!

[@PrinceWesterburg]

Blender - lets do this properly

[@markjenkins9424]

@@PrinceWesterburg Hahaha yes now, Blender any day

[@drone_video9849]

If this was a simple plug in for After effects, it looks commercial viable as is (for many projects).... Id buy it....

[@BrightAfternoonProductionsPlus]

@@drone_video9849 Same!

[@peacewalker3344]

ew, adobe product

[@dXXPacmanXXb]

just decrease the amount of movement and add it to phones when previewing photos and make it move according to the tilt of the phone. Would be gorgeous

[@televisionandcheese]

Absolutely! Imagine your photo viewer changing perspective slightly as you tilt the phone :0

[@thisisneil2.09]

Wait I'm sure I've seen this future before in a phone. Please @ me if you know what phone it was

[@mirageangelica5495]

ThisIsNeil 2.0 are you thinking of the parallax effect? if so it's similar but waaay different from this complex ai

[@zaparine]

Facebook already has this feature. It’s called 3D photo but you have to have the depth map in the first place while this method you let the AI guess and generate the depth map instead.

[@thisisneil2.09]

Just saw the Facebook 3d photo. Damn it's good(for selfies specially.)

[@Abossow77]

it is what they say: when science gets advanced enough, it is indistinguishable from magic this was incredible!

[@starrychloe]

They are like the Harry Potter newspaper moving images.

[@felipebrunetta2106]

Science is just like magic but. Nature that we thought to do cool tricks

[@the_hanged_clown]

it looks really good but I'm getting an uncanny valley feeling from some of them as it like the person in focus "slides" across the background, even while pivoting them appropriately.

[@allandavids9284]

Yeah same I think that is also because we expect to see the objects move when the image moves

[@amitanand498]

Exactly but yes! that can be fixed!!

[@kibbiking9122]

I'm not saying your lying but I don't know what your talking about. There are a good amount of issues with the 3D photos but for me at least this is not one of them.

[@TAOEXPRESS]

i don't have any uncanny valley feeling.

[@the_hanged_clown]

@@kibbiking9122 yeah well, 134+ people would disagree

[@ImprovingAbility]

What really blows my mind is how well our brains do that, how easy it is for us to see the differences in the four images at 2:20, we don't even have to think about it, we can spot differences between those images, attribute qualities to them, and it feels completely effortless

[@WangleLine]

The results are absolutely jaw-dropping. wow.

[@SnappCheck]

I usually do not comment on KZbin videos, but in this case I have to say I am so grateful for his channel, while so many people are complaining about our times this really shows in what a magnificent time we live. What a time to be alive.

[@MrSteeiii]

I am so happy to see that there are people like you guys, I relay like you channel

[@theforcefor]

OMG! I'm holding on to my papers

[@amitanand498]

Ha ha ha😄😄😄

[@immortaldiscoveries3038]

WOOSH

[@WangleLine]

me too!! this is super exciting

[@bntagkas]

this feels the start of a very important effect, increase alot the scale of it and improve it a bunch and it feels like this is something that will be very important in the mid future, thinking especially about ar and vr but also even applications in things like movies

[@markonfilms]

Amazing stuff. I love all this image processing stuff!

[@lebasson]

Your enthusiasm certainly shines through Károly!

[@akuma7616]

You're videos are the pinnacle of KZbin

[@__-tz6xx]

It would be crazy if it could make the entire scene if full 3D and you could walk around it in VR, from only a photograph. That would be amazing!

[@spuriousc]

@Wh0tch That's not too much crazier than the patchmatch type things already being displayed here. The computer may see that the photo is inside a room and generate three more walls with matching paint and adorned to similar style as the one shown. Outdoors, may create a garden behind you in the style visible in front. It wouldn't have to match the walls or garden that were there at the actual scene of the photo to be impressive or useful.

[@kobby2g8]

That'll require magic.

[@jimbernard3289]

Lookup Photogrammetry. It's possible, but not from a single photo of course. But the technology could be perfected enough so that with a minimal amount of time and effort you could capture your environment in 3D and walk around in it in VR. Drone Photogrammetry is pretty cool too.

[@SirusStarTV]

I think it's impossible

[@maxmustermann194]

360° cameras are a thing and not even that expensive. Grab one and start recording to hop in visited locations virtually later.

[@tobszakszotczak4132]

This would be an amazing way of stabilizing videos without cutting anything off! What a time to be alive!

[@kupeart3746]

you could easily do it with 3D software look up image projection on blender or maya, but yeah it will be way faster .

[@diezgp]

This channel is the gift that keeps on giving.

[@tonybaston3419]

If your goal was to make us feel astounded, you have certainly succeeded. This series has inspired my own journey into machine learning. What a time to be alive indeed!!

[@PepekBezlepek]

This is stunning and your amazement really does come over!

[@Yotanido]

What I find interesting is the sense of perspective it gives. In the second photo, I initially thought the kid was a lot bigger. Once the movement started, I immediately realised the camera is lower to the ground and the kid is smaller than I thought.

[@puskajussi37]

Wow. Im really impressed how well it interpolates the background! Not perfect, but then what is.

[@atallahahmed9983]

two minute papers is one of my favorite channels what a time to be alive.

[@russelpea]

amazing! In a next future events with a lot of pictures taken (Olympic games etc...) will be reconstructed using the large amount of materials available (pictures, films, from different point of views)

[@alexhutchins6161]

The kid with the waters hand was bending like crazy Edit: i see its the same issue as the wine glass at the end.

[@MrOrzech1]

1:16

[@iilwy]

Must be hard to understand transparency

[@RadenWA]

He is water bending

[@zakirreshi6737]

Fantastic...we learn from previous mistakes,and improve them to get better and better results.

[@ManthaarJanyaro]

Just imagine this technology in future...!

[@andie_pants]

We are living the future that so many imaginations through the ages dreamed of. :-)

[@andie_pants]

@@Zaydme Well, a large portion of what's covered on this channel focuses on graphics and image manipulation.

[@idjles]

It looks to me that the faces are flat without depth

[@Andytlp]

Yes 1 or 2 papers in the future.

[@theskeletonboi]

A similar technique is already being used and it's called Facebook 3D Photos.

[@jasonvick6617]

I don't think I've ever said aloud "Wow!" Watching one of these videos before, this is amazing

[@2Zemog]

I'm most excited about deep learning colorization and improvement of old archive footage. Although this already exists, I still feel there's a long way to go to match the quality of other deep learning applications, such as StyleGAN2.

[@mathusuthanvenkatesan]

This man is going places. Hats off sir

[@orik737]

This is yet another paper that leads me to believe scene building for video games and simulations will eventually simply entail drawing a picture of the scene. Eventually we will have technology that can recognize the objects, perceive their depth, model their geometry and choose their material from that single image. Of course multiple images would provide more information but the time save for the work flow for this kind of thing is incredible.

[@trinidad17]

Bobble-head technology has came a great way.

[@redcloak5230]

bro thank you so much, you are the most quickest ai news station

[@arnavrawat9864]

To think it's come so far that it's going into the uncanny valley is fascinating Bravo scientists

[@tmengucor]

I love the text to speech software you used in this video.

[@lauram5905]

One of the things Gabe Newell was talking about being interested in adding to Source, just after the 2006 HDR update, was a conceptual form of photograph projection like this, in addition to traditional models. It sounded silly back then because I had no idea what infilling was, or that extracting this kind of perceptual depth information was even possible.

[@arnavdas3139]

Thanks for providing colab links... incredibly helpful

[@omnia5-9]

"what a time to be alive!" Love this man

[@chedidkamal837]

Congrats 500 uploaded papers !

[@abstractn027]

I'm stunned! What a time to be alive!

[@lenskii755]

They have 2 of the same person that did the Consistent Depth Video and the 3D Photography. That's so cool! Can't wait for the consistent depth video codes to be released🥰

[@Er1ku]

This was such a cool article! Love it.

[@osenseijedi]

One paper closer to being able to analyse pictures like in Blade Runner !

[@txorimorea3869]

No because that wouldn't be different from using your imagination to fill in hidden or missing details.

[@AceDeclan]

What does this mean?

[@Lurker1979]

Now they need to figure out how to apply this to video. Then you could have 3D movies from 2D sources with out major labor involved. Or VR augmented movies.

[@youssofhammoud6335]

That idea just hit my mind AWESOME 👌!

[@eeni14]

This is so cool and the photos of the researcher and his son( I guess?) are so adorable

[@prajnadattameher6210]

Can all types of photographs be 3D painted (Parallax effect generated) ? If yes, Can we do it for contemporary Scanning Electron Microscopy images and Radiography and X-ray images?

[@phanikiran8689]

Amazing features and interesting things to learn.thanks for sharing

[@BillyEilish]

I remember doing something like this with my Kinect, but the quality was far away from this. Great paper share @Two Minute Papers!

[@reldies5364]

And now someone make an easy to use UI so that everyone can enjoy this technology.

[@theblah12]

One of the researchers works at Facebook so this technique seams like it could used to further refine the current 3D photos you can create on their service.

[@nickfratto2439]

This is my EXACT thought after every 2 minute papers video HAHAHA I relate so hard

[@reldies5364]

@@theblah12 I dont realy want that from facebook honeslty. But I would appreciate if it got main steam nether the less.

[@reldies5364]

@@nickfratto2439 Yeah its often so surpring what technology we have but dont use although its free.

[@dome8116]

actually the colab is so easy to use for everyone, assuming some knowledge about that kind of stuff

[@funny-video-YouTube-channel]

Photo filters will be great with this one. Now we can have 3D dog and cat selfie faces on instagram :-)

[@youngjin8300]

Love tour channel. You inspired me to start my own journey.

[@GeorgeDolbier]

This is amazing. I will revel in the day when I can apply this to my family photos, or run it on my phone camera that does not have a depth sensor (because It will probably be the better part of a decasde before I can afford a phone with a depth sensor)

[@pietervanderdussen913]

Now, add the AI that changes the lighting angle to make for the perfect AR experience!

[@theaninova]

Wow, I still remember the previous paper... It feels like that was just yesterday

[@DantalionNl]

It makes sense that neural networks can predict the depth of 2D images, after all we are pretty good at it ourselves so it follows that we must be able to program computers to perform the same task.

[@weegle.]

they need to make this into an app!

[@matteofalduto766]

This could be turned into the ultimate image stabilizer. Input any shaky piece of footage, and get a smooth sequence as if it was shot on a dolly or slider

[@wushurichardreallife4768]

NOW IMAGINE THIS WITH VIDEOS - AWESOME AND NO REASON WHY THAT SHOULD ALSO NOT BE POSSIBLE

[@Foxxey]

I didn't understand any part of the science stuff but this is insanely cool!

[@combatplayer]

this would be great for viewing pictures in VR

[@Centomila]

wooooaah. It's all wobbly now!

[@Tetsujinfr]

you should do a 2 min paper video on the MiDaS depth estimation model from stereo or mono images which is used in this work as a default, I think this is a key piece to make this 3D photograph really works in the general case. I think yo did similar depth from image paper review before though, but this one seems to work quite well.

[@KA-kl2ws]

Omg it is unreal in two papers down the line, we will use virtual reality to interact with multiple images in a 3D environment. What a time to be alived

[@PenitusVox]

This is all very interesting but what's the use case for this technique? I don't personally have much interest in the family photo examples. The historical photos are interesting but even that doesn't seem like it would get much use. The only real use that comes to mind is, perhaps, the film industry using stills as a fill-in for some amount of film they had to scrap during editing / reshoots. A possible example of where that might've been useful is outlined in Nando V. Movies' video on Artemis Fowl.

[@ThisIsTheBestAnime]

B-roll for videos or presentations perhaps. It doesn't need to be in the film industry, I can imagine this being used in many commentary/explanation youtube videos.

[@PenitusVox]

@@ThisIsTheBestAnime That's true, I could see that. Or even digital billboards.

[@dissolutevoid]

Deff i feel it... What a time indeed

[@lucensius]

The first using area would be documentary films of course and it is going be a great experience.

[@disky01]

Just waiting for the VR/AR realtime environment mapping implementation.

[@coyotetu]

this is going to be interesting, we may have a great depth of film content to experience in 360 VR, we simply need to develop the AI tools to fill in the information missing from 2D films.

[@joshuagoldshteyn8651]

Especially with the dolly zoom, I could see this getting used instead of the Ken Burns effect for documentaries

[@WO96OM]

What a time to be alive! Absolutely amazing!

[@jarredeagley1748]

I'd love to see this combined with lightfield displays

[@tommypaterson]

Ken burns will love this

[@SenshiSunPower]

Can we apply this to Magic card art? The swirling reminds me of Arena.

[@imjody]

Absolutely incredible!

[@fullmoongabo]

Question: are you sure it has the best LPIPS Score? It's neither the biggest nor the smallest in the column

[@fullmoongabo]

PS: Thanks for the awesome videos by the way!! This chanel is pure awesomeness! 😎

Got it to work in Windows 10 with Anaconda! The only bit I had, the code uses a deprecated function of Networkx module, so the solution is: pip3 install networkx==2.3

[@secnep]

This is Amazing!!! This kid one day is gonna be like "oh awesome "

[@antivanti]

I imagine this should be even better working on already stereoscopic content. Maybe one day we'll be able to view 3D movies in proper 3D like in a VR headset instead of fixed stereoscopic. Imagine small movements of the head giving motion parallax in addition to the stereo vision. A lot of our sense of depth actually comes from the constant micromovements of our heads

[@VicVegaTW]

what a time to be alive!

[@brettcameratraveler]

Do any of these methods translate well when used on video that didn't originally record depth information? Sometimes what works for stills doesn't look smooth at 30fps+ because each frame has been given it's own independent treatment without trying to match with it's surrounding frames.

[@LuxxyOfficial]

damn that accuracy is that great, i wonder what you could accomplish if it was used to 3d scan objects

[@ProtoMan137]

I didn't quite get the step from RGBD images to an (improvised) 3d scene. Does RGBD have anything to do with that? RGBD is still only 2 dimensional, there is no info about behind the object which is photographed.

[@celivalg]

while it still needs some work before applying it to consumer devices I'd say, it is truly incredible, I wonder if we can also improve the depth map network using another nnet that would judge if there are inconsistencies in the end result of these 3D movements, as these 3D movements amplify the depth issues greatly and I bet it would be easier to detect them once a movement in the image is done...

[@visualzaid]

The Google Collab file is no longer available if there is any service paid or three that can do the same?

[@glumpfi]

Imagine a Blender feature that allows to open a 2D image and setup the complete 3D scene, including texturing, materials and camera setup for you :) Shouldn't be far from that.

[@FlavioMartinez97]

Thank you so much for sharing this!!!!

[@itslilac5806]

I think that the biggest issue I see is that foreground figures, such as the humans in many of the photos, aren't rendered out as 3D objects, or at least it doesn't look that way. Instead, it feels like someone made a cardboard cutout of the figure and placed it in a 3D backdrop. Otherwise, this is stunning technology, and I can't wait to see where it goes!

[@mpty2022]

wow... i will definitely test this one

[@Simonjose7258]

I don't know anything about programming. I love this channel though 👍👏

[@lyomon9981]

i have this effect on my wallpaper its amazing

[@davidnunes299]

Hey, what about the analog or quantum computing for higher quality video calling? I can't find results on this.

[@VictorGallagherCarvings]

Would the output improve further if given more images of a scene ?

[@RashiPremal]

Wowww... blown away

[@ThePiones]

How do I try it out? I can't for the life of me understand what I'm supposed to do

[@jonathanz.9675]

Same here. I got to the demo page but then like boom; I have no idea. lol

[@ThePiones]

@@jonathanz.9675 exactly I feel like an idiot

[@jonathanz.9675]

@@ThePiones I found out they have play buttons that appear at the top left of each block of code and that definitely seems like the right direction. I tried playing each one in order and I think I almost got it but then an error appeared in the very last one I think because my apple computer had problems. The play buttons look exactly this [ ] until your mouse is over them for each code block. Also there is an in browser file system like it's own little computer where I think the image at the bottom shows you can take the new 3d image/video out of.

[@osamaelawad1428]

Hello , would you please make more explaining about how do it from the beginning for beginners?

[@sethbettwieser]

Imagine how much better it could be if it took the data from 2 photos of different angles.

[@eliaignacio]

When are we going to be able to see what's on the canvas being painted in "Las Meninas"? Anyway, the borders look a bit rough, I guess that's because of the input D not being 100% accurate. I wonder if that would be fixed with a depth information improvement pre-filter.

[@DantalionNl]

Could we get a video comparing LPIPS to Netflix their VMAF? both use neural networks with inner representations to give a score for similarity between two inputs.

[@IffeOnline]

Could you do this to one of those 360 degree sphere-images you take with a phone to create the flat-image "virtual reality" thing that you can see with things like Samsung Gear and the like?

[@GetOffMyyLawn]

I have been trying to get good depth maps from images taken with a Fuji W33D Stereo Camera using stereo photo maker and Dmag9B. Your results look great from a single image! Is it possible to create an even better depth map if your ai can be trained with stereo pairs?

[@papalouis9111]

that's mental! can you download some of these as programs on pc?

[@deep.space.12]

why would you need it on PC when it runs on the web? Just click the link in the description "Or try it out here - Author notebook"