Simulating Rolling Shutter (Behind the Scenes) - Smarter Every Day

[@AmiYamato]

The technique is called _"Slit-Scan"_ and was developed in the analogue days decades ago. It was used in Stanley Kubricks *"2001: A Space Oddysey"* for the star gate sequence.

[@5MadMovieMakers]

Ami Yamato There's a slit scan dolly zoom video that's very cool, have you seen it?

[@AleksandrMotsjonov]

Yes. Destin should definitely segway into this topic!

[@BeCurieUs]

Dude, when you google slit scan dolly zoom video, you get a crazy vimeo video!

[@mitchellmaytorena1137]

That Vimeo video is very reminiscent of a acid trip. The way your surrounding, or the walls around you appear to be breathing.

[@ThomasSchannel]

heres a good video talking and explaining for anyone vimeo.com/71702374

[@EvanMoon]

It's so cool that when someone says "you're probably cut this out", you use it because it's very important to what you're showing us. It's one of the key components to be smarter every day.

[@Pmcoelho22]

The type of curves you get are: y = x * tan( w * x + phi ) Where w is related to the propeller and shutter speeds, and phi is phase.

[@fzigunov]

I'd love to see the proof!

[@Pmcoelho22]

Well, imagine a vertical bar going from left to right of the screen (representing the shutter), and an infinite straight line passing at the origin, around which it can rotate (representing the propeller). If the vertical line is at position x = 1, then the interception of the propeller line and the shutter line is simply: x = 1, y = tan( alpha ), being alpha the angle of the propeller to the x axis, this is the geometric interpretation of the tan funtion. If x were to be a diferente value, then, because the propeller line has a slope of tan(alpha), the interception point is going to be y = x * tan( alpha ). Because the propeller is rotating with constant speed, we can write: alpha = omega * t + phi, where omega is the angular velocity of the propeller, t is time and phi is the phase. Now, if the vertical line goes through all values of x, from left to right, at constant speed s, then: x = s * t t = x / s So, from y=x*tan(alpha), we get: y = x * tan( omega * t + phi) = x * tan( omega/s * x + phi) y = x * tan( w * x + phi ) , with w = omega/s So the faster the propeller's angular velocity in relation to the speed of the shutter, the more separation lines you will see because the tan function cicles more often, like Destin and Henry theorized. Also, Destin was right about de arctan lines which is the result you get if you do this with an horizontal line moving vertically for the shutter. It's not a rigid proof but I hope it convinces you it's true :)

[@daversalstudios]

Gênio!! Valeu pela explicação! Brasileiro também, imagino ;)

[@Pmcoelho22]

Obrigado!! Na realidade sou de Portugal :)

[@SmarterEveryDay2]

This is the second channel. Please consider subscribing to it if you'd like to learn more about the technical side of what I do.

[@malachi5302]

Smarter Every Day 2 Try doing a collab with Grant Thompson

[@malachi5302]

(On your main channel of course)

[@22222Sandman22222]

Smarter Every Day 2 Yes, this is. Nice.

[@NathanDePaulo]

I think you and Henry need to do more like this. I'd love to see more tutorial type "this is how you can do this, if you want" video. I'd enjoy seeing the math and settings and all the boring things that don't usually make it into most videos. I mean we're here to learn! Also, thanks for this. You and Henry are great.

[@wobblysauce]

Henry’s getting his learn on via a sponsor.. niiiccee..

[@5MadMovieMakers]

The slit-scan effect looks amazing!! Also famously used to photograph the stargate sequence in 2001: A Space Odyssey

[@pkramer962]

FYI the background with all the glowing buttons was made by Jack Conte for his music video "Pedals" from 2013. He is one of the founders of Patreon and at the end of the video he talks about his new site called Patreon. He made many amazing music videos before Patreon too.

[@2nickso]

P Kramer thanks, I was going to ask !

[@WrenAkula]

Pomplamoose!

[@user32455]

I was craving to see how you achieved the rolling shutter effect in main video, was very pleased to see this video.

[@alessandrokameda8971]

1:42 "You're a wizard Henry."

[@multicandyfloss10]

1:43 “your a wizard” Henry

[@RomboutVersluijs]

He makes quite good money making videos 2 with his 2 patreon pages.. wauw

[@nimbus3957]

...I'm a what?

[@RTSRAZORBACK]

"my a wizard"

[@NerdyAardvark]

Amazing! You know you're doing something cool when you have to adjust the integer type.

[@Rodman200818]

Thank you both, for all the beautiful, varied and informative work and presenting it in a way that it can reach many and garner interest and curiosity. It is always a genuine pleasure to see these things, learn and see your enjoyment in your work.

[@celtgunn9775]

So fascinating to learn how the cameras work. Thanks guys!

[@klnsmn]

The thought process behind this is amazing!

[@KaiHowells]

This is amazing. If you think about it, this is what a 2D representation of a 3D representation of a 4D object would look like.

[@Dothackfreak11]

Awesome video Destin! I work in video and deal with rolling shutter all the time. The match up with the high-speed footage is amazing!

[@GoGreenMan]

love how you explain these concepts! also loved that you put Smarter Everyday on as much as you could, because you know this thing is going to blow up!! how it does!

[@rahulsundeep538]

This is pure genius! There's a whole lotta stuff you can do in after effects if you understand the technical side of things annd like henry says, they're all 'Stupidly simple' Cheers!

[@MrRolnicek]

I remember some people used high speed camera to detect minute vibration caused by sound in a static scene and turn that video back into sound. Then some undergrads (I think) found out that using the rolling shutter gets the job done too. So this super hight tech spy trick can now be used with any smartphone.

[@26CW128Jake]

Thanks Henry!

[@babis8142]

I didn't know you have a second channel. You should be advertising that stuff. In fact this might be even or more interesting as the main video

[@Fmily]

I think this kind of stuff is awesome! I really like seeing how different effects achieved in video editing, and now I'm going to have to figure out how to do this same thing in Sony Vegas!

[@LaneCarter]

I've actually been wondering how to do something similar to this for awhile, but couldn't figure out the banding issue. Thanks for the 16 bit tip, Henry!

[@zachpw]

Not to discredit the other video, and I know you put a ton of work into it, but I think I learned so much more from this one. Thank you for posting both! I'm sort of interested in video creation so I guess that's why this one is relevant to me.

[@mrkthmn]

Watched a 3:45 long ad for you brother, haha! Such a cool behind the scenes video! Im finding that I like these just as much and main videos. Thats not meant to be a diss of the main videos but a testament to how awesome these videos are despite being labeled as "behind the scenes."

[@neilsantos9339]

I've been hanging out for the next vid. Tots worth the wait!

[@ericfletcher8454]

Perfect example and explanation on "Temporal Distortion"

[@itsviney]

Could you use this technique in After Effects in reverse to 'fix' *actual* rolling shutter in a video? I'm thinking instead of telling AE to progress one frame for every line of pixels, could you get it to go back one frame?

[@mhvdm]

If you would try that, the video would be delayed from the bottom to the top. It would offset the video from the bottom to the top. It could work, but then you will have to 'fix' every frame in the video, or image

[@5MadMovieMakers]

Don't think so because you don't have that data if filming at normal speeds

[@mhvdm]

Again, you can do it, but you would have to manually fix the image. Build the image. Which takes days even weeks doing. And the skill to do so. +5MadMovieMakers

[@5MadMovieMakers]

Sorry I was responding to original comment, not yours

[@makkumeda85]

unfortunatelly it's not that simple. If you where to wind back each line of pixels by one frame, by the time you reach the end of the frame the last line would be 30-40 seconds back in time. The information is just not there, you would need 1079 more frames inbetween each "normal" frame to make that work and if you had so many frames you wouldn't have rolling shutter. I don't think an artifact like the one seen in the propellers shots is actually fixable. There is software that will fix (or at least it will try) the "jello" effect caused by rolling shutter by analizing the footage and creating a vector of each pixel based on it's direction and speed and move it in it's correct position.

[@805ROADKING]

Henry's a Genius!! Well done!!☺

[@KnowledgePerformance7]

I want to know how you did the smarter every day branding in that prop look so good... As someone who as experimented with CGI it looks sooo good

[@pastelab]

Beautiful work guy, just, the amount of work you guys do

[@Biel7318]

this is the same effect of the water monster point of view in The Abyss applied to slow motion shot in order to generate a controlable rolling shutter effect... pretty interesting!

[@dookiesmile]

I used this technique a few years back when I first started playing with After Effects to mimic the effect used in the music video for Unintended by Muse. Quite fun to play with :)

[@manojshiva13]

Oh. ya. it's freaking cool. You defly earned my subscription.. please talk about which dslr has less rolling shutter & compare em to videocameras..

[@TomGreene]

I believe this is exactly why when you look at a slow motion video of a hockey player taking a slap shot, you see the stick "bend" so much. It's not actually bending nearly that much but because of the scanning, it appears to be. Makes sense now!

[@momerathe]

Stuff like this is what makes the internet worthwhile

[@adamemac]

Yeaaaa I didn't understand a thing he was saying, but he did an amazing job on your video!

[@YetAnotherAaron]

How would it look with some other gradient patterns?

[@arirahikkala]

Yeah, that was my first thought when I saw he represented the time shift as a grayscale image. A circular gradient centered on a propeller seen head-on would give a spinning spiral, I'm pretty sure. More complex time shifts could get pretty psychedelic. Conceptually (as in, you could do it, but you'd probably have to implement it yourself) you could also make a colored time shift (building the result a channel at a time) or even an animated one, so you could technically time-shift one video by another. Doubt you'd get much out of it other than a distorted video, though.

[@JakeLauer]

I want to see them use a black and white photograph. That'd be insane.

[@MADjaHEAD]

Same interest here!

[@Gabe_A]

A depth map gradient would be interesting

[@itaidrory]

Not to take away from Henry's achievement here, but Vsauce did this exact simulation a few years back (with the gradient telling the computer what part of a frame to take to compile a new image). This was really interesting and kudos Henry and Destin!

[@BryGy]

Kudos to Henry! Great stuff.

[@ekwaTV]

wow yeah that is super cool. I was surprised that my guess on how you did it was correct. I wish I had a high speed camera to mess with stuff like this. Anyway, super cool video.

[@michaelkap55]

Love this collaboration! two fo my favorite channels

[@sylys]

Congrats, great setup and great video

[@zlyfer]

This man is truly a wizard! This is awesome and so clever!

[@Mindful2222]

great job henry!!

[@Lumpiluk]

I was confused at first when you mentioned aliasing at 4:40. Coming from computer graphics, I thought you were talking about the jagged lines.

[@locouk]

Wow, I want this guys editing skills!

[@SpeedyRK]

Awesome! Loved all the information!

[@xsauce3858]

these guys are GENIUS!

[@Gammaduster]

This is actually quality science right here

[@MattThompsonOnGoogle]

"The Reitch Rotation Simulation" sounds like a Big Bang Theory episode.

[@rainman2543]

This video is rich with 2 of my favorite subjects - VFX & SCIENCE! loved it.... And subscribe immediately👍

[@jokker03]

This is so flappin cool, I'm bubbling.

[@TheMonkeyboy401]

I wonder what it would look like if you change the gradient image into something that is not just a top to bottom linear gradient?

[@RomboutVersluijs]

You'd probably create patterns

[@alan2here]

Can also use just the middle line of pixels, often vertically, and move the camera on a rail. The results can be awesome. "Poloska" app is great for doing this on the fly.

[@iShockStudio]

Did he use any expressions? I would love to see a break down / tutorial of exactly how he did this. Thanks for the awesome videos!! Keep em coming!

[@KeithMarrocco]

As a computer science videographer type, this is the more interesting video to me. :)

[@vedant6633]

Please try other different gradient compositions for some amazing effects, the colour brightness of each pixel in the gradient composition tells when , at what time, will the pixel be fixed, this is such a good code!!!!!

[@minecafe]

It's beautiful.

[@snaggerdoodle4303]

1:42 you're a wizard Henry

[@chrisgreece52]

This is so beautiful!!!

[@josebarria3233]

All we want is to see Henry explaining rolling shutter with drawings and cats... Again.

[@TulioSounds]

Cool! Henry is right, that is simple! Im gonna try that sometime! :)

[@pavelp80]

Nice idea, I would probably write the program rendering propeller and change angle a bit for every horizontal line.

[@anishdas4127]

You are right Henry is the genius 👍👍👍👍👍👍👍👍

[@KulbirAhluwalia]

Holy Fundamental Forces ! This is Awesome !

[@menthalA]

finally, nicely shown how it happens

[@aluink]

"This will probably get cut out" essentially guarantees it won't. LOL!

[@slowmostuff3205]

i have always wondered why that happens

[@jasonstone1833]

I love looking at your high speed video perspectives since I'm a physics sort of person with an appetite for a deeper answer. I'm also a Christian and fully appreciate the 'concealed' meaning that is just a bit out of reach for those who approach a specific insight with, well, the wrong question. I recently came across a facebook post that was about a recording of cricket song, slowed down by many factors. It was quite beautiful. I wonder if you could look at the topic with both sight and sound?... Blessings

[@WADS4x4]

The "Mega Cool People" list is a good starting point, but I want my name under the "Ultra Cool People" list. ;-) Awesome stuff man, love your channels. Being a hobby video maker, this was very interesting to learn about.

[@psbbianforlife]

video copilot tutorial. time displacement with a gradient. something similar to that I think. well done!

[@ramoneescalante3644]

Henry is a genius? Yes he absolutely is! he explained exactly how he did it and I still cant figure out how it works

[@funny_monke6]

For once, this is the Parker Square of a method... This time Matt did it simpler.

[@logitech4873]

I simulated a rolling shutter in a 3D modelling program, and rendered it with an (nearly) unbiased renderer, basically a photon light simulator. I physically modelled the camera and made an analogue rolling shutter for it, and it's giving some very interesting (and accurate) results. It has wide potential for testing hypothetical things, like "what would X look like with a strong rolling shutter effect?" - even things that are completely impossible in real life. Tell me if you're interested, Destin :)

[@tomgibson5769]

Great video about digital video manipulation. Explaining rolling shutter not as much.

[@EisenFeuer]

That's what the main channel video does

[@andymcl92]

I think this is a great example of scientific/mathematical thinking: when you manage to combine a couple of fairly simple things in a clever way to do a really complicated thing that they just were NOT designed to do =]

[@deanwoodward8026]

"[If you only use 8-bit greyscale] You start to get weird effects..." Funny, when you're talking about simulating weird effects.

[@SVProductionGermany]

Vsauce once used the same effect in his video :D

[@hxcAMBERhxc]

Love MinuteEarth :O I didn't know about MinutePhysics tho! Thanks :D

[@TheAllroth]

First off: this was really interesting and I love the way you simulated the rolling shutter effect! Now the only thing I can think about is what effect other grayscale images gives! Could you do some simulations with other patterns, like dots or the same gradient but not linear in time?

[@EisenFeuer]

Other grayscale images one might image lead to more weirdness, but in practice the more complex the modifier image, the more incoherent your resulting image, turning it into unrecognizable blobs of color.

[@SaiKelly]

Love this.

[@MobiusHorizons]

I really enjoyed this video!

[@VerumAdPotentia]

You can't fool me Destin! You are on the Millennium Falcon!

[@jake_runs_the_world]

Brilliant video !

[@mathematicalmatt]

1:42 Yer a wizard, Henry! . . . I'm a wot?!

[@iansmellios1122]

I learned a lot Thank You

[@jmohara86]

pls tell me that I'm not the only person to see a cartoon airplane villain with a cigar hanging out of the side of his mouth and little shady eyes at 4:13

[@jarethclark2470]

This channel is a time eating rabbit hole

[@kito323]

you're a wizard, Henry

[@Pakoxon]

So basically you need to assign a "time table" to address specific vector of pixels from individual pictures. That's cool! Wondering what kind of software did he use.. or did he code one by him self.

[@Raptor05121]

@Smarter Every Day 2, I own an airplane with an adjustable-speed propeller and around the 7:30 mark in your video, I've done exactly that. On my Samsung S7 Edge, I've found that bringing the propeller speed down to 2400rpm EXACTLY will give me a stationary image of the propeller on my screen.

[@Insertnamesz]

Ooh, super cool to do it like that. I was thinking to just brute force it on matlab or something with a frame to row sample function or something :P this is more elegant

[@Kurukx]

Really awesome work :)

[@ArchesBro]

Well, I had minutephysics subbed, now I have earth too

[@jeetanand8407]

minute physics rocks!!

[@laika25]

Awesome. Saay, have you seen Lartigue's photos of passing cars?

[@valentinphillipp9264]

You are awesome

[@eric7538]

Very cool! What I'm curious about now is how does prop filters work? For example, on a Go Pro, when you add the filter to the camera, it gets rid of the prop distortion!

[@johnmeyer8078]

you could just set a looping video (frames for the prop to do one rotation) that should allow for the slower scans and get the whole blade

[@infinity1323]

Would it look the same if the scan rate of the camera was divisible (exactly) by the RPM of the propeller or would it show the blades as straight?

[@klanko64]

fez the blades would appear as if they weren’t moving!

[@infinity1323]

Are you sure about that? Because, the camera still takes time to make a complete scan of one frame. In my mind, the blades would appear bent straight and diagonally (to the connection the blades) and not rounded as they were in this video. I'm not sure if my description makes sense. I hope it does.

[@bachaddict]

they would be curved but stand still instead of moving or flickering

[@robertr7923]

the point of the blade would be in each frame i suppose, and in the middle you would see the whole blade. so you'd see a half circle and a horizontal line in the middle.