How Blurs & Filters Work - Computerphile

[@robmckennie4203]

now *this* is my kind of computerphile video. low level, technical, mathematical, great stuff.

[@Satchboy71]

+Rob Mckennie Agreed. I hope they do an "extras" video on this topic.

[@Crobisaur]

+Rob Mckennie If you want to learn more about image processing, a great book (and not too expensive) is ISBN-13: 978-0123965493.

[@indianakernick3788]

+xIsheyx Yeah me too!

[@matthiaswandel]

To do a realiztic blur, you really should convert pixel values to light values (de-gamma), then filter, then convert back to pixel values. Gamma is typically around 2.3. which means, light intensity is pixel value raised to 2.3. If you don't do this, dark smears over light in ways that it doesn't do in real life.

[@lukasdon0007]

+Matthias Wandel Excellent comment. Photoshop actually has an option for this (to perfom RGB-blends with linear gamma) However, the problem is not just with blurs. It's for pretty much all operations you perform. Which begs the question: why not work in a linear color space alltogether, and only apply gamma to the output. That's how lightroom tends to do stuff: gamma is only applied for visual preview and for output, but internally it's all a linear colorspace. In Photoshop, you can choose to do all your work in a colorspace such as LAB, which would solve most of your problems. However, sadly LAB is a bit of an underdog because it requires people to learn a new colorspace and adapt their workflow accordingly.

[@lcq92]

+Lukas don How's this option called in Photoshop ?

[@Tymon0000]

+Matthias Wandel Yea and by some reason this is not default option in many programs.

[@lesselp]

+Matthias Wandel You`re a woodwork guy,Wandel.This stuff is out of your area.

[@db112nl]

+lesselp Haha but you forget he also does the video editing !

[@LanIost]

I'm so happy now. I've been coding since I was a kid and never quite (had to but still...) understood how this sort of thing worked mostly because any time I tried to read/watch anything on the subject it SOLELY explained it using math too complex for me at the age I was researching it at OR was a snooze-fest. This was done in a way that IMMEDIATELY made sense and stayed that way the entire time. THANK YOU for finally solving this mystery for me!

[@DeJayHank]

+LanIost If you like programming and would like to try do some image processing of your own I would recommend the OpenCV library (available for C++ Python and Matlab). It's very user friendly and popular, so it's easy to google for help. Loading an image and applying a Gaussian blur is as easy as (using python as an example): import cv2 myImg = cv2.imread('path/to/your/image.png'') myImg = cv2.GaussianBlur(myImg, (5,5), 2) cv2.imshow('Blurred image', myImg) cv2.waitKey(0)

[@napillnik]

Having a square kernel is the correct way to do blurs, but it's also very computationally intensive. If you want something fast, but with the same effect, you do 2 passes through an image, applying 2 kernels each time. One time the kernel is a line vector, and the second time it is a column vector. So if you want a blur with a kernel that's 20x20, you need to make 400 multiplications for each pixel, and 399 additions, which is 799 operations. You could achieve the same with 2 times passing the images with just a 1x20 vector and 20x1 vector. For each pixel you make 20 multiplications and 19 additions, and you do that twice, amounting to 39 operations per pixel, which is 10 times faster than a square kernel. The advantage grows exponentially as the kernel size grows. The first approach (square kernel) has an algorithm complexity of O(n^2), while the second approach has the complexity O(n). Try it out. The effects are the same visually. The images will be different, but you won't be able to tell which one is the real Gaussian Blur.

[@gunjeetsingh90]

Mike Pound and Tom Scott..Ill watch any videos these guys make..

[@WhyFi59]

This video lead me to wondering something. Although some things are taught about how images are represented and manipulated by computers in CS courses and other such resources, nothing is taught about sound, that I could find. That is a shame. I would like to see something about digital sound on Computerphile, if possible.

[@TheBluMeeny]

+WhyFi59 Meto!

[@tobsco2]

Have a look at the videos on the Xiph.Org site, they're the guys that made the FLAC and ogg vorbis codecs.

[@quantumsmith371]

Digital sounds for the most part is signal processing with Fourier Transformations. It lends itself more to Mathematics and EE.

[@WhyFi59]

Quantam Smith Sure, but I'm sure there's a programatic way of dealing with sound, otherwise there wouldn't be sound files and you'd require a dedicated sound card just to process sound (which is actually opposite to how most PCs are, with dedicated graphics cards but no dedicated sound cards). Maybe I'm messing things up though. tobsco2 Thank you for the suggestion.

[@quantumsmith371]

***** most sound cards are advanced DACs they may do some sound processing, but for the most part they turn 1s and 0s into an analog signal. Programmatically sound is transformed using an FFT(Fast Fourier Transform). In which the constants of the Fourier series are changed which affects the sound wave. If you are interested in sound files I would say that is almost entirely under the field of compression as an uncompressed sound file is nothing more than a list of intensity values. If that is your thing then you should look at Discrete Cosine Tranforms and how they relate to compression

[@yawarjamal909]

I wish I had an opportunity to enroll in his class. I would have enrolled in each and every single class of his. Just simply love Dr Mikes teaching style and his passion towards teaching.

[@whileimgaming]

Dr. Mike Pound maybe the best. He has the patience to do arts and crafts and _really_ explain things. Superb!

[@EliteWheatProdctionZ]

As an outsider, this explanation was super easy to grasp and gave me a ton of insight into image processing, cheers Dr. Pound!

[@fukyougooglification]

i love the way this guy talks, awesome

[@Soundole]

This was a fantastic explanation, and I really appreciated the graphics used to demonstrate the processes.

[@AlexanderMcNulty92]

he casually gave us the most concise summation of Standard deviation i've ever heard.

[@xavinitram96]

pleaaaase more videos with Mike!!! These are great!

[@DroidTsuenik9]

3:06 @Computerphile The Average filter in Photoshop is NOT using the mean/uniform kernel. All this average does is it finds the average value of all selected pixels and assigns it to all of them. The mean filter that the video talks about corresponds to the "Blur" option under Photoshop's filters.

[@SyntekkTeam]

Great video, however I was a little dissapointed that no examples were shown. I would have liked to have seen how an image looks different when a gaussian vs a mean blur is applied and also how various standard deviations and radii would affect the image.

[@evilakah1]

+Arend Peter Castelein Download photoshop, try it yourself. It's really easy to do, and worth trying. Cheers! :)

[@joseph10097]

+Arend Peter Castelein Exactly! Thought the exact same thing.

[@B20C0]

Also not hard to include it. Of course I can do it myself but it kinda misses the point of an educational video to talk about the theory and don't display the outcome.

[@meepk633]

Also, there were examples. Just look closer.

[@Triantalex]

false.

[@PixelOutlaw]

A lot of color changing filters can be explained by simply using the intensity of a pixel to map along a range of colors. No kernel matrix required. (Colorize, Sepia etc). For a fun bonus, you can make a nice gradient of colors representing a hightmap landscape of ocean and hills and map that to Perlin Noise for procedural 2D continents or maps. :D

[@tminus200]

I suggest making a video discussing the compression and manipulation of audio files. There are some parallels between manipulating audio files and compressing JPEG files via Discrete Cosine Transform. It would be excellent to connect the dots! Great work!

[@glyochi8575]

6:51 i like how the editor cut to that guy closing his calculator even though its not necessary

[@jumanaalali8354]

you always save me before my finals ...thank you

[@logicalfundy]

An interesting optimization of the two filters shown here (and other "separable" filters) is that, even though this example shows a 2D kernel - you can actually use two 1D kernels. One vertically, the other horizontally. On small kernel sizes, this doesn't make much of a difference. However, as the kernel gets large, it makes a tremendous difference. Imagine, if you will, applying a 25x25 kernel on every pixel of the image. That's averaging 625 values per pixel. Now, imagine if you did it in 1D: one pass is 1x25, the other 25x1. Now you're averaging only 50 values per pixel, which is over 12 times as fast. I was actually experimenting with making poster-size images once and the open source software I was using had the 2D kernel, and it was quite slow for large kernel sizes. I was able to make the blur much faster by doing that plus using multiple threads.

[@logicalfundy]

You blur the image that was created by the previous pass.

[@Faxter313]

I've actually heard some lectures about this stuff. But this described some aspects from another perspective - that made some things clearer to me. Thanks!

[@ayushgupta5389]

Man you are awesome! Every video of your channel is so easy to understand and has in depth knowledge!

[@sayantansarkar4409]

That linking of sd, mean to gauss kernel was so good, it cleared a lot of confusions , thanks 🙂

[@Eugensson]

He forget to mention that if you make central value positive, and border values negative then you have a sharpness filter. and if you make centre value zero then you have a border filter.

[@rjfaber1991]

+Dmítrij Ačkásov Makes absolute sense, yes. Thanks for that bit of trivia!

[@martinsavc3202]

+Robert Faber Yeah, but not quite. If you want to find edges, you want to subtract values on one side from the values on the other. Or subtract the center from the surrounding values. You need to make sure that the weights in the kernel sum to 0 - this makes sure that when all the values are the same - a flat surface, the result of the convolution is 0. To sharpen an image you usually calculate the edge image and add it back to the original. This can be achieved using a single kernel, with the sum of the values being positive - usually you would increase the center value.

[@Roxor128]

+Dmítrij Ačkásov It also works the other way around: negative centre and positive everywhere else will also do sharpening.

[@DeJayHank]

+Martin Savc More more info about simple edge detection, I suggest people look up Sobel filter. One of the most basic edge highlightning filters.

[@shannonbruner542]

+DeJayHank decedent

[@somtoachu5704]

hey Computerphile this is the first of your videos am ever understanding

[@ZombieKilla96]

Aaaah now my linear algebra class is starting to sound a lot more useful!

[@knighty]

Something that probably should have been mentioned relating to gaussian blurs is that the window for them mathematically is infinite. The function tends to zero but never reaches it, and thus every pixel technically has an effect on every other pixel. The actual kernel size is determined by some arbitrary cut off where the function reaches a sufficiently low value that it has no perceivable effect on the result.

[@dominiccasts]

+knighty It would have been nice if they had mentioned this, as well as the number of standard deviations one should typically cut off the blur window. In my experience, 3 standard deviations is enough (at least for 8 bits per channel), but the reasons why one should choose that or another value, and the error that would result, would be an interesting video on its own.

[@michaelpound9891]

+knighty +Shadowfury333 We did record some stuff on this, but sometimes it better not to make them too long! I use +-2.5 standard deviations for my kernel radius, because that covers 98% of the weight under the distribution. Anywhere above two is going to look alright, but too large and obviously efficiency will be affected. The separability of the gaussian kernel (mentioned in some other comments) helps keep the speed up.

[@knighty]

+Michael Pound Oh yeah totally, it would be insane to try to go into every little detail, I just felt like it was notable because there was mention of kernel size changing because of gaussian radius and thought it might have been worth mentioning even briefly why we do change/restrict it.

[@dl6691]

+knighty but that's a property of the normal distribution that he used to explain it with.

[@AntOfThy]

For more information on image convolution search for image magic examples, convolution. Also closely related is morphology, which in a sense is a image manipulation technique that includes convolution.

[@username17234]

Kernel convolution is my new favourite fancy alias for a weighed average.

[@jstbillz]

Wouldn't mind a full image processing course from you guys !

[@SharpblueCreative]

As a photoshop user and designer I can relate to this. Interesting to see what's going on under the hood when I'm manipulating an image

[@Psychopatz]

I'm always fascinated about how our technology works, thank you sir for giving us some basic abstraction to it, its more of a marvel now than a magic to me

[@Tjekr]

The video's with Dr Mike are really good. Very interesting and nice pace in the editing

[@DeJayHank]

Love that you're doing videos about image filtering. You should do one about Canny Edge detection now that you've laid out the basics :D

[@michaelpound9891]

+DeJayHank Canny's next :)

[@buggaboo2707]

4:34 The area under the bell curve is equal to 1 ( 100% )... you can't just draw a bell curve fatter ( more variance ), without making it shorter at the mean Thank you for the informative video though, I learned from it :)

[@reda29100]

3:14 who needs a blur when you have a potato! jk CF but that montage was lit! 6:58 that's exactly what we humans call "not edges." Blurry images, well, blur the lines between high contrasting pixels of colors (thhe marginalized ones), which we would say about "where are the edges?" It's a "fault", or a flaw of the images/hardware (input) themselves, not the processing algorithm itself.

[@RisinT96]

Oh man, when he said kernel convolution it reminded me of the horrible "sampling and reconstruction" from "signals and systems" where you have to convolute the sampled signal with a kernel to get back the original signal. (and the whole Nyquist-Shannon sampling theorem)

[@darksteel78]

I feel your pain. After I took a signals and systems class I never wanted to do a Fourier/Laplace transform by hand or impulse response ever again. Lol.

[@RisinT96]

darksteel78 my exam is in a couple days and I hope that I will not have to do any of those again without the help of matlab. :)

[@darksteel78]

good luck :)

[@paabloojc]

+darksteel78 hnmm

[@danielgrace7887]

Dr Mike Pound is the dude!

[@reefstarnes839]

I was looking for a video like this. Thank you for the simple explanation!

[@fr33doom]

awesome explanation!

[@vabalokis]

always wondered how they worked , and its calculated easier than i imagined , cool stuff

[@sammcewan9544]

6:37 "This is where we cut" Couldn't do the quick maffs himself ha

[@Triantalex]

false.

[@BigyanDhital]

This was very interesting. Want more of these

[@alihsanelmas]

A great video for getting into image filtering and processing!

[@Tumbolisu]

But simply taking the average will give you something that doesn't look as good. All numbers stored in an image are actually the squareroot of the original real-life thing (if it would be a photo). So the math needed to get the correct value will need some x^2 kind of stuff. This is most noticable when bluring the two edges where one edge is red and the other is blue, or red and green, or blue and green.

[@L4Vo5]

+Tumbolisu 1. They already explained that in a video 2. this is how to handle an image, if some images have to be square rooted (im sure there are some kind of HD color images where that doesn't happen) or not is another problem (else i should say that even before blurring the image, you need to turn on the computer!! wich is true, but it's another problem).

[@nightblind]

Maybe a video for how cheby1 and Butterworth work? Basically a data filtering video. Cheers, awesome explanation boys.

[@mahoneg]

Blast from the past for me. In the 80s I was doing edge detection with Gould equipment at NYU. Then I went to finance. ...

[@normconel2907]

I hope they do more of this easy to understand stuff

[@eldelarechorga]

Very well explained

[@승현-r4q]

Very informative, many thanks!

[@HaiHoang-iv9rj]

A great explanation

[@AG-lc5mn]

Loved the video! I'm studying neuroscience and it helps a lot

[@eduardofernandes7889]

How so?

[@jackdumanat49]

2d convolution is also used to make john conway's game of life. I used to make it such that each reference cell check all the cells around it then decides to die or live. Then i came a cross this that does everything and is more efficient than my original code.

[@teekanne15]

What Kind of accent is his? I think that's one of the best sounding ones I've heard and the voice is really good too

[@sina_m_123]

Great Explained!

[@frankowalker4662]

Thank you for this. I've been creating and using these filter kernals (?) since the 90's, and never knew what they were or how they worked. LOL. PPaint on the Amiga lets you set your own parameters for some cool effects.

[@Aevatheone]

Awesome explaination! Exactly what I needed to fill i gaps!

[@boydzhang]

very illustrative, thanks!

[@konstantinrebrov675]

Great Lecture! Great Professor! Great Chanell!

[@chandusr1]

Thanks for the video, great explanations !!! I spent 1 hr in some online video lecture(I don't want to name it) to understand what filters are and this video just explained me in ~8 min. I'm learning computer vision, could you guys please make a video on 'Image descriptors'

[@rjfaber1991]

So how does the "Despeckle" option in Photoshop work? I always find that a much better way to remove noise than just blurring everything, because blur always preserves at least some of the noise, especially if the noise is coloured and not monochromatic...

[@Folopolis]

+Robert Faber Assuming it works similarly to the noise removing filter in GIMP, it's a layering of many filters. First, it computes the median and quartile values of the pixels in the original image under the kernel, to find outliers. It then applies a Gaussian filter, but applying where the outliers are.

[@tymothylim6550]

Thank you very much for this video! I learnt a lot from these explanations!

[@YalnekH]

Those Gaussian's should have different peak heights!

[@fluffigverbimmelt]

+1

[@spasovan]

very nice explaination

[@WiskerousPirate]

Great Video!! Very Informative!!

[@jostein6581]

Great video! Any plans on showing interpolation on images? Would be great to show the standard nearest, bi-linear and bi-cubic interpolation to regular grid images, but also scattered data interpolation to irregular grids for reconstructing images using RBF or Gaussian Process.

[@31337flamer]

best computerphile vids.. :O the standard gaussian blur in photoshop had bad edges cuz it used empty pixel data from outside the picture .. changed in cs3 ^^ box blur is still best but gaussian has nice edges now

[@World_of_OSes]

What do you do in your work?

[@AuthenticTerrificRickCastle]

wow this is completely amazing I was wondering how different blurs differ! could you please explain how so-called "Photoshop Lens Blur" works? The one with bokeh effect and different polygonal shapes of aperture. How similar it is to photoshop "shape blur"?

[@harchan6274]

Thanks a lot for this video, its was really helpful

[@noureldinmoustafatutorials2512]

amazing stuff👌

[@DannyJulian77]

How would you define noise in an image? An abrupt change in color from one pixel to another?

[@Vipenstrike]

Thanks for the video!

[@jdgrahamo]

Very interesting and helpful, thank you.

[@jorgevicencio3573]

thanks a lot for this lesson

[@rawlamb5197]

I would have liked to have seen more talk about how you get a picture to be expressed as a grid of numbers and how you would do this on an RGB image. For example, how does the pixel Mike first manipulates become just 64? I'm assuming this was done on a gray image because that's probably the easiest way to explain filters before introducing colour channels. Great video, anyways!

[@annanyatyagip]

thank you this was great!

[@zihanqiao2850]

Really nice!

[@TrebleWing]

This guy is my favorite

[@johnvonhorn2942]

So Dr Pound's Kernal of one's is called "Poundland" because everything is one pound. Coined a new SI unit there - The Poundland. Surely that's got to be worth a Nobel Prize? I'll swing by Nottingham later on and pick up my PhD. You're welcome.

[@Man0fSteell]

AMAZING !! THANK YOU SO MUCH!!

[@nixie2462]

Now please please, do a sharpen filter explanation!

[@harsh7704]

can someone explain me: while comparing mean blur kernel and Gaussian blur kernel , why we divide by 16 not 9 in case of Gaussian blue kernel

[@crunchyduck]

Harshvardhan Matta It's the value of all the multiplication done. 1+2+1+2+4+2+1+2+1.

[@omgimgfut]

super informative

[@alexj0101]

great video.

[@whiteshampoodev]

Thanks ALOT for this video!

[@Mohamed.wael7]

crystal clear!

[@sagnikdas7877]

4:21 are you sure of the diagram?got to preserve aea.

[@Deddiward]

What does 'intensity of the pixel' mean exactly? I mean, a pixel is composed of R,G,B,alpha values.. do you apply the matrix convolution to all 4 of them? Or do you do something different?

[@ThalesII]

What sort of optimization could be done to avoid having to look up several pixels for every one you have to process?

[@MattiEskelinen]

+ThalesII Look up the convolution theorem and fast fourier transforms. The theorem basically says that one can calculate a convolution of two functions (in this case, the image and the kernel) as a product of the Fourier transforms of the functions. Using fast discrete fourier transforms, this is more efficient than the naive convolution presented here.

[@Lanyovan]

+ThalesII If you cut the image into smaller pieces and process piece by piece a lot of pixel lookup will be faster due to caching (at least faster than going line by line on the whole image). Filters can also be parallelized well since the target pixels are calculated independently.

[@fafetico]

+ThalesII You could still optimize convolution by using separable and recursive filter algorithms. Basically, in separable filters, instead of using a nxn convolution mask you would use a 1xn and a nx1, making it more efficient. And in recursive filters, you would use the outputs of previously processed pixels in order to lower the number of pixels you have to look for the next pixel being processed, reducing the number of computations needed.

[@Theraot]

Search the document "Image Convolutionwith CUDA" you should find it on nvidia dot com as convolutionSeparable dot pdf - happy reading.

[@colinmaharaj]

If I take off my receptacles (I am short sited) what type of blur is that?

[@quenz.goosington]

It would have been nice to mention the new deep learning based filters like deepdream and neuralart.

[@RoySchl]

+44thats44oars well those are something entirely different, i would not even call those things filters at all.

[@quenz.goosington]

Well, Mike in the video describes a filter as something that will "take some image, process it, and come up with some output". Yes, they may be completely different an new, but I'd still say they're filters.

[@Theraot]

+44thats44oars the matter is that filters - at lesat conventionally - are pure*, in the sense that they ONLY "take some image, process it, and come up with some output". While those you mention has other inputs. *: deterministic, no additional inputs, no side effects.

[@quenz.goosington]

You could say that the actual algorithm is an extra input. You could also say that deepdream/neuralstyle's extra inputs are just part of their respective algorithms, and aren't necessarily "extra". I guess it depends how you wanna see it, but to me, they aren't /really/ that fundamentally different.

[@235deer]

For the gaussian filter how did you come up with that square of numbers (x1, x2, x1, etc.) A gaussian is intended to take numbers at random, I thought?

[@Roy192]

What a strange coincidence... I had a class around when this video came out, where my teacher explained exactly this. Are you guys following me?

[@Sh-hg8kf]

Sorry if my question is silly, I'm a computer noob. What do those pixel value nunbers really represent in the image? Color? Brightness?

[@hugabuga-il6125]

Undestood👏👏👏 thanks♥♥♥

[@ahmedaraby1113]

thanks very much

[@hamrikada3918]

thank you sir !

[@nhilario7638]

This is a very old video. But I'm actually starting at all this and I tried programming a function that does exactly that (an image convolution), but when I calculate a reasonable kernel size for the gaussian blur (as well as the kernel values themselves) the image just gets darker... It's so frustrating because I normalized it, so the sum of everything is 1. I'm getting pretty desperate to be honest!!

[@taylorhetherington745]

should you average the RGB values before hand or do you have to make 3 passes?