But what is a convolution?

[@reillymcdowell]

Ok… the fact that a 23 minute video about an advanced math topic is #42 in the youtube trending charts is incredible and represents the sheer explanatory power of 3B1B. Congratulations, Grant!

[@fredinit]

That gives me hope for the masses.

[@XTSonic]

@@fredinit I feel countries should be buying the rights to many of these videos to use in classes. The way 3b1b can help students understand math and make it more interesting is valuable and his view counts show that. Many of his videos definitely have their place in high school or university and it would make more sense to use that than some grimy old book if the goal of a country/university is to educate and engage, with 3b1b well remunerated for the licensing sales to all these potential buyers combined.

[@shaunoconnell9506]

I’d like to see the demographic breakdown on videos like this. It would be interesting who’s interested in learning

[@Cyclone1024]

To be fair a lot of engineers and engineering students are really familiar with convolutions, making this video more approachable for a lot compared to any other “advanced math topic”

[@ryuusensei8677]

how do you know that position (#42)? :0

[@liweicai2796]

22:20 Hidden fun fact here: the O(NlogN) algorithm is only discovered in 2019, and for it to be actually fast the number would be too long to fit in our universe. Naturally utilizing the FFT convolution introduced in this video yields an algorithm of complexity O(NlogNloglogN), known as the Schönhage-Strassen algorithm, discovered in 1971. It is fast for numbers with thousands of digits and is built-in for big integer computation in some programming languages.

[@Filipnalepa]

Is it one of so called "galactic algoruthms"? Algorithm optimal for sufficiently large N, but "sufficently large" means here it's too large to be ever used.

[@icosagram]

@@multiarray2320 that's one way of saying "i have no idea what this means and i think it's boring"

[@jamaluddin9158]

@@multiarray2320 fun != funny

[@notEphim]

Oh, I didn't know that the algorithm with roots of 1 had additional loglogN. I was told it's NlogN

[@arpyzero]

It's funny, I was looking at this galactic algorithm just yesterday! It uses a 1729-dimensional version of the FFT, which is hilarious... was trying to figure out why the taxicab number showed up, but I got lost in the math. I hope they find better versions of the O(NlogN) algorithm over time, it'd be neat if it ever sees actual use

[@MrSpeakerCone]

Great stuff as always! I'm an audio engineer and we generally use FFT convolution algorithms for creating reverb. Long story short, we can record how a physical space reacts to an impulse, then convolve other audio such that it sounds like it was produced in that space. It honestly feels like magic :)

[@djmips]

Jane street, has a good video lecture on this called 'Echoes of Fourier' @3Blue1Brown the topic would make a good 3B1B format video.

[@holomorphmusic]

Ohh so thats how convolution reverb works.

[@thebaker8637]

In particular: this is essentially the same thing that 3B1B did here with the moving averages. If you emit a sound with a step-like pattern and record the reverb in the environment, that gives you an acoustic fingerprint you can apply to any audio source! If you take any source of audio and convolve it with that fingerprint, you basically get how that audio source would sound like in the room. This is because a sound file is just a sequence of steps (samples) and your signature tells you how one such step responds to the room. The size of the step determines the granularity of the fingerprint -- ideally it's just an impulse like an infinitely short clap, then applying the signature to each sample trivially gives the exact behavior of the room.

[@anachromium]

That's just what I wanted to comment. This lecture is great and fantastic as always, but as a musician I just miss the audio part because this is such a beautiful way to make a reverb. (Or other sound effects as it is an extremely powerful tool!) Though I do understand that the visual way is much more in the style of 3b1b. Maybe a topic for a possible future SoMe3? :)

[@MrSpeakerCone]

@@thebaker8637 yup! there's two main ways we do this, 1) a sharp percussive sound which excites the air across the entire frequency domain like a small explosion for example, or if you're oldschool a balloon popping. 2) a long sine-sweep across the audible range from a really well tuned loudspeaker equally loud at all frequencies. We then deconvolve that against the time domain which we represent as a sine wave with a frequency of 1 cycle per second (time moves at one second per second, after all!) so that gives us a super accurate impulse response across the frequency domain

[@dude157]

I did my PhD in image processing, detecting the surface interfaces of different structures in volumetric imagery such as MRI data to support tasks such as segmentation. Convolution is such a key process for so many of the different stages. This video is the most elegant and clear explanation of convolution I've ever come across.

[@algeriapower7242]

Hi I also did my dissertation in image processing, my topic was about the osmosis variational method for image Fusion, top tip: the new chat gpt ai is insane and could help me code all image processing corses that i learned theoretically and didn't implement. Give it a try. What was your topic btw ?

[@algeriapower7242]

@@S1m0nTG i checked some of the articles you were involved in with a google scholars search, you are great i hope you keep up the good work and good luck.

[@stewartdahamman]

Interesting, but what do you get out of life?

[@mladizivko]

Any tips on making an ML model to detect boundaries in an image made up of polygonal cells? Are CNN the best way, or not?

[@johnnyreis6899]

@@stewartdahammanwhat do YOU get out of life?

[@misterjigolo]

I'm amazed by the fact that Grant is basically providing so much more intuition and understanding in 25 minutes about a complex subject than what I got from 10 hours classes during my AI master degree. And that's free. You really are a legend Grant, thank you.

[@ResilientFighter]

agree^

[@bagdis9000]

I think that animated video format is very helpful in studying. You directly see what comes from what. It should be implemented in courses. I hope it will have improvement in 20 years

[@leonardopsantos]

I'll 100 % second that. I was thinking about the "sneaky backdoor" way to calculate a convolution using the FFT/IFFT. Anyone who has studied the Fourier transform knows that the equivalent of a time domain convolution is just multiplication, so using the FFT/IFFT makes perfect sense. But how would I have loved to have access to these videos when I was buried in math trying to figure out that equivalency. Having that intuitive understanding before diving into the math would have been so, so great. The quality of these videos is simply astounding. Thank you very much for creating these videos.

[@hugodaniel8975]

This video should be taught in poor neighborhoods

[@SuperYtc1]

What did you do with your AI master degree?

[@lioreshkhar2752]

Anyone else would love to see a Stats series from 3b1b? I feel like that's a topic that's so often very confusingly and unintuitively explained.

[@djredrover]

No stats. lol I hate stats. Actually, maybe the reason I hate it is because I don't fully understand it. Soo... yeah! A 3b1b stats series would be AWESOME! PS. Are you doing a Duality section on the transforms? Don't get it to this day.

[@SiddharthPrabhu1983]

I'd recommend the channel jbstatistics. Brilliantly explained statistics fundamentals.

[@glokta1]

@@SiddharthPrabhu1983 +1. I used it when I took AP Stats and it was wonderful, especially the hypothesis testing parts.

[@will.isnull]

I would love for him to explain random variables, clt, and maybe some other obscure topics such as assumptions in models. That would be really cool :)

[@3blue1brown]

I've certainly thought about doing a probability/stats. In fact, I'd be very disappointed with myself if it didn't happen next year. What parts do you find most confusing?

[@nosy-cat]

That transition animation at 5:28 is a work of art!

[@3blue1brown]

That one was definitely very fun. Once the data was well organized it was one of those that was more emergent than planned, after setting the transition I remember thinking "Whoa! That turned out nicer than expected"

[@samgolden14]

was about to comment this exact thing; it is awesome

[@victor_creator]

I had to just stop for a moment and appreciate just how beautifully perfect that transition was when I saw it!

[@tedsheridan8725]

I especially loved the dimension transition at 10:19.

[@GabrielPettier]

@@tedsheridan8725 I was so not expecting it, that i got the impression my screen was reclining 😅

[@exylic]

10:45 One correction: Putting your lens out of focus tends to actually produce, rather than a gaussian blur, a disc blur, which is actually a lot *more* similar to a box blur, just circular in shape rather than square. It's harder to compute than a gaussian or box blur, because with those two, you are able to use the 1D kernel twice rather than use a more expensive 2d kernel. Using a box blur is often undesirable because it produces a "bokeh" square in shape rather than circular, and other directional artifacts. The gaussian blur, on the other hand, is more akin to putting a translucent film in front of the camera, or covering the lens in vaseline.

[@3blue1brown]

Very interesting! Would you have some kind of dissipation in the disc though? Surely for a large enough disc, the influence of different sources in the image don't necessarily have equal influence on a particular pixel after going through the (unfocused) lens. I can imagine exp(-x^2) is too steep a dropoff, but what's the more accurate dropoff?

[@stephaneduhamel7706]

@@3blue1brown As I said in my other comment, the bokeh blur is exactly the same shape as the aperture of the camera. It's often a circle, but can actually be a square in some cameras. This is because the rays of light which reach the camera sensor have to pass through the aperture to be able to go through the lens and then hit the sensor. When the lens is properly focused, all the rays that come from some point from the real object converge to a single point on the sensor, and this point is actually the vertex of a cone whose cross section is the shape of the aperture. But if the focus is wrong, then this cone of light don't hit the sensor at it's vertex, so the sensor is activated on an area corresponding to the cross-section of the cone. So the accurate dropoff is a step function, which is pretty steep.

[@Tumbolisu]

You could argue that due to weird quantum effects of light, that for instance give it the ability to bend around a sharp corner, it isn't totally a step function. You could also argue that light that hits the sensor at a 0.01° angle will be dimmer compared to light that hits it straight on. Both of these effects are completely insignificant though.

[@aka5]

@@stephaneduhamel7706 great explanation. I imagine there is also a subtle effect from the inverse square law. In the simplest case of a point source with the aperture orthogonal to it, the source is slightly further from the edges than the centre. Therefore the dropoff would be very slight until the step

[@stephaneduhamel7706]

@@Tumbolisu actually, diffraction around the edges of the aperture does produce some kind of blur, which we often refer to as "bloom". Is pretty insignificant when the image is exposed properly, and doesn't depend on focus anyways.

[@andrespulido8]

I believe this is my first time paying back to a channel for all the value you provide me. Thank you Grant!

[@debaelwyn]

As someone who's spent much of the last ten years doing complex signal processing professionally and using FFTs and convolutions in a bunch of contexts, this is phenomenal and does a great job of capturing the intuitions I built up over the last decade and summarizing them in 23 minutes. Thank you!

[@sharkinahat]

Fun trick for doing blur - it's separable. If you have a 9x9 kernel, you just sample along one axis using a 1x9 kernel and do the same using the results along the other axis (9x1 kernel). You go down from 81 samples per pixel to just 18.

[@AstroTibs]

I'm not an expert on this, but isn't this only true if the values in the kernel aren't correlated? That is-true for Gaussian (or flat), because the kernels are each symmetric about their x and y axes through their centers; but no longer true if you had a more "skewed" distribution within your kernel?

[@ShankarSivarajan]

@@AstroTibs Yes, that's essentially what "separable" is a technical definition for. Not all kernels are separable (obviously?).

[@Asuryante]

Separability property. It works good on Gaussian Filter since it reduces computing time by A LOT. Instead of 2(k×k)^2 operations you do 2×2×(k×k), where k is the kernel size.

[@azmodanpc]

The video I needed back in college when I was struggling to understand these concepts for my umpteenth math exam.

[@Wypipo]

Do not try to ”understand”, student. Just apply ze algorithm!!!

[@N0Xa880iUL]

@@Wypipo Spoken like a true engineer!

[@jimnewton4534]

I think umpteen is an imaginary number.

[@hyronvalkinson1749]

@@jimnewton4534 imaginary numbers are easy. Umpteen is complex Wait a second...

[@Wypipo]

@@hyronvalkinson1749 umpteen is such that when evaluated as an integer it yields at least like a bunch. Fun fact: the square root of umpteen is also umpteen.

[@Holko96]

Seeing so many different lectures on youtube on different topics is already an amazement in itself, but watching the highqualitity output, which has been kickstarteted by 3b1b with manim, transition to ofther channels, just increased youtubes potential as a learning platform. It is really amazing to see others using the library and creating their own content and I, as a humble viewer, am thankful for this addition.

[@kurtu5]

This was once the promise of TV. But with the small finite number of channels, TV degenerated into, well what we see today. KZbin with its larger finitity of channels allows both great learning content and also well what we see today.

[@jursamaj]

@@kurtu5 Ufortunately, YT *also* has flat-earthers, creationists, "Trump won" nuts, etc., many of whom are get *way* more views than Grant. The crap on TV *and* YT is there because there's an audience for it that is very profitable, much more profitable than educational info.

[@kurtu5]

@@jursamaj Thats fine. Let them have their ideas out in the open where you can clearly see them and be able to present counter ideas right back. People are always going to believe different things. My point is there is enough room for all the things. It hasn't devolved into a place where Grant is not able to reach millions and unlike TV, it never likely will.

[@jursamaj]

@@kurtu5 The problem with that attitude is that a large fraction of the population prefers a nice, simple, wrong explanation that they can understand, versus a a complicated but correct explanation that hey'd have to think hard about and probably still not understand.

[@mfShroom-z9x]

It's awesome that he programmed it and open sourced it. It's such a cool library.

[@janiKB]

You honestly bring me near to tears because I always thought I was "bad at math". But binge watching your videos and understanding concepts I never could have grasped have made me realize that I can learn math, with the right teacher. Thank you so much for making these videos!

[@konstantine8054]

We also use convolution in audio. It's the equivalent of adding one sound inside the "space" of another sound. It is highly applicable to the reverberation of spaces but not limited to just that. Excellent video. Subscribed for more!

[@sentinelaenow4576]

Imagine how many awesome technologies will derive from a few minutes of extremely sophisticated math visualizations. It's like a hyper efficient way to advance human race through talent, knowledge and a warm heart. Thank you very much Sir Sanderson. Humanity owes you a lot. Please continue for good.

[@liammccreary2941]

I love Grant’s respect for his students’ variable exposure to math concepts and problem solving. Instead of saying “you should know by now that…” he says “just know that there are certain paths you could have walked in math that make this more of an expected step.” This really helps me feel more validated in my own experience with math.

[@mikkolukas]

A lot of teachers could learn from this!

[@CarbonRollerCaco]

Actually sounds MORE guilting to me: "you not only remembered poorly, but you chose poorly as well". IOW, "you calculated poorly". :

[@simonmasters3295]

So I walked a SQL programming path select a, b, a*b from A, B -- full outer join group by a*b ...does something similar?

[@nodvick]

"you should know by now" from a teacher, as opposed to someone giving a presentation on something interesting online without the context of a proper course-structure, is a prompt that you missed something and to seek to fill the gap, it is necessary, and the "invalidated"(?) feeling you get is the intent of the assertion. Instead of getting mad at the sound, answer the door when opportunity knocks, find out what you missed and catch up to where you should be by that point in your course, the best person to ask is usually the teacher themselves.

[@jeremydeal230]

If you SELECT a*b, SUM(a*b), you might be on to something 😀

[@Phobos221B]

I've been struggling to visualise convolution for my University's Signal course, I've even watched some animations online and tried to get a hang of the discrete convolution and nothing, NOTHING comes close to this absolutely vivid imagery carefully chaperoned by your voice. You're the Richard Feynman of this century and Thank you for continuing making these videos. :)

[@SupaJay3]

As a guy that has never seen convolutions, i only came here because i saw Kirb in the thumbnail and wow I was enlightened

[@shahilhussain6466]

You must be kirby lover, even in your profile pic I see that

[@ranthean4357]

Do u watch terminalmontage? my family thinks the anime is canon ;_; c('.'c)

[@SupaJay3]

@@ranthean4357 yeahhh

[@Nossairito]

I cannot put into words just how outstanding your videos are. My absolute favorite thing about them is how they bring together notions that would always pop up in my daily life like gaussian blur and make them suddenly emerge from something I just learned, as if to reassure me that the world is always within understanding's reach if you don't let it intimidate you. I love it so much.

[@N0Xa880iUL]

It's the most simple yet convoluted concept in engineering!

[@michaelleue7594]

convolved

[@sacharjawellmer5530]

pun intended? I like it hahahaha

[@N0Xa880iUL]

@@sacharjawellmer5530 Yes! Thanks.

[@jasonreed7522]

Convolution is really powerful, yet its pain to do it normally so we actuvely avoid them by first converting to frequency domain where it becomes basic multiplication and then convert back. At some point in your education as an electrical engineer you realize that complex numbers are pure magic in just how much effort they save. (And I'm sure other engineering fields make heavy use of them too)

[@joyboricua3721]

I C wat U did

[@arjunraja8143]

When I was doing my bachelor's and studying the Taylor series you dropped your Taylor series video and saved me. Now I'm doing my master's and studying image processing and you've dropped this to save me again ❤

[@Nishantpatale9423]

Msc maths? Or stat?

[@Wypipo]

I’m a software engineer. You described this all AMAZINGLY.

[@Snowflake_tv]

He's genius at explaining Math lol

[@spacejunk2186]

I'm not an engineer and I agree.

[@benmaghsoodi2067]

I'm a construction worker and I concur

[@douglascheng9794]

I just graduated from school where I used convolution for image processing, signals, etc. and you just explained it more intuitively than anything I got out of school.

[@wyattr7982]

My final project for my MS in Computer Engineering was implementing an FPGA based guitar speaker simulator. The convolution of an audio signal with an impulse response of a particular speaker will impart the characteristics of that speaker onto the signal (though the actual implementation was FFT based so it would work im real time). Thanks for this explanation so others in my life can understand what i was losing my mind over for months

[@joelaul]

Made the same connection while watching this - in terms of the image processing example, you're BLURRING the amp signal with the impulse response. I made a similar albeit much simpler web app that simulates a tube screamer and I'd love to see how much deeper your project goes, if you'd share?

[@muhilan8540]

Yes convolution was a big part of my signals and systems course in computer engineering

[@jeffreygordonmusic]

Hey - I know this comment is a year old, but this sounds like a really cool project! I'm currently considering Computer/Electrical Engineering MS programs specifically because I'm interested in audio signal processing - would you mind sharing the name of the institution you went to?

[@Stewi1014]

Just wanted to say - I think the real power of what you're teaching is knowing that *certain mathematical tool set* exists, and understanding enough about it to go "Aha! I know what tools I need to deep dive into to solve this problem" when we encounter different problems.

[@vigilantcosmicpenguin8721]

That's what it's all really about.

[@richardpike8748]

Great way to put it. Extra tool in your toolboxes

[@Xoque551]

I've literally waited years to for 3b1b to make this exact video because convolution has never made sense to me! I'm so glad it's finally here and it's as illuminating as I'd hoped!! Can't get enough of this channel, a gem of the internet age :D

[@DrsharpRothstein]

I have come across convolution functions and have never understood them beyond a 'round robin' way of multiplication. Then I discovered its use for image manipulation and was confused by kernels. Thanks, you have guided my further interest in the subject.

[@lizzzylavender]

I’m a sophomore in a CS degree and it’s actually really exciting to understand a lot more of what goes on in these videos from all my college level stem classes. Even just being familiar with some of the concepts makes the video tie together so much nicer in my head.

[@MrSafeTCam]

I had an idea for a video game I wanted to make about 5 years ago that I never completed because I was trying to make an algorithm to do a particular thing, and it would never work properly. At 9:17 of this video I paused it and said "CONVOLUTION!" because I realised, THIS is the algorithm I needed. One of my two dream video games finally has a way to exist!

[@LegendaryKenneth]

1:42 That little snippet of animation in the bottom left was enough to give me an "ah-ha!" moment, and I finally understand how convolution reverb works in audio. Amazing!

[@mattp1337]

Man, two years of post-secondary math and this topic never came up once. Thanks for the brief and elegant introduction. I had a month's worth of ecstatic "aha" moments in less than half an hour.

[@ahmedhani9854]

I have to say I’m thankful that I’ve watched veritasium’s video on FT and FFT before hand. The same concept feels so much more familiar and easier to digest when viewed from two different but very similar takes that covers ever so slightly different aspects of the same topic. I’d highly recommend anyone to go watch the video on Derek’s channel if you haven’t already. The two videos just meld together so well it’s unbelievable!

[@adityamrai3892]

All these years of doing math and I never saw it in this manner. Probability or even simple multiplication of numbers. Always grateful towards 3blue1brown for bringing such awesome intuition out! 🙇‍♂️

[@donaldmcronald2331]

As a university student who currently learns the maths behind signals, this is a very interesting perspective. Visualizing the convolution as two functions or two sets of numbers which are sliding through one and another really helped me to wrap my head around it. It's always helpful to see a different perspective, many thanks for your effort!

[@arf9759]

This guy could put together a curriculum that could help a person finish a PhD with 12 hours of video! The density and simplicity of knowledge transfer of his videos is out of this world! If I had these videos during my BS And MS, I could've finished both in 2 years!

[@notEphim]

I once multiplied two 3-digit numbers by hand using fft just for fun. It took the whole blackboard

[@vigilantcosmicpenguin8721]

Mathematicians have a very interesting idea of fun.

[@jacemc9852]

This is so great. I would have loved if he included a bit on audio too. The natural phenomena of sound reverberating in a large hall or cave is a convolution as detected by the ear. The hall's impulse response can be captured (e.g record a gun-shot/ ballon-pop, in a large hall) and used as the filter kernel to convolve a dry audio signal. The result is to imprint the room's response on the input signal. Many famous cathedrals, theatres and basilicas have been modelled this way, so that singers and musicians can benefit in the studio with modern convolution dsp techniques. It also provides methods for echo removal in noisy recordings.

[@birdwalkin]

creepy, like simulating the ghost of a location using only its captured shadow 🤯

[@physics_hacker]

I also love the sound design applications of non-reverberation IRs, they always give such interesting sounds

[@jacemc9852]

For sure. You can model a rack of chained outboard effects, going beyond echo and reverb to things like EQ, delay and chorus etc., as long as they are linear processes. You can't model non-linear processes such as dynamic compression though.@@physics_hacker

[@huhneat1076]

18:50 PLEASE keep giving the 3 blues different personalities like this in the future. It's so cute.

[@posanijaswanth7274]

convolution shows almost every where in engineering and majority of students don't know what is happening. Explaining it takes huge amount of understanding and links between all fields. So kind of you to do that

[@sara_kaufman]

I just learned about your channel, and this is hands-down the best math video I've seen. Thank you so much!

[@mandarbamane4268]

Video: But what is a convolution? - LTI systems entered the chat. - Laplace transform entered the chat. - Fourier transform entered the chat. - Z transform entered the chat. - Correlation entered the chat.

[@Sugarman96]

AKA, the good stuff.

[@ME1D]

@@Sugarman96 The most complicated stuff

[@mandarbamane4268]

@@ME1D if that's "most" complicated, then study Electromagnetics, transmission lines, antennas waveguides, wave propagation

[@ME1D]

@@mandarbamane4268 Sadly, Im an electronics and communication engineer 😣

[@mandarbamane4268]

@@ME1D lol me too xD Good luck studying all those xD

[@larrysal8866]

It's quite funny. I study an engineering field, and we have a module called system and signal theory. I watched this video, and after maybe half a week, we had the topic "convolutions". I scratched my head, knowing i had heard it before, then remembered ir was the video. In the lecture it was really confusing, as we did the convolution as a new function on which the parameter was an offset of one of the functions compared to the other. Now i come back to this video, and it just suddenly makes sense. Truly amazing.

[@leparraindufromage366]

I really love the connection between the multiplication of polynomials and convolution of their coefficients, it arises a lot in the theory of (old school algebraic) error correcting codes as well and it's just so satisfying how probability, algebra and geometry are intertwined that way

[@danielecompangoni]

I can't express how greatful I am for how much clearer topics are after your videos, as an engineering student you have saved me quite a lot, thank you very much, keep up the good work !

[@LeoPlaw]

This was excellent! It was the missing piece in a jigsaw of what I previously thought of as disparate applications, (trading indicators, image processing...) but are all tied together with a few core mathematical concepts. I'm going to watch this again a few more times to fully absorb it.

[@TheF3AR98]

This month is awesome, thank you 3b1b for the inspiration to study.

[@justafish5559]

This is actually, hands down, the most interesting video you have uploaded recently. The recent content has just been superb so this is amazing! My class is about probability right now too so this was very cool to see.

[@hmm.3367]

In just 3 minutes you completed 23 minute video !! 😆

[@kasiphia]

You've figured out before watching, must have just been a great thumbnail.

[@frankjohnson123]

gotta get that early comment for the thumbs

[@justafish5559]

to all the comments talking about how i was able to watch the video so fast: i don’t know what’s so weird about watching videos so fast.. is that not normal? if not, i apologize for spoiling the video for the late comers.. sorry.

[@kasiphia]

@@justafish5559 Ah yes, we mere mortals are incapable of watching a 23 minute video in 3 minutes. I mean since you made your comment 3 minutes after the video, assuming it took you a minute to write it, you could have even watched the video in 2 minutes. Astounding.

[@SoniasWay]

As someone who studied engineering I’m really glad to see this

[@emiliomartineziii2980]

This was by far the most amazing video ever. Like this guy is a legend. I took only one class in this for electrical engineering and learned nothing. In this one video, I can see just how ridiculously useful all this information is. Like woah

[@aracelianayiese7027]

as a medical physicist who hates math but loves medicine and imaging, this was very easy to follow and learn from! thank you.

[@joyrc01]

I am really grateful to this channel 💖💝💝 I am studying engineering in electronics and communication systems So I deal with these theories everyday. And being able to visualise these makes things great to study.... Love from India.

[@whitenat]

As a uni student currently stuggling to understand what a convolution is, it's perfect timing ! thank you !

[@sacharjawellmer5530]

I think it is noteworthy here that the convolution operation in German is called a "Faltung" or translated a "Fold" as you fold one function/list into the other, a very fitting name I think. Great video by the way, keep up the good work

[@3blue1brown]

Every time I learn what the german word is for a math concept, it feels so wonderfully precise in contrast to English.

[@vigilantcosmicpenguin8721]

That makes it sound a lot simpler than the English term does.

[@Jack-tk3ub]

@@3blue1brown Or alternatively that the English version is more convoluted (pun intended)

[@appelnonsurtaxe]

What I find so satisfying with all your videos is that I don't just learn things; I really - more than ever - feel that I've managed to connect concepts in my head. - Convolutions ("learnt" as a math concept in annoying signal processing class) - Gaussian blur (I have average photo editing skills) - CNNs (of course it has something to do with convolutions but I like knowing the details: the kernel is what we're trying to "find" during training) - Sum of 2 dice (glanced over in stats class) - Polynomials (algebra class) - etc. I often feel powerless knowing about how much knowledge and skills I lack (even though I have a very decent education), and it' sooo incredibly satisfying to see that everything is actually linked together. Idk if that's actually the case but I also have this feeling that by connecting concepts together I'm less likely to forget them, as if my brain was a search engine with a page ranking algorithm.

[@adamzoiss638]

I'm taking a signals and systems class in college and after not understanding convolutions this made it incredibly clear, thank you so much!!

[@Karreth]

Fascinating! This would have been extremely useful when I was trying to learn this stuff in uni. I've always felt like this kind of math is a problem you just bang your head against until your brain changes to a new shape capable of understanding the problem, but these clear explanations with excellent visual helpers definitively helps. Great work!

[@reilandeubank]

As a CS major taking a break from studying for my test by watching some youtube, I really didn't expect to be hearing about big O time complexity right now!

[@alid9519]

Bro I JUST had this lecture today in Numerical Mathematics

[@JoeShmowYo]

Somehow I did grant’s homework assignment before this video came out. I do signal processing related optimization at work and just last week I was showing my coworker why I made the connection between squaring a string of 1s and the binomial coefficients / Pascal’s triangle, and how that relates to an iterative algorithm we were trying to vectorize (basically do it in steps of 4 at a time using special processor instructions).

[@MuditGupta07]

never in my life have i seen a math video and smiled out of joy the best explanation out there

[@leevimalmivaara5442]

Thank you for your incredibly beautuful visuals!

[@jjohansen86]

The conclusion about the FFT also suggests one more important thing: How to do a deconvolution. It's hard to see how to undo that integral of a product... thing... that is the definition of a convolution. But when you understand that the Fourier transform of a convolution of two functions is equal to the product of the Fourier transforms of the functions, then you don't have to undo that integral, you just take some Fourier transforms, divide, and do an inverse Fourier transform. So in addition to giving you a speedup, the FFT method suggests something that you just can't do otherwise. Why do we care? Maybe we have a signal where what we measure is a convolution of what we want to know about with the response of some detection method. That's because there's one other answer to the question "What is a convolution?" that comes to mind for me more than anything: Say you have a thing that has a known response to an input. Now replace that thing with a bunch of things with some distribution of important properties. The convolution tells you how the whole distribution responds to the input. For a concrete example, I'm an atomic physicist, and sometimes I want to know about the distribution of atoms in a gas, things like their velocities. I shine a laser through the cloud of atoms, and the velocity information is there because of each atom's Doppler shift, but what I measure directly is the convolution of that distribution of Doppler shifts of all the atoms in the gas with the Lorentzian response of an atom to a laser. The deconvolution can let me take out the Lorentzian response of an individual atom and leave me with the actual distribution of Doppler shifts, so now I actually know what the cloud of atoms is doing. Or maybe you are trying to detect some radio signal with an antenna, but what you read just isn't as clean as you'd like it to be because the response of your antenna is actually being convolved with the original signal; a deconvolution can help you remove the way your antenna is influencing the signal from what you measure. There's a ton of different signal and response problems like this, and an FFT deconvolution can let you get a better signal in each of those cases.

[@bejcsmith334]

There was a great video from a guy named useless game dev who made video game graphics filters using those vertical and horizontal line detector kernels in order to make any 3d space look hand drawn, the video was called mobius-style 3d rendering Super cool

[@QuinnDuPont]

I love how Grant speaks about different journeys in math, and doesn't presuppose a traditional schooling. He's the best.😃

[@sethcarver6275]

the phrase "there are certain paths you may have walked in math that may have made this a more expected step" is such a certifiably heartwarmingly affirmational statement to math enthusiasts of all backgrounds and paths. Thank you 3Blue1Brown for warming this Political Science major turned math nerd's heart!

[@jorge_cazares]

I just finished taking 6.046, Design and Analysis of Algorithms, at MIT, and the last lecture was about convolutions and FFT. This video was literally a summary of most of the things covered in lecture, in just 23 minutes. Amazing!

[@hammadsafeer4283]

What a coincidence, ... Just learnt about convolution today in class was searching for detailed video... And here you are , real saviour ❤️ Thanks man

[@threemr01]

Grant, this time I literally screamed at the screen “nooo, not yet” when you got to “the clicky stuff”! This was the intuition I was missing back in signal processing class, and loved that explanation of the FFT trick. Awesome video (as always). Please don’t keep us waiting two years for the next one 😉… still waiting for the next video after the one for the binomial distribution.

[@NigelThorne]

These videos are gold. Thank you for putting in all the time to think up ways to explain complex concepts so clearly. Also thanks for all the patreons that support you to allow you to make this stuff publicly available. We are so lucky to be alive at such a time.

[@agenticmark]

as a kid who hated math and then became a programmer and the an ML engineer. you were sent by angels my guy. I have now been officially watching your vids for 4 years and I have a deep respect and even a budding liking for maths.

[@abhasoodan7982]

i am studying Digital Signal processing in university right now and this is the most well explained video of convolution. thank you so much!

[@glitchy_weasel]

My professor just introduced convolutions in my differential equations class, so this is a perfect coincidence. Hoping to see the continuous case. Fantastic as always.

[@PtakubJ]

Important in audio processing too! Analogous to blur in graphics, we use convolution in audio for reverb. You may record your electric guitar with some interface. Then record an impulse response of some cathedral or something. Impulse response is the reverberation that the room gives you in response to an impulse - ideally that impulse should be an infinitely high pressure for infinitesimaly small time, but you may use a gun shot, balloon pop or a specifically constructed noise (sounding like white noise) to calculate it. And then you convolute the guitar track with the cathedral response and here you go - you have a track of your guitar playing in the said cathedral. And then of course it's done not with the pure convolution, but rather multiplying the FFTs - which are in and of themselves important for audio as the sound spectra.

[@nicolasoche7993]

A Laplace transform video would be great. Congratulations for your excellent content.

[@MSDhoni-pz5wc]

Sir, whenever i get to hear your voice, it feels like magic! Like the neurons in my mind blow up that we are gonna learn something new! Thanks for stimulating our curiosity sir! 😊 you are literally the best in the world

[@Mahdi-gz3fk]

As an audio engineer with 12 years of experience, I have heard of Convolution Reverbs all my life. Now I have a better understanding of how they work

[@luisfernandooliveira5301]

Sincerely one of the best channels I have found so far. I have ADHD and keeping attention sometimes (Always actually) even on something I like is somewhat challenging. All the elements in your video are perfect to keep my attention. The music, the way you speak, the synchronized animations, even the speed with which you explain. Congratulations on your work!

[@pyrobeav2005]

For any other ADHD afflicted peeps: have you tried videos at double speed? It's built in to KZbin and some others (Audible, Netflix). I've found that at double speed it requires all of my focus to keep up; it really helps me not get distracted. Have to dial it back to 1.5 for 3B1B videos due to the density!

[@genessab]

I’m in the middle of a class on quantum field theory and I was literally asking just yesterday “well..what exactly *is* a convolution?” As always, the best math KZbinr out there has got my back :)

[@ashwinnicholas497]

3B1B never ceases to amaze. Truly one of the best channels on youtube.

[@miroslavstevic2036]

We learned the convolution at the University, but it was just another formula in a sea of formulas. This video explains everything much better. Thumbs up!

[@rttuo1259]

I've been studying EE and Physics for a while now, and I had my mind completely blown away after less than four minutes, when that dice analogy clicked. Absolutely amazing, thank you!

[@antoineprosper8645]

Amn, I wish those videos had been around during my undergrad, you make some brain twisting concepts appear in such a clear and motivated light, I have been following your channel for a few years now and I truly love what you are doing. I sincerely thank you for this, and for all the content you make

[@kidgroovie]

This used to be a nightmare to wrap my head around during college, thank you for creating this

[@saisiril96]

These videos are gonna be here till the end of time, inspiring millions of students to pursue mathematics and engineering.💯

[@akanksha0613]

As an electronics major, I have always faced difficulty to visualise convolution to Images. Thankyou so much for this amazing explanation.

[@ElNachoMacho]

Excellent video! I have to admit, I haven't heard FFT in over 20 years since I took a DSP class in my undergrad studies, that brought up some 'interesting' memories lol. Thank you for putting together this amazing videos. It sheds a new light on how CNNs in deep learning work.

[@giovannironchi5332]

There is a form of convolution in category theory also, it's called "Day convolution"; hope one day you'll introduce category theory to the world with your amazing style!

[@김영준-i1u4z]

This is excellent!!! More people should watch this video.

[@mjrmls]

I love the recent wave of "DSP explained" videos in the YT education space, hope to see more!

[@samcarswell9890]

Ooooh if you have any recommendations for any other videos, I'd love to know 🙂 Really enjoying diving further into DSP at the moment!

[@ti84satact12]

What is DSP?

[@samcarswell9890]

@@ti84satact12 Digital Signal Processing :)

[@lawrencedoliveiro9104]

Next: how to explain to all those computer graphics noddies that “pixels are not little squares” ...

[@mjrmls]

@@samcarswell9890 check out Veritasium and Reducible's recent videos on FFTs, + stand-up math just posted a video about using convolution to identify whales

[@Yone.Yovender]

Thank you sir. What a video! I've been stucking with the convolutions for months. Luckily youtube recommended your video. Thanks youtube algorithm

[@antonioprovolone2815]

I’m an aerospace engineer and I’ve been working on a Visual Odometry project to estimate the trajectory of a rover through a video taken with a stereo-camera. I had some trouble understanding the theory behind different detectors (Harris, SIFT, etc.), especially the convolution operations and the Kernels definition. I have to say, this video has been extremely helpful, the animations make it so clear and simple, so thank you for giving us such amazing content!

[@kyay10]

This video feels a bit different, idk how to explain it, but it's a nice, refreshing change. Keep it up!

[@stevenclarke2655]

I'm glad I'm not the only one who feels this way. I am a huge fan of all Grant's videos, but there is something about this one that is just making it "pop" a bit more for me. I'm still trying to figure out why that is.

[@MaxxTosh]

I can’t thank you enough for the way that you are creating the next generation of math geniuses and chipping away at the disdain math gets among students, don’t ever quit! ❤

[@hitarthk]

I've seen the video by Reducible over and over and every time it feels so satisfying to see how many things just fall in place just the right way. It seems almost like a conspiracy in the favor of mankind.

[@gwenturo9550]

0:48 The only time i think I've ever heard 3b1b say the word _"um"_ in a video. Dude your speech is immaculate.

[@gusthomas6872]

I am doing image processing research as an undergrad and I have taken linear and discrete but never learned about convolutions. This video helped me when I got to a python function called "signal.convolve2d" and I couldn't understand the documentation. Thank you so much!

[@iamsushi1056]

Oh, boy, I can’t wait to fftconvolve all my matrix operations

[@hobbified]

When I was a kid in the 90s I had a computer and a copy of Paint Shop Pro (long before Corel bought and ruined it). It had a bunch of built-in filters, but it also had a "Filter Matrix" option that would let you put in an arbitrary kernel, up to 5x5. The manual explained what it did mathematically, but not really anything on how to use it - just sort of "you'll know if you want this" - but the first things I figured out were that something like [[1, 1, 1], [1, 1, 1], [1, 1, 1]] would blur an image (there was an option to automatically divide the kernel by the sum of its elements, which made things easy when you didn't know what you were doing), [[-1, -1, -1], [-1, 8, -1], [-1, -1, -1]] would do an omnidirectional edge detect, and things like [[0, 0, 0], [0, 1, -1], [0, 0, 0]] would do an edge detect in just one direction.

[@MathiasVerhasselt]

ohh I remember this! as a kid I didnt really understand what was going on but it was fun to play with!

[@AW-mv2wq]

that sounds so cool!

[@peterboneg]

Something I find amazing is how image convolutions / correlations can be done entirely optically because of how an image encoded in a laser beam can be Fourier transformed when passed through a lens. It doesn't seem possible that all that maths gets reduced to lenses. A follow up video to explain that would be great!

[@donmoore7785]

This is excellent. Back in the late 1980's I worked as an engineer for GE on an airborne infrared search and track system. We implemented the spatial processing with a convolution to detect point source targets (aircraft, missiles). I wish I had this explanation at the time to help understand what was going on.

[@wcweb5574]

I love all the 3b1b stuff. Thank you for it. Apologetically, I will offer a picking of the tiniest imaginable nit: at around 3:21 instead of "And if we slide that bottom row all the way to the right" it should be "... to the left". I'm really not a jerk, just appreciate the work and thought you may want to add it to your errata :)