Convolutions are not Convoluted

EXTRA NOTES:
-The convolution matrix is a Toeplitz matrix.
-If the bakery's impulse response was to give cookies periodically (e.g. giving cookies every week), you would get circular convolution.
-Another intuition of flipping the impulse response is that if you DIDN'T flip, the system would be using signals from the future input to calculate the present output, which is impossible in a time-dependent system.
- Fourier Transform video mentioned in footnote at 8:08: • The Linear Algebra of ...
- Origin and History of Convolutions: www.slideshare.net/Alexdfar/o...
Timestamps
0:00 Introduction
0:47 Bakery
1:38 Fancy Word #1: Impulse Response
2:13 Fancy Word #2: Time-Invariant
3:21 Fancy Word #3: Linear
4:12 pausing/replaying highly encouraged here
4:39 Linear Algebra
8:56 Fancy Word #4: Convolution
9:17 Practical Applications
Clips:
Oscilliscope Footage: • EEVblog #396 - Bode Pl...
Conv. NN Animation: • Convolutional Neural N...
Fourier Series Animation: • Fourier Series Animati...
Pascal's Triangle GIF: daugerresearch.com/vault/para...
Circuit Animation Website: www.falstad.com/circuit/
Tacoma Narrows Bridge Collapse: • Tacoma Narrows Bridge ...
Music:
Winter Lane (Remix) - Nujabes
Investigation Core 2001 - Ace Attorney
Sweet Mystery Galaxy - Super Mario Galaxy 2

[@BoZhaoengineering]

This is the most understandable convolution interpretation I have ever seen on KZbin. Just awesome.

[@Michallote]

I went through university and this was way more understandable than whatever we were taught

[@kartikvishaldeshpande7422]

Many hours put into understanding this concept during my undergraduate days, watching your 10-min video now provided far better understanding. Simple and to the point explanation. Thanks, and please keep updating with intuitive teachings. Great work !!!

[@blisslen]

Wow, one of the best explanations I've heard of one of the trickier concepts in signals classes (one that you really need to understand going deeper into DSP or anything). Hope this was submitted to 3blue1brown's contest, this is definitely a contender.

[@sigfyg8384]

It was submitted to the contest, but I already know I'm not in the top 100 lol

[@TheRookieNerds]

Omg you are criminally underrated

[@thunderstriketrading]

This is underrated educational content! Great video!

[@surendrabarsode8959]

Brilliant. Very well explained!

[@utkarshmishra7416]

I was surprised to see only 565 subs, keep going, with the quality, i hit sub in like 30 seconds into the video. This is a major channel with millions of subs in the making if you keep makng. Keep up the good work

[@tymofei8586]

Very good and intuitive explanation. As a mechanical engineer who pursue into mastering control theory and systems,sometimes i find its math difficult to grasp intuitively but with those cookies you made it just so easy :D thanks

[@blacklightning7227]

Thank you for making this.❤

[@ghassanehajji7591]

This is an excellent video, you have a real talent for explaining the intuition behind "convoluted"" ideas. Please continue making explainers. I'll be eagerly awaiting the next installment, and hopefully you'll make a series exploring the topics you listed at the end.

[@kaemmili4590]

yea please , its awesome, and feels like a nice fresh breath =) edit : the last sound effect made me hit the bell button .

[@MartinBurg0s]

Thanks for making this video, now I finally feel able to explain this to someone else!

[@danielweber4917]

Love the phoenix wright background music; absolutely fits the tone!

[@biffbum8221]

This is the video that lighted the bulb 💡! Much appreciated keep going ❤

[@sebastiansanfunas4674]

This video is amazing, beautiful explanation: instant subscribe

[@SDX9000]

Outstanding video. Thank you.

[@praveenb9048]

Congratulations, Sir!

[@sauravgupta92]

The example is Gold!

[@commonsenseinmathphysics3560]

Great video! Really liked the humorous presentation;)

[@user-vp6dd1fz6p]

Cool explanation, thanks

[@aamirm]

Great explanation. Thanks!

[@4733R70]

man you just proved that maths can be much much easier if explained from people with your talent!! if you like it keep going with your videos!!!

[@aadiduggal1860]

thank you bro, this was an excellent video.

[@shimaalcarrim7949]

Excellent 👌

[@chrissysevigny2462]

I watched two other videos on convolution prior to this (one of them being the fairly notorious 3Blue1Brown), and I think this helped me understand the concept of convolution the best. It just seemed like it stripped away all of the (in my opinion) more challenging vocabulary, and brought it down to layman's terms. Thanks for sharing!

[@omaralamin1948]

God bless you and others like you.

[@Bravo_L]

Im in signals and systems and we doin this stuff rn -.- This is sooooo good btw kudos!!!!!!! Ee major here

[@FinalDestinationAllah]

Best and Best video Love you So much❤

[@alexsere3061]

Subscribed after watching this and the linear algebra matrices, i'm gonna share it with all my engineering classmates

[@bolhasani]

Really helpful, thanks a lot.

[@elyepes19]

A convoluted million thanks and subscribed! Thanks for the cookies as well

[@h2ogun26]

some might say it's distracting, but the selection of back ground music was aesthetic and well reconciled.

[@exoticcoder5365]

I love the background music as well, it helped me focus

[@ozif-csgo7746]

Awesome video

[@TheJayenz]

Great job!

[@MrPabloguida]

Very good.

[@timurdosmurzayev6220]

man, you are genius indeed!

[@pitiwatkittiwimonchai4656]

excellent great sir (Thx for explain !!!!)

[@Mridul_Paul]

Great video man....

[@miguelramirez187]

Thank you so much

[@radicalsaled5756]

amazing video. please make more videos on sytems and signals

[@okal7706]

Great video!

[@skatomort]

The only thing I'm convoluted atm is the number of subscribers ... Well done!

[@X.C.11]

Great video sir

[@MrFischvogel]

Thanks a lot !!

[@nabeelsherazi8860]

Holy shit the moment the meaning of linear time-invariant clicked for me... instant subscribe. I don't care if you never make videos again, I have NEVER understood linearity like that.

[@TheFolkRevival]

I have never seen the output on the left in LA, the equation is usually written Ax = b in most literature

[@lowlevelgamedev9330]

Very nice video

[@kevinmathewson4272]

For anyone confused: at 7:36, each term in the c vector is really a summation. The first term, for example, should be read as "sum, for t values 0 through 6, of the expression: d(t) x b(-t)" where the x is a multiplication symbol. The second term should be read as "sum, for t values 0 through 6, of the expression: d(t) x b(-(t - 1))." And so on.

[@TheZ10Z]

Really good video

[@albinjose8272]

Wow😮...really get the idea

[@xandersafrunek2151]

In my entire study of linear algebra, I have never put the output on the left. But it looks good there, really!

[@sigfyg8384]

I'm used to putting it on the left because I use "state space representation" of diffeqs a lot. Also, when you assign a variable when programming software, the variable you want to assign goes on the left, and therefore, when you model these equations in code the outputs go on the left.

[@xandersafrunek2151]

@@sigfyg8384 Yeah. I like that notation because it reminds you that the matrix-vector product is just a vector. Or that a scalar product simplifies to a scalar value.

[@holomurphy22]

I also put it on the right, never seen otherwise. I study math in France

[@smolboi9659]

Extra Question 1 and insight into Question 4: We can express polynomials in 2 ways: (1) (n+1) coefficients uniquely represent the polynomial of order n. (2) evaluating the polynomial at (n+1) points uniquely represent the polynomial (see lagrange polynomial). For example, 2 points uniquely define a line, 3 points uniquely determine a quadratic... In the coefficient representation, to multiply polynomials P and Q, we convolve the coefficients of P and Q to get the coefficients of the product. In the evaluation representation, we multiply the evaluation of P and the evaluation of Q to get the evaluation of the product. These 2 representations are fourier transforms of one another.

[@sigfyg8384]

Nice! I was actually thinking that if you treat use the fourier transform's discrete version, you can apply that to the convolution. You'll start to see the terms line up in a way that makes frequency multiply. I never expected it to connect back to question one.

[@fail_fast]

Thanks!

[@Alexander-oh8ry]

The step from the finite to the infinite integral representation with infinitesemals is missing. This is a very good explanatio for beginner´s understanding, but without that last step it is heavily incomplete. Just show how the convolutions is the overlap of functions shifted by a different amount

[@lounes9777]

hey i am the 1000th subscriber !

[@wellid5163]

Finally, I found out that Background music is the reason why I am not interested in math.😆 Great video!

[@alfderbabybenz7092]

our professor is using this to explain convolutions in our signals and systems course.

[@themightyquinn100]

Great video. One thing that is confusing to me is how you are taking the dot product of d and b. d is defined as a 7 element vector and b is defined as a 4 element vector. How can you take the dot product of these two vectors if they're size is not the same?

[@mendelovitch]

For a little while there were no obnoxious memes, but then he had to add them. Alas.

[@anishsharma6702]

amazing , and my brain seems to get it

[@ranam]

Brother you are correct for linear time invariant systems that they stack and scales response but in filters ther are used to attenuate some frequency can you please explain them with an amplifier used in some filters using convulution

[@ranam]

Telling the application of convolution in terms of signal processing may help a part of people in the telecommunication and eee engineers but when convolution meets signal processing it also is used as a filter but my question is I have read convolution even it's useful for mechanical engineers such that in resonance when a failure occurs not due to massive force hitting an object results in large deformation could cause failure or an large impulsive force acting on it for a duration of time could cause failure but there is an another phenomenon where the natural frequency of any object is reached the energy builds in it very high and could cause a failure in this manner a small disturbance which accumulates over a time and causes a high energy to build in the system due to energy very high it causes stress and the system collapses this is highly different from stability perspective of control system being not stable does not mean it's accumulating energy inside it but in case of amplifier there is an capacitor or inductance device which causes the attenuation in the electrical signal and filters some frequencies but in other perspective amplifier amplifies the signal such that it stack ques and scales the signal but I don't know this is done by capacitor or am inductor but convolution is useful to both mechanical civil eee ece and every applied scientist and engineers hence it's used as a filter in an circuit or used to amplify but even transistor amplifies the signal without an capacitor or an inductor I guess also mechanical engineers can use it to model resonance hence the energy inside the system build high by periodic accumulation of the system reaching its natural frequency which leads to failure and I can also tell you that when amplifier filter or amplifies the signal it used convolution hence it's useful to every applied scientist and engineers but not to mention the pure Mathematicians use it of convolution of kernels thankyou guys some of my inference could be wrong if somebody or the author of the video is familiar with it please correct the above and educate me thank you for the wonderful video sir

[@ranam]

Brother I know mechanical engineers could find resonance but when I had a deep thought on this resonance Is an slow accumulation of energy which is accumulated very high in small installments when the frequencies match if you strike a turning fork of 50 hz you get the same frequency of vibration on another tuning fork so they both vibrate if you strike it harder the amplitude changes hence loudness is a human factor the frequency is the same the languages that human speak through out the world the sound only resonate your ear drum for few seconds my question is that the harmonics is the fundamental frequency and overtones are the frequency that follow it take a word in any language you spell it according to convolution the thing scales and ques and stack the signal so convolution can be used to model resonance so when your ear drum vibrates it vibrates so the electrical signals are carried to brain like tuning fork ear drums vibrate within the audible spectrum 20 hz to 20000 hz hence resonance is caused by the word we speak and within the audible range the ear drums vibrate and we make sense of words I have seen in one videos on KZbin that due to harmonics in any sound causes resonance which could be modelled by convolution recalling the resonance its destructive because slow and steady accumulation of sound on the mass causes high stress and high energy to build inside and stress increase and the system fractures or collapses but our ear drum hearing the sound from human languages try to vibrate but why our ear drum when subjected to continuous exposure of sound does not fracture or rupture like a wine glass iam not telling about high loud sound higher than 80 db but a audible range sound within the frequency of 20 hz to 20000 hz under continuous exposure why it's not damaging it again not failure by high energy but low one in synchronisation on air . But I tried it in my students when I told them to be quite in class they did not listen to me so I took my phone and set an frequency 14000 hz and they told it was irritating the idea of resonance is "small effort but large destruction " just like Tacoma bridge where the wind just slowly accumulated energy on the bridge and it collapsed it so my conclusion is if an audible frequency at continuous exposure to an human ear can it cause bleeding again "small effort but great destruction" sorry for the long story I you are able to reach hear you must be as curious as me so still not finished the ear drum is shook by harmonics in the sound we make by the words( or )overtones in the sound we make by the words I know harmonics is the fundamental frequency and overtones are following it which under slow and steady accumulation of sound energy resonates and could damge the ear drums again "small effort but big destruction" not to mention we assume the person is in coma or brain dead hence when the sound irritates him he or she could not make a move so my question is so simple normally human ear responds to harmonics or overtones according to convolution which could be a disaster but with minimal effort 🙏🙏🙏🙏 at here I could be wrong because harmonics can also be used to construct sound so can it be destructive or the overtones which are the trouble makers and which one according to this gives a response curve when two signals convolved by harmonics or overtones which is destructive but with minimal effort and convolution happens when ear drums oscillate is by harmonics or the overtones or also the trouble makers there

[@lzh00]

The beginning killed me😂😂

[@simonegrandinetti7306]

when the convolution appeared i went "whoah"

[@mrpachotis]

nice

[@CodicologiaUCM]

The impulse function is b(t) or d(t)? Thanks. Great video.

[@sigfyg8384]

The impulse function/response is b(t). You could call d(t) the total response, which is made up of many impulse responses.

[@milenamijuskovic2340]

excuse me but where is the next video? PLEASE continue making these:D

[@sigfyg8384]

I might have a new one ready by the end of December.

[@tulio829]

Me focusing on the lesson Also me: isn't that ace attorny soundtrack?

[@Speed001]

7:35 I don't really understand this step. I see the relationships, but why do you end up subtracting?

[@sigfyg8384]

f(-t) is f(t) but flipped around the vertical axis. f(t-1) is f(t) but shifted forward by one. Combine these two transforms, and you get f(-(t-1)), which is a flipped and shifted version of f(t). It's ok to be confused here. I suspect that the source of the confusion is trying to comprehend why the flip and the shift are not commutative. To see what I mean, compare f(-t-1) vs f(-(t-1)). f(-t-1) Flip first, Shift second f(-(t-1)) Shift first, Flip second When you switch the order of the transformations, you get two different results. Now you are thinking along the lines of Group Theory, where this question of commutating becomes important! Another note: I could've distributed the negative sign in and got f(-t+1) or f(1-t), but that makes it harder to see the flip and shift. Different representations of the same phenomena can tell two different stories. For example, a/b and a*(1/b) are mathematically equivalent, but they communicate two very different ideas. Similarly, f(-t+1) and f(1-t) don't communicate the idea I want to emphasize as clearly as f(-(t-1)).

[@viniciusgoncalves1238]

First off this is the best video about this ever. But I have one doubt, at 7:45, when you say c(t) can be written as d(t)*b(-t), it's weird because that doesn't depend on past imput. compying your first bakery, let's say d(4) (dollars given at day 4) is 1, and b(-t) (cookies given at time -4?) is zero. The result would be zero cookies yet you could've given a dollar before that is just now receiving cookies.

[@sigfyg8384]

It is not possible to receive cookies from the bakery on day 0 for a dollar you give on day 4, UNLESS the bakery can predict the future. Last I checked, bakeries do not have divination powers, so c(0) cannot depend on b(-4). Since the impulse response b(t) get's flipped around to b(-t), the negative sign actually refers to a point in the future. The LATEST part of the impulse response interacts with the EARLIEST part of the input signal, which is why you need the flip.

[@CyberGamer1539]

yes they are

[@martinsanchez-hw4fi]

Where do you make the animations?

[@subtlethingsinlife]

3b1b recommended so I had to come

[@marshall7253]

Thank you so much for this, you are a fuckin g

[@Speed001]

Is this just matrix multiplication in a different form?

[@sigfyg8384]

What do you mean by "different form"? The addition 1+1 = 2 is technically a matrix multiplication of the vectors [1, 1] and [1, 1] transposed (so a 1x2 times a 2x1). Does this mean that 1+1=2 is just matrix multiplication but in a "different form"? Now, that's not a convenient way to represent 1+1=2, but in other scenarios turning something into a matrix multiplication is super useful. So matrix multiplication isn't really a "form" as much as it is a representation. Here's a blogpost I wrote that digs into this some more: sigfyg.medium.com/the-philosophy-of-linear-algebra-ac2d9ce14619

[@Michallote]

@@sigfyg8384 thank you for the material!

[@nomercytv9733]

the like and view ratio !!!

[@brennanh7140]

Spooky music

[@hhehe24]

nujabes.!

[@notsober2030]

Lost it from the moment he said linear algebra

[@olivertaltynov9220]

I prefer the second bakery.

[@mariocolon95]

At 7:37 I think you made a mistake since c(t) should be reversed. For example c(0) should be d(t).b(-(t-6)) and c(6) should be d(t).b(-t)

[@sigfyg8384]

c(0) is supposed to be d(t).b(-t), as shown in the video. At t=0, the only part of the input d(t) that can have any impact on the output is d(0). When you suggest that c(0) should be d(t).b(-(t-6)), that means the system is somehow reading signals from the future!

[@mariocolon95]

@@sigfyg8384 Yeah, I guess i got confused because you said you took the rows of the matrix to determine c(t) so for c(0) you use the first row which is b(-(t-6)) and for c(6) you use b(-t).

[@brockobama257]

No linear algebra related class I’ve taken has the input, output on the right, left. I’ve only left, right. Are you smoking?

[@sigfyg8384]

Look up "state space representation". The derivative vector, which is the product of a matrix and the state vector, is written on the left side.

[@sigfyg8384]

Furthermore, when assigning a variable in software the variable you want to assign goes on the left. Therefore, if you want to write matrix multiplications in code, your output goes on the left.

[@shahinrab]

Do the 4 follow ups. I will pay you. Like seriously. Set up a Patreon page... now.