The Spectrogram and the Gabor Transform

Here I introduce the spectrogram, which is a moving-window Fourier transform, giving insight into the time-frequency content of a data set.
Book Website: databookuw.com
Book PDF: databookuw.com/databook.pdf
These lectures follow Chapter 2 from:
"Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control" by Brunton and Kutz
Amazon: www.amazon.com/Data-Driven-Sc...
Brunton Website: eigensteve.com
This video was produced at the University of Washington

[@AllElectronicsChannel]

Is the Gabor Transform a special case of a STFT ? What are the tradeoffs of using other windows functions in place of the gaussian?

[@vinilsimoes]

You never cease to amaze us Dr. Brunton!!! Keep up your magnificent work!!

[@JulioDiaz614]

Thank you Dr. Brunton for your always insightful and inspiring lectures. I may find a way to use this in my research

[@Eigensteve]

Wonderful!

[@bobspianosbffl]

I was trying to understand spectrograms recently and finally thanks to this video it clicked! A few clear visuals do a wonder for elucidating the maths and concepts

[@neb5615]

Thanks for all lectures, I really appreciate your explanations

[@Lrexmo]

The description is great, the ideas are clear and the logic is coherent. Thanks for your work.

[@Eigensteve]

Glad it was helpful!

[@hugeride]

This is just amazIng. He can pass the overall concept in 10 minutes better than one can read in books in 1 hour. Conceptual understanding is crucial to guide learning. After understanding what this concept is about and where you are within the topic and how can it be used in practice, its much easier to absort the material and guide the learning by connecting the detailed concepts you learn afterward, by digging deeper. But having this overall knowledge is essential and most books don't give that.

[@erikgottlieb9362]

Thank you for clear, concise, organized presentation. Appreciative of how much time and effort such a presentation / explanation takes to create and deliver. Appreciative of the format you use and precision in getting explanation correct. Explanation of terms and where terms originate has always been helpful in your presentations. Thanks again. (Erik Gottlieb)

[@succirasuccira]

Dr. Brunton's concise explanations of all these transform and compression algorithms are first class, and the visual here just incomparable! In this video, on the topic of Shazam's algorithm, you mentioned that it has the caveat that: when the song is stretched in time it makes it harder to match peaks in the power spectrum. This got me thinking about the other dimension: a slightly transposed (pitch shifted) song also breaks the algorithm given the spectrum was measured in fixed frequency. This could be me imagining things but: it might be useful to have spectrum that measures relative frequency. That way you can match songs even if it's transposed to different keys.

[@RealMcDudu]

You're really an amazing teacher! Explained very clearly. Shows you also understand it very well.

[@zoheirtir]

Thanks a lot for your inspiring lectures ! Zoheir TIR Algeria

[@hunters.dicicco1410]

i wish i had this when i was first learning the math required for signal processing. great stuff!

[@twangist]

Thank you Professor Brunton, you're really excellent at this.

[@pieterjoubert4346]

Thank you Dr. Brunton! Clear and concise. Liked and subscribed.

[@gamedeeds]

teaching tech is beyond this gen keep it up prof, very useful and understanding

[@dragoncurveenthusiast]

You are such a good lecturer! Thank you!

[@lancelotdsouza4705]

Thanks Dr Brunton ...The Gabor transform was very well explained ,,,,needed the code for the same

[@davidtolle9533]

The bit about Shazam using the power spectral density property to accurately identify songs was interesting. Thanks for the content

[@spkt1001]

Best video for understanding the intuition of spectrogram!

[@danielnagy6360]

Awesome videos, really great content and great quality, and also a great topic.

[@AnilAgiral]

This is really neat!

[@jushkunjuret4386]

This is a wonderful lecture!

[@marcogelsomini7655]

Wow that's awesome! Thank you for introduce it to me Dr. Brunton :)

[@BruinChang]

I like the whole series very much.

[@geraldheinig1473]

Excellent explanation. I'm particularly happy about the mention of how Shazam works: that's something that's intrigued me for a while now. Thank you!

[@-E42-]

I like the creativity with the transparent wall between the lecturer and the camera, on the other hand the presentation seems strangely surreal due to the fact the presenter is only visible as floating head/shoulder and arms

[@mehdis.7404]

Excellent quality!

[@nisanaryal156]

I really love this video. I am working in audio classification and I have learned the basic about Spectrogram(STFT), Mel scale and mel spectrogram, MFCC, Consant Q transform etc but I still cant figure out which spectral representation should I use at which condition . Apart from the representation there is the selection of window length and the hop length of the window (trade off between temporal and frequency resolution). At the end of these series I would love to see the comparison and your view on these different representations.

[@abhishek_sengupta]

Very nicely explained!! Thanks!

[@rodrigomesquita8741]

Isso é uma das coisas mais lindas da engenharia. Fundamentalmente você vai calcular a transformada de furrier para janelas de tempo específicas, e vai poder ver quais as componentes de frequencia naquele instante!

[@josueprieto7371]

Beautiful explanation

[@Anorve]

I love your videos and explanation

[@aalselwi]

I am enjoying your trip of learning process

[@Eigensteve]

I'm so glad!

[@udomatthiasdrums5322]

love your work!!

[@Thejus_5511]

Great explanation

[@abdellahsellam912]

Thanks, this video was very helpful for me

[@kaxxamhinna5044]

Thank you very much 🎉🎉 you saved my weekend 😂 Have a great day

[@trinetram707]

Great video sir please keep posting such videos

[@yuep6808]

I can't help but think about how you do inverted writing so well 😂

[@tech01q]

If the music is shrinked or stretched, it should still be easy to recognize the music, if the program is adjusted according to the percentages of time intervals between peaks of sound, than a unique pattern can be generated. This has a great potential in the future… Thank you …

[@mohamadhamoudy8232]

Thanks a lot Prof. Steve , please could you upload a video for using Spectrogram on sound classifications and feature extraction , regards

[@user-px2nq6rk9i]

Please video on Mel spectrogram and why it can't be reversed, thanks for the book and the videos.

[@qilinwang5889]

Hi Steve, I can't thank you enough for making these beautiful videos. I have purchased the book as a way to say thank you. The book is beautifully printed, and if I can give you some feedback from a reader's perspective, I would like the book to have a larger font. They are too small to read for a long time. Anywaysl, thank you for your work!

[@brendawilliams8062]

I work with number theory. The teacher is powerful.

[@danielparra6902]

Thank you very much for the clear intuitive explanations Dr. Brunton. I was wondering if there is a Gabor transform analog that uses a data driven approach like the SVD? In the case of SVD would the power spectrum change along with the basis or should one compute the basis with the whole signal and only then with a fixed basis apply the transform?

[@StefanT41]

Wow ! Thanks a lot!

[@SRIMANTASANTRA]

Hi Professor Steve, Nice.

[@Eigensteve]

Glad you like it!

[@Jo-ce6gd]

Perfect! thanks

[@banggiangle8258]

best explanation ever!

[@Eigensteve]

Wow, thanks!

[@sandras2624]

... i'm just silently wondering why this was in my recommendations; i am a social science major and i spend most of my time here on youtube watching cat videos. oh, the yt-algorithm. however: keep up the good work!

[@JohnVKaravitis]

SUE KZbin! You've been scarred for life!

[@yousifyahiaahmed8206]

Thanks so much indeed

[@NaveenKumar-gs8xn]

Thanks a lot ...

[@Yang-YTchannel]

Super good explanation!! May I know how do we get the power information if the y-axis is frequency and x-axis is time? Like how large the signal is for each frequency at an instant time?

[@zhihuachen3613]

like your video, especially programming in both python and Matlab

[@Eigensteve]

Glad you liked it!

[@aefieefnvhas]

Amazing

[@user-ml6gr1rk1m]

Thanks a lot for the series of videos. They are very useful for my projects. How about S-transform? Thank you again.

[@Eigensteve]

Thanks! Maybe I'll make one on the S-transform sometime.

[@Via.Dolorosa]

thank you

[@GabrieleNunnari]

This videos have an incredible quality, really. Content and graphical. My only real question is: how are you able to explain and write mirrored making it look so natural!!!

[@evanritchie6195]

You can record the video with backwards writing, then mirror the video in an editor afterwards. :)

[@pavelkonovalov8931]

Thank you so much for your labour. Do you mind to make a video on harmonic distortion?

[@Eigensteve]

I can't promise I'll make one, but I will add it to the list.

[@bhaveshamarsingh1656]

Hello Steven sir, I have gone through wavelet transforms back in the day and i wanted to ask that is it not similar in the sense that they too have evolved/developed because Fourier Transform fails to specify the time at which certain frequency occurred in the original signal. And moreover, please do bring up a short video lecture series on wavelet transforms as well. Thank you.

[@philipq6906]

Great explanation can’t wait for the next vid. BTW this is just like a music score. The wave are decomposed by windowed Fournier transformer. I am wondering in real control or identification system, how do we update the realtime signal? We cannot wait a long period and the windows g(x) size also matters. How do we choose a appropriate length

[@Eigensteve]

You are right -- and yes, in control applications, the spectrogram will be computed continuously with a sliding window.

[@aliadams516]

Wonderful :)

[@Eigensteve]

Thank you! Cheers!

[@jm3415]

what are the HUP implications on time and freq uncertainty for the Gabor Transform?

[@haowang4306]

thank you for your lecture. how to make this kind of video in which the drawings can be shown in front of lecturer?

[@lorenzosparacio1184]

Beautiful video! I have the following questions. Why does the weight function have to be a gaussian? What would happen if this function is, for instance, a constant of unitary value (so, I'm applying a wight of 1 across the entire window)?

[@shashidharmuniswamy2620]

How do I locate the fundamental frequency at that particular instant? and what do I do to find the ratio of the harmonics to the fundamental frequency as it evolves with time? :)

[@carlossama2191]

This is sweeet! How can this be applied to voice recognition? Go dawgs!

[@Eigensteve]

I think so. Modern voice recognition uses recurrent neural networks, but the spectrogram can be very useful here too.

[@udayanbanerjee5271]

Dear sir Is W vs t a continuous function? Can we identify the nature of the change in frequency and then invert it back to get that part of 'f'?

[@Space-Audio]

I have a few out-of-this-world examples of spectrograms you just might be interested in.

[@ffelixvideos]

I would like to know: what are the advantages of this spectrogram over the STFT algorithm or the mel-spectrogram?

[@MrKrvo]

Mr Brunton said some a little misleading information. Gabor transform was the first time-frequency representation of signals, it is special case of STFT (because STFT (Short-Time Fourier Transform) can have any sliding window, Gabor transformation is done using only Gaussian window; there is also Triangular, Hamming, Hann, Blackmann and many others). Each of them has advantage and disadvantage - e.g. wider first lobe in frequency means suppressed the others and vice versa. It always depends on the use and purpose of the analysis. For audio signals, the most common window is Hann window (it doesn't have so sharp edges, e.g. like triangular, it is made by harmonic function - cosinus, so it is more smooth but its also wider for the first frequency lobe). The most common spectrogram is computed with STFT (Fourier T. in general), not Gabor transform. Mel-spectrogram is little different because it uses "mels" - Mel is a unit from psychoacoustic for a subjective melody; it also uses cosine transform but in short, it is another time-frequency representation of a signal and it tries to simulate or imitate human hearing and musical perception. Mel-spectrum (and kepstrum) is commonly used for research purposes of MIR (Music Information Retrieval) because the signal representation is usually closer to the subjective aspects of human hearing and thus is better for most of the applications (so far).

[@ffelixvideos]

@@MrKrvo Ok, I got it. Thanks for your time.

[@MrKrvo]

@@ffelixvideos No problem.

[@Eigensteve]

Thanks for the great extra information!

[@tobi3497]

@@MrKrvo Thanks for this !

[@mohammedal-haddad2652]

What about the width of the Gaussian function?

[@wesleytaylor-rendal5648]

How to find the time resolution? Look at the width of the Gabor function?

[@miguelfernandesdesousa7784]

is there any difference between Gabor Transform and STFT? Is it just a particular case with a gaussian window and unitary gain?

[@neuronneuron3645]

This is behind the uncertainty principle?

[@trinetram707]

Doesn't a Gaussian window alter the frequency content of the signal ??

[@amruthgadag4813]

So basically it is the short-time Fourier transform-based spectrograms. Please reply if yes or no.

[@trinetram707]

Why do we use Gaussian window can't we use a rectangular window ??

[@numoru]

i know its late but we ahould definitly have two, since left and right channels/phases can be different . would be a good extension. yYet, even more so 3d phases allow for darn near infinite phases so would lobe a transform that splits that u0 based on color and intensity to see the intricacies. I mean if were not talking polyphonic it should be possible, monophonic yeah . but honestly I dont get why its a problem for multiple phases and poly phonic if we take each initiation of intonation as a phase of a fractional phases that doesn't multiple/algorithmically interact with/ into the greater whole/nor parts (if given the total data first). Further, this is based off of first principle in the idea that we can generate moire patterns even with aperiodic data (Glass patterns) easily yet it is the microstructures (local) that we aren't able to inversely appropriate/segment, yet if the phase data is there we should be able to,..-wrong?

[@numoru]

typos will not be dealt with, it to much to read. so guess

[@JohnVKaravitis]

Zsa Zsa or Eva Gabor?

[@carlosvargas2907]

You rock

[@Kong9901]

So the spectrogram just shows the frequency played at each time, but there is no information about the amplitude of these frequencies ?

[@Kong9901]

I got my answer from the next video. Thank you :)

[@arnabprophet]

Did anyone else realize that Dr. Brunton is writing backwards on a screen?

[@AbhilashIngale]

Pleased to meet you, the Real Men and Women of culture ;)

[@rolandstefan5204]

Fun fact: Gabor transform was named after by Gábor Dénes who was a hungarian physicist and electrical engineer and he also got the Phsics Nobel Prize for inventing holography. Sorry for the grammatical mistakes.

[@melihozcan8676]

@11:50 I miss the times when KZbin had dislikes, and we had a chance to avoid the garbage...

[@Ting3624]

so one can technically write an algorithm to listen to music and generate sheet music.... holy there are plenty after google it

[@Eigensteve]

This is indeed one of the big open challenges that people are working on. Can you imagine if researchers could create an algorithm that would generate new Vivaldi?

[@leif1075]

@@Eigensteve what exactly is the open challenge? If you can write an algorithm to write mew music from scratch? But it would probsbly just be random stitching together of notes from existing pieces wouldn't it?

[@treksis]

@6:45 low 22222 high~~~😁😁😁

[@kelvinxie1029]

Music sheets are spectrograms from Gabor transformation LOL

[@v8pilot]

Did you mention the Gabor transform? I must have dozed off if you did. I've been re-reading his paper and want to understand its relation to the spectrogram.

[@frankservant5754]

KZbin University Freshers where you at!