Shannon Nyquist Sampling Theorem

Follow on Twitter: @eigensteve
Brunton's website: eigensteve.com
This video discusses the famous Shannon-Nyquist sampling theorem, which discusses limits on signal reconstruction given how fast it is sampled and the frequency content of the signal.
For original papers:
Shannon, 1948: people.math.harvard.edu/~ctm/h...
Nyquist, 1928: bayes.wustl.edu/Manual/Certai...
This video was produced at the University of Washington

[@Eigensteve]

A viewer made the excellent point that we technically need to sample at "strictly greater than" 2*omega. For the example, if you sampled at exactly 2*omega at the wrong phase, you could easily get something that looks constant.

[@juliangomez8162]

The equality holds only for even functions, e.g., a cosine.

[@tushermyth]

This is actually the reason I came to look at this video and it answered that. Thanks!

[@tushermyth]

@Leland Hugh seriously? Is this the place for this? 😂

[@Anon54387]

4:20 It's a little bit nit picky, perhaps, but f is in Hertz or cycles per second. Omega is angular frequency and measured in radians per second. Omega=2*pi*f where the units on pi are radians per cycle and the units on f are cycles per second. The cycles part, being in the denominator and numerator respectively, cancel out leaving radians per second.

[@joaovitorjoaovitor]

Wow, I was just thinking about this having finished the video. Thank you for all the info!

[@hyperTorless]

it's crazy how clear everything becomes when the teacher knows how to make pretty drawings.

[@Eigensteve]

Thank you so much 😀. Who knew art was so important for math!

[@Via.Dolorosa]

I am really impressed by how you teach, your students are lucky. You make science really attractive. Thanks prof.

[@Eigensteve]

Thank you!

[@freecheese4143]

I have been a graduate student and have seen many presentations by Docs, PhD's, experts but this was a superb explanation and best backdrop.

[@its.bonart]

Why isn't this man getting millions of views?!

[@raghavinder2161]

How many people study signal processing??

[@Eigensteve]

Thanks -- I appreciate it!

[@antonkot6250]

actually in average around 1 million views for all videos per year for the last 4-5 years

[@nias2631]

@@raghavinder2161 compared to the total population they are likely sparse LOL

[@Pewpewlazorz112]

Fun fact: In Solid State Physics, the periodic spatial arrangement of atoms in a crystal basically "samples" the physical properties related to waves traveling in the solid. One then uses so called "reciprocal space" which is just a 3D fourier transform of the crystal for easy mathematical description. The Nyquist frequency in that 3D reciprocal space denotes the so called "Brillouin Zone", which plays an important role in Solid State Physics. For example, it dictates a condition on the occurence reflexes in diffraction experiments for determining crystal structures.

[@gooblepls3985]

cool! thank you!!

[@DJmates]

Steve, I really never comment on videos, but you are the best. If this can serve as a bit of motivation. please keep doing these amazing videos. You nail them every time. Let me finish by simply saying: Thank you!

[@celine-6934]

Thank you so much for your great explanations! You are making very complicated topics easy to understand - without losing information and oversimplifying it. Please never stop teaching :).

[@skaalduggery]

WOW...I have spent 3 years at university and they have failed to explain control/signals anywhere close to this. Love the energy dude!

[@Bombad2]

Great anecdote about why MP3s files are sampled at 44kHz. However, the extra 4kHz are there not because we can hear frequencies in the 21-22 kHz range, but because of how filters behave in practice. The extra frequency padding provides a good transition band for anti-aliasing filters. Also, love your videos btw. Been watching since taking AMATH 301 :)

[@_boris]

So lucky to have found this channel, top notch 💎

[@ashwiniganesh662]

Hands down, one of the best presented videos. Tools used + narration = Game on point.

[@Eigensteve]

Glad you liked it!

[@user-rt9pw2jk8r]

What he said and clarified at 10:56 made me understand what I could not understand from so many formal texts. Thanks a lot.

[@weir-doe3205]

You should conduct an entry level communication module or signal and systems, this is really a masterclass quality explanation

[@aswathik4709]

excellent teaching, i should say! I was troubling with sampling theory and cam across this video. Certainly this is a great kickstart for me to continue! Thanks Steve and please don't stop making videos!

[@juliangomez8162]

Thank you Steve for making the most of this COVID year.

[@DGSEM]

Your videos are really good. Keep up the good work. It's fun to watch your videos.

[@patrickayivor2406]

Very simple and clear explanation. Thanks Prof Steve

[@mhawan5784]

Great job by adding attractive background history. You explain nicely, and relate it with real life examples. I understand and learned something new because of you! Thank you!

[@andrezabona3518]

Soooooooooooo cool!!! OMG I loved it! Thank you Professor, you saved my year with your videos!

[@balajimk5067]

One of the greatest lecturers on youtube. If possible can you please take few classes on nonconvex, convex optimisation?

[@PirateFunk]

I never heard it explained so well!

[@steaminglobster]

Thanks for the video with intuitive. After watch your video, I think this way: Think the system as a black box, sample it at the 1X of highest frequency means you get a amplitude with no phase information, but with 2X sampling rate, you get the phase information, this amplitude and phase information gives you everything about this signal. It is like some aliens want to check what season the earth it is now, they need to check twice per year(they need to make sure within a year, the season). My two cents. Thanks again, I benefit a lot from your video.

[@afonsorafael2728]

Your classes here are looking really good!

[@Eigensteve]

Awesome, thanks!

[@jonconhaim6258]

Thank you for creating this very clear explanation. Very helpful!

[@devashreemsutre3339]

What a beautiful explanation and a beautiful teacher😊

[@chrisesprey6361]

The point at which frequencies fold is at omega over two e.g. Nyquist. In the PSD plot you show the folding to occur about the sample frequency of omega. For instance a frequency at 0.6*omega would fold down to 0.4*omega. Great video though with a great explanation. Subscribed - looking forward to watching more of your stuff!

[@rammsund]

You can get a flat signal if you measure at exactly twice the frequency. Imagine a pure sine wave at 1 Hz. You measure at 2 Hz and only find a flat signal. Then you managed to start measuring at the inflection points of the sine wave. Thats why I sample above twice the frequency. Excited about the compressed sensing series!

[@MikhailBarabanovA]

That's why Nyquist frequency upper border is not included in what it is possible to be measured without aliasing.

[@ericyip947]

How much higher than twice? Is it appropriate to do 2.5X or something? Must it an integer multiple?

[@kamalmanzukie]

​@@ericyip947no, it doesn't need to be an integer multiple. even just a couple hz would most likely suffice

[@rrrushan]

Thank you so much! A few minutes needed to understand it intuitively!

[@danielhoven570]

Love the new intro!

[@PaulinKantue]

I've learned something new today. Human hearing is up to 22 kHz. I've always found Nyquist Samping Theorem very fascinating. Thanks as always Steve!

[@davidasher22]

Actually 22khz is pushing it. The range is from 20hz to 20khz but by the time we are adults most of us lose those highest highs. 14khz is considered normal for adults over 30. The actual reason for 44.1khz is to allow for filtering at the high end. A low pass filter is put onto the signal starting at 20khz and sloping down to null at 22khz. This is needed to prevent aliasing.

[@mostafahasanian5421]

Very nice illustration, thank you.

[@hadriencornier4433]

This video is gold. thanks !

[@amirfallahzadeh5099]

A very nice intro! Thank you.

[@krishmemes7639]

Great explanation, thank you very much. gave my gratitude.

[@YoungBuddhaEzuk]

Great Breakdown! Deserves more views!

[@rybread5718]

Great video. I understand this stuff so much better.

[@Eigensteve]

Awesome, I'm so glad!

[@banachjoanna7055]

Thanks to this video I understood in 10 minutes theoretical points that I would probably have to meditate on during 2 weeks if I had to perfectly grasp the ideas behind it on my own :)

[@omsingharjit]

6:10 as you promised you will explain it in many ways 6:10 this part is Helpful to me and still watching..

[@vivekdabholkar5965]

Nice presentation!

[@AICoffeeBreak]

Such a great video, thank you! 💪

[@Eigensteve]

Awesome, thanks so much!

[@Michael-zn4oq]

Very informative, thanks a lot. I would love to see some content on digital signal filtering.

[@Peter-fy3zj]

You’re the absolute man.

[@RainbowRising]

you would not dream for better explanation than this

[@samirelzein1978]

Awesome as usual!

[@dishaaa-studywithme8909]

Thank you so much, very clear explanation.

[@davidgriffin79]

Very good. I was playing along at home in Mathcad.

[@jebbi2570]

I was doing HandmadeHero and the sound part was giving me some trouble, this helped a lot. Thanks.

[@sebastianhelmle6630]

Steve, I like your videos so much. Thank you! In this video, however, there might be a small incorrectness: The angular frequency Omega is given in 1/s, which is technically the same as Hz. However, the unit of frequency can be given as both 1/s and Hz. Whish you all the best!

[@annyone3293]

Great explanation, thanks.

[@yuchinglee5555]

wonderful explanation!!

[@catasstrophy7807]

Thank's for the video. Only thing I'd like to point out, is that saying "the highest frequency you care about" can be confusing. It should rather be "the highest frequency you receive as Signal". Otherwise one might think, that you automatically only measure the band below f/2, but actually you have to add a lowpass to manually cut off all frequencies, that are above f/2. Of course you explained aliasing (quite well), which is why I said "confusing" and not "misinformed". ^^

[@alexanderskusnov5119]

This is Vladimir Kotelnikov's (USSR) theorem (1933).

[@aliscander92]

Именно

[@djtovys]

Buen video. Me gusta la forma de la presentación. Perfecta. Y el contenido de maestría.

[@Aemilindore]

Whow. More and more signal processing theory from you. Please

[@aliscander92]

Also called Kotelnikov theorem

[@MikhailBarabanovA]

Whittaker-Nyquist-Kotelnikov-Shannon!

[@JamesTJoseph]

Whittaker-Nyquist-Kotelnikov-Shannon saplings theorem. 👍

[@chenqu773]

Very concise. Thank you very much professor. I am just wondering that in the example, if I sample @ω，and the actual freq is 2ω, will the "aliased" signal be still 0.5ω ?

[@adrianphotovisions2308]

Thank you for the tutorial. Can you explain the Nyquist sampling rate needed for a still CCD image used in Astrophotography?

[@schawsk]

can you say anything about the tech how you filmed this? I was really confused when you could see that compressed was misspelled as compred. I assumed all those graps where just edited in in post? or did you actually project it somehow so clearly visible onto the glass? Or did you just have the graph on a teleprompter behind the camera and then didn't bother to fix the mistake in editing when you actually put the graph into the video?

[@danielh2833]

very interesting and anschaulich, thank you.

[@TheMechatronicEngineer]

Great! p.s. according to wikipedia Nyquist was a Swedish physicist and electronic engineer.

[@sudqi]

Amazing stuff.

[@PrzemyslawSliwinski]

4:20 - in a few textbooks the theorem is also named after another scientists: Whittaker and Whittaker (supposedly unrelated!), and Kotelnikov. So it would sound really serious: "Today we will prove the Nyquist-Whittaker- Kotelnikov-Whittaker-Shannon theorem!". BTW, in seventies (Masry, Shapiro and Silverman?) proposed "alias-free" random sampling. This was, as far as I remember, used (together with ocular microtremor) as a model of our alias-free vision due to random distribution of our rods and cones.

[@gazitec]

thank you very much Sir.

[@bchenley]

apparently, for nonuniform sampling intervals, you can reconstruct a signal if the average sampling rate is twice the max frequency of the signal, as long as you have enough samples.

[@bchenley]

oh, didn't know he'd say that at the end...lol

[@melodytune5619]

Hi, I really liked your presentation, but I wonder how your "blackboard" actually works. I love it.

[@thotayateeshchandrasai3828]

So good!

[@valentinfontanger4962]

thank you very much !

[@azizketata3241]

No actually THANK you . You sir are a very good teacher !

[@car0lm1k3]

i like 2.2 -> 2.4 x. I find the 10-20% oversample really great in helping shorten sample time to ensure full wave replication

[@car0lm1k3]

if i have a really short (duration) signal i am trying to catch, i go all the way to 10x if its a highly unpredictable signal (noise issues tho). If it is a predictable short duration signal, i like 5x.

[@bighugejake]

I have a question: how does the theorem account for the theatrically infinite frequencies between 2 Hz values? For example, between 100 and 101 Hz there is also 100.1 and 100.01 and 100.001 and so on, forever. How do we choose the "precision" of how finely we sample the amplitude, and which values, and the distance between the values? Can we pick between 100.00000001Hz and 100.00000002Hz? What if we stored information within that range? Could we accurately recreate it using a sampling rate of 202Hz?

[@omsingharjit]

14:28 so FFT spectrum analyser is best example for that part ??

[@flybekvc]

I would really like to see a better grouping of your videos or an continious online course. There are some playlists but certain items like this don't appear in any.

[@Geenimetsuri]

Remember folks, N-S only applies to an ideal case, and frequencies near the limit get distorted (aliasing & non-ideality) with any real signal sampled on any real machine.

[@rylanmcneely2243]

Wow great video

[@maxbaugh9372]

So having watched this one and the one on beating Shannon-Nyquist with compressed sensing, it looks like the reason that works for sparse signals is that the random measurements by dumb luck will have very high sampling rates within sub-windows of the overall measurement window. Would a measurement schedule with a fractal pattern be "optimal" in terms of highest reconstructible frequency for a given number of measurements made?

[@davidsobel6870]

Steve, these are great videos. However, at ~10:20 you misstate the nature of frequency folding. Recall that if you sample at omega, then the folding occurs around the omega/2 line.

[@Eigensteve]

Thanks David, great catch!

[@chriscockrell9495]

America's Alan Turing. (Shannon) Communication Theory and cryptography are tied together. Twice the highest frequency is the sampling rate. Nyquist rate = optical sampling rate. Broadband signals works with Nyquist sampling. Compressed sampling.

[@superciliousdude]

Can you please talk a bit more about aliasing in general, and how to detect that it might be taking place? I've noticed on modern mid-high end oscilloscopes that they seem to be able to detect that the signal is aliasing even though they have a limited sample rate. If I feed a spectrally pure (50+ dB over the noise floor) sine wave at 30GHz into a scope that can only sample at 8Gs/s it can still somehow detect that the signal is aliased and warn the user. I wonder how it does that? Is it heuristic or is there a deterministic way to tell?

[@BartWronsk]

I’m not sure how oscilloscopes do it, but you can also highpass signal in the analog domain and compute it’s power spectral density. If there is a lot of energy - you have severe aliasing. Another sign of aliasing is if changing the sampling phase slightly, you get different frequency power.

[@marlonjareck7367]

How you done the video? Did you mirrored it in post?

[@BaruchNissenbaum]

Is this video mirrored and lecturer is left handed, or is he actually writing in mirror mode (like Leonardo)?

[@IAMDIMITRI]

10:30 that's why Audio Engineers use oversampling when creating music.

[@2wordU]

Is the video flipped vertically to show the direction of notes correctly?

[@MagnumCarta]

Yes the video gets mirrored/flipped to show the notes correctly to the audience.

[@user-hq7bp1kx4o]

If I have a signal sampled at 2*omega, then I will expect anything shown below omega in psd plot as real. But how can I be sure that it is not created by frequency folding of a real signal between omega and 2*omega? Say, I sample at 2kHz, and in psd plot I see a peak at 500 Hz. It can be either a real component at 500 Hz or caused by frequency folding of a real signal at 1500 Hz. In such case there may not be even a component at 500 Hz. Right???

[@tay3707]

Wouldn’t the sampling clock be constantly falling in and out of phase with frequencies close to nyquist? How come u don’t get amplitude modulation artifacts from that?

[@pentachronic]

Great lecture. Surely random sampling is actually sampling at a higher sample rate than 2W. If you look at the period between samples (or the delta from an integer nyquist) you effectively are sampling above nyquist and just picking up aliases! You just aren't doing it periodically but over time you will build up the periodicity. The tradeoff that it takes longer to gather the data. So really you're not disobeying Nyquist sample rate. You are using a stroboscope effect where the strobe is non periodic so you will eventually see the whole signal.

[@x161x]

I love it all and yet the stupid question I'm left with is : wait how did you write aliasing backwards? 0.o

[@codypredum2059]

Ok, cool, it wasn't just me that thought that was crazy. Although now that I think about I'm pretty sure he didn't, but instead just flipped the video after recording it. The gig is up Steve, looks like I just caught you left handed!

[@unrealreality5419]

wow the explanation! Silky smooth. How you do that

[@marcsfuller]

This video confuses the Nyquist Rate ( the rate you sample at) with the Nyquist Frequency(Half the rate you are sampling at). Frequencies are folded around the Nyquist Frequency (omega/2 in this video) not the Nyquist Rate. Another small point, but one that confuses my students is that omega is measured in radians/second- Hertz stands for cycles/second.

[@anilcelik16]

Two times the angular frequency or two times the cyclic frequency. I am a little bit confused

[@dariocline]

Flipping GOAT

[@amber1862]

09:35 is confusing me: the true signal is reflecting around omega, not omega/2? Love your content btw! Will be getting your book for Christmas =]

[@davidboles1692]

That seemed like a bad explanation from my understanding... You only get mirroring across the y axis (and only for real signals), aliasing causes shifts by multiples of the sampling rate: 1.5-1=0.5 (note that there'd be two peaks being summed there actually if the signal was real, the other being from -1.5+2) *Edit: they would be square rooted, summed, and squared since you're dealing with power in this case I believe.

[@davidboles1692]

Also, if you're familiar with fourier transforms, a useful way to think about sampling is as multiplying your signal by an impulse train, where each impulse is spaced by the sampling period. That then turns into convolving with an impulse train spaced by your sampling frequency in the frequency domain, which in turn explains the whole shifting and adding process. Then you can think of aliasing as having spectrum components that will overlap when shifted (such that after adding you can't tell whether it's part of the original or a copy), so to not have aliasing your signal has to be band limited to half the sampling frequency. Edit: explaining the conclusion a little better: if you're signal is properly band limited to f_s/2 (a.k.a you're sampling at twice the maximum frequency component of the signal), your original signal will be centered around 0 between -0.5f_s (from reflection (ish, potentially different phase but same magnitude if we're talking straight fourier transform) across y axis for real signal) and 0.5f_s, a copy will be at f_s between 0.5 and 1.5, a copy will be at -f_s between -1.5 and -0.5, etc. No overlap, no aliasing!

[@XtopherBryson]

Is he actually writing on that board invertedly?

[@josesaenz8471]

thanks....

[@Rudenich911]

If we measure at 2omega, but not on peaks, then we also have some data corruption?

[@nowrozimohammad]

like your videos

[@Alexagrigorieff]

AKA Kotelnikov theorem

[@SisayMekonnen-md2hg]

Nice

[@vadiks20032]

6:40 holy shit he draws on a screen, and he drew a perfect sinusoid