My Problem With (Most) Spectrograms

Get Minimeters: minimeters.app/
Spectrum Reassignment articles:
en.wikipedia.org/wiki/Reassig...
people.ece.cornell.edu/land/P...
arxiv.org/pdf/0903.3080.pdf
hal.science/hal-00414583/docu...
Follow Au5:
TW: / au5music
FB: / theau5
SC: / au5
IG: / au5_official
Store: au5.bigcartel.com/

[@skybreakedm]

I love that wave candy is in the bottom left corner just dunking on these inferior ones the whole video 🤣😅

[@zweizweidreizweidreinull]

Hey, audio plug-in dev here! I'm currently working on a plug-in UI component library and I'll definitely check out how spectrum reassignment is implemented. I've always wondered how the really clean spectrum analyzers and spectrograms that retain so much detail in the bass frequencies work. Thanks for this awesome video!

[@Mefistofy]

Heh, I just started playing with the vst3 dev kit, coming from audio DSP, mostly in research. I guess this is something I might want to try as my next project. Possibly interesting to learn Vulkan.

[@nexusobserve]

all i can say is check our airwindows, he stream his coding

[@kanzakimusic]

Let's goooo

[@Tsarpf]

@@Mefistofyprobably better off learning CUDA rather than Vulkan if you just want to parallelize stuff. Numpy (jax) are close to matlab if you know that and just want something more convenient. The thousands of lines of boilerplate to create a single triangle in Vulkan is not really worth the time unless you are a big team creating a game engine or something. For OpenGL its only two hundred but still rarely worth it. Use libraries!

[@beardyman]

Ooooh?

[@MrBillsTunes]

"today i'm not gonna do a tutorial" *does a tutorial*

[@au5music]

lol “Today’s tutorial: how to get frustrated over sub-par spectrograms”

[@jenbanim]

5:25 this is actually quite literally the uncertainty principle. In quantum mechanics, the position of a particle is related to the momentum of that particle by a fourier transform, just like how an audio file's waveform is related to its spectogram. The tradeoff between frequency resolution and time resolution in these VSTs exists for exacty the same reason that there is a tradeoff between knowing where a particle is and how fast it's going in our universe

[@au5music]

i know, its crazy!

[@LarsWilms]

That is so cool

[@jorgepeterbarton]

Its more than that in the quantum world. Wee don't affect the observed object here, its just a tradeoff. In quantum systems your measurement actually collapses the particle into a state of one variable measured which causes another variable to become uncertain probability. It occurs with spin measurements too, due to the effect of things at scale being affected by measurement and observation. Spin doesn't use fft yet its still operating under uncertainty principle because what we are measuring is collapsed into a condition of that variable due to being fundamentally so small, plus particles can not have continuous values of spin so measuring in new direction orients the particle in a new way . Here its continuous and doesn't change the state by observing its just our knowledge of the continuous state. It definitely just a classical uncertainty.

[@morgan0]

@@jorgepeterbarton it’s still about wave properties, just with the added issue of only getting one measurement. if the wave isn’t affected by the measurement, each measurement has this tradeoff, but you can just do it more than once

[@Bobbias]

​@@jorgepeterbartonthat's only one interpretation of QM though. Not everyone agrees on that interpretation. What we see does appear that way, but there are other possible explanations for that behavior beyond the wave function collapse theory.

[@ethanlkelly]

I rarely comment on videos, but as a musician, mathematician, DSP lover, and general fan of your videos, this information has been too absolutely mind-blowing to NOT leave a comment. I'm definitely going to check out minimeters ASAP! Love your music and can't thank you enough for the awesome videos!

[@JakeFoster01]

Shoutout to Direct for creating MiniMeters! Amazing producer

[@jesseeatsbrains]

For Ableton and other daw users it’s basically all we’ve ever wanted for an alternative to wave candy. Also i have a Mac on 10.14 still, and he still supports the OS, what a king.

[@SjarMenace]

He didnt create it . The external vst builders dis…

[@SjarMenace]

Minimeters was perfect if the osciloscope wasnt so small

[@LarsWilms]

​@@SjarMenace what do you mean by small

[@SjarMenace]

@@LarsWilms you cant change the osciloscope like oszilloscope from ben schulz

[@dazacman40]

I use Wave Candy on my default template. Glad to see someone showing the love it deserves

[@Holovegram]

Your first plugin SpectrAu5 ⚡✌🏻

[@DJPastaYaY]

Interesting to hear this. You don't see many videos covering this topic!

[@nathanempty]

This is an EXCELLENT video. this has always driven me nuts and I have never known there was a better alternative until JUST now. THANK YOU

[@deadpixel2907]

Honestly the person who coded Wave Candy (Didier Dambrin aka "Gol") is just a Savant, so I'm not surprised to see it here. I mean all the major FL Plugins were coded by him; Sytrus, Maximus, Harmor, etc. It's a shame he's no longer really involved with the company, but they are still delivering solid content.

[@filipenicoli_]

For those interested, the FFT groups ranges of frequencies inside "bins" (basicaly ranges) and those bins are linearly spaced in frequency. Since when dealing with frequencies we tend to use log scales, you're basically zooming in at low frequencies and what you see there is "pixelated". You'll have the same amount of bins between 0-1kHz and 10-11kHz. Increasing the FFT size increases the amount of bins and thus the "resolution" improves, but the FFT size is correlated with the amount of samples the FFT uses.

[@CmoIsDaNam3i]

Dang. Honestly shows how much IL plugins are ahead of their time in some regards but also can agree with that they should be more open to where you can use it. :/

[@rynabuns]

I've been waiting for a VST version of Vocodex for nigh on a decade 😅 Someday my prince will come

[@lmaoanatorhilarious2582]

@@rynabuns I think I’ve actually run vocodex on ableton before

[@hanshanshansans]

Thanks for making us aware, I always figured that mind of low end resolution must simply be imposible, didnt even bother looking

[@jack_digital]

JEEEEZZZZ,,,, you are just exactly when and where i need you to be. I sat down with the intention of buying minimeters last night and just when i did i saw this video thumbnail. Soooo,, thank you Mr, Perfect timing.

[@gursach4435]

12:32 you designed that sounds to look like a bifurcation plot in a spectrogram? crazy also... watching this video just made me realize a link between the content of THREE courses I'm taking at school. Fourier transforms, how they are a more fundamental cause of the uncertainty principle in QM due to position and momentum being conjugate, and how this sense of being conjugate is like the sense of conjugate variables in the Hamiltonian formalism of classical mechanics. And I have no business learning about audio production tools, I just watched this video because I like your music. Insane.

[@donotoliver]

Finally someone is talking about this One more thing! Edison is even better than Wave Candy! Along with enhanced frequency, it has 'enhanced time' (shortcut: T) which totally sharpens up the transients, at no expense

[@kinka16]

12:31 isn't that a famous math fractal with a chaotic behavior ?

[@jinto-music]

Definitely, that's the Mandelbrot set as a bifurcation diagram, Veritasium has that one video on it

[@kinka16]

@@jinto-music oh thanks for info, i thought it was a different set, no related to the mandelbrot set. ngl it sounds pretty good btw

[@gursach4435]

@@kinka16 it can actually arise in many other ways (though they could always be related to fractals in some deeper mathematical sense)! Example: Free-swinging pendulums generally behave periodically - if you keep track of the pendulum's position over time, as well as its speed over time, you'll find that its position and speed return to the same values after every natural period passes. Now if you drive/push the pendulum in an oscillatory fashion at a different frequency than its natural frequency (think pushing someone on a swing but NOT necessarily in the way that maximizes their speed), you can introduce chaotic behaviour; by increasing the driving amplitude (how hard you push it), you will find that the behaviour tends to become 'more chaotic,' corresponding to the pendulum taking a larger number of its natural periods to return to the same position and speed values. Increase the amplitude enough, and eventually the pendulum will NEVER revisit a single position and speed configuration value, i.e. its motion is completely unpredictable. The bifurcation plot arises when you plot the driving amplitude on the horizontal axis and the position value(s) it takes on at every period on the vertical axis. At small amplitudes, it is periodic in the normal sense, revisiting the same position after every natural period, so the plot is a horizontal line here. Increase the amplitude, and eventually it takes two periods to revisit the same point, meaning every even-numbered period it will visit one point, and every odd-numbered period it will visit another point - the plot here is two parallel horizontal lines. Increase some more, now it will visit four distinct points, and so on. The plot splits and splits (bifurcates), generating that distinct shape.

[@kinka16]

@@gursach4435 yeah i did a lot of mechanical engineering and i have a lot of interest on science, physics, but im not too familiar with in some way, i really need to go deeper and somewhat experimental. it is absolutely facinating how simple things can be this chaotic !

[@ZesterFrost]

The frustration in finding a good spectrogram is so relatable. Like you realise how valuable simple and powerful plugins like Wave candy and Harmor are once you switch to a different DAW. As always, thanks for the valuable insight Austin.🙌

[@sentinelav]

There's a recent reassignment method called Synchrosqueezing that would be dope to see implemented in spectrum analysis. Thanks for bringing awareness to the issue!

[@au5music]

Oh I believe I’ve seen this in Steinberg Spectralayers

[@jeff-meyer]

@Au5 if you have any interest, i threw together a really simple M4L device which will hide the IL Wave Candy vst when the Hint/Info box isn't showing. This way it automatically goes away and gets out of the way when you are scrolling through the library and such on the left side of the screen. It only appears when the bottom "Devices/Midi/Audio" area is visible, which is when the Hint/Info box is visible.

[@UndulaeMusic]

Really hope Image-Line opens up their walled garden. Currently an Ableton user but I used to use FL Studio and I really miss some of these plugins (Wave Candy, Harmor, Maximus)

[@MrReeTart]

Harmor is just incredible.. mind blowing. Zebralette3 seems promising

[@izzurumusic]

Agree

[@stereokuuji]

Harmor is a stand alone plugin

[@MrReeTart]

@@stereokuuji Harmor only available for windows

[@UndulaeMusic]

@@stereokuuji unfortunately Image Line discontinued updates for the standalone VST a few years ago, so you can currently only use it within FL Studio unless you happen to already have an older version and you're not running on Apple Silicon

[@43tss]

Music producing turns back to be science. Finally. Thanks for this video, can't wait for more!

[@Ignight]

This has been driving me crazy for years! I just picked up minimeters a few weeks ago, didn't know there was a sharp setting. Absolutely clutch 👌 Thank you, stay lit 🔥

[@dohpam1ne]

So happy that you pointed out how this is similar to the uncertainty principle. The mathematical reason is exactly the same for both. Unfortunately, better plugins won't be able to get rid of this tradeoff :)

[@Derpcat]

its about time more people start to give a shit bout the spectrogram standards, Wavecandy is ahead of the curve, seems like a pattern with Image Lines plugins, they always been ahead of the curve, insane stuff!

[@xtrullor]

Really weird how long I've been sleeping on Wave Candy even though I've known about it for ages. Never knew how good it actually was, just took it for granted.

[@SlvYT]

I have the same, lol

[@SteveAcomb]

as an electrical engineer with a focus on signal processing, I’m honestly blown away by that wavecandy “enhanced frequency”spectrogram. how tf are they doing that? this is really hard for math reasons… *edit wow! thank you so much for mentioning the papers! you’ve given me something to geek out about for the next few weeks 😂

[@alsoeris]

Most of them not being able to display the side channel is my main problem. It works if you set up a utility before it though.

[@eosmusic9872]

Came here to say this

[@offsetemusic]

there's a reason though: the side channel can phase cancel the mid channel depending on the information it has the same thing can happen when you sum the left and right channels to mono. the most ideal solution is to have separate spectrograms for each channel

[@user-gg6kc2ls6k]

i think you can just split the mid side into two tracks

[@au5music]

There was a very old iTunes visualizer plugin called SpectroGraph that displayed stereo information in a spectrogram by coloring the L and R channels red and green (yellow for combined/mono). I never saw such a thing implemented since and would also love to see that as L/R and M/S analysis spectrograms along with enhanced frequency. Would be the end-all for me.

[@RaquelFoster]

I'm a developer and I've done a lot of signal processing work. Reassignment is not a secret, and neither are the tradeoffs. It's doing multiple DFTs and using the phase derivative to estimate the frequency. It takes a lot more CPU, and all those estimations turn into error when you add harmonics or multiple sinusoids. It totally falls apart vs. white noise. But if you want a sharp spectrogram of a monosynth, definitely use reassignment. Spectrograms are kinda like compressors. Everybody just wants them to work, but dialing them in is very situational.

[@Plikso]

Its incredible how he got the best intro i've ever seen with just 5sec

[@dr4gon8it3music3]

pushing the industry forward once again - inspiring being and great sensei

[@adamluhring2482]

The reassignment method is cool, but it's not the only solution worth considering here. Two common DSP methods: 1) You can estimate the true frequencies of peaks using splines that are only calculated near maxima in the spectrum above a certain threshold. This is pretty computationally efficient and is used in a lot of algorithms that extract partial data from audio (like pvanal in Csound). 2) You can take a wavelet transform. You're right about the uncertainty principle between time and frequency. The best time resolution and worst frequency resolution is the untransformed signal (just the list of sample values). The worst time resolution and best frequency resolution is an unwindowed DFT across the whole signal. Spectrograms are a series of shorter snippets of the signal Fourier transformed; the longer the snippets, the better the frequency resolution and the worse the time resolution. There's also a whole different class of transforms called wavelet transforms that gets the same excellent frequency resolution at the lowest end that a windowed Fourier transform would have and good time resolution at the high end (with a slight loss in frequency resolution).

[@Venthorn]

It's honestly frustrating how few spectrograms give you the option of different wavelet transforms instead of the short time Fourier transform.

[@filipenicoli_]

Couldn't the FFT window be slided a few amount of samples instead of the length of the whole window? I guess you'll be doing a higher amount of transforms and using more processing power, but it could improve the time resolution. In DSP courses we've also studied zero padding/stuffing. I'm not sure if this could be applied in some way here.

[@adamluhring2482]

@@filipenicoli_ Sliding the FFT window is regularly done, but it doesn't improve frequency resolution.

[@filipenicoli_]

​@@adamluhring2482 but wouldn't sliding improve time resolution for bigger fft sizes?

[@adamluhring2482]

@@filipenicoli_ Oh sorry, read that as frequency resolution, not time. Still, the answer is no. For a window of a given length, the FFT will always be the worst time resolution. The fact that those extra time steps can provide practical value is because human hearing works more like a wavelet transform; we'll never actually hear a sinusoid of infinite duration. But in the sense meant by time-frequency uncertainty, you can't beat that limit

[@fidgetgadget3475]

Very coherent message. Thank you for bringing awareness.

[@farty555]

...And loudness is represented by the brightness of the image Great video Au5. Love the insights you have. And yeah FL is definitely infringing on our freedom by limiting their software to their daw

[@DANETRONICMUSIC]

No wonder why I couldn’t find that spectrogram. This was very informative thanks for sharing!

[@_Polymorphiq]

I love your videos so much man. I literally just bought Vision 4x. 30 minutes ago and came across this video...... wish I viewed it before. Also I am one of those who have also wondered what spectrogram you use in your videos.

[@user-uq5qw1fk3d]

Loved this video! I've always been unsatisfied with the low frequency precision of Izotope Insight's spectrogram, so thanks for recommending the alternatives!

[@mike777881]

legit working on my first DSP project and this seems very applicable for what I'm doing. Thanks for spreading the word

[@rottenelk]

This is so important, thanks for sharing. So glad minimeters added an enhanced frequency option

[@eputty_audio]

Absolutely love this video!

[@feeltheflow9383]

12:34 omg it plots the logistic map!

[@kinka16]

i was going to sleep but i had this call from Austin !

[@officialmaximedes]

great video as always!

[@official-phuh]

Wave candy in Live myself too. I didn't know IL stopped the non-FL version. I shall cherish this old version. Good analysis

[@truthordarrenmusic]

🤩 1024 FFT FTW 🩵

[@exacoustics]

Thanks for the links. I'll check them out

[@turbinchik]

Much respect as usual❤

[@ebrann]

As someone who works as a sound analyzer under 3khz we have to change our sample size many times throughout an analysis. If I am working with board-band frequencies we have to change our spectrum output to get more detail as opposed to a narrow-band frequency source. The short of it is don't rely on a single processer setting and move through the options provided to get the detail you want.

[@cookiecap_465]

great explanations!

[@Jaysterr]

Dang I always wondered why Wave Candy looked so good lol. It’s awesome that you got the minimeters dev to add it though! Love that tool

[@s.t.p.1236]

Thanks for sharing, peace

[@Illkacirma]

thanks for sharing

[@_Pancake992]

Thanks, just implemented this into a tool of mine👍

[@Ravarcheon]

ive been annoyed about this for years! thank you for this video and spreading awareness

[@yuxanne.]

The minimeters one gets a higher resolution depending on the windows size, i like to keep it vertical, with a really wide window size, its on my second moniter so space is never an issue

[@Znernicus]

This is very interesting. I'm curious about your wavecandy preset to get it to look like that, because the default spectrum view does not look quite that sharp. Thanks!

[@_XRMissie]

Wavecandy enjoyer for years. I was grateful how good this is, but I didn't realise how crap the competition was lol

[@fidgetgadget3475]

Electrical engineers: Why would you need that much performance? Sound designers: I need it.

[@SolarLiner]

I thought I was the only one that cared about how Wave Candy was the only spectrogram I knew that did such a thing, and when moving from FL to Bitwig, I was seriously missing having it.

[@chainsawteddybear]

Preach it brother

[@Terrapastel]

Incredible. I have 4x and thought it was the best thing out there. Seems IL Wave Candy already nailed it, although sadly obsolete. It's amazing that mathematicians and scientists have found a way to analyse the phase to get super accurate frequency measurement using FFT, with magnitude as a trade-off. I'm going to get Minimeters while it's still in beta. Looks great.

[@CanaDan]

I think alot of it was that it just wasnt heard about. It may also be really difficult to implement (fft analysis alone isnt easy. some very spooky binary manipulation and instruction set utilization going on). I've implemented fft spectrograms into projects of mine, and yet have never heard of spectrum reassignment, so thanks for bringing this to my attention. I was planning on implementing wavelet analysis, but theres not alot of info on that algorithm so maybe this is a better alternative instead to get better resolution. btw, if you can find a wavelet transform visualizer, give it a go. It may be similar to spectrum reassignment in terms of quality, as it essentially breaks past the fft uncertainty principal by cheating lmao. You get extremely good low frequency information and really good time accuracy because it treats each frequency as its own independent calculation

[@benkirkpatrick6343]

So happy rn, being a Windows FL user. Love being the unsung hero of the music game lol

[@mirrorengine]

thank you!

[@dustinmorrison6315]

Hey Au5 you should also check out the Wave Candy vectorscope so you can fine-tune your stereo shaping! I have been using an FL Studios' patcher patch that I made to listen to melodic dubstep for the last year and it sounds incredible. I also combine it with the convolver to make it sound like I'm listening in any venue I want.

[@au5music]

I have, but its not nearly as wild as Signalizer's vectorscope. It's got a time dimension so its actually a 3d oscilloscope.

[@Djenerater]

FL Out here flexin

[@PaulEubanks]

I feel like having more accurate spectrograms will also help to fine tune AI models for generating accurate representations of what music sounds like.

[@VorsoMusic]

nice bifurcation diagram at 12:32 :D

[@derekepperson2448]

It seems like vision 4x makes up for the low FFT resolution by having the horizontal bar graph that shows the volumes of frequencies

[@7minutesdead]

Extremely helpful. I haven't been using spectrograms at all and I really should because I lose sense of my mixes so often and so quickly Now I gotta figure out how to get the wave candy version for Windows since I'm on Ableton too lol

[@billpodolak7754]

This is a cool video - heads up that the reassignment method spectrogram is available as an option in iZotope RX, but not yet implemented into Insight.

[@CameronWindheuser]

that alien weapon sandwich at the end was yummy tasty

[@WDShorty]

Also custom color gradients would be the cherry on top

[@AliAtash-dh5dv]

perfect

[@SALEENS7GTR5]

As an FL user, thanks for saving me money by sticking to Wave Candy

[@StarOnCheek]

Omg, I've started doing sound design and using spectrum analyzers in a huge part thanks to you, especially the spectrum analyzers, and I've been trying Vision X4 and I am literally so disappointed by it. I didn't even understand why I was not like what you do in your videos until I saw this

[@menglin7432]

Mathematically it's impossible to increase resolution of low frequencies without sacrificing time resolution. But what you described seem more like a UI issue, where the detected frequency isn't communicated to the user well.

[@menglin7432]

Using morlet wavelet transform, it's possible to get a better balance of time and frequency resolution

[@iKiZaaH]

I love that Ableton skin, is it from Livethemes?

[@SjarMenace]

Thx for being a voice we need more vst be adressed such things. Inhope they will get canceld

[@HLRxxKarl]

Image Line needs to bring the Juice Pack back! Even as an FL owner who uses it 50-75% of the time, I'd drop more money on the Juice Pack in a heartbeat.

[@ColinLeet]

It looks like it's performing a peak picking on the fft following up with phase correction on the peaks, and then is just graphing those results. (Hence why it's not showing side lobes around the peak for the improved spectorgam.) I wonder if it's bases off of quadratic peak interpolation, or if it's more along the lines of a phase vocoder.

[@adriankohres5623]

Id love harmor on mac so much. Maybe there is something we can do? If enough musicians sign a petition to bring it and other vsts to mac too.

[@hjkhhjkh3635]

Hey Austin, Would it be possible for you to do a tutorial on how you use Morph? Only a 2-3 min video would be great!

[@muffinmaster9037]

Nice video! I just found a new purpose for Minimeters (i only used it for making twitter clips lmao)

[@jeff-meyer]

I've been building my own M4L one the past 3 weeks. And my goal has and still is, to create a IL Wave Candy style one. With a few extra Ableton specific features. Random to see this video come out today

[@ridethewavz]

anywhere we can follow your work? I'd be super interested in buying a M4L device that checks all the boxes

[@jeff-meyer]

@@ridethewavz I will update this video when its ready. I'm hoping to figure out this spectrum soon.

[@SlyceCaik]

wave candy gang

[@NaviRetlav]

I'm curious if this can also be applied to all fft based processing plugins to improve their sound?

[@Milinor]

That's my battleground!

[@MFKitten]

Some kind of compositing of FFT sizes would probably work decently.

[@Kallyn]

Does the reassignment method only affect visualization? Or is it possible to get finer detail with audio processing as well? I wonder if this would help improve the detail of spectral plugins while keeping a smaller window size to reduce latency

[@au5music]

Its for visualization. Most FFT audio processors are already using the phase spectrum data in the resynthesis. I think RX has a different approach for enhancing freq and time in their processing like the noise module. To my ears it sounds like a series of different FFT sizes though.

[@gctechs]

Also something like this is available in Renoise.

[@DragomirDraganov]

Any idea why the sound at 12:33-12:34 produces a spectrogram like a Bifurcation diagram of the Ricker model or logistic map?

[@GBRen-xc2ow]

Does it apply to EQ’s as well? I’m not sure how Fruity Parametric EQ 2 compares to other EQ’s, but it has a similar feature where you can enable “Enhanced Frequency” and “High Precision” to better track the frequencies on the heatmap with thin lines.

[@StanleyGurvich]

JuicePack Forever!

[@GoatPepper]

Hows the cpu usage with MM? Running V4X can get fairly intensive.

[@Deadspacedub]

Can you do a video on how to read wave cand/ mini meters? I get that it shows a graph, but how can one benefit from using it?

[@jay7dsi]

its so annoying that the crossplatform wave candy is discontinued, i miss it every time im working in ableton

[@MaxKoko]

I got my minimeters copy a month ago and I think I will never switch to anything else ever again. Wonder when the skin option will be available tho

[@valentinpedersen6144]

Is it because it would need to clean up noise down there to only show the most dominant frequency?