Making a Pitch Shifter

[@diplomaticfish]

This is really well made! I had no idea pitch shifting required this much complication.

[@wendolinmendoza517]

Same here. I really did not suspect so much math was needed for something this (apparently) simple.

[@Cineenvenordquist]

It doesn't. He should have done this with op amp circuits and it would be fine. Even if he skipped the window fudging thing to cut the pop, he could pick better transforms and meet success. I mean, why didn't he match the song effectiveness with what he demands from the filter; blam, a sing wrong fixer that has funkadelic qualia.

[@LuLeBe]

Funny that you say that, because this seems rather simple to me. No, I don't understand it at all, but the result is great for how "little" he did. I expected this quality to already need a huge amount of processing that only the gods at Antares (autotune company) have figured out.

[@DuckPerc]

@@LuLeBe The wizardry in autotune is all about interpolation between notes. The notes they sung, and the notes they wanted to sing. The repitching part probably isn't all that great if you just want a static effect. Like, imagine the math in this video, except the ratio between original pitch and target pitch is a function of the frequency content of the signal you started with.

[@MatthijsvanDuin]

@@Cineenvenordquist Ehm, no, what he's shown in the video is pretty much as simple as it gets for performing pitch shifting, which is just inherently difficult. I can't even guess what you could be referring to with the extremely vague "op amp circuits"... the function of an op amp in basically every circuit that uses one (except when used as a comparator) is to solve an equation, but the actual equation it solves is entirely defined by rest of the circuit, so describing something as an "op amp circuit" is utterly non-descriptive. Regardless, the fundamental problem of pitch shifting are not specific to how the audio is represented, and so are the solutions except they're much harder to implement using analog hardware than digital filtering, and getting phase to match for transient sounds seems like it would be particularly painful.

[@ianbelletti6241]

The original Alvin and the Chipmunks albums took advantage of speeding up the playback to change the pitch because they weren't being digitally processed. This means that the artists had to sing at an appropriately slower beat to account for the time shrinkage.

[@charleslambert3368]

And then you can play it on a 16RPM turntable to hear the singer sing really slowly and the band play down an octave or so

[@ianbelletti6241]

@@charleslambert3368 the way these things were recorded the singers sung lower notes at a slower beat. The instruments were played at normal rate. The singer track and the instrumental track were combined later.

[@raksipulikka]

Last semester did a project about phase vocoders at an audio signal processing course. Your explanation about is absolutely superb, we had to give a presentation at the end of the course and it was train wreck compared to what you managed to explain in 15 minutes. Well done!

[@TheGamefreakr]

As a semi-professional producer and programmer, I am deeply fascinated by this.

[@MysteryPancake]

6:28 The reason pitch modification is listed as "yes" for PSOLA is because it's the only algorithm that doesn't change the formant, which is important to produce natural sounding speech. Unlike SOLA and WSOLA, it doesn't introduce a chipmunk effect when speech is sped up. Because of this, it's heavily used in many autotune plugins today. It's possible to tell if a plugin uses PSOLA by shifting the vocal down: - If the formant sounds the same and there's gaps in the waveform, this indicates PSOLA was used, as it has no way to fill in those gaps. - If the formant sounds the same and there's no gaps, this indicates a frequency-based approach was used, as it can fill in those gaps. - if the formant sounds different, it's likely another approach was used.

[@mavaction]

Super smooth. Clear, concise and thorough... and then having a real demo audio to follow examples is a great presentation feature. As someone interested in spectrograms I'd like to see a video of which audio properties create the most problems for a shifter vs which properties are easiest to work with. I've got some guesses but I imagine you could demo it in a similarly illuminating video.

[@davidvomlehn4495]

Nice. My degree is in Physics. I assumed this was done with Fourier transforms, but I had always wondered how the finite sample sizes were handled. Basically, how they were glued together. This was well presented and I can see how much more there is to learn.

[@bentpen2805]

Do check out wavelets (including the continuous wavelet transform), as well as spectrograms. It’s all so cool.

[@davidvomlehn4495]

@@bentpen2805 Wavelets are on my list of things to check out, this is probably a fun place to start.

[@a52productions]

This is an incredible explanation! The visuals are great, and the way the process is laid out ("lets try this -- oh that causes such and such problem") really helps understanding the rationale behind every aspect of the algorithm. I knew, as a physicist and electronic musician, that pitch shifting was hard. But I never knew how it worked, just that from a math side I couldn't come up with a good solution and from a musician side that choosing the right algorithm for a given sample was often difficult. This explains so much!! What a satisfying algorithm and a satisfying explanation.

[@theb1rd]

I've wondered about this for decades. Great explanation, thank you!

[@HunterHerbst]

This video is truly amazing. I hope you make more coding content and simplify other random topics for literally anything. If it's just a one-off you still did amazing and I wish you well on future endeavors

[@amyvo2278]

Thanks

[@msclrhd]

The Overlapped Add (OLA) algorithms also form the basis of several text-to-speech vocoders like MBROLA, PSOLA, etc.

[@TehBrian]

I lost it right around 11:00-ish. Nice to see where my current math knowledge tapers off. Hopefully I can revisit in a couple years and be able to understand the whole video. :)

[@PhilipSmolen]

Thanks. I've always wondered how this was done. I didn't realize it was so complicated.

[@LaszloKorte]

One of the best signal processing videos I have seen yet

[@rb8049]

I’ve always wondered how this was done. Thank you. I could only really understand if I did it myself, but I get the gist.

[@lnerqad--5268]

this taught me more than a whole half a semester of uni. good video. sent it to everyone and their grandma

[@samuelwyeth4178]

This is insane quality and content for such a small channel. Hoping you get the attention you deserve!

[@Fasteroid]

I remember watching a video some time ago where this guy deconstructed a square wave with FFT, phase-shifted the frequencies to minimize the constructive interference between them, and then put them back together. The result was a signal that still sounded like a square wave, but could be pushed far louder. I wonder if certain "compressor" type filters also use this approach, and what they might do to transients having seen after seeing the results of naive pitch shifting with phase shifting.

[@gabrieldoudna6570]

isn't a pure square wave the loudest possible waveform to begin with? i would guess that you could only decrease its loudness with this technique (ie increase its crest factor)

[@aotmr1604]

@@gabrieldoudna6570 indeed, phase-shifting the components of a square wave by a quarter turn produces in theory a signal with infinite crest factor

[@SamAlton]

I also don't like reading words. Thanks for the pretty animations and explanation!

[@Cineenvenordquist]

Yeah, so why weren't the Jupyter notebooks in Sugar or a new Jaron Lanier language instead of python?

[@user-yv6xw7ns3o]

This is something I’ve wondered about for a long time! Nicely done. It gives me hope when I see quality videos from small or new creators show up in my recommendations 😃

[@CK-3K]

I like how almost every hurdle in audio processing comes back to phase when you trace it back far enough.

[@4.0.4]

The piano tunes in the background were a nice touch!

[@morgan0]

9:15 expanding on this, for N samples of time domain data, you get N samples of frequency domain data. however each frequency is a complex number, using two samples each, dc offset gets one. the nyquist frequency also gets only one, because phase shift between nyquist sampling frequency and input frequency at nyquist results in amplitude change. for every phase shift there is an amplitude that will make it fit the samples, and hence only one can be represented (and we never know which it is).

[@MatthijsvanDuin]

Though note that FFT outputs N-1…N/2+1 (note descending order) are the complex conjugates of outputs 1…N/2-1 and therefore are redundant, so if you discard those you've got a real numbers for DC and N/2-1 complex numbers for frequencies between DC and nyquist, and a final real number for nyquist, which all combined conveniently fits in the same amount of memory as the original N real samples :-) (All this is assuming N is even; if N is odd then there's (N-1)/2 frequency samples between DC and nyquist and no frequency sample _at_ nyquist.)

[@__MINT_]

​@@MatthijsvanDuinExactly, values are mirrored with nyquist bin being the center point, real part of mirrored values is the same, while for imaginary part the sign flips. So mirrored value of x = a + bi would be y = a - bi. For example, with 1024 frame size, 0th coefficient is DC, 512th is nyquist, and every other coefficient 1024 - n, where n is coefficient number from 1 to 511, is a mirrored version of coefficient n. 513 is mirrored 511, 514 is mirrored 510, etc. I assume that you know this, but maybe someone will learn something new.

[@mikecase2033]

I love when I see something I take for granted (viewing youtube at x1.5) has a ton of math behind it. Awesome video

[@robertphilip1832]

I've understand so much more and so much better about pitchshifting by Your video. Thanks for it, really great job done!

[@yuvalne]

the demonstration are what really make this video. truly well done.

[@Aio-Project]

the amount of math videos i had to rewatch before returning to this

[@michaelharrison1093]

Long before the existence of auto-tune there were pitch shifters that were used in live performance situations to eliminate audio feedback from the speakers to the microphones. These units had a very clever approach which I wonder why is this not used for auto-tune? The approach was based on something very similar to super-hetrodyne theory. The audio was multiplied by a carrier signal and the modulated signal was filterd to remove one of the side bands and then this filered signal was multiplied by another carrier signal that was slightly frequency shifted from the previous carrier frequency. This demodulation proces gave an output signal that was slightly frequency shifted from the original.

[@schmitzbeats6102]

What you describe are frequency shifters, not pitch shifters. When you apply too much of that, it messes up the harmonic relations of the sound. (A couple Hz of shift is not very audible, but removes the audio feedback.)

[@Cineenvenordquist]

That's a kinda DAW in the making, just add a tape loop or 7. Knocked away common mode and room resonance noise?

[@Etobio]

This video is so well put together! I’d love to see this but with formant shifters. Love the content

[@moralboundaries1]

Thanks for sharing this! Easy to follow and really interesting! Your pitch shifter sounded great!

[@ri-gor]

Love this video. I took a college level course on digital signal processing, and we never even covered those hann windows. I definitely saw some leakage in my projects, but thought it was unavoidable (similar to the Gibbs phenomenon).

[@Descenacre]

Subscribed instantly! This is an incredible video, thank you for visualizing something I've been incredibly curious about but had no clue how to find out! Beyond just being a math nerd I'm also a musician so a lot of this information might come in handy in the future...

[@cornoc]

amazing video, took something i thought was magic and made it simple enough to the point where i feel i can implement it myself

[@karlkastor]

Great video! I always thought pitch shifting was just translation in the frequency domain. I like how you explained the problems with the example.

[@MarkMendell]

This is fantastic. The animations and code are phenomenal for keeping up with the concepts. Great job!

[@JonHarris77]

Really enjoyed this! I would love a video on formant shifting please.

[@byte.observer]

Excellent video! I've implemented these algorithms many times by now but having this video 5 years ago would have saved me a huge amount of time and headache :)

[@Cineenvenordquist]

Wait, why do you reimplement this bunches and why don't you just use an op amp circuit or get a singer who likes working to spec? Or a marimba with motorized dampers? Do you have to teach elliptical equations on python driven instruments?

[@shinyless]

best explanation of pitch shifting I've seen so far, this is a 3blue1brown level of quality video, awesome work !!

[@danielchin1259]

Content presented around 8:00 is a bit misinformed. STFT preserves all info. Discrete time-domain repr doesn't include any info above Nyquist freq either. When pitch shifting, freqs higher than Nyquist is supposed to be lost, so scaling the frequency axis is not problematic. HOWEVER! The video up to 8:00 was very cool and taught me a lot of time-domain methods I didn't know about. Very inspring, and I appreciate that!

[@danielchin1259]

I just finished the video. 12:00+ was imprssive as well! You are utilizing assumptions about natural music to strike beyond theratical limits.

[@Ignisami]

Going to have to watch this a few times to really grok what’s going on, especially after the DFTs enter the picture. Been a while since I’ve done math to any serious level

[@sirati9770]

I am watching this video at about 4 Times speed. This actually becomes very important to me because these artifacts dictate how fast I can watch a video on a device (or in general)

[@vilkillian]

5:32. To be precise, auto-correlation is a convolution between signal and itself, but delayed, this is what auto- means, in this case it is just "correlation"

[@1495978707]

I’ve been wondering about this for a long time, but haven’t gotten around to reading a digital audio book to understand it

[@JJRicks]

The silence after "the guy's name was just... Hann?....." is so perfect haha

[@calebdeming5515]

Jupiter notebooks unlocks alot of the mystery to this video.

[@CFSworks]

My DSP experience is in digital telecommunications (OFDM! Costas loops! LMS equalizers! Oh my!), where this kind of operation isn't really ever necessary. I'd always wondered how my compatriots on the audio side of the DSP world managed to develop an algorithm to do this. Turns out it's an even more complex problem than I thought and I should perhaps rethink saying "I do basically the same thing, just at a higher sample rate." Excellent video editing and visualizations! With production this good, I'm surprised you only have ~1.57K subscribers - but count me among them now. :)

[@mintsjams8862]

I've been watching a lot of SoME videos recently and this is one of my favs. Well done!

[@ngideo]

This is really well done, I just feel like you could have given a quick recap of DFT, since you took a long time with the time domain explanation, but went super fast in the frequency domain explanation.

[@playbyan1453]

Very technical details that I love, clrear explanation and very inspiring. The vocoder result sounds so high quality.

[@ReginaCæliLætare]

"It's full of big word" brother, most of the words you used flew right over my head I understood almost nothing

[@zorggn]

I wish i found this video sooner tbh, would have saved me some time, i believe. :3 After toiling for quite a bit of time trying to implement a realtime pitch shifter/time stretcher (hence i didn't want to touch fft or other things that would eat performance), i arrived at a solution where i only ever use two windows overlapping exactly 50%, but for each step to generate a new chunk, the windows are both resized to a size that's the chosen one plus/minus a small (uniformly or normally, i tried both) random deviation; Considering that a static window size was producing a very noticeable tone with my code with window sizes that were small enough to make the crossover points fast enough to go into audible pitch territory, the randomness introduced seemed to make that less noticeable quite nicely, basically akin to turning a pure tone into white noise. I still have issues with fluttering though, that i couldn't find time to implement a fix for yet.

[@mumiemonstret]

That was a concise walkthrough of the steps necessary to pitch shift. I never thought there were so many, and apparently you left some out. I can say that your end result still sounded way better than what Cool Edit Pro (predecessor to Adobe Audition) could accomplish. Of the left out steps, I would like to see formant preservation as the topic of a follow-up. It would be interesting to hear what happens when you move formants and pitch independently in a vocal sample.

[@aze4308]

WOAH this is incredible for your first video! i hope you blow up!

[@made.online2149]

Best video on an audio DSP topic I've ever seen tbh!

[@riccardorosso2795]

Do more video like this one please!! Incredible work!

[@TheLaxOne]

Wow I've always wondered how pitch shifting algorithms work and you did a really great job illustrating how some of them do so with this vid!

[@hughmathimati]

Yo, I remember you from Khan Academy! Keep making awesome stuff dude.

[@WangleLine]

I've been hoping to see a video like this one day, thank you so much for making it

[@Rajkumz]

Wonderful video. Excellent idea to calculate the volume ❤❤

[@triplezgames3882]

Damn, this channel is an underrated gem... I've been here at 661 subs, see you at many thousands soon

[@joelpetersson8742]

Amazing video! Thanks for putting so much effort into it!

[@fakesilver746]

really nice video. I appreciate that you took the time to define the jargon words.

[@camerongrenier5405]

I studied physics for 3 years, and I’m currently studying computer science. That was wayyyyyyyy more complicated than I was expecting and a lot of it went over my head 💀

[@pepedecoatza]

What a video. I just saw its your first video in your channel, so, congratulations and keep it up!

[@therockafire]

Well, that clears that up! (For Albert Einstein!)

[@tomatosauce5775]

I’ve struggled to get anything with pitch shifting done without sounding bad which is why I prefer the “just doing it in the right key/in tune during recording” strategy. At least now I know why it’s so complicated 😭

[@apotheotic]

Excellent video, I hope to see more from you soon

[@FelixArchive]

Amazing video! I was actually looking for this type of video a year or so ago and couldn't find one. Thank you for making this!

[@wentropia]

wow! "Hey kid, you know those things you didn't care to learn about? I'm going to show you what they're for and why they're cool"... fascinating.

[@1.4142]

I've noticed how much better chrome is than safari at 2x speed audio

[@kaishang6406]

Thank you very much, the ideas in this video are useful for more than just music.

[@ThingsMadeSimple]

Excellent video! So clearly explained. I hope you will do many more like it!

[@dittilio]

Excellent video. I really hope the phases didn't smother your interest in making more videos. Instant subscription.

[@DokterKaj]

So glad I clicked on this video, beautifully made!

[@ag-k]

only watched 4mins as of yet and i already learned more than i did in school lmao you deserve a sub

[@AK-vx4dy]

Your knowledge level is intimidating... Also talent to present it clearly. Thank you for your work. I once tried "pitch shifter" using doppler effect method, hardware couldn't even dream of Fourier . I wonder if you have knowledge how pitch preserver works like in youtube speed change

[@geraintluff]

4:22 The fluttering is related to how much the output blocks cross over. Since you've fixed the analysis hop size (instead of deciding the output hop size and then figuring out what the input one should be) compressing in time means the output blocks overlap a lot more.

[@JentacularGent]

This seems right; I just tried since fixing the output hop size, and it does introduce fluttering when shifting down. But shifting down decreases hop size, which creates more overlap, so shouldn't it have had more fluttering?

[@geraintluff]

​@@JentacularGent More overlap reduces fluttering - the more blocks you're adding together, the more their fluctuations average out to be smooth.

[@a.w.n.i.s]

i would not have the balls to do such a well made video at 1,7k followers

[@seatyourself7082]

My Favorite SoMe

[@fano72]

Awesome video and easy explanation of such complex stuff👍

[@kruvik]

I implemented a pitch shifter and time stretcher in 2018 or so and was surprised how much non-trivial math was involved in doing that...

[@adrien5568]

Two years ago, I had to do this project. The explanations were less than clear. So, I started to dig into the academic papers. Once I was done I wanted to do a video just like this one for SoME2. Unfortunately, I never finished the project.

[@sbanner428]

I'm a nerd when it comes to sampling. You are a legend for doing this, I hate words too ^_^

[@gooball2005]

Incredibly interesting and well explained!

[@backpackvacuum9520]

Great video! I got a little lost with some of the Fourier stuff, but I haven't found a resource yet that doesn't lose me on that topic 😂 I'm surprised to see python doing this! I did a project doing DSP in python a few months ago and started hitting the limits of what could be done fast enough. Although I was doing real-time, not sure if you are. And I wasn't making extensive use of libraries written in C.

[@ttc1045]

This is awesome and I need so much more, you’re like if 3blue1brown was a producer

[@georgeabraham7256]

Changing the pitch without affecting ADSR ❤

[@ksaw3212]

In Alvin and the chipmunks the actors were actually speaking slowly and then just sped up (yes, they were singing like that as well)

[@Cineenvenordquist]

Why won't it die? Lol silly voice tech keeps changing. Also actual chipmunk training bc. it sucks if they ruin your house, connections, yard, etc. despite looking cute.

[@tibr]

Fantastic visualisations! Tnx for the video!

[@parlancex]

Informative with nice animations, thanks for making this.

[@DKDiveDude]

Very nice video. I would really like to see the code C or C++, maybe even using the JUCE platform.

[@melody3741]

Its funny because this was always my first thought on how to do it (only overall not the details) but it always seemed too complex and like it would never work, so I assumed it was something crazy like fourier transforming then changing the pitch then changing back or something

[@Cineenvenordquist]

Soon: Oh just use a phase mux SBN (sulfur boronitrite) inlet and bias vacancy decay with the new pitch. Or: I only use transforms named after musicians, but I'm still surprised when SZA is inverted through ASZA. Seriously though, did he have to not pitch shift the percussion?

[@timothydrewfilms5272]

WAIT! How the hell do you only have 174 subscribers!? I only noticed after watching the whole video and then going to check your channel for more xD Consider it 175 now

[@wareya]

Could you improve the time-domain algorithm by splitting the signal up into several frequency bands with a brick-wall bandpass, timestretching each one, then recombining them? You could use smaller window sizes for higher-frequency bands, so transients would be less disturbed by bass frequency phase alignment.

[@NicosLeben]

I like this video a lot because I like to visually see stuff like that. Now I am interested in how to not only pitch shift audio with a static value but with dynamic pitch shifting like autotune does it.

[@mavisbavingson2299]

this video is seriously awesome and inspiring thank you for putting in the effort! 😊

[@RaccAttacc01]

This is awesome! I have been planning an onboard active pitch shifter for my electric guitar for about a month, so thanks for the video. Now would you happen to know how one would make this work on an Arduino micro?

[@RealNovgorod]

It's all about that phase, 'bout that phase.

[@wChris_]

Wait there was an almost identical presentation at the ADC22, just 4 Months ago! Coincidence? I think not!!

[@Cineenvenordquist]

Analog Semi does a conference? No, it's just audio not radar etc. so ...whoever they are, they have a YT Channel ADC. Or not, but some talks are there.

[@geraintluff]

That was me, here you go 😄 kzbin.info/www/bejne/nHu4nqCZfLCNg6s Yeah, this video hits many of the same points in a pretty similar order to mine - but presentation counts for a lot, and Jent's done a good job here. My goal is always for a train of thought to feel natural and obvious in retrospect, so if that's what happened here I'd be flattered.

[@eqwerewrqwerqre]

This is amazing!