How autocorrelation works

[@RashmiBanthia29]

Autocorrelation 7:02 Partial Autocorrelation 9:45

[@Ikarus2985]

And this is where the info on autocorrelation starts: 6:57

[@davidyt9365]

You're an hero, bro.

[@ranitdey7369]

This is simply excellent. I haven't seen a better explanation of this concept.

[@anighosh920]

This is the best video ever on autocorrelation! Thank You!

[@BrandonRohrer]

Thanks Ani!

[@MrVishyG]

Well explained! Very natural transition from topic to topic until I accidentally learned autocorrelation.

[@daesoolee1083]

Woah, thank you, Sir. This short tutorial gets me to understand this entire auto-correlation thing than 10 page pdf explaining with all kinds of formulas .

[@moesaleh12]

Nice and clear explanation. I will add at 4:30. That the points can fit the line very well but there is no correlation. It is the case when the fitted line is parallel to the x-axis or y-axis. It is the case of near zero variance variables.

[@joeaustinathimalamaria624]

Came to revise Autocorrelation. Ended up revising regression too. Great video.

[@visualdad9453]

One of the best videos about acf

[@bazoozoo1186]

Brandon's videos deserve a lot more likes and views. Very underrated.

[@valerianmp]

Thank you very much. This give the insight for ACF, not just mindlessly calculating them

[@VictorGoncharov-ln1dp]

Nice video, the best one I've seen yet about the concept of partial autocorrelation!

[@wexwexexort]

I watched this 2 years ago. Didn't understand the concept. I watch again in quarantine days.

[@yungrabobank4691]

Where you use the word "shift", when describing autocorrelations, wouldn't it be better to use the word "lag", as I've seen this appear in literature more often and am now wondering whether they're the same thing

[@kapricun]

they are used interchangeably, as well as the word "delay"

[@AONomad]

You covered regression better than videos on regression 👏

[@checkpeck]

best explanation found till now

[@AbrahamSalazar210]

Excellent explanation with an easy-to-follow example!

[@stephanie_ong]

Great video! I liked how you presented the python code that would perform what you're explaining.

[@antuirno]

The kind of explanation I was looking for! Thanks!!!

[@rodrigocarvalho6426]

Very good video. In Numerical Methods it always helps to visualise what you are doing

[@mohammedsaeed7241]

Amazing video, but the autocorrelation loop contains a bug. for shift in range(10) will start from zero, and when you temp[ : -0] you will get an empty array, and your code won't work, because you will do a corrcoef of an empty matrix with another that contains data, thus, the correct loop will be something like: for shift in range(1,10)...thanks a lot for the vid

[@oliverbalbuenahernandez43]

Best video on this topic. Thank you!

[@1Bugatti1]

Hello Brandon, thank you for the explanation. Especially the code helped to understand whats happening.

[@BrandonRohrer]

I'm glad to hear it Bartosz. Thanks.

[@kshanepaswett261]

Thank you for existing.

[@DebatingDog]

Instead of using list slicing it is a bit easier to use the "shift" method from the pandas library. The video in general is great :)

[@RagibShahariar]

I really liked the autocorrelation part!

[@leonsponzoni]

Auto correlation sounds useful for finding data cycles. Like every year temperature correlation, around ~365 days shift~, there is probably a max again. You can find cycles in shifts where correlations peak. Also thanks for showing how it can be refined the same way as tailor series. Didn't cross-concept that correlation can give differentials weight for interpolation.

[@mohammedalmarakby1221]

thanks for the great video, @4:57 should the greater than and smaller than signs be flipped in the moderate correlation plot ?

[@BrandonRohrer]

Thanks mohammed, you're exactly right :)

[@TheAmolchavan]

I was looking for good video to understand the concepts and this video has done that. I have the following questions: - Can we do correlation to compare stock market bottoms or tops? If we compare the market price by its 1 day 1 week prior price then will this help in prediction of top or bottom? - How can we compare two asset prices? I am thinking of below methods - Do a ratio of two asset prices and run the n-1 vs n comparison of the ratio - Do a ratio of two asset prices and compare the ratio vs one of the asset prices - Compare the % return of each assets against each other on X and Y axis Let me know what do you think

[@siddharthrawat7205]

Brendon, I would so love to learn ML from you. teaching code along with concept would definitely decrease the MSE between ignorance and knowledge. Thanks

[@BrandonRohrer]

Thanks Siddarth! In case you're interested, there's a whole collection of concept-with-code courses at e2eml.school/catalog

[@whenmathsmeetcoding1836]

this is best available in you tube thanks it was very helpful

[@armitosmt5753]

Sounds like George Clooney explains it

[@S4KDK]

Great explanation with such a good example. Easy to understand too. Thanks!

[@silentsnooc]

Hi! Great video but there is an error at 5:24 - The correlation is not negative if just the slope of the line is negative. Correlation is negative if the data points and the line have converse slopes. %matplotlib inline # JuPyter Notebooks import matplotlib.pyplot as plt import numpy as np n = 10 x = -np.linspace(0, 1, n) y = x + (0.3 * np.random.rand(n)) - 0.3/2 neg = np.corrcoef(x, y)[0, 1] plt.plot(y, marker='o', markersize=4, ls='None', c='k') plt.plot(x, label='negative ({:.2f})'.format(neg), c='r') plt.legend(); plt.show()

[@mankindh9183]

No error there, video is correct. you are plotting x and y independently on the same plot, not y against x. correct code should be: plt.plot(x,y)

[@alexeykokh8240]

Cool description, very detailed, with lots of graphs

[@AnkitGupta-zk6pz]

Great explanation, Thank you.

[@m.raedallulu4166]

Very nice demonstration!

[@carbon273]

Intuitively, how does one gather the data for the temperature scatter plot for autocorrelation? This is my only stumbling block.

[@BrandonRohrer]

To see a detailed example in practice (including step-by-step data collection and organization), check out e2eML course 212: e2eml.school/212

[@niveyoga3242]

This was explained excellently!

[@Hank-ry9bz]

7:00 autocorr

[@stepover12]

I think you might be conflating 2 different concepts: 1. de-trending the timeseries before forecasting with AR, 2. partial auto-correlation. For example the first de-trending step may not be just fitting a straight line, it could be non-linear or have seasonality etc. This is unrelated to PACF which teases out what marginally new information are gained with each auto-correlation step

[@everything_is_on_fire3155]

Really helpfull And straight forward U r genius man

[@cypherecon5989]

temp_change is not defined in your code is it? Something is missing there.

[@KAUSHALRMODI]

In partial autocorrelation, the graph for shift 2 is incorrect, it has to show the points tilted along the 1st fit.

[@alteshaus3149]

Very clearly explained! Great video

[@BrandonRohrer]

Many thanks!

[@carlosnavarro9588]

Amazing explanation, subscribed!

[@BrandonRohrer]

Thanks Carlos :)

[@darmstadtvideo]

nice and clear explanation for this useful concept! for me its just not clear how to determine whether or not a value is still whithin the 95% confidence interval? R is giving some limits but there it remains unclear as well how to calculate those boundaries..

[@BrandonRohrer]

Thanks darmastadvideo! This is important, but unfortunately didn't make it into this video. I'll have to save it for a future one.

[@IslandRai]

Good job on this video! I wish it would have showed up when I searched for information on ARIMA, all the ones that did were pretty poor.

[@bulletkip]

fantastic explanation

[@ruthgerrighart1669]

Very nice and clear video!

[@Arthur-nf9dm]

Is the autocorrelation with zero lag always 1? Because I've read that Gauss-Markov processes have the following equation for autocorrelation: autocorrelation(lag) = variance*exp(-constant*lag) In the case of this formula, autocorrelation(0)=variance (!!!) why is it different in this case?

[@eloboa5355]

no question about this. with no lag, you're basically computing the correlation of a variable with itself. Any algorithm that gives you something different than 1 in this case simply fallacious.

[@Patapom3]

Another great and super clear explanation! Thanks.

[@yungrabobank4691]

Excellent explanation, thank you!

[@BrandonRohrer]

Thanks yung!

[@anindadatta164]

It is not clear in the video how much residual of the previous periods to be added to the time series equation of the present period.

[@matthiasrichter9264]

Very nice and explanatory video! Thanks! One thing that you could also do is mention real-world applications. A toy model is always very helpful for me to *understand* a topic but real-world application helps me to also *motivate* it. Besides that keep it up your videos are super nice! :)

[@dribblersmy]

I am also currently learning so I'm not sure if my idea is entirely correct, you could use this for any data set, to see how it changes. You could use the auto-correlation on gas prices to see how it changes with time, on flight tickets to see how it changes on a day to day basis or even on the price of a big mac to see how it has changed over the years.

[@PikesCore24]

What is the payoff of all this autocorrelation stuff? Once you have fit the data to the line and calculated the correlation, it seems like that is the all the useful information. I can't see where the autocorrelation ads information that would be useful.

[@sebastianliendo6579]

Autocorrelation helps understand seasonality/periodicity in time series data, which can determine when will a value repeat itself in the future. A high autocorrelation means the selected shift/lag is likely to be the period; and instead of guessing which lag value will yield a high autocorrelation, we can checkout something called the correlogram. This can be useful, as sometimes the time series has both a trend and seasonality, and the correlogram decomposes it (sort of).

[@JagoBridgland]

im confused! So when you talk about correlating the shifted (lagged) dataset. Lets say in an AR(1) process you shift the Y values by 1 day, does that mean the Y-1 dataset will have 1 less data point than the original Y dataset? And by extension if you specify an AR(10) process are you then knocking off 10 data points and hence correlating two datasets where one dataset has 10 less data points than the other? Sorry if i explained that like a moron, maybe this will help: let's say y = [1,2,3,4,5,6,7,8,9,10] y-1 = [1,2,3,4,5,6,7,8,9] y-2 = [1,2,3,4,5,6,7,8] y-3 = [1,2,3,4,5,6,7] y-n = [.........] so for example if i'm interested in specifying an AR(3) process, to determine the autocorrelation would i just find the correlation coefficient between y & y-3: y = [1,2,3,4,5,6,7,8,9,10] y-3 = [1,2,3,4,5,6,7] i am confused as to whether the fact you are correlating a [1,10) with a [1,7] is correct? or an issue? or what! sorry this was really long!

[@dhirendrasingh7379]

yes . Think of it like this , when you are using a lag of 1 , you are comparing every day value to its previous day value therefore automatically for day 1 you won't have anything to compare it to. But it doesn't matter , the data points left are enough to give an accurate correlation value.

[@JagoBridgland]

Dhirendra Singh amazing, thanks so much for the response. I have since experimented with some data and it makes much more sense now! Cheers mate

[@rezaulkarimmamun6211]

How to compare two voice signal similarity in python?

[@anindadatta164]

If in the time series , the residuals on temperatures day i after fiting temp(i-1) are correlated with actual temperatures of day i , then can it be said the there is partial auto correlation with lag beyond (i-1). If that method holds good ,then it could be much easier method of detecting PACF. But I have liked your video also.

[@Mark_Isaiah]

do you know how to do a white noise time series to test for seasonality?

[@Tom-qk4jq]

really helpful video, thank you for this

[@romellfudi]

Thanks a lot, view the code helpful to understand quickly

[@zagwask78]

The graph at 10:46 and the graph at 10:52 are the same but the x-axis is labeled differently. Which is accurate?

[@zagwask78]

And does the correlation coefficient, r, from the regression performed on the graph at 11:02 give you the point on the PACF (2, r)? And the graph at the right I'm confused by. If you are regressing the series with lag 2 against itself it would be a perfect fit. Why are there residuals at all?

[@keek1337]

@@zagwask78 doesn’t answer questions, sad :(

[@pv1234567890pv]

Amazing video! Very useful. Thanks!

[@alimahmood4158]

can i have the data sample sir on the basis of which you are calculating thanks

[@nirjhorchakraborty1861]

Stellar explanation!

[@ferozshahquriashi3021]

Close your eyes and you can hear Matthew McConaughey explaining autocorrelation. The only thing missing in here "Alright, alright, alright"

[@xenonmob]

he literally sounds nothing like him

[@mikel27180]

No

[@pigrebanto]

good but i think the way you represent the days shift in the code using [1:] and [:-1] is wrong (i do not get the meaning) or please explain what you meant. tks

[@cetrusbr]

It isn't wrong, its just python notation to slice a array. Just google it :)

[@brandonp3991]

This is amazing, thank you

[@tariqahassan5692]

Very very nice explain .. many thanks

[@BrandonRohrer]

🙏

[@neelabhchoudhary2063]

this was super helpful

[@Considerationhhh]

nice and concise video!!

[@sohailhosseini2266]

Thanks for the video!

[@BM-uf4pp]

great video man

[@Alexander-pk1tu]

very good video!! Thank you

[@bilalkarsl671]

THANK YOU

[@anaskhan4841]

Great Video

[@gagnon124]

great video

[@ishan7975]

wonderful!

[@VY-zt3ph]

Give this video a like

[@pypypy4228]

I like it! Thank you!

[@thechhavibansal]

Thank you :)

[@totochandelier]

Thanks a lot!

[@dribblersmy]

Thank you! :D

[@dnyanadaarjunwadkar7441]

This is great. Thanks

[@Lyndanet]

Appreciated

[@alirezafarhadizadeh5674]

Excellent

[@marlonzapata5429]

Damn. So mf nice!!! 10 stars

[@BrandonRohrer]

Aw, thanks Marlon. I'm really happy it clicked.

[@zeracyb3318]

Thanks !

[@astherphoenix9648]

6:48 you'll pull off??

[@ShiyanAkalankaJayanath]

Nice

[@zoozolplexOne]

cool !!!

[@omkarsalunke7117]

👏

[@typicalidealist9138]

🤓

[@chrisidema]

"degrees" is not a unit

[@nexyboye5111]

a degree is a dimensionless unit of measurement

[@charbelhamouche8016]

Bad

[@parthagarwal4592]

Poor Explanation

[@timetraveller3647]

Thank you!

[@omaramil5103]

Wow thank you

[@andrewluo6598]

Thanks!