Forecasting Future Sales Using ARIMA and SARIMAX

[@abhisheksharma8798]

first differencing (of order=1) has been done to de-trend the data. Once it is de-trended, it should further be deseasonalised by differencing again (of order =12). Thus, we have original data-> order1 differencing -> order 12 differencing. The final data will now start from t=14, and it is then checked for stationarity by ADF. The values of PACF at lag=1, lag=12 (for the final transformed data, after two levels of differences) are comparatively higher than PACF values at other lags (as evident from figures). Thus p has been taken as 1, implying AR(1). actually it is written as 1,1,1,12. p is taken as 1, because it reflects the highest PACF value, means 1 lagged value is highly correlated with its subsequent value as compared to other lag values.

[@kanchanwelcomes]

Kindly make video sir....nd explain on stock market data..take 10 company and kindly make a video

[@priyaarora4436]

Every body is using very easy to see "seasonal" dtaa to make youtube videos. If you wanna teach, teach with a highly random data!!!

[@nikhilvishnuvadlamudi]

at 10:05 - You mentioned that we will accept the null hypothesis. There is correction here - you never accept the null hypothesis, its just that there isn't enough evidence to reject it.

[@viveksivalingam9181]

Yeah you either reject H0 or fail to reject H0 due to lack of evidence

[@crazy_man_007-mz9ho]

Thank you so much sir .❤❤... even after 4 years in 2024 your content is helping alot of people including me .....❤

[@arjyabasu1311]

Pretty complex topic sir...need an intuition video of this !!

[@poissongirrl]

Thank you for this great material. Amazing videos that you create and open-source codes helped me land a dream job as data scientist. Thank you ❤

[@mahesh.khatai93]

Hi Krish , Thanks for the video on ARIMA time series analysis . I have few doubts from the video 1> Regarding Hypothesis testing -- how many times do we need to test inorder to get idea about data being stationary . 2> if ARIMA does not support seasonal data , do we have to make the raw data stationary like in video using differencing . And directly do modelling . 3> What is start , Stop dynamic parameters used in predict functions . Thanks .

[@dikshitlenka]

Hi @Mahesh, Please find the below answers of your question . 1st question- If your data is not stationary by the help of differencing you can make them stationary. In most cases time series data becomes stationary with d=2. 2nd answer- No model supports seasonal data because most of the Time series data are made based on the assumption that time series data is stationary. So you have to make them stationary before using any algorithm. You can make them stationary by differencing. As I mentioned with d=2, most of the data becomes stationary. 3rd answer- start is from which index you want to start the prediction and end is till which index you want stop. It's like a range of index. I believe it make sense now. Let me know if you have any further question. You know how to reach out to me. :)

[@chillbro2432]

@@dikshitlenka I'm very much new to time series. I want to learn Time series. Could you please suggest me any place where i can get to know about time series in detail. Thanks

[@dikshitlenka]

@@chillbro2432 Hi Tammany, you can follow Krish’s videos as well as check out videos from other KZbin channels. There are good blogs on towards data science/medium. You can check out them also.

[@mahikhan5716]

@@dikshitlenka 1. could u please tell me why did he select differencing order for both arima and sarimax as (-,1,-) since he selected seasonal differencing where he shifted 12 times so according it should be d=12 , am i right if wrong what the logic here ? 2. i am clearly seeing this here acf plot and pacf plot sharply declined after starting and it is 1 so how actually define exponential decrease for acf plot in MA ? what is actually shuts off ? 3. on which basis the start and end are selected for forecasting? what's the rules here

[@alanpalacios7784]

I never understood this at college and now it is really clear with your example. Thanks a lot!

[@huxleyhudson2261]

i guess Im asking randomly but does anybody know a trick to log back into an instagram account..? I somehow lost the login password. I would appreciate any tricks you can give me.

[@johnathanedwin6696]

@Huxley Hudson instablaster ;)

[@DP-od4yr]

Thanks a ton Krish Sir, got a job in Flipkart in Analytics coz of ur helpful playlists! Please help supply chain guys like me with problems and tools in that sector also... Plz plz plz

[@shubhankarray2515]

hi can you tell me how did u apply etc?

[@maheshkarigoudar117]

I think we don't accept null hypothesis but it's failed to reject null hypothesis so accepting status quo, it doesn't make difference in output but correct way of seeing it

[@AMANRAJ-jl5ub]

Very true, one should note that you cannot accept the null hypothesis, we can only reject the null or fail to reject it.

[@viveksivalingam9181]

When you do differencing once ( so Integral of order one ), the series 'Seasonal First Difference' is stationary as per ADF test. Then when you make a estimation by using SARIMA model, you should use the transformed series and not the original non-stationary 'Sales' series. Correct me if am wrong Krish ! Cheers

[@aswinaravind2801]

actually no. There are two things which you can do. If you are specifying d= 1 or 2 or any number as per order of difference, then you should provide the actual series. Otherwise you can feed the transformed series and then keep d as 0. Because d will internally do the transformation.

[@viveksivalingam9181]

@@aswinaravind2801 Yes, you are right that can be done as well.

[@sharifalmahmud8071]

@@aswinaravind2801 but in this case he made the series stationary by differentiation 12 steps, doesn’t it make the d=12 ? I am confused

[@lanslans6409]

@@sharifalmahmud8071 i don't think so, d= 1 implies that the series is differenced once

[@abhisheksharma8798]

first differencing (of order=1) has been done to de-trend the data. Once it is de-trended, it should further be deseasonalised by differencing again (of order =12). Thus, we have original data-> order1 differencing -> order 12 differencing. The final data will now start from t=14, and it is then checked for stationarity by ADF. The values of PACF at lag=1, lag=12 (for the final transformed data, after two levels of differences) are comparatively higher than PACF values at other lags (as evident from figures). Thus p has been taken as 1, implying AR(1). actually it is written as 1,1,1,12. p is taken as 1, because it reflects the highest PACF value, means 1 lagged value is highly correlated with its subsequent value as compared to other lag values.

[@ashishmishra7506]

Most most awaited video for me , Thanks a lot sir 🙏🙏🙏🙏

[@dramekandya4918]

Very good teacher, his explication is clear and efficient thank your very much

[@maryamfarzad4123]

Is there any tutorial for Multivariate Time-Series Forecasting?

[@aryangupta4372]

hi im from 3 years later can you help me please?

[@vaibhavpandey7398]

Ese teacher pehle mil jate to bht acha hota

[@riyasmohammad9234]

I read an article about sarimax and was really confused. But this video helped me to understand easily. Subscribed

[@VIVEKYADAV-gc1ti]

I read it same but in a very complecated manner but you make it is easy and orgnise way

[@vzinko]

says sarimax in the title, but at no point were exogenous variables discussed

[@eBuddha33]

I am studying on time series from last few days. Thanks for adding this video in correct time.

[@hamzamehmood1318]

Hi I need some research topic for My MScs thesis related to time series. have you any??

[@surajjanampally7023]

good video. It gives a clean and good understanding . It is very useful if you are beginning to understand timeseries data analysis.

[@sowmyatushar7487]

good one!! however id like to know how do we predict 3 months of sales for 50 different items at 10 different stores.

[@lns8940]

You need to run this model for specific store and specific item

[@nwabuezeprecious457]

@@lns8940 how do you predict 6 months of sales of different items

[@haydnmann7736]

Thank you for Krish-ening me with your knowledge

[@ganeshkharad]

its nice explaination but...you should also explain why we are doing what we are doing...like you didn't tell why we want data to be stationary??? what will happen if it is not stationary.... kind of stuff...

[@ambar2595]

You are great. Some feedback, write everything you say in the notebooks and slowly and steadily read them. Form the thought with clear explanation write it down and then make a video, your channel will explode after that.

[@roopchoudhuri7755]

Good content, but you should have explained the part where you decide the value of p, d, q in ARIMA with the help of differencing, ACF, and PACF analysis. Should we change only one, or all of it? Lets say in my data the differencing(shifting by) 1 is giving a good stationary graph, and shifting by 12 is not, so in model the d should be 1 right! Here you showed that shifting by 12 gives a good stationary data in you case, then why you chose d as 1. Please explain that part and add it.

[@ancydcunha8121]

I think that the d value is not the number of digits you have shifted rather it is the number of shifts . Since the differencing was done only once that's why its 1. Even the variable name for the first shift was 'Seasonal First Difference'. Hope this helps.

[@Emotekofficial]

I would say "Failed to reject the Null hypothesis" rather than "Accept the Null Hypothesis".

[@ruvitkon]

Thank you very much. This really helped me on completing my final year project :)

[@thangasamyp6011]

Super explanation sir. I have thanks to you for my doubts clear from this lecture. Thank you sir.

[@SimranBansal-z8b]

Excellent Video with appropriate explnation

[@mks7846]

Please keep explaination with code of any real time deep learning project ?

[@trainsam22]

Feedback: This is great content, but you move the screen too much. Go slow while moving screen and mouse please.

[@ruhulhaque3407]

Hey @Krish Naik. Nice Explanation! I have one query - when to use dynamic=True or dynamic=False , while predicting using SARIMAX inside future_df['forecast'].

[@ruhulhaque3407]

@Tomislav Primorac thanks for replying.. I used both parameters dynamic=true and dynamic=false and my predictions were similar. Only I can find difference in graph. I have used both train set to predict test set for checking predictions. Also I have predicted for time period beyond test set for future. I asked similar query below in stack overflow but didn't get satisfactory answer stackoverflow.com/questions/68092670/approach-while-using-dynamic-true-and-dynamic-false-in-sarimax-forecasting

[@ruhulhaque3407]

@Tomislav Primorac Sure buddy .. I got your point but unfortunately my results were same for both dynamic and static prediction for next 24 months ..Please provide ur mail id , I will ping u.

[@theprashantprabhakarjaiswal]

Superb Exppanation Sir. Hats Off.

[@raghvendra60]

Hello Krish, I am a Healthcare consultant currently working in the Pharma and Medical device Forecasting (Mainly Excel ) I want to learn these platforms like ARIMA and PROPHET. I would be highly thankful to you if you guide me regarding this.

[@fasttimeboy]

We have to conduct residual test called Portmanteau to check the model adequacy ! That's missing in your video ! And Also there is no analysis on model reliability on future forecast in terms of confidence intervals !

[@teched1803]

Hey Krish , i couldn't understand the part on how to choose the value of p and q from graphs .Can you show some variations so we could get to learn the abrupt drop and exponential decay part in the ACF and PACF plots to choose the values of p and q.

[@viveksivalingam9181]

There is not much relation with past values, post 1 lag in acf and pacf plot. Thus you take p =1 and q = 1. The nature of ARMA is that these two will show exponential decay. For AR or MA, either one shuts off to zero.

[@devayanbasu2218]

Basically you understand the nature of the acf and pacf at each lags and check if it's declining sharply or exponentially. This is rather prone to error and time consuming. Normally in industry we use pyramid arima where we run a grid search to find the optimum value based on the akaike information (aic) or bic depending upon your selection parameters. To be sure aic penalizes models with higher complexity so your optimal model may not have the least aic.

[@duztv5370]

@@devayanbasu2218 please could you direct or drop a link that has a video on this method of you know of any. Please

[@TechyScientists]

good content, just a clarification, non-stationarity is related to trend, not seasonality and same is true for the ADF, which can check for unit root and hence stationarity but has no linkage with seasonality, please confirm if this is correct.

[@ravindarmadishetty736]

I hope we also need to remove the trend if it occurs. As airpassengers data contains both trend and seasonality. If we remove seasonality still we can see an increased trend in data

[@sachinborgave8094]

Thanks, please upload Deep Learning further videos.

[@aishwaryanarkar2954]

ua just FAB Thnak you very much for your guidance

[@deghanandreddy7168]

Can you please make video on multiple seasonalities in time series forecasting by day wise holiday wise weekend wise sales . Thanks in advance

[@yonathanwijaya2316]

Hi, I've been wondering.. isn't your plot looks pretty good because you included the forecasted date as training data? cmiiw and thanks!

[@nathanborel2597]

So, are ARIMA models supposed to NOT be "fit" on non-stationary data? Or just not derive order from? Because you did the seasonal difference to achieve stationarity and then just applied SARIMAX to the original non-stationary data

[@teched1803]

Which are the best forecasting techniques often used in Industries ?

[@hamzamehmood1318]

Hi I need some research topic for My MScs thesis related to time series. have you any??

[@kvafsu225]

Very nice presentation. Very clear

[@SP-db6sh]

Very useful. Please start a series or a paid course on Algo trading.

[@KARANKUMAR-pd6gl]

Oh, bro, I have got placed in the HFT domain in my campus placements at a decent package of 10+ LPA. Can you tell me how is the future scope of HFT/Algo Trading??

[@SP-db6sh]

@@KARANKUMAR-pd6glit's great job, but in this fast moving world one quant need to adapt itself with a latest tools & techs.

[@paraskumar693]

I am getting good predictions on this dataset using ARIMA

[@asmareadane3647]

Hi, Krish Naik How to filter-out columns record value using python. For example the column name is HC71. It have 10873 records. the record values are -119,-443,-164,300,250,50,-200,200,...etc. I want to give value >=200 "over",-200 up to 199 "mild",

[@monikakj7469]

Thank you so much, this video really helped me a lot:)

[@aji2847]

Very well explained, but you are very jittery. I couldn't follow the sometimes because you scrolled so much highlighting and moving the mouse.

[@shibangibarua2285]

at 10.13 you said to look at p-value to satisfy the condition that it is less than or equal to 0.05. and this condition is not met by the p-value hence it should go to your alternate hypothesis and declare it as stationary? clear this out please

[@tyagiFit]

if p-value is less than 0.05, then we reject the null hypothesis at 5% significance level; So, in this case, null hypothesis is "series is non-stationary" and the p-value is way bigger than 0.05; therefore we don't have enough evidence to reject null-hypothesis; so we are going to accept null-hypothesis; that's why we are assuming that series is "non-stationary" because this is what null-hypothesis states.

[@erinbai8510]

Even if the ADF p value is less than 0.05, we can only say there is no trend but not to say that the data is stationary right? Since being stationary means no trend and no seasonality. ADF cannot detect seasonality and cycle. Am I understanding right?

[@someshkb]

Thank you for the explaining it so simply...

[@victoriaharant103]

Great video, very helpful! Thanks!

[@kateeileen6840]

Fantastic, the explanations are very clear. Do you have a whatsup group?

[@sid321axn]

really awesome. That is what I m looking for so many days. Good job thanks :)

[@technospider1917]

Hey! Krish can you suggest to me which model gives me better accuracy if I have only a 15min dataset (performing time-series dataset).. plz I am waiting for your answer.

[@nita02215]

Can you please provide reason for why did we use adf test and not kpss test? Also what to do if adf test and kpss test yield contrasting results?

[@karthebans2420]

Hi Krish, Can you able to make video on pmdarima

[@piratetechie2411]

Hi sir, The part where you mentioned p,d,q is equal to P,D,Q , i don't think that is true. For eg, d= 1 in First order differencing, and D= 1 in First order Seasonal Differencing. Similary p,q is not similar to P,Q.. Both have different calculations..

[@ssvipl64]

Hi Krish, Good coverage of the ARIMA workflow. If the screen is zoomed , it would have been more easy for the visibility of the code.

[@prathmeshshinde5683]

Sir as you said that the hypothesis ' h0 ' is an assumption that we do . I have a doubt regarding that, what if we in the first case assume that our hypothesis 'h0' is stationary(reverse of what you have assumed) and go on with further discussion, are there in pre analysis done for assuming our hypothesis?

[@samratkorupolu]

I have the same doubt all the time, how do we assume H0, if we just assume viseversa, everything will change, I'm clueless

[@sreenivasshrihaan1318]

super explanation...sir...could you provide ARIMAX with weather parameters...

[@pushkarshukla9409]

great content. mouse movement is too much/fast and sometimes not necessary :)

[@rajeshjose7496]

Hi Krish, this is a great video. While running the python file, not sure why do I get this error "Cannot interpret '' as a data type".

[@mishabp3815]

Change u r numpy version to 1.18.1. It would help you

[@YogeshBiguvu2208]

Hi Krish, I have one doubt here @7:58 Mins. How did you take Null Hypothesis as "Not stationary"?. Cant we take Null Hypothesis as "Stationary" & alternate is "Not Stationary".?? What is the criteria for selecting null hypothesis? is Null Hypothesis always should have negative assumption like "Not stationary", "Not same", Not etc....

[@daniyal657]

i want to ask that why are you using jupiter instead of spyder because please do live stream on weather forecast and live stock exchange

[@naharaldamer2416]

thank you , I have one question , what is the purpose of converting data to stationary if you will going to use non-stationary data to fit the model and do the prediction?

[@erinbai8510]

I think it is to find the optimal values for the hyperparameters

[@ashwin_.0710]

Do you train the model on the original values and not the differenced ones?

[@r4rajiv1979]

Finding Error - fig = plt.figure(figsize=(12,8)) ax1 = fig.add_subplot(211) fig = sm.grahics.tsa.plot_acf(df['Seasonal First Difference'].iloc[13:],lags = 40, ax=ax1) ax2 = fig.add_subplot(212) fig = sm.grahics.tsa.plot_pacf(df['Seasonal First Difference'].iloc[13:],lags = 40, ax=ax1) NameError: name 'sm' is not defined

[@David-rb9lh]

I will explain for those who will pass on the video . sm refer to statsmodel library.

[@galymzhankenesbekov2924]

I have faced the problem of scientific notation in y-axis, how can i convert it to normal one? i am using df.groupby ....sum().plot(), where can i use .format()? thanks

[@faizrazadec]

Sir, if we have to forecast two variables let say generation and load, how to do that ? seperately? and if we have the time too in the timestamp along with dates, what in this senerio, why we set the index to the timestamp. Kindly responce

[@PRASANNA-vd6xo]

dataset is cooked dataset or taken from any published paper pls reply

[@yopiandrew622]

In ARIMA flowchart we should transform the data before differencing. Why you just differenced it ?

[@ankurpratap1968]

Hello Sir, What should I do if I have to predict that a player in gaming industry will come tomorrow to play or not ? This is for multiple players and the number of players are around 80000. Please guide me to overcome from this problem. Thank You.

[@bcr5430]

Can you do a video about sarmiax too? I was working with exogenous variables in a time series data and the function wasn't accepting the two variables I passes in the argument.

[@krishnaik06]

This includes sarimax

[@viveksivalingam9181]

Krish has used SARIMA, but you can use SARIMAX for exog variables with the same package. Syntax : SARIMAX(data1, exog=data2, order=(0,0, 0), seasonal_order=(0, 0, 0, 0))

[@dungvan7251]

I saw you don't split train and test, you put all data of sale column in model, if it's good when we test the model?

[@rahulbagal6741]

if anyone at this point got the error just replace your code by for value,label in zip(results,labels): print(label+ ' : ' +str(value) ) in the video it is shown as value;label it will give you error is running as value;label

[@nishantjindal4394]

Hey Krish QQ for forecasting which is better Arima/Sarima or RNN is there any comparison?

[@mohitpande2006]

great teacher, many thanks sir

[@shrikanthsingh8243]

Was good until 13:55. After that, it's messy and incomprehensible. Seasonality was already removed, then you could have fed that data to ARIMA.

[@AbhishekMishraiitkgp]

Thanks for wonderful video :)

[@maazansari9774]

You have taken a "seasonal first difference" hence capital D=1, why is small d=1? You haven't taken the first difference

[@MacronageChain]

i think that when he does the seasonal first difference, he needs to subtract from the first difference and not from the original Sales data.

[@CreatingUtopia]

@@MacronageChain right

[@stonesupermaster]

Hello Krish, thanks a lot for your video. I wanted to ask you if you've read how to apply forecasting models to time series with multiple SKU (like 500 - 2000) considering the efficiency while running it, thinking of using the forecast once every week. I would really appreciate if you can indicate me a study case or real case in which I can take a look at the approach within the code. Thanks in advance!!

[@harikrishnanrajesh3118]

Did you get any help with this for multiple SKUs??

[@swagatamandal7917]

did you get the solution to this problem

[@stonesupermaster]

Not yet! I've been trying on my own but the running time and calibration for each SKU is a huge problem to make it work and using it in the long run...

[@aditidalvi9105]

Hey! even I am working on similar project comprising of multiple SKUs. If you have any idea how to go about it kindly share !

[@muhammadadeelsiddiqui8235]

How you copy path of this data set Can I get it through keggle directly without download

[@AK-qt3sk]

thanks for the video, though the speaking is too fast, abrupt at times not clear. and dataset is too quickly being jumped from one area to another

[@manavshah2119]

Sir What is the difference between the d value of ARIMA and What is seasonal_order parameter Value of SARIMAX

[@siddheshambre5787]

I want to know that how to check the accuracy of this model and how to save the model for deployment on the website?

[@txx8302]

Krish, there are so many overlaps of career in data science that I want to know does a demand planner in retail company considered a data scientist as well?... As they are also predicting sales.

[@Irhtayagradnus]

Hi, How will you process this model with user input like if user give , year = 2000 then this has to feed to the algorithm dynamically and then forecasting needs to happen . how can we do that?

[@harshithm1739]

Why is lags conaidered as 40 while plotting autocorrelation and partial autocorrelation graphs?

[@magicmushroom9670]

Why did you used predict method again and not forecast ? as we are going to see unseen observations. model.forecast(6)

[@ErSonuSinghh]

Need simple tutorial on multivariate time series forecasting

[@HemanthKumar-lb4xt]

Good one 👌can u do for R also?

[@akhileshgandhe5934]

Great. This is very helpful 👍

[@cyberprit]

Hi, lags = 40, how did we arrive at 40 ? Thanks

[@amanpatkar7009]

ACF & PACF plots must be of the original timeseries before differencing.

[@ottolunam]

I am totally confused. In the differencing you selected d=12 to make the series stationary and then in ARIMA you select d=1. Can anyone explain this?

[@AmericanHorror43]

Hi! I am trying to replicate this model into my dataset, but where the "forecast" column came from?

[@lifestyle_leap-n8x]

which forecast column? are you talking about Seasonal First Diff..?

[@polash1978banerjee]

How do I make SPSS accept triennial intervals (Like 1989, 1992, 1995) in the 'define date and time' options?