Stock Price Prediction & Forecasting with LSTM Neural Networks in Python

[@GregHogg]

I offer 1 on 1 tutoring for Data Structures & Algos, and Analytics / ML! Book a free consultation here: calendly.com/greghogg/30min

[@RishiKahtri]

please provide link for that big Window code tutorial. Thanks

[@Santhosh_sandi]

go through his notebook which is the first description@@RishiKahtri

[@elbala777]

Thank you so much for this video. IT taught me LSTM and how to use Python

[@RI-zn3ju]

I have a LSTM Model that I think is good but I want it to run multiple stock tickers at the same time to save time and give me all their individual results. I am having a tough time figuring out how to do this and am constantly getting errors can you help?

[@godspowerbunor2638]

HELLO I AM INTERESTED CAN I CONTACT U?

[@jtrobotics5421]

the value at the end is constant because theres a big logic error in your foor loop. as you always set it back to x_test[-1] before the prediction. s.t. the LSTM will output a constant value. Non the less if you do it correctly (i just tried it) the prediction will ofc diverge from the stock price anyways (it ll keep going in the direction of first prediction

[@benjamintenbuuren9652]

Can you show the example on how you change it? I also noticed that it is incorrect but have no idea how to correct it.

[@TheRafler]

I believe your final recurrent forecasting is wrong. last_window should be initialized outside the loop, and then for each loop do np.roll(last_window,-1) and update last_window(-1) such that you have a continuously-moving window. In your case you only update the last value, but the rest remain fixed and that is why your prediction remains flat.

[@clintonletsoela1479]

@TheRafler can you please paste the code, I tried using np.roll as you suggested but the prediction is still flat.

[@obedrolland3259]

@@shengchuangfeng227 you can do even better just by doing last_window = np.concatenate([last_window[-2:], np.array(next_prediction)]). You won’t need anymore to specify last_window[-3:] in your model.predict() function.

[@obedrolland3259]

We don’t need a np.roll() function. He should just change the last_window expression in the loop by : last_window = np.concatenate([last_window[-2:], np.array(next_prediction)]).

[@emmanueloloyede5774]

Using the price alone won't be a good idea. How about other variables like MA

[@MuhammadAlQaeda-z2o]

I'm gonne make a "course" about machine learning but I actually don't know how it works so I'm gonna copy-paste this function and voila, working like a charm. Way to go

[@alexchirapozu3234]

Hi Greg, I believe that you made a mistake at the recursive prediction. If you print your windows, you'll see that every time you predict with the same 3 values, instead of changing it at every iteration. That is why your plot is completely flat, which it shouldn't. Here is my correction (just for last 14 days), I think it's OK: recursive_predictions = [] recursive_dates = dates_test[-14:] last_window = X_test[-14] for target_date in recursive_dates: print(last_window) next_prediction = model.predict(np.array([last_window])).flatten() recursive_predictions.append(next_prediction) new_window = list(last_window[1:]) new_window.append(next_prediction) new_window = np.array(new_window) last_window = new_window If you try to predict with it still doesn't fit to the actual values, but at least is not a straight line hehehe

[@DevilErnest]

from copy import deepcopy recursive_predictions = [] recursive_dates = np.concatenate([dates_val, dates_test]) last_window = deepcopy(X_train[-1]) for target_date in recursive_dates: next_prediction = model.predict(np.array([last_window])).flatten() recursive_predictions.append(next_prediction) last_window = np.append(last_window[1:], next_prediction)

[@arpadikuma]

@@DevilErnest this one almost worked, the last line needs to be last_window = np.concatenate([last_window[1:], [next_prediction]]) otherwise it errored out on me. But yeah this is the way how last_window actually gets updated with the last three predictions of each iteration Still the result is almost a straight line...too bad :]

[@rushilsingh]

00:00 Forecasting Microsoft stock using LSTM neural networks. 03:56 Convert date column to datetime objects and make it the index of the dataframe 07:44 Convert the given data frame into numpy arrays for input and output 11:28 Performing univariate forecasting on closing value over time 15:06 Create and train a sequential model with LSTM and dense layers 18:35 The LSTM model for predicting data shows poor performance on validation and test sets. 22:00 Training LSTM model on a subset of data to improve prediction accuracy. 25:19 The model recursively predicts future values based on the available data.

[@liornisimov9367]

Hi, great video! I have written a more efficient function to window the data using the pandas "shift" method. Feel free to use it! def window_data(data, n=3): windowed_data = pd.DataFrame() for i in range(n, 0, -1): windowed_data[f'Target-{i}'] = data['Close'].shift(i) windowed_data['Target'] = data['Close'] return windowed_data.dropna()

[@GregHogg]

Thanks so much, well done!

[@JustAdrianYT_MTB]

This doesn't have an integer index so the resultant shape of X is wrong

[@MrDaniloDj]

​@@GregHogg Pin this!

[@majilarohit4]

@liornisimov9367 You should start a channel. Thanks:)

[@OtRatsaphong]

Great video, Gregg. Best one I’ve on how to implement LSTM on stock price prediction. I’m going to see if I can replicate this LSTM. Thank you 👍🙏

[@GregHogg]

Great to hear! Perfect!!

[@1990lietuva]

Hi, I might be wrong here but I think one of your issues about recursive prediction is - that you always replace the last element, meaning that it tried to predict from the same values with the last one changed, and that might be why it predicts the same. I would change the part to be (append prediction to the end) and pop the first one out,

[@LawrenceReitan]

Which is pretty much the same copy paste mistake all these stock predictors wannabe do Appreciate the effort in confusing thousands of people though...

[@ryanmilgrim6427]

I know little about machine learning but I think you could fix the extrapolation probelm by making predictions on the stock's log return rathern than price. with a pandas dataframe it should be a line like np.log( 1 + df.pct_change() ). The goal is to get the data down to a stationary time series, so modeling returns should get you closer but the data is not quite symmetric, which is why you also take the log. of course, you outout then becomes a series of logreturns, which you will have to apply this line to fix, np.cumprod( np.exp(df) - 1 ).

[@kilocesar]

I thought the same

[@mrrolandlawrence]

wow amazing. for a tiny mortal without a PHD in maths i actually understood all what you said. amazing work

[@GregHogg]

Glad to hear it :)

[@___OmerAJ___]

The problem is with experienced Ai developers they always wanted to predict the future price, but with real traders or investors they all wanted to take a profitable trades, no one wants to exactly know the price and nothing can do that

[@folashadeolaitan6222]

Hi Greg, thank you for another awesome video. I am working on a multivariate LSTM and i have scaled my data (containing both the input and output variables) as in the code below. scaler = StandardScaler() scaler = scaler.fit(df_train) df_train_scaled = scaler.transform(df_train) Now, doing inverse_transform as below in order to be able to compare my predicted values with the actual, i am getting an error. prediction_copies = np.repeat(prediction, df_train.shape[1], axis=-1) #This will repeat the prediction the number of times columns it was originally scaled with y_pred = scaler.inverse_transform(prediction_copies)[:,0] #Here i am taking all the rows and just the first column, since the other columns are replications of the first. ERROR Found array with dim 3. StandardScaler expected

[@gauravnarkhede3622]

I am also working on LSTM

[@folashadeolaitan6222]

@@gauravnarkhede3622 Nice one Gaurav, how is it going with u?

[@gauravnarkhede3622]

@@folashadeolaitan6222 Good, need some company to work

[@jpaulexo]

​Hi ​@@folashadeolaitan6222Did you found the error?

[@jakob4371]

Its an extremely basic one, so it should be resolved by now.

[@yan200go]

this actually explained things that I didn't understand by other people's explaination

[@GregHogg]

Glad to hear it!

[@quanxu1]

awesome video 👍 the flat line from recursive prediction is most likely caused by an glitch in the code. "last_window = deepcopy..." should be moved up and outside of the loop. After that's fixed, my guess is that the recursive predictions will converge going forward into an almost flat line

[@Algardraug]

Yeah, I noticed that too. Glad I'm not the only one! The loop is overwriting the last_window at the start so predicting on the same data in every loop should reasonably result in the same value. Also setting last_window[-1] = next_prediction every loop seems wrong. Realistically, you would want to shift the first two values before doing that otherwise, you'd be stuck with the first two values in the list the whole time. It should probably be something like this last_window = last_window[:-1] last_window.append(next_prediction) (I'm not very good at python)

[@YoungMoneyInvestments]

Glad someone else caught this.

[@idrisseahamadiabdallah7669]

@@YoungMoneyInvestments , have you tried this ? Because i want do forecasting, like predict after the test_dates that we already have ?

[@YoungMoneyInvestments]

@@idrisseahamadiabdallah7669I have and haven’t been able to get anything over 60% on shorter timeframes. I’m using 15 min candles with about 80k candles of data. It has been harder than I anticipated. I’ve tried 3 variations of LSTM’s,, 3 different random forests, a perception, and I’m not yet at the point where im pleased with the results.

[@37gippo]

You're a legend I was looking for ages for a explanation like this one

[@GregHogg]

Glad I could help!

[@Nedwin]

Yay, 2nd comment!! My Saturday night is now completed with this video! Thanks Gregg.

[@GregHogg]

You're very welcome and have a great night!

[@britox.6216]

still hate how you left us on a cliffhanger greg! please make a part two!

[@timseed4489]

One of the best explanations on the subject - I would do some of the pandas stuff slightly different - but overall 10/10

[@GregHogg]

Yeah, I would too looking back. But thanks - great to hear!

[@xiaoyang4521]

The lines are close to each other becuase you are always feeding "future" data, into prediction. The predictions are actualy 1 day shifted to the right, which means the so called prediction is actually delayed. This won't be helpful to stock price prediction at all.

[@luisaurso]

Hey Greg, thanks for sharing your knowledge. Amazing video and very well explained. Following you onwards

[@timnilson]

The first 6 minutes could be shortened to: df['Date'] = pd.to_datetime(df['Date']) df.set_index('Date', inplace=True)

[@arsheyajain7055]

Was waiting for this one!!

[@GregHogg]

Great to hear!

[@senna_william]

The final catch was fantastic, thanks for the lesson!

[@GregHogg]

Yeah I feel like adding that in there is very useful to know haha

[@senna_william]

At first I thought "I do not believe that he will be able to predict the market only with this information", a good lesson, congratulations!

[@GregHogg]

@@senna_william yeah! :)

[@cesarfierro4792]

Great video, it's really well explained. I was expecting the recursive predictions to work a little bit better since I will apply this to a school project more oriented to failure detection but I understand that's a hard task. But still is a great base to start with. Thanks so much.

[@carrocesta]

Greg, you are the man! greetings from Spain :)

[@GregHogg]

Thanks so much! I'd love to visit Spain some day

[@alrey72]

Nice, very detailed. I just have a few questions: 1. I suppose Sequential means Time Series. If that's the case, is there still a need to create 3 columns for the last 3 closing prices? My understanding of Time Series is it automatically looks on the order of values (output of last record is input of current record). 2. Believe LSTM is an optimization technique so that if there are many layers, the vanishing gradient problem will not be encountered. I'm also confused since you mentioned Adam (I'm not familiar with it) as optimizer so it might be in conflict with LSTM? 3. You mentioned LSTM(64) means 64 nodes and Dense(32) means 32 layers .. so its just 2 nodes per layer? Thanks.

[@lorryzou9367]

the last 3 closing prices are our input features, and the closing price for today is the output/prediction value.

[@alrey72]

@@lorryzou9367 Yes that's the intention. However, the neural network used is time series so the last 3 closing prices is redundant. If a feed forward neural network is used then the input of last 3 closing prices is logical.

[@rileyclubb]

I'm trying to adapt your code to hourly and minute time series. I think I need to modify the complicated section that you mention @6:40 can you help me find that tutorial to understand this section? I don't see that in the video description above

[@tariqmahmood2734]

Hi Greg, its a nice video to learn. Can u explain what changes we should made to forecast the values after the dates_test, mean after the end date (mentioned in the 2022-03-23) for few days or weeks.

[@1234567PokemGaming]

Guys, at the end of the video, it should not be a constant value. Just a little mistake. You guys can use my code for reference. In the video, the constant trend happened due to the last_window always keep constant. # Predict future from copy import deepcopy recursive_predictions = [] recursive_dates = np.concatenate([dates_val, dates_test]) last_window_new = deepcopy(X_train[-1]) for target_date in recursive_dates: next_prediction = model.predict(np.array([last_window_new])).flatten() recursive_predictions.append(next_prediction) last_window_new[0] = last_window_new[1] last_window_new[1] = last_window_new[2] last_window_new[-1] = next_prediction print(X_train[-2:]) # Print the last 2 elements of X_train print(np.array([last_window_new])) # Print the value of np.array([last_window])

[@gayanc6193]

Great content solved one of my major problems. many thanks❣

[@GregHogg]

Super glad to hear that!

[@Canada_Facts]

Thank god I found this video, I tried the same and really poor performance over long term

[@cbassett123]

6:40 "If you need to know what that code is, check out the tutorial"... Can you please add the tutorial to the description? Thank you!!

[@JeffPohlmeyer]

I have a concern. It looks like the predicted vs observed values are offset by a day. In other words, when zoomed in the predicted values do seem close to the observed, but they're delayed by a day. From the perspective of actual trading this is very detrimental.

[@GregHogg]

Hmm I'll have to check this out.

[@chris285as]

I watched another video on neural training and it said that the model is not good for this purpose as it just predicts the last day - or + on the close value that’s why it’s real close. Their reasoning is that if you take the difference each day it can’t predict and the error rate is huge.They suggested that it’s better to use other techniques for this reason.

[@chris285as]

For me the jury is still out I would do lots of testing and your own research

[@JilN660]

Hi thanks a lot for this video! How would you move from forecast to a Signal 1/0/-1 column to be applied to the price variation? I have issues with potential future leakage and the condition to use to create the signal. 😊

[@anmolpreet8959]

Congrats on Hitting 22k Subs.

[@GregHogg]

Aw thanks Anmol, and nice dab!!

[@peralser]

Amazing video. You were so clear to explain! Thanks!

[@GregHogg]

Great to hear!

[@ivandozoretz3031]

Great video! I've just one doubt: if I want to put the model in 'production' and predict prices for the next month, I won't have the previous 3 prices for most of those dates (in that production dataset). So, is it right to include those features in training,validation and test set?

[@otaviocoutinho2855]

I have the exact same doubt. Shame nobody aswered. I this ideia we could only predict the next day, but what you can do is try to make the same he did in the validation, you must try to predict a larger range of days and the performance will drop quickly, but problably enough for a few days ahead. Which I find interesting besides being limited.

[@rohanaryan2231]

hey! love the way you deliver your content sir! learnt a lot from you. just a quick question, can we use a direct type conversion to convert the type of date (pd.to_datetime or astype(datetime64)). if yes why didnt you use it and create a function for the conversion, how is it different from doing it directly as i mentioned above. thanks!! good day!!

[@RafaelRivetti]

Hi, Greg! In the MLP network, data from independent variables from date t are used to predict a future value t+n. In the LSTM network, instead of using only data from time t of the independent variables, it uses data from time t, t-1, t-2, ..., t-n as desired by the programmer, and after that, generates the prediction for a future time t+n? Is this reasoning correct? Thank you very much!

[@zeroj7492]

really nice tutorial and thank you so much for making it brother

[@sivaramjallu001]

Just Awesome! very helpful😀😀😀

[@antoniosaltoborsodicarminati]

You just made my project for bitcoin prediction in college, thanks man

[@GregHogg]

Sounds great!

[@luisaurso]

Greg if I understood correctly I can include more variables (multivariate) to make predictions (e.g. Predict the sales for Icecream using the Units per Day + Temperature) and for that I would need to include these 2 variables into the dataset and reshare the feature (X) accordingly, but I got confuded on how to windows the data ?

[@manuelnovella39]

Man, at 6:40 you refer to a tutorial in the video description that explains the function df_to_windowed_df. What link is that, exactly? Can't find it

[@obedrolland3259]

There is a huge mistake in your loop which predicts the test & validation closing prices according to the last 3 values of the training data. You should just change the last_window expression by : last_window = np.concatenate([last_window[-2:], np.array(next_prediction)]) And see what you will get.

[@obedrolland3259]

Awesome video by the way ! 👌🏽

[@heribertoquintanilla743]

Thanks a lot Greg, very interesting. One question, once you have your model, how to add new data, without re-train the whole model, but train just the new data and predict under the whole dataset? does it make sense? thanks a lot!

[@andrewlichte6734]

I think at some point you have to retrain your model. like how Greg threw out all the old data because microsoft went parabolic in the more recent years. the model didn't know how to cope with that. it's like budgeting gas money for a trip, but your're using gas prices from 5 years ago. You're just not gonna get where you want to go.

[@prashant8762]

Hello Greg, great video again as usual, can advice ,whether can we have a LSTM neural network for predicting stock price of multiple companies say 1000 companies ?

[@jarrodmautz159]

Yes compile each stocks data into a key value pair of a dictionary, the key will hold the stocks name, and the values will hold the data frames that contain the dates and prices/more indicators etc., then turn each into sequence, then train your lstm on each stock in your dictionary

[@prashant8762]

@@jarrodmautz159 thats very impressive explanation , do you have any sample notebook( if you free to share ) on similar use case or any article link ? iam really looking for one sample notebook, thanks for the reply ,it helps :-)

[@MrLam-lx7td]

Thank ， so much ! Very good project .

[@GregHogg]

You're very welcome

[@DevilErnest]

Hi Greg, great video and I learnt a lot here. May I ask whether detrending and deseasonalisation (if they exist) are needed during time series modelling. There are resources out there which suggests these should be done and the series must be stable before feeding time series to the model, while others (like yours) don't really care about trends and seasonality within the model and let the model does its work. My guess is there's no right or wrong answer to it, but what's the reasoning behind these decisions in time series modeling using machine learning algos? Thanks.

[@GregHogg]

There's always multiple ways of doing things. In theory you try everything and do what works best

[@fluctura]

majjjggic😂

[@sandhusukhdeep]

that is one excellent video to setup a basic LSTM model! great work and thank you

[@GregHogg]

Awesome! Thank you, and you're very welcome!

[@jeraldgooch6438]

Greg, Thanks for this. Two questions 1. Will adding in more target values (e.g. go to Target-10) help? 2. What would shuffling the windowed data set do? At least some of the data in the training set would be after 2016 then. Thanks

[@GregHogg]

Changing the window may help. Not sure what you mean by shuffling.

[@jeraldgooch6438]

@@GregHoggsklearn.utils.shuffle i think is what i mean. In your windowed dataframe, the 2nd row might end up as the 100th row, the 3rd as the 250th, the 1000th might end up as the second and so on. Obviously I do not have enough of the vocabulary to truly express myself and typing on an iPad is a bit clunky!

[@jeraldgooch6438]

@@GregHogg After shuffling, the only time related data would be within each sample. The shuffling would destroy any time related dependency between samples. This might not be desirable with the LSTM layer? Also, what might be the effect of just using % change on a daily basis, rather than absolute value of the stock. And this is where you get to say “Why don’t you go ttrry it out and let me know what you find”?

[@arian_apply]

I've done it before. It depends on future forecasting and some other parameters. There is no clear relationship between past data and the accuracy of results. Generally, you should test different scenarios based on your project and your data.

[@onlystudy4645]

at 11:41 why we have to convert it to 3-D matrix, can't we train the lstm on 2-d X matrix?

[@marselloormanda1525]

thankyou for the video, but i have a problem it seems like the prediction that i have is just a straight line, does that mean i need normalization for the data? cause the stock price in my currency is so big it reaches thousand

[@SakshiDwivedi-p8e]

My question is , how will one model fit all stock, every stock will have different behaviour based on news and companies end results, so will it fit all??

[@DawidMichna]

Great stuff! Why did you set date as an df.index in min ~5, was it necessary/what's a rationale behind it?

[@GregHogg]

See the other comments haha

[@AbedMotasemi]

Hi Greg thanks for this great video. I tried this on SP500 futures market and have three questions: 1- what did you do to Handel missing values for weekends and days that the market is closed? 2- what you did not do any normalization on your data? 3- I can see that my prediction and observation charts got pretty similar trend but looks like prediction has a lag meaning prediction is behind the actual which is not good at all. How can I fix that?

[@GregHogg]

Please don't attempt to actually predict the stock market with this :)

[@FRKHuang]

Hi Greg, my MAE is getting pretty huge, do you have any idea how to fix it? And wondering why you can achieve the low MAE without standardization, thanks!

[@anatoliyzavdoveev4252]

Super tutorial 💪 great code💪💪🤝

[@GregHogg]

Thanks so much and great to hear!!

[@ivancostabernardo7670]

Thanks for the brilliant work!

[@GregHogg]

Thanks so much for the kind words Ivan!! :)

[@someotherstuffs]

hi, why did you not include the other columns open, volume, low, high in the training dataset?

[@ouanesachouche6785]

because he actually doesn't know what he's doing

[@robottalks7312]

Will there be any advantage if we scale the closing stock price between 0-1

[@GregHogg]

You could definitely try that :)

[@teegee7801]

Im getting a syntax error on the def df_to_windowed_df(dataframe, first_date_str, last_date_str, n=3): where its telling me its incomplete.

[@konstantintomilin1826]

Would be interesting to see how inclusion of other features like trading volume and maybe some common technical analysis indicators could affect prediction!

[@fluctura]

I'll do this and keep you posted

[@dnas5629]

I have backtested many of the TI and found they are broken clocks. Not that they are not useful to help filter data, but as for predictions they ideal. The oscillators help with sideways markets, but not trending. Detecting trends before they happen is tough. As for inputs into predictive models I do not think it will help the model. But definitely try. TALIB is best library for that.

[@SanjayMantati]

@GregHogg, If i want 'Close' and 'Open' both (last 3 days data) as input for single row. What code I need to change. I tried to update middle_matrix.reshape and the InputLayer with 2D. The predictions went awful. Can you please guide?

[@ignessrilians]

Amazing guide again , Thank you so much for these REALLY helpful guides. I appreciate it alot 💯 🙏 I'm actually doing the same kind of task with autoregression forecasting (where the next inputs are previous output) for an arbitrary time series , but no matter how hard i try to find the best model, i still end up with my autoregressive time series being not even accurate at all and after days and hours of searching I'm still lost... Seems kinda impossible to do , but anyway , thanks alot for the guides again 🙏

[@GregHogg]

You're very welcome! And yeah it's pretty tricky

[@hulunlanteworku9341]

Awesome video , but how do we know our window size in this video case 3 is there a way to know that. I think it's most crucial while working with LSTM

[@GregHogg]

You choose

[@hulunlanteworku9341]

@@GregHogg Based on what? or by seeing the data visualization to see some pattern that repeats

[@ShuhaoGao]

What is the value for test_val?

[@tiagobrito9765]

Another great video! I have a doubt: My date column is in the format day/month/year hour:minutes. What should I change in the str_to_datetime def function to show hours and minutes?

[@fkeb37e9w0]

Hi Gred, I had a doubt, for an unseen data (which contains only the date), how can I send that data into our LSTM model and predict for the next 5 days, when I have no input data apart from the DATE? Please help.

[@pixusru]

Your input for LSTM is not a date, but prices for previous N days before now.

[@efhndz7990]

i have the same question what can i do? if i want to know values for next days

[@ffinalttrip]

Amazing work Greg!!👏 Do you know any good community to share and talk about projects like this one? discord or something...🧐

[@GregHogg]

Thank you!! LinkedIn and Facebook have some very large groups :)

[@uveuvenouve7680]

Can someone explain why the prediction is so flat when the train data is big? as the video show , it should be using the 3 previous days' price to predict the next day close price. that's mean it always predict a dramatic drop of few hundred dollars? Is that the cause of LSTM? would it be better to use the price differences to make this model?

[@aadarshsingh2247]

I have a doubt asked by my professor "Why are we using only one attribute "close price" when the stock market can depend on volume too????

[@GregHogg]

You absolutely could

[@moekhaled]

thank you , where is the windowed_df code explanation ? i could not find it in the description ?

[@mahdis-hs6bn]

Hi the reason that it couldn't predict correctly was because there was never a rise like the test set in train set right ?

[@mingchannel9672]

I guess the correct recursive prediction should be : from copy import deepcopy recursive_predictions = [] recursive_dates = np.concatenate([dates_val, dates_test]) last_window = deepcopy(X_train[-1]) for target_date in recursive_dates: next_prediction = model.predict(np.array([last_window]), verbose=False).flatten() print(target_date, last_window[:, 0], next_prediction) recursive_predictions.append(next_prediction) last_window[0] = last_window[1] last_window[1] = last_window[2] last_window[2] = next_prediction However, it not working as well.

[@alexandrverenchuc7524]

it does not work because you get an array of single arrays and the code is good, thank you

[@G5Locks]

Can you implement this in a streamlit interface

[@skinderspike7564]

Error: Window of size 3 is too large for date 1986-03-18 00:00:00 How to Resolve?

[@kkololp]

if your data is not linear, you can use the log to make it linear

[@afseeniqbal]

hello I am new to this so I was wondering if this exact model would work on say VTSAX stocks and not just microsoft? if I were to just import VTSAX data instead of microsoft data as a csv in the beginning

[@marcosagustinmel2294]

Hi Greg, Thank you very much for the video! It's excelent! I have a doubt regarding the windowed_df function. What is the meaning of the if statement about df_subset? I'm talking about this part: while True: df_subset = dataframe.loc[:target_date].tail(n+1) if len(df_subset) != n+1: print(f'Error: Window of size {n} is too large for date {target_date}') return Because I'm getting this error when I pass to the function a period higher that 2 years (+/-) . For example from 2020-12-01 to 2022-12-4 is retrieving me the correct array (with the 3 timesteps), but if I pass 2020-05-01 to 2022-12-4 I'm getting the error "Window of size 3 is too large for date 2020-05-01 00:00:00" So how can I solve it ? Why this statement is present? What is trying to avoid? Many thanks! Regards Marcos

[@ricardodahuss9939]

Same here :(

[@ismaelbastos4097]

Very good video!!! I have a question, when we separate the validation data, because of the the size of the window is three, the first row of the validation tdata will carry out the last 3 values that are in the target of of the three last rows of train data, and the second row will carry two values of the the target of last two rows of train data and so forth. Is this considered a data leakage? I mean, I have the target of train set as inputs of the validation set, is this ok?

[@GregHogg]

I see what you're saying, and thanks for the kind words. I wouldn't really consider this is a leakage. Formally? Maybe. Practically? Not really

[@ismaelbastos4097]

@@GregHogg Thanks for the answer. I really appreciate your job here on KZbin and on LinkedIn. So I have to be careful when choosing the size of the window, right? Because, if I choose a larger window, the amount of data that will "leak" will grow.

[@GregHogg]

@@ismaelbastos4097 you're very welcome! And yeah, I guess I agree with that. I really wouldn't worry too much though, when you're in a company these things will sort themselves out.

[@rizkamilandgamilenio9806]

amazing video, why does your model on this video perform differently from the actual code.

[@GregHogg]

Thank you! Probably just randomisation

[@poojithachalla8106]

@Greg Hogg Please provide vedio link for window function

[@Wissam-rk7tv]

thank you for the vidéo , it's amazig, how if we wont to forecast stock prices for different company ? can you make a video about this case

[@GregHogg]

Exact same thing

[@ntepelaletsoela812]

Danko means thank you🤟🙏

[@GregHogg]

Danko! :)

[@SowgandBakka]

Where do I find windowing the data frame video the size doesn't support my dataframe and how do I change the dates according to the size of the window?

[@heylucyhere588]

What type of LSTM it is ? Vanilla , stacked or CNN ?

[@GregHogg]

Good ol' regular lstm

[@Jipzorowns]

I don't get the part on 20:25, how is it possible that it gets the beginning of the prediction so right? I just don't see how this is possible, am I missing something?

[@pranaliredgaonkar8154]

Which mathematical model we you have used for prediction?

[@ffinalttrip]

LSTM

[@liuyi8713]

great video, but it never pass following part running. import numpy as np def df_to_windowed_df(dataframe, first_date_str, last_date_str, n=3): first_date = str_to_datetime(first_date_str) last_date = str_to_datetime(last_date_str) ...... ...... # Start day second time around: '2021-03-25' windowed_df = df_to_windowed_df(df, '2021-03-25', '2022-03-23', n=3) windowed_df

[@VT-Rathore]

Have u got the solution?

[@явыиграл-д6ф]

Thank you

[@GregHogg]

Very welcome!

[@srishtisdesignerstudio8317]

very helpful

[@GregHogg]

Thank you!

[@ethanriver8393]

idk if someone else had trouble converting the date column, but there is a solution: df = pd.read_csv('TSLA34.csv') df['Date'] = pd.to_datetime(df['Date']) print(df['Date'].dtypes)

[@Devmaxxer]

I don't understand why he didn't use the built in function of panda to_datetime to convert

[@chaunguyen5378]

what is the meaning of prediction for every single day, nothing, your prediction give back the value with big standard error

[@dananjayaidabagusgede5144]

i copy your code in description, why my data is error? "window of size 3 is too large" what thats mean? thnks sir i hope you answer soon 🙏

[@SafwanAlselwi]

I got the same error, but I fix it by downgliding the entire max dataset

[@ricgondo]

Just Thanks!!!!!

[@GregHogg]

That's super nice of you, thank you so much!

[@ricgondo]

@@GregHoggNot at all! Thank you sir! Jupiter was fine but, now trying to figure out why I can’t run in my Ubuntu terminal lol!

[@ricgondo]

OMG, nicely working on a terminal, but I will stick with Jupyter. First time using this thing... so much better!

[@Paul-mk3jt]

I think you made a mistake at the end with the recursive predictions, you are not updating the last window correctly. Great video nonetheless!

[@GregHogg]

Hmm thank you I'll have to take a look at that!

[@felipegiacomel9751]

@Alejandro Pachon it worked for me, thank you! But I had to change the last line to last_window = last_window[1:].reshape(len(last_window)-1,1), or I would get an exception Also, although it works, I didn't understand why you make a copy of X_train only to deepcopy it later

[@eshaangupta8637]

Great Video

[@GregHogg]

Thanks Eshaan

[@abhirajranjan1518]

Why is that windowed_df function taking so long to run, it isnt stopping...