Stock Price Prediction & Forecasting with LSTM Neural Networks in Python

[@rushilsingh]

00:00 Forecasting Microsoft stock using LSTM neural networks. 03:56 Convert date column to datetime objects and make it the index of the dataframe 07:44 Convert the given data frame into numpy arrays for input and output 11:28 Performing univariate forecasting on closing value over time 15:06 Create and train a sequential model with LSTM and dense layers 18:35 The LSTM model for predicting data shows poor performance on validation and test sets. 22:00 Training LSTM model on a subset of data to improve prediction accuracy. 25:19 The model recursively predicts future values based on the available data.

[@TheRafler]

I believe your final recurrent forecasting is wrong. last_window should be initialized outside the loop, and then for each loop do np.roll(last_window,-1) and update last_window(-1) such that you have a continuously-moving window. In your case you only update the last value, but the rest remain fixed and that is why your prediction remains flat.

[@clintonletsoela1479]

@TheRafler can you please paste the code, I tried using np.roll as you suggested but the prediction is still flat.

[@obedrolland3259]

@@shengchuangfeng227 you can do even better just by doing last_window = np.concatenate([last_window[-2:], np.array(next_prediction)]). You won’t need anymore to specify last_window[-3:] in your model.predict() function.

[@obedrolland3259]

We don’t need a np.roll() function. He should just change the last_window expression in the loop by : last_window = np.concatenate([last_window[-2:], np.array(next_prediction)]).

[@emmanueloloyede5774]

Using the price alone won't be a good idea. How about other variables like MA

[@emrevurgun8328]

ive corrected the code but the model acts incredibly naively like if the prices are going down at the end the model says the price will go down some more and flatten. i think the issue is much more complex, it might be best to use a more complex model and use different techniques. the model just cant create the chaotic randomness of the actual market, it might actually be impossible with this method but as stated in the video it is a decent indicator of the next day or something perhaps

[@jtrobotics5421]

the value at the end is constant because theres a big logic error in your foor loop. as you always set it back to x_test[-1] before the prediction. s.t. the LSTM will output a constant value. Non the less if you do it correctly (i just tried it) the prediction will ofc diverge from the stock price anyways (it ll keep going in the direction of first prediction

[@benjamintenbuuren9652]

Can you show the example on how you change it? I also noticed that it is incorrect but have no idea how to correct it.

[@MuhammadAlQaeda-z2o]

I'm gonne make a "course" about machine learning but I actually don't know how it works so I'm gonna copy-paste this function and voila, working like a charm. Way to go

[@manuelnovella39]

Man, at 6:40 you refer to a tutorial in the video description that explains the function df_to_windowed_df. What link is that, exactly? Can't find it

[@alexchirapozu3234]

Hi Greg, I believe that you made a mistake at the recursive prediction. If you print your windows, you'll see that every time you predict with the same 3 values, instead of changing it at every iteration. That is why your plot is completely flat, which it shouldn't. Here is my correction (just for last 14 days), I think it's OK: recursive_predictions = [] recursive_dates = dates_test[-14:] last_window = X_test[-14] for target_date in recursive_dates: print(last_window) next_prediction = model.predict(np.array([last_window])).flatten() recursive_predictions.append(next_prediction) new_window = list(last_window[1:]) new_window.append(next_prediction) new_window = np.array(new_window) last_window = new_window If you try to predict with it still doesn't fit to the actual values, but at least is not a straight line hehehe

[@DevilErnest]

from copy import deepcopy recursive_predictions = [] recursive_dates = np.concatenate([dates_val, dates_test]) last_window = deepcopy(X_train[-1]) for target_date in recursive_dates: next_prediction = model.predict(np.array([last_window])).flatten() recursive_predictions.append(next_prediction) last_window = np.append(last_window[1:], next_prediction)

[@arpadikuma]

@@DevilErnest this one almost worked, the last line needs to be last_window = np.concatenate([last_window[1:], [next_prediction]]) otherwise it errored out on me. But yeah this is the way how last_window actually gets updated with the last three predictions of each iteration Still the result is almost a straight line...too bad :]

[@rileyclubb]

I'm trying to adapt your code to hourly and minute time series. I think I need to modify the complicated section that you mention @6:40 can you help me find that tutorial to understand this section? I don't see that in the video description above

[@ricgondo]

Just Thanks!!!!!

[@GregHogg]

That's super nice of you, thank you so much!

[@ricgondo]

@@GregHoggNot at all! Thank you sir! Jupiter was fine but, now trying to figure out why I can’t run in my Ubuntu terminal lol!

[@ricgondo]

OMG, nicely working on a terminal, but I will stick with Jupyter. First time using this thing... so much better!

[@cbassett123]

6:40 "If you need to know what that code is, check out the tutorial"... Can you please add the tutorial to the description? Thank you!!

[@1990lietuva]

Hi, I might be wrong here but I think one of your issues about recursive prediction is - that you always replace the last element, meaning that it tried to predict from the same values with the last one changed, and that might be why it predicts the same. I would change the part to be (append prediction to the end) and pop the first one out,

[@LawrenceReitan]

Which is pretty much the same copy paste mistake all these stock predictors wannabe do Appreciate the effort in confusing thousands of people though...

[@onlystudy4645]

at 11:41 why we have to convert it to 3-D matrix, can't we train the lstm on 2-d X matrix?

[@skinderspike7564]

Error: Window of size 3 is too large for date 1986-03-18 00:00:00 How to Resolve?

[@ryanmilgrim6427]

I know little about machine learning but I think you could fix the extrapolation probelm by making predictions on the stock's log return rathern than price. with a pandas dataframe it should be a line like np.log( 1 + df.pct_change() ). The goal is to get the data down to a stationary time series, so modeling returns should get you closer but the data is not quite symmetric, which is why you also take the log. of course, you outout then becomes a series of logreturns, which you will have to apply this line to fix, np.cumprod( np.exp(df) - 1 ).

[@kilocesar]

I thought the same

[@liornisimov9367]

Hi, great video! I have written a more efficient function to window the data using the pandas "shift" method. Feel free to use it! def window_data(data, n=3): windowed_data = pd.DataFrame() for i in range(n, 0, -1): windowed_data[f'Target-{i}'] = data['Close'].shift(i) windowed_data['Target'] = data['Close'] return windowed_data.dropna()

[@GregHogg]

Thanks so much, well done!

[@JustAdrianYT_MTB]

This doesn't have an integer index so the resultant shape of X is wrong

[@MrDaniloDj]

​@@GregHogg Pin this!

[@majilarohit4]

@liornisimov9367 You should start a channel. Thanks:)

[@JeffPohlmeyer]

I have a concern. It looks like the predicted vs observed values are offset by a day. In other words, when zoomed in the predicted values do seem close to the observed, but they're delayed by a day. From the perspective of actual trading this is very detrimental.

[@GregHogg]

Hmm I'll have to check this out.

[@chris285as]

I watched another video on neural training and it said that the model is not good for this purpose as it just predicts the last day - or + on the close value that’s why it’s real close. Their reasoning is that if you take the difference each day it can’t predict and the error rate is huge.They suggested that it’s better to use other techniques for this reason.

[@chris285as]

For me the jury is still out I would do lots of testing and your own research

[@LikeWaterBeWater]

awesome video 👍 the flat line from recursive prediction is most likely caused by an glitch in the code. "last_window = deepcopy..." should be moved up and outside of the loop. After that's fixed, my guess is that the recursive predictions will converge going forward into an almost flat line

[@Algardraug]

Yeah, I noticed that too. Glad I'm not the only one! The loop is overwriting the last_window at the start so predicting on the same data in every loop should reasonably result in the same value. Also setting last_window[-1] = next_prediction every loop seems wrong. Realistically, you would want to shift the first two values before doing that otherwise, you'd be stuck with the first two values in the list the whole time. It should probably be something like this last_window = last_window[:-1] last_window.append(next_prediction) (I'm not very good at python)

[@YoungMoneyInvestments]

Glad someone else caught this.

[@idrisseahamadiabdallah7669]

@@YoungMoneyInvestments , have you tried this ? Because i want do forecasting, like predict after the test_dates that we already have ?

[@YoungMoneyInvestments]

@@idrisseahamadiabdallah7669I have and haven’t been able to get anything over 60% on shorter timeframes. I’m using 15 min candles with about 80k candles of data. It has been harder than I anticipated. I’ve tried 3 variations of LSTM’s,, 3 different random forests, a perception, and I’m not yet at the point where im pleased with the results.

[@OtRatsaphong]

Great video, Gregg. Best one I’ve on how to implement LSTM on stock price prediction. I’m going to see if I can replicate this LSTM. Thank you 👍🙏

[@GregHogg]

Great to hear! Perfect!!

[@Jipzorowns]

I don't get the part on 20:25, how is it possible that it gets the beginning of the prediction so right? I just don't see how this is possible, am I missing something?

[@muhammadfauzan6970]

Could I add additional data to build the LSTM model? For example, macroeconomic data (inflation, interest rate) along with ROE, PBV, and EPS. I am thinking of combining technical data (historical price data) with fundamental data (macroeconomic indicators and balance sheet data). Is this possible?

[@timnilson]

The first 6 minutes could be shortened to: df['Date'] = pd.to_datetime(df['Date']) df.set_index('Date', inplace=True)

[@marcosagustinmel2294]

Hi Greg, Thank you very much for the video! It's excelent! I have a doubt regarding the windowed_df function. What is the meaning of the if statement about df_subset? I'm talking about this part: while True: df_subset = dataframe.loc[:target_date].tail(n+1) if len(df_subset) != n+1: print(f'Error: Window of size {n} is too large for date {target_date}') return Because I'm getting this error when I pass to the function a period higher that 2 years (+/-) . For example from 2020-12-01 to 2022-12-4 is retrieving me the correct array (with the 3 timesteps), but if I pass 2020-05-01 to 2022-12-4 I'm getting the error "Window of size 3 is too large for date 2020-05-01 00:00:00" So how can I solve it ? Why this statement is present? What is trying to avoid? Many thanks! Regards Marcos

[@ricardodahuss9939]

Same here :(

[@37gippo]

You're a legend I was looking for ages for a explanation like this one

[@GregHogg]

Glad I could help!

[@___OmerAJ___]

The problem is with experienced Ai developers they always wanted to predict the future price, but with real traders or investors they all wanted to take a profitable trades, no one wants to exactly know the price and nothing can do that

[@luisaurso]

Greg if I understood correctly I can include more variables (multivariate) to make predictions (e.g. Predict the sales for Icecream using the Units per Day + Temperature) and for that I would need to include these 2 variables into the dataset and reshare the feature (X) accordingly, but I got confuded on how to windows the data ?

[@SakshiDwivedi-p8e]

My question is , how will one model fit all stock, every stock will have different behaviour based on news and companies end results, so will it fit all??

[@tariqmahmood2734]

Hi Greg, its a nice video to learn. Can u explain what changes we should made to forecast the values after the dates_test, mean after the end date (mentioned in the 2022-03-23) for few days or weeks.

[@Nedwin]

Yay, 2nd comment!! My Saturday night is now completed with this video! Thanks Gregg.

[@GregHogg]

You're very welcome and have a great night!

[@mrrolandlawrence]

wow amazing. for a tiny mortal without a PHD in maths i actually understood all what you said. amazing work

[@GregHogg]

Glad to hear it :)

[@adityasaxena3359]

When i tey to convert date to int , the function gives a value error , invalid litral for int base 10 , how to troubleshoot ?

[@someotherstuffs]

hi, why did you not include the other columns open, volume, low, high in the training dataset?

[@ouanesachouche6785]

because he actually doesn't know what he's doing

[@RafaelRivetti]

Hi, Greg! In the MLP network, data from independent variables from date t are used to predict a future value t+n. In the LSTM network, instead of using only data from time t of the independent variables, it uses data from time t, t-1, t-2, ..., t-n as desired by the programmer, and after that, generates the prediction for a future time t+n? Is this reasoning correct? Thank you very much!

[@alrey72]

Nice, very detailed. I just have a few questions: 1. I suppose Sequential means Time Series. If that's the case, is there still a need to create 3 columns for the last 3 closing prices? My understanding of Time Series is it automatically looks on the order of values (output of last record is input of current record). 2. Believe LSTM is an optimization technique so that if there are many layers, the vanishing gradient problem will not be encountered. I'm also confused since you mentioned Adam (I'm not familiar with it) as optimizer so it might be in conflict with LSTM? 3. You mentioned LSTM(64) means 64 nodes and Dense(32) means 32 layers .. so its just 2 nodes per layer? Thanks.

[@lorryzou9367]

the last 3 closing prices are our input features, and the closing price for today is the output/prediction value.

[@alrey72]

@@lorryzou9367 Yes that's the intention. However, the neural network used is time series so the last 3 closing prices is redundant. If a feed forward neural network is used then the input of last 3 closing prices is logical.

[@ShuhaoGao]

What is the value for test_val?

[@folashadeolaitan6222]

Hi Greg, thank you for another awesome video. I am working on a multivariate LSTM and i have scaled my data (containing both the input and output variables) as in the code below. scaler = StandardScaler() scaler = scaler.fit(df_train) df_train_scaled = scaler.transform(df_train) Now, doing inverse_transform as below in order to be able to compare my predicted values with the actual, i am getting an error. prediction_copies = np.repeat(prediction, df_train.shape[1], axis=-1) #This will repeat the prediction the number of times columns it was originally scaled with y_pred = scaler.inverse_transform(prediction_copies)[:,0] #Here i am taking all the rows and just the first column, since the other columns are replications of the first. ERROR Found array with dim 3. StandardScaler expected

[@gauravnarkhede3622]

I am also working on LSTM

[@folashadeolaitan6222]

@@gauravnarkhede3622 Nice one Gaurav, how is it going with u?

[@gauravnarkhede3622]

@@folashadeolaitan6222 Good, need some company to work

[@jpaulexo]

​Hi ​@@folashadeolaitan6222Did you found the error?

[@jakob4371]

Its an extremely basic one, so it should be resolved by now.

[@moekhaled]

thank you , where is the windowed_df code explanation ? i could not find it in the description ?

[@JilN660]

Hi thanks a lot for this video! How would you move from forecast to a Signal 1/0/-1 column to be applied to the price variation? I have issues with potential future leakage and the condition to use to create the signal. 😊

[@virajtharuka4983]

How can we use Machine learning to detect elliott waves ?

[@ivandozoretz3031]

Great video! I've just one doubt: if I want to put the model in 'production' and predict prices for the next month, I won't have the previous 3 prices for most of those dates (in that production dataset). So, is it right to include those features in training,validation and test set?

[@otaviocoutinho2855]

I have the exact same doubt. Shame nobody aswered. I this ideia we could only predict the next day, but what you can do is try to make the same he did in the validation, you must try to predict a larger range of days and the performance will drop quickly, but problably enough for a few days ahead. Which I find interesting besides being limited.

[@luisaurso]

Hey Greg, thanks for sharing your knowledge. Amazing video and very well explained. Following you onwards

[@uveuvenouve7680]

Can someone explain why the prediction is so flat when the train data is big? as the video show , it should be using the 3 previous days' price to predict the next day close price. that's mean it always predict a dramatic drop of few hundred dollars? Is that the cause of LSTM? would it be better to use the price differences to make this model?

[@taruchitgoyal3735]

Hello Sir, Thank you for the tutorial. I am getting the error when we start using the function: df_to_windowed_df() I get error:- Error: Window of size 3 is too large for date 1986-03-12 00:00:00 None Thus, need your help to understand about this error and how to resolve it. Thank you.

[@hybriddude007]

If you want more previous days, then take at least two years worth of data or more, that should work

[@arsheyajain7055]

Was waiting for this one!!

[@GregHogg]

Great to hear!

[@marselloormanda1525]

thankyou for the video, but i have a problem it seems like the prediction that i have is just a straight line, does that mean i need normalization for the data? cause the stock price in my currency is so big it reaches thousand

[@fkeb37e9w0]

Hi Gred, I had a doubt, for an unseen data (which contains only the date), how can I send that data into our LSTM model and predict for the next 5 days, when I have no input data apart from the DATE? Please help.

[@pixusru]

Your input for LSTM is not a date, but prices for previous N days before now.

[@efhndz7990]

i have the same question what can i do? if i want to know values for next days

[@timseed4489]

One of the best explanations on the subject - I would do some of the pandas stuff slightly different - but overall 10/10

[@GregHogg]

Yeah, I would too looking back. But thanks - great to hear!

[@Hodlybooogaaas]

is the info in the video dependable to trade in real life?

[@SowgandBakka]

Where do I find windowing the data frame video the size doesn't support my dataframe and how do I change the dates according to the size of the window?

[@afseeniqbal]

hello I am new to this so I was wondering if this exact model would work on say VTSAX stocks and not just microsoft? if I were to just import VTSAX data instead of microsoft data as a csv in the beginning

[@yan200go]

this actually explained things that I didn't understand by other people's explaination

[@GregHogg]

Glad to hear it!

[@SanjayMantati]

@GregHogg, If i want 'Close' and 'Open' both (last 3 days data) as input for single row. What code I need to change. I tried to update middle_matrix.reshape and the InputLayer with 2D. The predictions went awful. Can you please guide?

[@zabrinasmith5087]

I'm trying to replicate this model with another dataset but the code for 'f_to_windowed_df' that was pasted at 6:40 doesn't seem to be working, I have mirrored the code using the linked provided. Any suggestions?

[@GregHogg]

Sorry, not sure.

[@21morpha]

```def targets_df(df, n): df = df.reset_index() names = ['target_date'] for i in range(1, n+1): names.append("target"+str(n-i+1)) names.append('target') new_df=[] for i in range(n, len(df)): new_df.append([df[df.columns[0]][i]]) for j in range(0, n+1): new_df[i-n].append(df[df.columns[1]][i-(n-j)]) return pd.DataFrame(new_df, columns=names)```

[@21morpha]

I made this targets_df function because I didn't want to just copy what was done by others. It works, but it considers all of the dates. I believe it is not hard to adapt it for you to select the window by yourself. Maybe you could just slice the close dataframe before applying the function on it. In my case it worked just fine because I got a dataset with as many as 120 rows. Don't know if it would work well on huge datasets, with millions of lines, but I believe it wouldnt, because there is a nested for loop in there.

[@aadarshsingh2247]

I have a doubt asked by my professor "Why are we using only one attribute "close price" when the stock market can depend on volume too????

[@GregHogg]

You absolutely could

[@G5Locks]

Can you implement this in a streamlit interface

[@mahdis-hs6bn]

Hi the reason that it couldn't predict correctly was because there was never a rise like the test set in train set right ?

[@heribertoquintanilla743]

Thanks a lot Greg, very interesting. One question, once you have your model, how to add new data, without re-train the whole model, but train just the new data and predict under the whole dataset? does it make sense? thanks a lot!

[@andrewlichte6734]

I think at some point you have to retrain your model. like how Greg threw out all the old data because microsoft went parabolic in the more recent years. the model didn't know how to cope with that. it's like budgeting gas money for a trip, but your're using gas prices from 5 years ago. You're just not gonna get where you want to go.

[@teegee7801]

Im getting a syntax error on the def df_to_windowed_df(dataframe, first_date_str, last_date_str, n=3): where its telling me its incomplete.

[@poojithachalla8106]

@Greg Hogg Please provide vedio link for window function

[@cesarfierro4792]

Great video, it's really well explained. I was expecting the recursive predictions to work a little bit better since I will apply this to a school project more oriented to failure detection but I understand that's a hard task. But still is a great base to start with. Thanks so much.

[@hulunlanteworku9341]

Awesome video , but how do we know our window size in this video case 3 is there a way to know that. I think it's most crucial while working with LSTM

[@GregHogg]

You choose

[@hulunlanteworku9341]

@@GregHogg Based on what? or by seeing the data visualization to see some pattern that repeats

[@zephaniahqamer]

Can't you do all the numpy window creation stuff by simply using the pandas .shift() method? smth like this: for i in range(1, n+1): df[f'Target-{i}'] = df['Close'].shift(i)? and then dropna()?

[@rohanaryan2231]

hey! love the way you deliver your content sir! learnt a lot from you. just a quick question, can we use a direct type conversion to convert the type of date (pd.to_datetime or astype(datetime64)). if yes why didnt you use it and create a function for the conversion, how is it different from doing it directly as i mentioned above. thanks!! good day!!

[@senna_william]

The final catch was fantastic, thanks for the lesson!

[@GregHogg]

Yeah I feel like adding that in there is very useful to know haha

[@senna_william]

At first I thought "I do not believe that he will be able to predict the market only with this information", a good lesson, congratulations!

[@GregHogg]

@@senna_william yeah! :)

[@FRKHuang]

Hi Greg, my MAE is getting pretty huge, do you have any idea how to fix it? And wondering why you can achieve the low MAE without standardization, thanks!

[@robottalks7312]

Will there be any advantage if we scale the closing stock price between 0-1

[@GregHogg]

You could definitely try that :)

[@britox.6216]

still hate how you left us on a cliffhanger greg! please make a part two!

[@abhirajranjan1518]

Why is that windowed_df function taking so long to run, it isnt stopping...

[@carrocesta]

Greg, you are the man! greetings from Spain :)

[@GregHogg]

Thanks so much! I'd love to visit Spain some day

[@anmolpreet8959]

Congrats on Hitting 22k Subs.

[@GregHogg]

Aw thanks Anmol, and nice dab!!

[@1234567PokemGaming]

Guys, at the end of the video, it should not be a constant value. Just a little mistake. You guys can use my code for reference. In the video, the constant trend happened due to the last_window always keep constant. # Predict future from copy import deepcopy recursive_predictions = [] recursive_dates = np.concatenate([dates_val, dates_test]) last_window_new = deepcopy(X_train[-1]) for target_date in recursive_dates: next_prediction = model.predict(np.array([last_window_new])).flatten() recursive_predictions.append(next_prediction) last_window_new[0] = last_window_new[1] last_window_new[1] = last_window_new[2] last_window_new[-1] = next_prediction print(X_train[-2:]) # Print the last 2 elements of X_train print(np.array([last_window_new])) # Print the value of np.array([last_window])

[@bloodio4237]

Hi 👋👋 I am looking to create a model that predicts token bonding curves. In order to do that, moreover to predict token price, I need to predict the supply. The majority of youtube videos are talking about the price. I have some trouble finding tutorials explaining how to predict token supply and which features would be better used. Any help, please !!

[@gayanc6193]

Great content solved one of my major problems. many thanks❣

[@GregHogg]

Super glad to hear that!

[@yavuzhancakr6592]

sometimes in the big datas my epochs value be nan i mean loss : nan, i haved normalize the data but it didn't change anyting, what should i do ?

[@DevilErnest]

Hi Greg, great video and I learnt a lot here. May I ask whether detrending and deseasonalisation (if they exist) are needed during time series modelling. There are resources out there which suggests these should be done and the series must be stable before feeding time series to the model, while others (like yours) don't really care about trends and seasonality within the model and let the model does its work. My guess is there's no right or wrong answer to it, but what's the reasoning behind these decisions in time series modeling using machine learning algos? Thanks.

[@GregHogg]

There's always multiple ways of doing things. In theory you try everything and do what works best

[@fluctura]

majjjggic😂

[@chaunguyen5378]

what is the meaning of prediction for every single day, nothing, your prediction give back the value with big standard error

[@rizkamilandgamilenio9806]

amazing video, why does your model on this video perform differently from the actual code.

[@GregHogg]

Thank you! Probably just randomisation

[@dananjayaidabagusgede5144]

i copy your code in description, why my data is error? "window of size 3 is too large" what thats mean? thnks sir i hope you answer soon 🙏

[@SafwanAlselwi]

I got the same error, but I fix it by downgliding the entire max dataset

[@sivaramjallu001]

Just Awesome! very helpful😀😀😀

[@Pankajgoyal41521]

Unable to download dataset from yahoofinance website and drive link both. Can somebody help me with it?

[@muhammadquliyev9189]

i face same problem

[@sameerulhaq4066]

how can we check the rmse score?

[@jeraldgooch6438]

Greg, Thanks for this. Two questions 1. Will adding in more target values (e.g. go to Target-10) help? 2. What would shuffling the windowed data set do? At least some of the data in the training set would be after 2016 then. Thanks

[@GregHogg]

Changing the window may help. Not sure what you mean by shuffling.

[@jeraldgooch6438]

@@GregHoggsklearn.utils.shuffle i think is what i mean. In your windowed dataframe, the 2nd row might end up as the 100th row, the 3rd as the 250th, the 1000th might end up as the second and so on. Obviously I do not have enough of the vocabulary to truly express myself and typing on an iPad is a bit clunky!

[@jeraldgooch6438]

@@GregHogg After shuffling, the only time related data would be within each sample. The shuffling would destroy any time related dependency between samples. This might not be desirable with the LSTM layer? Also, what might be the effect of just using % change on a daily basis, rather than absolute value of the stock. And this is where you get to say “Why don’t you go ttrry it out and let me know what you find”?

[@arian_apply]

I've done it before. It depends on future forecasting and some other parameters. There is no clear relationship between past data and the accuracy of results. Generally, you should test different scenarios based on your project and your data.

[@arfasobirhanu4801]

Where can I get the description function for df_to_windowed_df function, sir?

[@DawidMichna]

Great stuff! Why did you set date as an df.index in min ~5, was it necessary/what's a rationale behind it?

[@GregHogg]

See the other comments haha

[@prashant8762]

Hello Greg, great video again as usual, can advice ,whether can we have a LSTM neural network for predicting stock price of multiple companies say 1000 companies ?

[@jarrodmautz159]

Yes compile each stocks data into a key value pair of a dictionary, the key will hold the stocks name, and the values will hold the data frames that contain the dates and prices/more indicators etc., then turn each into sequence, then train your lstm on each stock in your dictionary

[@prashant8762]

@@jarrodmautz159 thats very impressive explanation , do you have any sample notebook( if you free to share ) on similar use case or any article link ? iam really looking for one sample notebook, thanks for the reply ,it helps :-)

[@xiaoyang4521]

The lines are close to each other becuase you are always feeding "future" data, into prediction. The predictions are actualy 1 day shifted to the right, which means the so called prediction is actually delayed. This won't be helpful to stock price prediction at all.

[@tiagobrito9765]

Another great video! I have a doubt: My date column is in the format day/month/year hour:minutes. What should I change in the str_to_datetime def function to show hours and minutes?

[@konstantintomilin1826]

Would be interesting to see how inclusion of other features like trading volume and maybe some common technical analysis indicators could affect prediction!

[@fluctura]

I'll do this and keep you posted

[@dnas5629]

I have backtested many of the TI and found they are broken clocks. Not that they are not useful to help filter data, but as for predictions they ideal. The oscillators help with sideways markets, but not trending. Detecting trends before they happen is tough. As for inputs into predictive models I do not think it will help the model. But definitely try. TALIB is best library for that.

[@pranaliredgaonkar8154]

Which mathematical model we you have used for prediction?

[@ffinalttrip]

LSTM

[@obedrolland3259]

There is a huge mistake in your loop which predicts the test & validation closing prices according to the last 3 values of the training data. You should just change the last_window expression by : last_window = np.concatenate([last_window[-2:], np.array(next_prediction)]) And see what you will get.

[@obedrolland3259]

Awesome video by the way ! 👌🏽

[@kind-reminder]

whats about predictions in here? you showed us a regression. it doesnt have anything in common with predictions of any kind.

[@ismaelbastos4097]

Very good video!!! I have a question, when we separate the validation data, because of the the size of the window is three, the first row of the validation tdata will carry out the last 3 values that are in the target of of the three last rows of train data, and the second row will carry two values of the the target of last two rows of train data and so forth. Is this considered a data leakage? I mean, I have the target of train set as inputs of the validation set, is this ok?

[@GregHogg]

I see what you're saying, and thanks for the kind words. I wouldn't really consider this is a leakage. Formally? Maybe. Practically? Not really

[@ismaelbastos4097]

@@GregHogg Thanks for the answer. I really appreciate your job here on KZbin and on LinkedIn. So I have to be careful when choosing the size of the window, right? Because, if I choose a larger window, the amount of data that will "leak" will grow.

[@GregHogg]

@@ismaelbastos4097 you're very welcome! And yeah, I guess I agree with that. I really wouldn't worry too much though, when you're in a company these things will sort themselves out.

[@peralser]

Amazing video. You were so clear to explain! Thanks!

[@GregHogg]

Great to hear!

[@Canada_Facts]

Thank god I found this video, I tried the same and really poor performance over long term

[@kkololp]

if your data is not linear, you can use the log to make it linear

[@Wissam-rk7tv]

thank you for the vidéo , it's amazig, how if we wont to forecast stock prices for different company ? can you make a video about this case

[@GregHogg]

Exact same thing

[@eightonecapital6554]

BRAND NEW TO CODING. Is there a way to make the chart interactive to see exact predicted prices?

[@GregHogg]

Yes use plotly instead of matplot

[@탕멍위엔]

how to determine the parameter values in LSTM model? such as LSTM(64) and Dense(4）

[@GregHogg]

Generally through trial and error, but you could automate a search

[@탕멍위엔]

@@GregHogg 😳thank you for reply

[@_KobbyOb]

How is related to RNN and LSTM. Are they also the same?

[@GregHogg]

Lstm is a (good) type of rnn. It's structure is indeed recurrent.

[@liuyi8713]

great video, but it never pass following part running. import numpy as np def df_to_windowed_df(dataframe, first_date_str, last_date_str, n=3): first_date = str_to_datetime(first_date_str) last_date = str_to_datetime(last_date_str) ...... ...... # Start day second time around: '2021-03-25' windowed_df = df_to_windowed_df(df, '2021-03-25', '2022-03-23', n=3) windowed_df

[@VT-Rathore]

Have u got the solution?

[@ffinalttrip]

Amazing work Greg!!👏 Do you know any good community to share and talk about projects like this one? discord or something...🧐

[@GregHogg]

Thank you!! LinkedIn and Facebook have some very large groups :)

[@anatoliyzavdoveev4252]

Super tutorial 💪 great code💪💪🤝

[@GregHogg]

Thanks so much and great to hear!!