hi i'm working hard since one month to train a profitable model. my question is why are you trying to make a model to do so much trades when i backtest my model trying to predict the next price, i have around 52% chance that my prediction is in the good direction. but i succeeded to narrow down certains market conditions where i have 60% + (these market conditions happens less than 1% of the time but i can compensate this low amount of trades using leverages) i'm still not testing with real money, i have to do adjustements first, i'll let u know

Interesting work! Thank you for sharing. One key consideration when working with financial data is context. Your model is trained on relatively old data, but the market is extremely dynamic, influenced by numerous external factors that change over time. From my experience in forecasting energy prices, I have found it helpful to train the model multiple times on progressively larger time windows to evaluate its performance. Afterward, I train the model on all available data up to the present, incorporating as many relevant external signals as I can find. I then train a set of diverse models, such as gradient boosting models, decision trees, neural networks, and even transformers. These trained models are used to make out-of-sample predictions for the entire dataset. Next, I build an ensemble model that combines the external signals with these individual predictions. While the resulting model is quite complex and challenging to interpret, it often performs well even on similarly stochastic and complex data, such as stock prices.

Classic

Is it possible to train a model using computer vision to recognize trading patterns and candlesticks on screen, and then use reinforcement learning to train an agent that can trade based on what's happening in real-time on the chart?

Why are the RMSE values so low?

Hey Jin, do you think transformers can do a better job than LSTM on stock prediction

I always feel like Neural networks are overkill for trading. Feeding it random things like OHLC and technical signals will not get you very far, they almost always overfit them to the training peroid as well, and will often only see them perform for a couple months at best. Contextually markets change, markets do not stay static it's why almost all algorithmic systems must be monitored and swapped out due to different contexts. There are not random things in the market but trying to fit models to bar data isn't one of them... it will get you nowhere.

Hey Dr Choi, great video. Would love to hear more detail about any data driven value investing approaches you are aware of. Also would be interested to hear you compare the uses and popularity statistical modelling vs machine learning in quantitative analysis.

Hey Jin, if I want to add volume tick & RSI, what the best way to shape the data?

Since this is the last video on the channel and it came out 6 months ago, I'm assuming the model works :DD

Congratulations for being very clear in your explanations and very honest in the model evaluation.

8:57 how did you link this code of your model to the main code?! Because the cod of your model is in another file!!

We learn more from mistakes than successes, so thanks for sharing Jin!

Interesting concept! Thanks Jin

Torch 2.0 isn't available anymore and changing it to torch 2.2 causes the program to crash.

I wonder if you chose stocks for your model that didn't have many or few options contracts if it would remove some of the randomness.

would using a transformer based model be better? would love a video on that topic

Amazing video! is the code available like your other videos?

Great video! How is the model doing?

This is amazing, im just starting out with testing my own model and your points are clarifying, would love more videos on this

The videos you produce are super helpful for me, since I am working hard on understanding the math and logic behind neural networks. When you were talking about it not being suitable for predicting just the next it got me thinking. You could try and predict the price tomorrow and the day after with the same input data. If the price is down tomorrow you do an close market order otherwise you could buy at market open. I am curious what you would think of such trading strategy and maybe issues that arise like for example it being harder to train maybe? Also won't this model perform much better if you were to add loads of extra data like indicators and such. I am so happy I found you, you are super helpful on this journey with your videos!

I did a similar project using CPC + GRU. The results were great for training, evaluation and testing, however the problem is with the trades for I used RL with PPO to simulate a trading environment and to make the agent make his decisions based on the policy optimization, despite my good hardware training the PPO was very long and the results didn’t work that well but it was a fun experiment

Great video

Insightful video. As you point out, certain things may be simple in concept but complex and difficult in execution. You make a comment about using AI to predict future values to be derived from a company. I presume you are talking about future free cashflow predictions using some form of an ML model? Have you done any videos on this? Thanks!

Great video, Jin. I'm collecting minute data on the total microcap market right now (over 11 GB a month) to train my first neural network model. In my previous attempts with random forest models, I was able to get about 60-70% precision on live data, but as my dataset grew, my model performance got worse. It made me think there may be some merit in having a model that is only trained on short term data, like the last 30 trading days for example. Do you think such a model could be viable?

you should use better input variables maybe related to technical indicators and macroeconomic ones

Hope you find the model that make you money!! Hard work pays off, if it was easy everyone would be rich. Well done!

I love your content but I wish you made more videos and explain the codes

Stumbled upon your video today after learning about backpropagation. Really appreciate the clarity in your presentation and being honest about the model's results. Sadly, 53% success rate is no better than flipping a coin. Another factor that lead to many fund managers giving up on Machine Learning in recent years is market reflexivity. Quants are able to predict with high probability using an ensemble of algos but once they place a trade, the prediction goes haywire, due to the trade meddling with the chart pattern. Stock Price prediction is possibly the only ML endeavour that the analyst "poisons" the data after acting on it. Weather prediction on the other hand, a time series I believe where LSTM is deployed, successfully makes predictions because meteorologists are simply observing the atmosphere. Still early days, but I have seen publishings about success predicting an Emerging Market: Vietnam, claiming that it was achieved through multiple inputs from Closing Price to Technical Indicators. No evidence of people profiting from it so it seems like just theory at this point in time... like you mentioned, these papers are meant to make the PhDs look good.

that's exactly the reason why i am an algorithmic trader...

Great video. Thank you.

Awesome! Look forward to the upcoming videos regarding LSTM + CNN!

I love your content 🫶 please do moreee

Looking forward to your future videos Mr. Choi. Really interested in creating one of these models myself. Do you have any book recommendations to learn this type of stuff?

Love your vids Dr Choi

enjoyed the content.

Great to see hybrid models in action. Would love to have the code available to look through offline.

Hey bro. I love your content. Could you create a video showing what a ML model would look like for a classification Ny session bias problem. Thank you in advanced.

Goated

In other words. I need to develop the solution myself. And it have to be out of the box, groundbreaking strategy that nobody would guess for. Interesting challenge even though that sounds impossible. But still the biggest question is still remains. How do regular traders are still earning money on market if mathematicians and data scientists are swarming WallStreet with all fancy tehnology?

Really very practical. A lot of people try to predict price instead of return which might appear to give results but is an illusion and is not what we trade on. It sounds like you are not a big believer in the utility of these models but I am curious as to how well they might do at predicting return over a week or a month. It seems counterintuitive that they would be more successful at this but then if looking for patterns that might work and in some ways looking over a time period might actually eliminate some of the noise. In addition I am thinking some of the other potential inputs like sentiment, interest rate future prices, equity option prices might then have more relevance as inputs. Thoughts?

Great video! I appreciate how you delve into various aspects, unlike other videos that skim over crucial details. One aspect that caught my attention is the delay in prediction. Correct me if I'm wrong, but you utilize a sequence of returns up until a certain date, let's say February 16th in your example, to forecast the return on February 18th (calculated as the closing price on the 18th minus the closing price on the 17th, divided by the closing price on the 17th). My concern is practicality. If the model predicts a +10% return, does it mean one should buy the stock at the closing price on the 17th? But is it feasible to execute a trade at the exact closing price on the 17th? I'm curious about how this works in real trading scenarios.

Thanks for the great video Jin! Would like to see a future video about how to incorporate CNN with bi-directional LSTM to predict the stock prices. Keep up with the great content, all the best!

Can you predict the low and high of the next day?

Hi thanks for the tutorial very useful! I think all are waiting for the version with multiple stock

Dude this is great. Its funny I was watching videos of people making LSTM's speaking about the issues. One guy in comments pretty much said what you explained. He has been getting great results with a very complex LSTM, lots of dropout & batchNorm. Leaky relu activation and then uses Mean Squared Error %. I found it interesting. Then I find your video! Going to go play around with some models now :D

Could you please make a follow-up video or reference materials where you trained an LSTM using multiple stocks?

This is an extremely useful tutorial; very clear and info-rich. Two questions: The two-week time-sequences you use seem to all be the same length. Could you please share any thoughts that you may have on: a) using varying-length time-sequences, ie different history lengths and b) using time-sequences w/ varying history lengths initially, but that are subsequently padded with zeros so that we end up with the same fixed history length across all time-sequences?

I really enjoyed the clarity in your video. It would be interesting to see how this model performs with dollar bars. I am new to this area and have been reading time bar sampling has inherent limitations. Thank you for sharing your code, I'd like to dig deeper and this helps tremendously.

incredible video, where can i discuss more about this with you ?