Price Trend Detection Using Machine Learning For Trading

[@kaushikchakravorty103]

The content that you create is actually smart. It is to the point and covers many misconceptions that people generally have with ML models, especially in trading where passing future data into ML models and coming up with good predictions is nothing short of a crime. Phenomenal, to say the least.

[@CodeTradingCafe]

Thank you for your support. Yes ML is far from being easy to use in this case.

[@jorgecasas9263]

Best video ever I watched in algorithmic trading, sometimes I get crazy trying to use ML thank you mate!

[@CodeTradingCafe]

Happy you liked it, good luck with your trading

[@100Ticks]

Your content is phenomenal! Thank you for putting so much time into them. Would be interesting to see ML models in combination with TA signals.

[@CodeTradingCafe]

Thank you for your support, TAs with ML are on the way, many requested this

[@mpty2022]

thank you for sharing your work... i come from similar background and i can see how much work you put into it... while doing some research, one paper that caught my eye was the "Encoding candlesticks as images for patterns classification"... the authors encoded candles and solved it as image processing rather than regression and got better results

[@CodeTradingCafe]

Thanks a lot for your support and sharing, I will check it out, in fact the image analysis method is something I didn't investigate yet for some reason I didn't think it has potential for such a problem. But it might be worth it.

[@homealone75]

Good content! I came almost to the same conclusion. Can’t use ML for trading in a straight forward way like many think. Would not mind to see more videos from you on this topic.

[@CodeTradingCafe]

Thank you for your comment, yes infact I have been trying ML in many different ways and neural networks are even worse in this for trading (maybe I will show it in a video).

[@PouyaMofidi]

I realy enjoyed from this video content and points. It was 30min but l it need long long time to test to observe it. Thanks a lot. With wathing again this video im sure it makes me a lot of questions to do. 😉

[@CodeTradingCafe]

Glad you enjoyed it, thank you for your support! really appreciate it!

[@Jamil.francis]

Seriously thank you for this vid. It saved me A LOT of time !!

[@CodeTradingCafe]

Glad it helped, good luck

[@rverm1000]

back to this video again . while taking a deep learning course(pytorch). now it makes sense on how difficult it is to predict the direction of stock prices using machine learning. ive recently started using the 5day 5minute chart in a scanner program to find stocks the are going up. if it passes that test i run it threw a 1 month 15 minute scanner. if it passes that its most likely to keep going up.

[@CodeTradingCafe]

Hi thank you for your comment, it's nice to know someone shares the pain :) prices contain a lot of random noise that messes up the AI models pattern reading capacity. I believe that ML alone can't make it, it needs to be supported by a human like algorithm (passive conditions that we use in our trading). Your idea about trend detection is good just needs a safety exit in case of reversal. Good luck!

[@mehranabdivand9368]

I've collected over 200 ISI papers in this domain and read, you mention valueble points that all of them overlook. One point, the preprocessing phase and elevating data to be more like information, so the model can obtain knowledge, is so overlooked! We can evade overfitting by dimention reduction techniques. We can use graph representation and embedding techniques. . .

[@CodeTradingCafe]

Hi , thank you, I think it's better to avoid "old" papers in this field, it seems to me that a lot of people were new to these algorithms and published with limited knowledge. My assumption is that recent papers should be better because the whole set of AI tools is now democratized and more reachable to a much wider audience/scientists.

[@mehranabdivand9368]

@@CodeTradingCafe you are so humble! Most of those papers are for recent 3 years. For the record, I mostly collected ML and DL for Bitcoin price prediction papers! Just a few of them have a new look on handling ohlc prices, by for example removing boundries that includes logarithms of those as entry features, but, as yiu said, those are random numbers. A very few considered data graph representations like Visibility graph and embeddings like Node2vec as features. An all of them are for calculating MSE and MAPE!

[@CodeTradingCafe]

What really surprises me, sometimes videos and comments on KZbin have more rigor and better scientific approach than academic studies in this field, usually it should be the opposite! (of course I don't mean all YT videos).

[@bosypuspus]

ML works best in the direction of the trend. However, its not good in understanding trend reversals, therefore you have to implement risk and money management strategies when the ML model fails at predicting.

[@CodeTradingCafe]

Interesting, you might be right didn't think of it this way... I will mention it in the next video for good information

[@blasdelezo8396]

A way of improving the lagging is using Signal Processing concepts. Ehlers work is seminal in this case. In my experience, it could work yet the granularity has to be adequate. On lower time frames, noise to signal is so low that is useless.

[@CodeTradingCafe]

Hey thank you for sharing, so far ML was a failure in forex for I have tried, will take your idea in consideration maybe later we can still find something useful.

[@DinoSatto]

I finally get the ML concept! (at least I hope, lol!) I never get it with other example or reading docs and books about ML but now I have understood the x and y kind of data and how to formulate the prediction The entire concept in my mind was wrong and never explained so well Thanks for your content. Super helpful! I've worked in IT for more than 13 years as a software developer and recently I've started to approach this field. Isn't easy! Do you recommend me Datacamp or other resources to approach data analysis/ML? (obviously I'm already trying to watch all your videos :P )

[@CodeTradingCafe]

Thank you for your supportive comment, I wish I had the time to put a full course together but I don't think it will happen soon. For data analysis and ML, I suggest starting with "data visualization" courses/books these will take you through essential modules like pandas numpy matplotlib seaborn pyplot. Then move on to machine learning, it's a wide field try to stick with applications of interest to you, I suppose regression and classification problems. Good luck! (Coursera is great and you can audit any course for free no need to pay for anything)

[@DinoSatto]

​@@CodeTradingCafe Thank you for your response! If you do a course it will be awesome and I'll be more than happy to join the class :) Thanks for the advice! Really appreciated

[@randomdude79404]

More technical indicators means more features but then you can do feature selection to select the best combination of features for the ML model

[@CodeTradingCafe]

Yes I see your point I failed in couple of previous tries I will find more time for this

[@randomdude79404]

@@CodeTradingCafe Machine learning just for accurate signal generation , can generate a trading strategy around this

[@CodeTradingCafe]

It's always curious idea I think I will build a code just for the sake of trying it and know the potential

[@randomdude79404]

@@CodeTradingCafe Well we know that using Pure ML to make trading decisions is probably not the best idea , instead have a ML model that generates some buy or sell signal of different strength and then design some technical strategy around this to trade.

[@CodeTradingCafe]

Exactly I think the final decision shouldn't be left for the ML model, instead ML an be used as one input indicator among others... This looks more promising to me

[@ozan4702]

Deep reinforcement learning works and performs very well.

[@CodeTradingCafe]

Hi thank you, did you try it?

[@ozan4702]

@@CodeTradingCafe Yes.

[@CodeTradingCafe]

Good to know I haven't invested in reinforcement learning yet

[@nasratullahzazai7269]

@@ozan4702 how well did it perform?

[@musiclover1067]

Love your work! Just a suggestion... How about using machine learning to detect uptrend, downtrend and no trend. Then use Bollinger Band trading strategy for no trends and Moving Average with RSI strategy for up and down trends? Will love to see you demonstrate the coding and backtesting for this... Thank you!

[@CodeTradingCafe]

This is nice actually!, I like the idea of combining both ML and indicators depending on the trend. Will add your comment to my list.

[@tapas20051]

You can use sample weight in fit function for imbalance signal

[@CodeTradingCafe]

Hi Yes thank you, I think in this example it wouldn't affect the results.

[@MO-gf6hu]

Have you thought about making a model for only up and/or downtrend and apply a trailing tp & sl ? I’m very curious how that would work out. Or do you think sideways market would be a problem there ?

[@CodeTradingCafe]

Hi, thank you for your comment, yes at some point I tried it and you are right the problem is sideways market, so ideally you can assess the market and launch the algo only when there is trend, otherwise use specific algo for sideways, it's human assisted though not fully automated. But anyway this trend detection is something I will need to revisit at some point for now we are using the moving average.

[@deuxsoixantedouze1670]

Great content! I was wonder something about the validation of the model: wouldn't it be better to evaluate with a time-series cross-validation? I tried to do it and the average score along the different folds was a bit lower than the one used with a single 70/30% split. Using CV seems more robust and reduce randomness on the results.

[@CodeTradingCafe]

I agree with you CV is the way to go for proper testing, and maybe also trying the model on completely new/unseen data

[@fitcyx6952]

Great content, this type of videos is very interesting too

[@CodeTradingCafe]

Thanks! I didn't think it might be of interest since it's not a positive return model yet, but yes it's still good for understanding how it works (or not)

[@qwertasd7]

The conclusion here should be that the model mainly is a layer around the MSI indicator. And the other indicator has less impact (and thus better to be avoided). Although nice to see a code sample, right as you are with one shouldnt use direct price data one could add (1/pi)*arctan(newprice-oldprice) + .5 angle input. where oldprice could be x days or set to last transaction date price.

[@CodeTradingCafe]

You're right I like the formulas you are proposing might put these in practice at some point.

[@qwertasd7]

@@CodeTradingCafe also did you know that there exist a module for candle patterns? Ta-lib Although correlation seams lower, your model can find the relations. Maybe though this needs to be solved by a Qsolver like those game ai that tries to stear car in a parcours of (trading info)

[@CodeTradingCafe]

Yes I knew about ta-lib but I don't know how it's coded I wanted to avoid a blackbox usage type, manual code gives you more control

[@qwertasd7]

@@CodeTradingCafe how can I send in the code for you to see how to do that, as I did it yesterday.. just fork or .. ?

[@maciekpaciarski9343]

Great content. It would be nice to see more on this topic

[@CodeTradingCafe]

Thank you for your comment, it's nice to see that you are interested although it's not a winning method yet... I will try neural networks for the continuity

[@sailisolo6256]

Amazing video to give insights on using passive features, where can i download Gold futures historical data?

[@CodeTradingCafe]

Thank you for your support. You can get data from yfinance, dukascopy or also your broker. Good luck!

[@dominiquedunlap9223]

Can this be modified, not to automate but to give a directional signal?? Also can the model be trained with historical data??

[@CodeTradingCafe]

Definitely yes, but I can't forecast the results before trying, I did try it in the past I should have the code somewhere, and yes we can train the model on historical data.

[@dominiquedunlap9223]

@@CodeTradingCafe What kinds of percentages were you getting by using ML for price trends??

[@dominiquedunlap9223]

@@CodeTradingCafeCan it also be trained for News?? I'm not looking to automate the trades, more so to be a directional Guide

[@CodeTradingCafe]

I was getting a win rate of around 52% so just 2-3% advantage, not impressed at that moment so I didn't carry out much on it

[@CodeTradingCafe]

Yes it can be trade to read the news and the market sentiment, and you are absolutely correct not to fully automate the process just to have custom complex indicator, this is a safe approach in my oipinion.

[@ФівМиа]

Great video, thank you

[@CodeTradingCafe]

Thank you for your support 😊

[@jeremyross7641]

excellent video

[@CodeTradingCafe]

Thank you very much!

[@ckchao5150]

This is not a normal regression analysis. 1. You did not do a proper preprocessing for your data. 2. target's frequency is closer to the RSI more than the trend (non-trend) frequency. Meaning it will choose a quick root to optimize its target objective, therefore RSI is a choice. 3. Candle is in reversal nature. Meaning it is made to counter the trend, but you made it to following the new trend. There will going to be more and more reversal to realize just a few real trend. 4. Candle also itself an imblance signal, therefore they shows up on your imbalanced report. 5. Your target variable is not good enough. It has no predicting power itself. So your predictors follows a dummy leader to death. I am one of your course student. I hope you can listen more to learn. Thank you for sharing your thought processes! Cheers

[@CodeTradingCafe]

Hi, thank you for sharing, I would like to take this discussion a bit more in details, (proof of me listening to learn:) ). Regression has nothing to do here, this is a classification model. 1. Which part of a proper preprocessing is missing? 2. I didn't understand how is a frequency close to RSI ? and which is the frequency of the trend non trend, isn't it the same as the target because it's supposed to be trend? 3. I don't see the problem, if a pattern is rejecting a key level then a new trend is established and the model validates this in case it's statistically sound, but it might not be. So basically we are training the model to detect a specific pattern which is a complex rejection. 4. Imbalance is ok for Gradient Boosting most academic research papers used it this way, as long as it's not extreme results shouldn't be affected. 5. The target variable is also used in academic papers for financial models testing sometimes it's called the triple bar method, if you have any alternative please let me know I will introduce it in future videos.

[@vivosereni]

Where can i learn this course ? how much does it cost ? must i have some computer knowledge to learn this? and how long does it take to learn.

[@CodeTradingCafe]

Hi, I have courses starting from beginners level up to intermediate and trading level, check my channel about page the links with vouchers are there. Now to be honest with you if you haven't coded before don't expect to write trading codes before 6 months of learning and experimenting. Regarding your other question yes it's easier to have some computer knowledge but I know some students who were not very proficient in coding and they did go through the learning process and now they can code with the help of chatGPT. I hope this gives an idea... now if you feel discouraged, don't worry you can still check the strategies on this channel and see the results of our backtests, at the end it's the result that matters.

[@ndumisoradebe5256]

Can you please share the link of one of the videos where you show how to import data. I'm new to the channel, I haven't seen that video.

[@CodeTradingCafe]

You are welcome, check the second video in this playlist Python Learning: kzbin.info/aero/PLwEOixRFAUxZVV0XNWlapem1yoRA4rt_O

[@ndumisoradebe5256]

@@CodeTradingCafe Thank you

[@randomcandy1000]

thank you

[@CodeTradingCafe]

Thank you for your support!

[@osazemeusen1091]

I couldn't find the csv file but thank you so much for sharing. I will purchase your udemy course.

[@CodeTradingCafe]

Hi, I wouldn't be able to find the file I used in this video, it has been a while, however check this video kzbin.info/www/bejne/aqPFeKmXnqlsjc0 you can get the datafiles from the description and use them for any code with small parameters tuning.

[@rverm1000]

wow thats good to know. i recently realized its a math problem and it deserves a math solution.

[@CodeTradingCafe]

Out of experience simple methods worked much better than complex systems

[@rverm1000]

@@CodeTradingCafe you seem to know alot about machine learning . If you can do a few more videos explaining how you arrive at your conclusions would be great

[@CodeTradingCafe]

@@rverm1000 I will do my best to provide information, time is limited though for the moment it's only one vid per week max...

[@LilyCutie97]

did everythying currect but not given the source csv execel file

[@CodeTradingCafe]

Hi sorry you are right, I am sure on other videos I am sharing csv data you might want to download one from a different video and use it for this program. You can also download any data from yfinance or dukascopy. I hope this helps.

[@miguelangeldiazoviedo9523]

Very goog realy!!

[@CodeTradingCafe]

Thank you for your support 😊

[@guanguangua]

I think one potential issue with your implementation is that you didn’t normalize your data. The custom signals you provide will be either 0 or 1, but RSI is from 0 - 100. That probably explains why RSI has such huge importance, since numerically the number can be much larger.

[@CodeTradingCafe]

Hi, thank you for sharing, xgboost as any decision tree based classifiers do not need normalization, but just to make sure that it's not required we can try the same program using normalization.

[@alexisgraff2049]

Hi, thank you for your videos, im learning a lot, did you read the book Machine Learning for Algorithmic Trading? It's a really nice one because it combines machine learning with trading, adding also fundamentals. If you don't have it, i can give it to you :D. I stop reading it because i have to learn more the base of python with trading before I can understand something of that book.

[@CodeTradingCafe]

Hi, thank you for sharing, I didn't read the book yet I will go through in the coming holidays hopefully, who's the author?

[@alexisgraff2049]

@@CodeTradingCafe I thought i answered this. The author is Stefan Jansen, if you want a copy i have one. I hope it helps you !

[@CodeTradingCafe]

That would be good thank you, if anything powerful is within I can put it in a video later on 🙂

[@top10top54]

challenge try implement wycoff strategy

[@CodeTradingCafe]

I will check it out might take some time to put into a video

[@ahmedhoussien3068]

frinds can any body give my advice to learn how to make program boots,deep learinig patterns and start learn in this path ?

[@CodeTradingCafe]

Him you can start with python a lot of resources (check my python learning playlist on this channel). It's a long way but it's really worth it. Good luck!

[@kodzuvu]

Why don't you use Tensorflow to build this bot!

[@CodeTradingCafe]

Hi, because it was not needed I could do it directly with sklearn

[@RobotInnovationandAI]

am i going to understand this one day how i wish, is it possible to write the code from scratch

[@CodeTradingCafe]

6 months of learning from the basics up to advanced level, give is 30 min a day not more. Good luck :)

[@RobotInnovationandAI]

@@CodeTradingCafe could visualize the trend predicted on the chart so that it give one more interest

[@RobotInnovationandAI]

@@CodeTradingCafe i have basics in python could maybe you could write the code from scratch in each video or if you are referring to a previous video maybe you add link to the description

[@nygasyskaczalepiej]

mate if you have 20% recall winning that mean in opposite way you have 80% win rate just trade it other way around....

[@CodeTradingCafe]

🙂 doesn't work this way unfortunately

[@J7-Steny]

@@CodeTradingCafe yeah! doesn't work that way surprisingly.. I tried.

[@CodeTradingCafe]

@@J7-Steny these are called non linear problems and their dependencies in many parameters raises the challenge, it doesn't work in the opposite way because of the SL and TP values, they have to be equal to get opposite results. But don't waste your time on it unless for learning and understanding purposes.

[@Aram-SEMO]

🤖🤖🤖

[@CodeTradingCafe]

Looks like you like bots

[@Aram-SEMO]

@@CodeTradingCafe Yes I love technology very much 😇😇

[@top10top54]

you can also write a research paper

[@CodeTradingCafe]

:) to be honest I have written and published plenty, I am doing KZbin to do something different and somehow do it my way without sticking to the "rules", I find the reactions and the comments more rewarding

[@thechoosen4240]

Good job bro, JESUS IS COMING BACK VERY SOON; WATCH AND PREPARE

[@CodeTradingCafe]

thank you

[@nimazare782]

how can I get EURUSD_Candlestick_1_D ASK_05.05.2003-30.06.2021.csv ? because I want to Run the code myself...

[@CodeTradingCafe]

Hi, I will make a video about this now just need couple of days to finish and upload, I am getting this question a lot, so I guess the video is worth it. Stay tuned, and good luck with your trading!

[@yamitanomura]

Thanks,great video

[@CodeTradingCafe]

Glad it was helpful! Thank you for your support.