I Used Data Science to Buy the Dip

Training a machine learning model to predict the bottom of the market and buy the dip!

[@MrMoore0312]

I can never have enough Ritvik stock market videos in my life!

[@ritvikmath]

Aw thanks!

[@SparaPengar]

Would love to see a model taking the most common indicators as features. Something like RSI, MACD and Moving averages.

[@jarvadetoro7004]

can we get the github repo for this please

[@eramy1]

Hello, thanks for the explanation ! Could you please talk about the PI prediction intervals and how we make it for time series prediction?

[@jollyrogererVF84]

I'm not surprised this video drew the conclusion that it did😀 If you could explain the market in a 20 minute video, you'd literally be a genius But I still enjoyed, please keep them coming 👍

[@ritvikmath]

Honestly I’d be skeptical of a data science KZbinr who claimed to have a silver bullet stock market model explained in 20 mins 😂

[@AndrewT]

Do you have to guard against collinearity when selecting features for this kind of model?

[@mayanklal9892]

Hi Ritvik! Really informational video. Would it be possible to share the code so that we can recreate it using our own dataset for better understanding. TIA!

[@hantt]

Would love to see this applied to volatility instead of reversals

[@karthikrajeshwaran1997]

Thanks for the explanations!

[@ritvikmath]

My pleasure!

[@MyMy-tv7fd]

pro traders also call your 'bottoms', pullbacks or reversals, and the most common technique is to define to two moving averages (eg, 20 and 50 SMA) and buy when the price is in the zone between at a 'discount' or 'fair value'. A lot of eyeballing goes on normally. Also, your 'gains' are all paper gains, you can only lock in the profits by selling and taking the money out of the market, so at the very least you should specify a trailing stop loss which will execute to get you out of the market and go to cash in the case of a stock market crash - recalling that the Great Crash of 1929 saw some stocks fall by 90% in a day.

[@dennisestenson7820]

I've noticed when the price crosses over the 72 SMA, and stays there, that's usually a reversal. When the price nears that SMA it'll often seem to bounce off and return to continue the trend.

[@poisonza]

no passive investor would train decision tree to time the market bottom. they just buy whenever the price dips like 5~n% or more so maybe compare with that approach.

[@ritvikmath]

This is a really important comment. Especially with passive investing, we should compare with simple heuristic type strategies. I will say that we feed features like these into the model so we can think of it as coming up with these rules on its own. For next video will keep in mind using more simple baselines!

[@ResilientFighter]

Agree on both these comments

[@cru2426]

I love this guy would find out the reasons if the method really works. Not a click bait video just telling you how successful their methods are.

[@hpix123]

1wk growth is important because of short term mean reversion. Shorter periods will probably be even more important (but harder to monetize)

[@wariodoors9481]

Well, if you are forced to invest every month which day would you choose? I.e. Historically Monday are better than Fridays, you just use the short term prediction features to guess a day, predict daily.. if no good answer you just invest end of the month... Is it worth the hussle, not really, is it fun, yep

[@ritvikmath]

interesting experiment, might have to make a video on that one!

[@riccardoronco627]

I love these videos but there are some points that are not clear. In the consistent model we invest $100 per week. We do that for the whole duration of the backtest, I imagine, and this will cost us $100 x number of weeks in the test (let's say 10 years or 520 weeks). That is $52,000 as overall investment. Ok. Now how does the bottom or top model invest? Imagine we have 3 tops and 3 bottoms: do we divide $52,000 in 3 parts and we invest exactly 1/3rd at each top for the "Top strategy"?. But then how the Model invest? if it does not know a priori how many bottoms there are, how does it know how much to invest in each point? thank you for the fanstastic work in explaining.

[@vineetbhagwat4256]

7:54 Am I understanding correctly that you are looking 1 year backwards AND forwards to define a bottom or top? You can't use data in the future to retroactively define a top/bottom! You have to be able to predict it in real-time using only data available as of that date!

[@ritvikmath]

This is a great question. To clarify, for the model’s labels we do look backwards and forwards in time to define a market bottom but the features of the model look only backwards which means we aren’t “cheating” here

[@vineetbhagwat4256]

Thanks! Love your videos, esp the ones related to finance@@ritvikmath

[@bonob0123]

the last 12 years were mostly characterized by unusually low near-zero interest rates helping prop up equity prices so not representative of general SP500 performance. Pick a different 12 year period and the conclusions may be quite different (investing at the tops may not give as decent a return etc)

[@VadimChes]

Fear and greed are highest at the end of the trend. Therefore, last week return can have a sense. More volatility, more trading volume

[@ritvikmath]

Interesting theory 🤔

[@MarcusPereiraRJ]

Question of a beginner: so did it really work?

[@ritvikmath]

well ... we were able to build a model that gave us better return than consistent investing under very specific parameters but the moral of the video is that the risk associated with such models is very likely not worth the possible gains.

[@AB-zv6dz]

So the models were better. Anyway its a nonsense comparison, Sharpe ratio (i.e. consistent, buy and hold, whatever you want to call it) is a very misleading KPI. Also you didn't cross validate your models so the results are going to be suspect at best. Also there are clear benefits to using a model when it comes to risk and positon sizing that are not expressed here, that make it superior to buy and hold. So I like your results, but they are probably wrong and you're really only creating about 30% of a trading strategy by only comparing the raw signal.

[@poisonza]

i think his conclusion were to DCA every week consistently. so we should not rely on the model to time the market bottoms as strategy return with model is more volatile

[@ritvikmath]

hey, I totally agree that a lot more work needs to be done if we're seriously trying to build a model for buying the dip. Lot's of cross-validation, backtesting over different time intervals/periods, etc. Thanks for the insights!

[@mayanklal9892]

Any recommendations to learn about building better models. TIA.

[@pm-qf4oq]

ritvik id be your bottom if u talked data science in the same room as me

[@josee.1709]

Ha Haw!

[@ekaterinaburakova8629]

Awesome content, thanks for your videos! I still didn't understand why did [redacted] outperformed the [redacted] strategy (18:40)

[@ritvikmath]

basically because the strategy where we know the dip relies on a certain definition for the "dip", we might miss out on the less obvious dips thereby missing investment opportunities. The strategy where we consistently invest does not miss those opportunities.