Do stock returns follow random walks? Markov chains and trading strategies (Excel)

Markov chains are a useful tool in mathematical statistics that can help you understand and interpret probabilities. Interestingly, Markov chains can also be applied to test whether the market is efficient, and if the test shows it is not, can help develop simple trading strategies to exploit this inefficiencies. Today we are going to explain Markov chains and the Markov chain test for the random walk hypothesis in Excel and simulate some investment strategies based on Markov chains for S&P 500.
Don't forget to subscribe to NEDL and give this video a thumbs up if you want more videos in Finance!
Please consider supporting NEDL on Patreon: / nedleducation

[@NEDLeducation]

You can find the spreadsheets for this video and some additional materials here: drive.google.com/drive/folders/1sP40IW0p0w5IETCgo464uhDFfdyR6rh7 Please consider supporting NEDL on Patreon: www.patreon.com/NEDLeducation

[@ariesclark3272]

instablaster...

[@KarolKapuscinski]

wow, thx! subscribed! its the best video about markov ch. I found as far....

[@vanbach852]

Your channel is gold. Please continue to enlighten all of us.

[@NEDLeducation]

Hi, and thank you! Glad you find the videos helpful :)

[@DavidElstob73]

Can't help but admire your amazing Excel skills. Superb!

[@mikiallen7733]

rarely I find high quality content such as this one , thanks sir , I may get the chance to work with you in the future on some similar ideas , am working still but not yet finished with the first step , which is data preparation / and variables which may provide a hint in to short-term percentage returns .

[@dchoudhary5449]

Thanks for your valuable input. I have been trying to understand the Markova state for long. Thanks for your help.

[@huskyjohnson31]

This is a phenomenal video!

[@camkrik5812]

Fantastic video. Your excel skills are amazing!

[@NEDLeducation]

Hi Cam, and thanks so much for such kind words. Stay tuned for more videos!

[@nikosje]

Absolutely superb video

[@selvasair]

Thanks for sharing the spread sheet, it help me in my understanding, by repeating the calculation as i watch the video

[@NEDLeducation]

Hi, thanks for your feedback! Glad you found the spreadsheet helpful :)

[@drsjamesserra]

So awesome you share the files!

[@sreedharma]

Just Brilliant ! thanks a lot

[@joeaoun6321]

Thanks for another great video and providing the spreadsheet. If only I could watch it in slo-mo!

[@Knud451]

Just slow down youtube.

[@muntedme203]

Very nice.

[@philippe2402]

Thank you!

[@lubiaescalante4107]

You should work on fractal Dimension, calculating area of Rectangles that is a good input for trading strategies too.

[@salardelavarqashqai]

please give youtube or book reference for better understanding for me. thanks

[@onda4165]

Thank you very much Savva. I am still impressed by this video and your explanations. However I think there is a future leak in the strategy because you tested starting on 2015 the inefficiency discovered when you have all the data. So the strategy is using information not available at the moment.

[@NEDLeducation]

Hi, and glad you liked the video! You are absolutely right, a robust estimation of such a strategy would revolve around an estimation period and a testing period where information only from the first period would be used. I mainly presented Markov chains as a market efficiency test here and the strategy part was mostly an illustration and not a guide for algorithmic trading :) I started doing tutorials on algorithmic trading recently though, so check it out if you are interested: kzbin.info/www/bejne/oKe9YamreJ6igM0

[@SurrenderPink]

NEDL, I've learned much from your past python tutorials. Wanted not to learn, but understand Markov Chains and your channel was a logical place to start. Amazing! Mad excel skills! You got it front, back and down! Many thanks for sharing your considerable knowledge as well as taking the time to record, edit and share your expertise. First rate!

[@yaxunyang8487]

Hi NEDL, Thanks for the wonderful video. When you construct your strategy, do you think it's better to apply it in a different data set?

[@NEDLeducation]

Hi Yaxun, and glad you liked the video! It is hard to say in advance which strategy would be best applicable to a particular dataset without applying the tests in the first place. As a rule of thumb, however, strategies exploiting dependencies of returns tend to be more profitable on less liquid markets. Hope it helps!

[@yaxunyang8487]

@@NEDLeducation very helpful!

[@ilijagjorgjevic9354]

How about using VOM 24:57 (variable order Markov chain). Do you find them more suitable for boosting of prediction accuracy?

[@veshderr]

Thank you for such great work! Is there a specific paper in your drive folder that goes over the Markov chain test for the random walk hypothesis?

[@NEDLeducation]

Hi Ergys, and glad you liked the video! This paper, for example, provides a quite simple outline of the methodology: dergipark.org.tr/en/download/article-file/50208

[@veshderr]

@@NEDLeducation Thank you!

[@veshderr]

@@NEDLeducation Can you comment on why the runs test, variance ratio, and Markov can sometimes calculate different p-values? Do you have any insight on the nature of the signal itself (for example, length of signal or sampling rate) that may explain some of these differences? Thank you again for such thorough explanation for these tough concepts!!

[@NEDLeducation]

@@veshderr Hi Ergys, and thanks for the excellent follow-up question! These tests conceptualise various facets of efficiency/inefficiency/predictability. For instance, variance ratio is about parametric predictability (here I show how variance ratios can be thought of as aggregated autocorrelation coefficients: kzbin.info/www/bejne/bJSZnmWLhc6GkJo). Runs tests are about directional predictability (whether the signs of returns and not necessarily their values can be predicted). Markov chains put this concept further and show whether return quantile buckets are predictable. So long story short, bar multiple testing concerns, if at least one test shows inefficiency, you can conclude the market is inefficient.

[@veshderr]

@@NEDLeducation Thank you!

[@cathyqinKoreanEnglishSongs]

what is the meaning by the number of states

[@marekdziubinski850]

Very valuable content! I couln't find the Markov spredsheet in your google folder (?)

[@NEDLeducation]

Hi Marek, and glad you liked the video! The file is called NEDL_Markov, here is the link just in case: docs.google.com/spreadsheets/d/1Fe-qstzOqE40wo4Q28D_gl8-zGWLDV3-/edit?usp=share_link&ouid=113436662715404606257&rtpof=true&sd=true

[@marekdziubinski850]

@@NEDLeducation Thanks!

[@salardelavarqashqai]

Please model this awesome example with python and plot the result for better explaintion because it was very complex. thanks alot

[@annmaryalexander]

As part of the results what all should we report?

[@NEDLeducation]

Hi Ann, and thanks for the question. For such analysis, you can report the Chi-squared stat and the p-value as well as, potentially, the transition probability matrix. Hope it helps!

[@annmaryalexander]

Ok Thank you

[@empemitheos]

Your strategy is somewhat like using a moving average strategy 👍

[@tiagontop7472]

lol

[@empemitheos]

@@tiagontop7472 what's so funny?

[@AttilioPitt]

Amazing video! Could you make this on python? Thank you really much

[@NEDLeducation]

Hi Attilio, and glad you enjoyed the video! Yes, I was planning to return to market efficiency tests and Python implementation of these at some point in the future, so stay tuned! :)

[@salardelavarqashqai]

@@NEDLeducation yes please model this with python

[@terencewinters2154]

Consider a squirrel reaching a fork in a tree branch . Does he get to the nuts in his hutch most efficiently by following the left fork or the right . But supposing a big wind blows that branch down and he must leap to another . There are variable probabilities of success and not all states can consider all possible variables . N - x variables is possible to give some consistency . But x is not the infinite set. The market isnt. fficient .

[@mikiallen7733]

I don't think Jim Simons fund uses returns to classify in to states , it must be something else ! am sure soon I will crack this up inschallah

[@NEDLeducation]

Hi Miki, and thanks for the comment! I am pretty sure whatever Jim Simons uses is proprietary :) But you definitely can build state transition matrix on other characteristics rather than returns.

[@mikiallen7733]

@@NEDLeducation we will see but thanks for your prompt response , but do you happen to work on projects related to industry right now ? Or is it just pure academic research though I suspect it , I would really love to know!

[@MrMcaff]

This modeling suffers from look ahead bias, doesn´t it? You do not know now what tomorrow´s return will be in order to calculate the empirical distro.

[@jasonwayne4647]

I found treasury