Why Most Trading Strategies are Fake

There is an extremely high false discovery rate in both the academic and financial industry for trading strategies that “produce” alpha. In fact, most of these strategies are false discoveries due to research bias, multiple testing and the true probability of finding a new investment strategy being very low (less than 1%) due to competition.
As stated by Marcos Lopez de Prado with a true probability of a backtested strategy being profitable at 1%, and 80% power (rate of identifying true strategies), in testing 1000 trading strategies using a standard threshold of significance level at 5% would imply at least 86% false discoveries!
Today we investigate issues of multiple testing and false discovery of a profitable trading strategy. We develop a momentum-based trading strategy on Apple stock and show the issues that can arise from unknowingly completing multiple testing on the same dataset.
Papers discussed in this video
Evaluating Trading Strategies: www.stat.berkeley.edu/~aldous...
The Pitfalls of Econometric Analysis (Marcos Lopez de Prado): www.quantresearch.org/Lecture...
Scientific method: Statistical errors: www.nature.com/articles/506150a
Moving to a World Beyond “p less than 0.05”: www.tandfonline.com/doi/pdf/1...
An investigation of the false discovery rate and the misinterpretation of p-values: royalsocietypublishing.org/do...
★ ★ Code Available on GitHub ★ ★
GitHub: github.com/TheQuantPy
Specific Tutorial Link: github.com/TheQuantPy/youtube...
★ ★ QuantPy GitHub ★ ★
Collection of resources used on QuantPy KZbin channel. github.com/thequantpy
★ ★ Discord Community ★ ★
Join a small niche community of like-minded quants on discord. / discord
★ ★ Support our Patreon Community ★ ★
Get access to Jupyter Notebooks that can run in the browser without downloading python.
/ quantpy
★ ★ ThetaData API ★ ★
ThetaData's API provides both realtime and historical options data for end-of-day, and intraday trades and quotes. Use coupon 'QPY1' to receive 20% off on your first month.
www.thetadata.net/
★ ★ Online Quant Tutorials ★ ★
WEBSITE: quantpy.com.au
★ ★ Contact Us ★ ★
EMAIL: [e-mail]
Disclaimer: All ideas, opinions, recommendations and/or forecasts, expressed or implied in this content, are for informational and educational purposes only and should not be construed as financial product advice or an inducement or instruction to invest, trade, and/or speculate in the markets. Any action or refraining from action; investments, trades, and/or speculations made in light of the ideas, opinions, and/or forecasts, expressed or implied in this content, are committed at your own risk an consequence, financial or otherwise. As an affiliate of ThetaData, QuantPy Pty Ltd is compensated for any purchases made through the link provided in this description.

[@QuantPy]

Let me know if (like me) you've ever backtested a profitable trading strategy that you have since realised was a false discovery? Or equally as interesting, let me know if you've found a working trading strategy - and how you did it :p

[@Static006]

Wyckoff and Elliott wave are great strategies

[@drek273]

@@Static006 hahaha

[@rozsadnymarek5988]

Why most trading strategy fails? Because in 99% time markets are random. Simple. You can't develop profitable strategy in random market -> It doesn't exist. Simple.

[@rozsadnymarek5988]

@@Static006 Maybe they're great but you can't use them to make money.

[@Static006]

@Rozsądny Marek yea I can and do use them, to make consistent money..

[@Bobylly]

Let me say this for future watchers : your work is so underrated, it's borderline criminal. The target audience is so niche, it's like we're in a secret society club. Lopez "s work is a work of art.

[@QuantPy]

Mate, it’s the legends out there like you who keep me motivated to continue making these videos! Cheers

[@MuhammadAlfiansyah]

Most people dont know the right keyword to type in the search bar to arrive at this channel.

[@QuantPy]

Feel free to send the link to your mates

[@pugausa]

I'll second b's comment - unbelievable work you're doing here. For me not since Charles Cottle (of Think or swim fame, and early days trader doing conversions/reversals and 'boxing' on the trade cards before anyone had the 'tech') has anyone had such a complete view. Just great.

[@TonyVuolo]

@@MuhammadAlfiansyah I typed "is monte carlo analysis useful for stock prices"

[@Phantom4416]

Found your channel by chance!!! Your content is really well researched. Thank you for your dedication.

[@kbisht3680]

this channel is extremely underrated - so much evidence based discussion here

[@josephwehby9313]

Great video! I have been bingeing your videos lately!

[@GraceJesseandCats]

I would say one of the most underrated channel in all the trading/coding channels.

[@brandonjohnson8880]

Great video! Correct me if I'm wrong, but I think you should be using risk adjusted return as a performance metric. Sharpe is the ratio of two random variables and is likely to follow a Cauchy distribution. I'm not sure if your hypothesis tests are valid on the Cauchy

[@BlackSwan-sq2iw]

Yup... exactly true and a well known fact among experienced traders. Lots of scammers target beginner traders with by selling them strategies that are overfitted to historical data. Beginners who are lazy to do their own work and try to short cut their learning journey often fall prey to such scams but they continue to jump from one product to another until they lose all their capital.

[@Freud973]

Thanks for your content Jonathan, its the most underrated quantitative channel in Australia by a real quant

[@andrewliu8906]

Great video, which covers part of the question whereas the other part I think is when to drop a trading strategy that is no longer true. Even for the same stock the changing market dynamics could make or break a strategy, let alone you have social political aspects adding into the equations. Yet there are signals on community trading platforms showing great solid stable annual return(10-20% in resent low inflation years) , I wonder if that is true? Would love to see you doing a video talk about that.

[@TehBr0]

Great video, will need to review it again. My own research indicates that a 1% success rate in discovering an expectant system is indeed generous. The markets are extremely efficient.

[@biomedicalit]

Thank you for the brilliant summary. I should have watched it before spending days and weeks on research by using simple technical analysis. Answering your question - my most profitable strategy was tinkering with different technologies and believing in people with innovative ideas (entering during Ethereum crowdfunding and holding).

[@Tom-qq5bk]

Excellent assessment, and very well stated. Marcos is an old friend, former coworker and an unimpeachable source of research. Well done.

[@notonlygeek]

Hi, Maybe i missed smt from the paper "Evaluating Trading Strategies" but if you have 10000 trades in your backtest or 100 trades with the same sharpe ratio the std error is not really the same. Meaning the sample size should take into account somewhere is the equation that evaluate the confidence in your strategy ? Many tks for high value content and the ability to explain it very well !

[@noimnotnice]

I did not really understand well what you were attempting to explain from minute 11:00 and going forward. Maybe there were some leaps in logic. But everything became clear after having read Harvey, Liu [2014]. The video section is a synopsis of their paper and I suggest everyone read it if you have trouble following the video.

[@vladk9152]

Could we apply the same t statistic if using the sortino ratio or adjusted sortino as our test parameter? Perhaps finding strategies from the perspective of downside deviation could lead to better discoveries? I'm not sure, what do you think? The way i would go about finding strategies would be to program a couple of dozen indicators, combine three of them at a time and backtest cherrypicked datasets (trending or ranging). The indicators would not be "optimized" but instead run on default settings (what most people use), maybe limit the type of indicators to use, for instance, the script wouldn't be able to use two momentum indicators since they basically display the same information. Depending on the amount of indicators this would probably take a couple of days to run. In the end you'd have a list of combinations of indicators that MIGHT generate alpha. What do you think about the idea? I don't really have time right now to do a project like this, but if you see potential tell me what you came up with

[@tonycrouse6544]

To sum up: Consistent, successful trading is hard when trying to beat the market averages. These presentations should be listed as "Understanding financial statistics."

[@afaf6478]

I just started with algo trading and I absolutely love your focus on theory and stats.

[@jessojoy8899]

Do you use backtrader for your back testing or developed your own custom one ?

[@darcebrooks5409]

A good question to put to you: can non-quant trained people make consistent returns in the market? (not including statistical anomalies/ lots of happy accidents)

[@sorsdeus]

This might be the best video I have seen in the subject... eye opening.

[@dawei6697]

Question, can you safely assume your strategies performance follow normal distribution?

[@kanejiang2938]

Great idea, I think one of the reason is that the public information not working! if you put your strategies on the public side. It's will be not working

[@michchanel19]

Pls bare in mind the data is not static. the profitable strategy that you define is only good for certain time period or data set coz the participants in market are evolving. Making money in market is not about strategy. And models only give pricing with wrong assumptions so market players at least have agreed framework to trade with.

[@BOSprodz]

Great video But I don't understand why we use the number of tests/optimizations (216) to make the original p value smaller. I could hypothetically reach the same parameters you reached for the "optimization" in just 20 tests or even the 1st test by pure chance, and the p/n would not be a sufficient warning in this case despite the strategy still being a false positive

[@Dialga9187]

The trouble with finding a winning trading strategy is in not knowing what and how variables will affect future pricing. Based on how many traders (who are of a consistent, or I suppose "agreed upon", set of market knowledge) exist in a market, a heuristic of trader psychology as a basis for a trading strategy is often more useful. Put simply, if a tree falls in the woods and no-one's there to hear it, it will not affect the price of lumber. That is all. Just felt the need to express myself on the internet, get these thoughts out of my head.

[@esInvests]

Great video. It's not sexy at all but it's the work required to understand quantitative trading with a focus on pragmatic information vs a ton of theory.

[@abdulghani9186]

Do you have course algo trading for beginners, want to learn trading in statistical way.

[@GODBLESSAMERICA]

Try Replaying the Market data to check on your strategy entry exit.

[@iniquuisorbis4824]

Interesting video. But say your out of sample and your in sample had a sharp of say 3. Does that mean your onto a winner?

[@neilrichardson7454]

😂😂whilst I watch this, I recieved 3 trading KZbin commercials 😂

[@masoncnc]

Impressive video done in one take. I'm subscriber based on this one video. Did you create your own false negative in the fact that you're running the algorithm on one single stock. Trying to be mathematically predictive on what emotional humans do to their favorite love hate stock is too small of a sample size. Wouldn't it make more sense to try and be predictive more on the macro scale such as broad-based ETFs with a long short strategy?

[@user-wr4yl7tx3w]

in 5:56, how do you get 86% false discovery rate?

[@HitAndMissLab]

problem is low quality strategy thesis. For example, there is no reason why moving average crossover would work, its just a bad thesis to start with. If you change thesis into something more realistic you'll get strategy that works.

[@zefree1312]

Really good video and i am just been discouraged to be an independent quant researcher with such little chance of making profits😢😢😢😢

[@patryknextdoor]

what books (of any kind of topic eg.: stats , ML etc...) do you recommend for currency traders?

[@QuantPy]

I’ll just point you to my video on what is a quant. Gave book recommendations here for quantitative finance. I’d also point you to Quantitative Finance Stack Exchange

[@patryknextdoor]

@@QuantPy Thanks G

[@alexandterfst6532]

Thank you for this great video

[@nikitoci]

Shall it be 1000 instead of 10k at 4:08?

[@andresg297]

What is your opinion on Reinassance Technologies and Statistical Arbitrage?

[@QuantPy]

Big fan, hard to debate their results over a long period of time with the 10K reports. Long track record

[@leoeduardo3016]

If there is only 1% of a succesful strategy, why there is no 99% of strategies are wining money by only switching the direction of the entry at market?

[@abdirahman_9ines33]

what advice would you give to a one who has a trading system that has been back tested on last 10months and its successful but can only be traded by an algo but the trader ha no skill of coding...what advice would you give interms of automating the trading system and finding people to trust or any route he can take interms of partnering up with a firm? Thank you

[@mast3rbug]

I made over 20 strategies that was profitable in backtesting for more than 10 months, and at any little change in the market, the strategy does not work anymore. I'm making bots traders coded in C++ for a year from now and none of them was able to generate profit.

[@DavidElstob73]

In the study one broker did covering millions of trades, they found traders were good at picking winners 68% of the time. It was DISCIPLINE that let them down, cutting winners short and holding losers. So poor human psychology not bad strategy.

[@paulnyagini]

😂😂😂😂😂😂 I love your broker. Mathematical all trades should be at break even after a distribution of wins and losses regards there physiology levels. The problem comes is the spread/slippage which gives your broker edge. In other words your broker has over 99% chance to make money from you. And you have 99% chance to lose money from your broker regardless of your strategy or your phycology he will still beat you until you find that 1% . Which is not easy because will mean that you understand the laws of distribution in person and you understand the market beyond avarage trader. And you can easily spot errors and use them to your advantage.

[@DavidElstob73]

@@paulnyagini you're missing the point. The traders could have done much better if they didn't close winners too early. Most traders know when a market has topped or bottomed, but they don't take full advantage of this, as they trade with fear and not enough emotional control - discipline. Blaming brokers fees is just an excuse.

[@paulnyagini]

@@DavidElstob73 let as be realistic son. Betting on fevorate doesn't guarantee long time success same goes to underdog. Let your winners run and cut your losses fast will also not put you on green zone. Because you miss calculate the dead bodies on your way to Mt evarest. If it was true don't you think that automated trading would be profitable.to be a consistent profitable trader you have to look for advantage that is connected with the distribution. In other words if you start behind the 8-ball you will never get Infront.

[@isabellafiorito316]

Very helpful video

[@brockobama257]

I don't understand how there exists profitable traders if most positive trading strategies are false positives. Are they lucky? Obviously a subset of their trades will produce positive results, and it is likely a subset of all traders using false positive strategies will accumulate a net gain, but the success stories are typically in the millions of net gain. How do successful traders consistently get net gain if their trading strategy is false positive?

[@paulnyagini]

The real secret is to be able to define edge. Every thing else is the same only one thing that makes the difference. 🤯

[@EnergyChat]

They dont exist...its more profitable to sell signals then do trades.

[@user-xy1uq8qy3v]

Great video

[@Bright9-ne1zw]

Wonderful channel Help me a lot The ending of the stock is mathematics

[@RockyMountain947]

Hi, I just found your channel two days ago. I was thinking about applying to the HEC Montreal masters of financial engineering, University of Waterloo masters of quantitative finance and University of Calgary's masters of mathematical finance. Can you give me an idea of the job prospects for these types of degrees? I don't trust boards like Wall Street Oasis as I get the feeling half of the people on them are living in their mother's basement.

[@parsamoaven2677]

Hi, Did you apply for them sir?

[@dongmei3810]

How can any of these conclusions be relevant if you do not first define what a “winning” or “losing” or “false” or “true”trading strategy is?

[@QuantPy]

Winning strategy- one that is statistically significantly different from a strategy with a zero sharpe ratio. Losing- one that isn’t. True, False hopefully these don’t need further explaining in a binary tree for hypothesis test outcomes. Example is from Marcos López de Prado

[@Russellmichaelnew]

Interesting, thanks for sharing… what do you think of buy and hold value investing?… apparently Edward Thorpe is a huge fan of warren buffet…as Edward Thorpe was one of the pioneers of quant trading, and he gives his money to buffet, is value investing a superior strategy to quant strategies?

[@TonyVuolo]

Edward Thorp did fascinating research on not just the market but also the game of blackjack.

[@TrumpForPrison2023]

Of course value investing is superior to trading. Trading is gamling. Holding long term is basically guaranteed exponential profits over time, unless Putin drops a nuke on USA, ending the market as we know it lol.

[@Pushapk]

What about a strategy which is not rigid rule based but bend in according to market sentiment and follow trend….

[@alexeyborodin4545]

Hi, what you say it too fair for "true quant", because all who talks about trading should say that it's actually pretty easy to win, otherwise there will be no random people losing their last money on which you can get rich

[@kaisy2340]

Sounds as if the OOS test is redundant and some Monte Carlo analysis with wide percentages of ranging are to be preferred...

[@tjij-mbai]

Code not available anymore?

[@Vito099]

So what would you say about day trading channels like Warrior Trading, Options Millionaire or my favorite Trader Tom and many more like them. They all claim to make money from the stock market and show that they do by trading live with course members. Some might say "why would they need a course if they can make money from stocks?" but lets just assume they actually like teaching people to trade and the courses they sell is just some extra change.

[@egglyph]

Well, you should really be asking yourself something like “are there any of their students that made it as good as these teachers?”

[@johnp7739]

Ask them for 3rd party audited performance and you'll get your answer. Rockwell Trading is involved in a lawsuit over this right now.

[@alexCh-ln2gw]

they make money from pump and dump schemes "I just bought this stock. you should also and inflate the price so I can exit early"

[@wantoxy1407]

Nice! 😍

[@57ashdot]

So if almost every strategy is b.s., feels like the only winning move is not to play

[@sylwek1177]

Because you learn by looking at others play and most of them do not make money on the log run. You have build wrong intuition.

[@drfranz29]

The thing is that market behaviour changes over time for all assets ,which means application of even the same strategy on an asset will tend to not work in the long run. What is required is to at least backtest a strategy for like last 3 months like weekly to see if it still works _ that's the easiest way to make positive turnovers over time in the markets

[@selfless727]

@@drfranz29 Unless enough people are backtesting also, then that strategy won’t work either. Truth is everyone can’t be daytraders….day trading only works when investing ratio is high enough, relative to HFT.

[@daveb4446]

This is extremely accurate. I had to do literally thousands of tests to find models that actually outperformed the market. When he says less than 1% of models are successful, that’s not an exaggeration. It’s actually an understatement.

[@drfranz29]

@@daveb4446 true man, I have also coded several strategies and most underperform in the long run , most work in consistent trendy conditions which are rare for example most strategies work on the USDJPY only since early 2022 but won't work on other symbols

[@dgoullet1]

This is more relevant to ML strategies. Good video anyway

[@tonycallender7670]

Any thoughts on ICT trading?

[@johnp7739]

"Pure price action" and "market structure" type stuff? I've heard a few people promote that, but I think it's too subjective to backtest. The only way to prove it would be a long-term audited performance record or account statements. Is anyone providing that?

[@henryjoseph9564]

This makes more sense than what KZbin so called millionaires say

[@TheVd101]

What do you think about technical analysis? I know a lot of it can be B.S, but it does seem to serve as a good tool to understand current market dynamics. There does seem to be a good amount of people who have found success trading using technical analysis and their discretion, but a lot of it seems to come from years of market experience. It's definitely not as simple as most make it out to be, but do you think someone who spends years studying and live trading markets could find "intuition alpha"-- basically internalized understanding of markets and human psychology through years of experience and in the framing of technical analysis concepts?

[@alexairplanner]

Well, you have to spend a lot of years studying and practicing before expecting any result. Otherwise it's like trying to become a lawyer in a couple of weeks. The thing that everyone confuses about trading is that it is about reading a chart like a pro and predicting market moves. That's not the case. We just use the odds and try to turn them in our favor

[@AB-zv6dz]

I love guys who claim to be quants who make videos implying that trading strategies are hard to find or elusive. Clearly you don't know the first thing about actually trading, otherwise you'd know that every quant fund is independently operating 100s of uncorrelated and unique strats. This video is so misleading. Maybe there are many false discoveries, sure, when I program a mean reverting strat it goes through 500 different interations. Every research paper does the same. Does that mean there aren't many ways to make money? Of course not. The video lacks context, it's just an edgy "trading is so hard brrrrr look how smart I am" video.

[@alexCh-ln2gw]

Basically, you don't know what works until you get lucky that it works on real money.

[@camitrading4938]

He explorado diferentes estrategias en diferentes mercados y considero que un enfoque adaptativo como el de Andre Hunger puede ser una solución optima para generar dinero constantemente.

[@RealLexable]

There is only one strategy for nowadays and ever lasting which works. Measure the real values coming in and out the market, understand the pressures the market can build up by yourself or others and grab them as fast and long as you can with modern techniques and mathematical shortcuts on given indicators. Easy as that. Done

[@Faraz70]

Was risk management like 4 to 1 (lose 1 to make 4) taken into consideration in these studies?

[@andrewxzvxcud2]

nope he uses sharpe ratio which punishes positive volatility

[@paulnyagini]

​@@andrewxzvxcud2 the problem with that 4:1 ratio does not give you eny edge since your win rate will not be 50% most likely would be bellow 15% since your break even would be 20%. 🤦

[@hplovecraftmacncheese]

Why would you try to create a new trading strategy when there are known valid strategies that people have been using for decades?

[@g59fan50]

First!

[@algoodsfx939]

All strategies are profitable a 30%trading strategy can be profitable its the risk management and the individuals Psychology

[@paulnyagini]

Not really because if it was true ALGO trading would be profitable.

[@banemen27]

So you're saying those tik tok finance gurus are false?

[@trustfultrading]

Good video, but it is entirely possible to develop real profitable strategies.

[@ronin6158]

this man clearly did not go to Harvard.

[@MarksmanSpecialist]

the only way he can beat the market is to become a professional coin flipper

[@mrreal3853]

NOT MOST ALL ARE FAKE PRICES ARE RANDOM

[@sandk]

I guess Jacob Amaral (KZbinr) found the holy grail and makes $100K a month now with automated trading strategies, a bit hard to believe...

[@allyncraig7419]

I once did live paper trading of multiple trading strategies with specific rules for entering and closing trades. All strategies had 100% failure rate - in other words, all trades lost money. So I modified the test to both buy and sell with each open signal, in other words, trying both positions. Again, all trades, both regular and contrary also failed.

[@Quado0s]

Soooo you’re saying we’re being click baited for views? Got it

[@basumataryscience]

Nothing perfect strategy for option trading

[@hughstryker5637]

This is gonna make me sound like such a hater but if your strats were that lucrative perhaps this vid wouldn't be rammed full of youtube ads. It's hard I know but I'm just venting, thx for your content

[@QuantPy]

Haha too funny, the strategy doesn’t work - whole point of the video 👌

[@hughstryker5637]

@@QuantPy Lol, touche

[@hughstryker5637]

Are you planning to do more ergodicity related stuff in future?

[@kbisht3680]

bruh I think your comment is the classic example of "think before you act" - the video is literally titled "Why Most Trading strategies fail." ....

[@steeneugenpoulsen8174]

Sounds like you are talking nonsense, since you seem to think there is trading strategies. What you are doing is trying to use the last 10 coin flips to predict the next 10 coin flips, except you are trying to predict the winning lotto numbers for next week based on all the weeks already drawn. You do NOT have access to data that predict the future.

[@andrewxzvxcud2]

You're the one talking nonsense; a better comparison is if you modeled all of the intricacies of how the coin gets flipped, etc., etc., basically all of the physics, and you tested that model strictly on whether or not the coin will be heads or tails at whatever timestep you had statistically significant and correct predictive data. i.e., the market isn't completely random, and we know that for sure because of companies like Renaissance Technologies or people like Warren Buffett.

[@crazyhead1588]

The only way you going to learn trading is by doing it yourself look at the chart one market for one hour a day with fulll focus the market always makes the same moves over and over again and you will start seeing patterns after a while

[@paulnyagini]

🤭🤭🤭🤭 the problem with that you will still won't be consistent profitable. Because the markets have already discounted intuition long time ago.