Reinforcement Learning for Trading Practical Examples and Lessons Learned by Dr. Tom Starke

[@Bill0102]

I'm immersed in this. I read a book with a similar theme, and I was completely immersed. "The Art of Saying No: Mastering Boundaries for a Fulfilling Life" by Samuel Dawn

[@Otvazhnii]

Yet another improvement. You cannot rely on one attempt of training. Initial weights are random. You have to implement a multiprocessing logic and run 10 attempts at a time and print the 10 pictures of profit and loss curves at the end.

[@Otvazhnii]

And yet another improvement. You regularize a state data by substracting mean and dividing by standard deviation. It is a good thing to regularize but a state data is made of 141 values, including ohlc prices for M5, H1, D1 bars and different indicators ranging from -10 to 100. I do not think you can merge quantities of products with prices for products and then regularize, as they say, altogether.

[@attilasarkany6123]

yep, you are right, that part of the code is wrong.

[@scalbylasusjim2780]

I think neural networks start to truly outperform SVM’s as the decision boundary becomes more and more nonlinear. Kernel tricks would have to become more and more complex.

[@AlexeyMatushevsky]

Thank you for great presentation!

[@Otvazhnii]

I come up with a number of improvements to the code. Firstly, the epsilon calculation runs out to zero after trade 5, while 99% of random numbers fall between 0,1 and 0,9. So no exploration after trade 5. Secondly, H1 and D1 bars are made from M5 bars by choosing only the 2 left and right M5 bars. This is correct for open and close prices, but not for high and low prices which move very noisily during every hour and still more so during the day. Thirdly, the way the code is built, it may take well over a month to run 11500 games (trades) as is indicated in your code. By converting the pandas data to numpy data and by then building a numpy array of states before training, you can speed up the code literally 10,000 times. And finally, the Apple stock goes bluntly up at some point so your strategy which drops exploration after trade 5 and starts learning the replay memory of the last trades, does it not fit so nicely to the growing trend of the stock?

[@zhibindeng8723]

why not share your code for us to backtest? Thank you.

[@michelletadmor8642]

I wonder how he smoothes the data - perhaps "now" timestamp was already including partial info of the next data point. If it was smoothly only backwards then the next timestamp at exit might be completely off than the real exit price.

[@blackprinze]

SVM has some type of geometry element which responds well to any freely traded market

[@AtillaYurtseven]

30:14 you are updating state and applying the action. When we chose an action, first we need to apply than we need to update the state and get the reward. Let's say current price is 100.20. When agent decides to buy, it's has to buy from the price 100.20 (excluding spread/slippage and commission). In your example, it's buying with the next price. Am I wrong?

[@hanwantshekhawat4314]

Executing at the next price sample (tick/bar) is a common way to model time delay. Simplistic but adds a some noise which may be more realistic than assuming execution happens at exactly the price where the decision was made

[@chrisminnoy3637]

Just as with any other AI algorithm, you need to clean your data before you give it to your reinforcement learner. But you can make a neural net that cleans that data for you, with relative succes. Noise is also an issue in other domains, not just finance. Ofcourse you are creating a feedback loop. When you buy/sell with succes your competitors will adapt, and so the problem shifts to a more difficult state...adding overal noise (randomness) to the system.

[@niallmurray2915]

How do your competitors know you are successful?

[@harendrasingh_22]

3:32 guys please upload his talk too !

[@andy.robinson]

There's quite a few Tucker Balch vids on YT. You can probably pick up a lot from those 👍

[@williamdad]

does this really work for stock trading? any trackrecord to check for last 5 years?

[@aricanto1764]

NLP is the way forward 💪

[@AlexeyMatushevsky]

On 19:53 you mentions the Right Regime - 'does it end up choosing some training process ' It's easy to understand what is Mean Reversion process, but what does it mean 'training process' ?

[@alute5532]

I am doing deep learning but now I'm thinking of integrating it with a reinforcement learning as an ensemble on the outside. (with a money management system on the side) Is there Anyone in California interested in my project?

[@Scrathzerz]

How’s it going?

[@suecheng3755]

Hi there, my research area is reinforcement learning, maybe I can give you some ideas. May I have your contact information?

[@alrey72]

but the technical indicators are from past prices also. isnt it better for the nn/rl interpret the prices themselves?

[@a_machiniac]

09:37

[@monanica7331]

BTC for $75K by end of this year& Control of The Currency is already Decentralised And now the China disruption would simply Decentralise the Mining setup for the better

[@norabelrose198]

"LSTMs, they're somewhat new" they've been around since 1997 lol

[@block1086]

attention is all you need

[@polonezu9576]

can we use this on MT 4 platform?

[@andresg297]

MT5

[@polonezu9576]

@@andresg297 is not posible

[@joysahoo7470]

Thank you sir for good explanation! Please help me to solve this error - ImportError: cannot import name 'sgd' from 'keras.optimizers' am not able to fix this error and if anyone to fix this error please help me

[@eliastheis5265]

from keras.optimizers import SGD not sgd

[@eliastheis5265]

@@joysahoo7470 Try from tensorflow.keras.optimizers import SGD

[@joysahoo7470]

@@eliastheis5265 Thank you

[@polonezu9576]

i get errors on mt 4

[@gogae22]

Why can't we give reward in every time step?

[@randomdude79404]

From my basic knowledge the reward only comes in when a decision is made whether that be a buy or a sell. I may be wrong but this is just from my basic understanding.

[@henrifritsmaarseveen6260]

95% of the trades are made by big money they can hire and build the most advance systems and programmers .. and they fail . so it probably will not .. will not ever work ..

[@oliverli9630]

Siri was triggered at 1:40, hahaha. Time to rethink about ML?

[@theappliedcoder9824]

Hi, I work on Reinforcement Learning too, anyone hiring reply !

[@Tradinginthezen]

how much salary are you expecting?

[@theappliedcoder9824]

@@Tradinginthezen depends on the work flow, sir

[@redcabinstudios7248]

If AI works, Quant Trading will go down I guess, beware!

[@alute5532]

23:55 when you said random walk most of the time I just realized you're not a real Quant my dear one.

[@rshsrhserhserh1268]

why

[@williamqh]

@@rshsrhserhserh1268 the market is moved by big institutions usually, so they will make it appeared to be random but not really.

[@guardtank4877]

I thought reinforcement learning is shit for trading

[@100xspaceai9]

dont think do , test and validate