Reinforcement Learning for Stock Prediction

[@ukimalla]

with the number of videos he's putting out these days, i'm convinced that he's built an AI that edits videos for him. keep it up man! loving all your recent content!

[@NickKartha]

This should be true at this point. Would be dissapointing if not.

[@saiskanda]

Yup defined true! Siraj prove to us you are human!!!

[@RahulKumarAitian]

I thought the same..

[@lionelarucy4735]

Currently my favorite youtuber. It's weird but everytime I think of doing a project on AI somehow youtube always suggests the topic and it's usually by Siraj. Am beginning to think the AI thing might be true lol.

[@ItzGanked]

nah just plagiarism

[@samram2950]

Man u are not human being, i'm hardly finding time or energy or motivation to learn new concepts everyday, you are not only learning but mastering and releasing videos on constant basis, you are some thing else dude.

[@SirajRaval]

Thanks Sam, I'm definitely trying to transcend my human capabilities and do the same for Wizards everywhere

[@samram2950]

Wow, wasn't expecting this, you made my day (technically night) by responding to my post, thanks man.

[@TheBobjovi]

sam ram it's easy to give the illusion of having mastered these topics

[@abramswee]

i think Siraj is a borg from star trek. there is no way a mere mortal can do all these video in such a short time.

[@ThunfischXXx]

Well, ou go on medium, read a few articles and make the video!

[@dp0813]

Haven't watched one of your vids in a long time; so glad to see you've slowed down your presentation speed! Keep up the great work!!

[@mikedavinci6731]

Siraj Raval is a A.I created by Google he is way too good to be human !

[@12WeMet1]

We need more programmers like this gentleman educating people!

[@donnidarko465]

I like this style of video better than others. Very easy to follow along with , perfectly paced

[@vijayabhaskar-j]

There is actually a full specialisation on reinforcement learning on finance. On Coursera

[@alexrosellverges8345]

Vijayabhaskar J lol I didn't know it, thanks man!

[@talalzahid2241]

can u provide a link

[@illustriouschin]

You are the second smartest man on KZbin.

[@jkrai9684]

Gordon Chin whos the first?

[@SirajRaval]

i am the sum of the people that have influenced me

[@M.G.R...]

*Very very thanks sir for this awesome gift* 😀😀😀😀

[@loganathansiva7063]

Hi Siraj very kind of you. Best wishes for your future journey. Hats off to your wonderful and amazing videos. Very much stunning and impressing and shocked by the way the stock predictions are done in this manner. Thanks a lot. Advance congratulations to you and your ventures. May the almighty bless shower on you.

[@memoriasIT]

Your videos are always great, and in the last years I have seen your content getting better and better. Good job 🙌🙌

[@naseef2075]

You uploaded it just like you said yesterday, Thanks, I really wanted to learn this.

[@RobTheQuant]

Siraj, I'm a big fan of yours, but looks like you're churning videos like crazy with popular keywords like 'stock prediction, making money etc' to attract subscribers, while not really caring about the subject of trading and it shows in depth and quality. The code doesn't even work. Yahoo finance api was discontinued in 2017. Did you just put up an old code without even testing it?

[@SirajRaval]

give me 30 minutes fixing now

[@SirajRaval]

Thanks for the feedback Robert! I went ahead and fixed it on GitHub. I'll admit I didn't test it beforehand because I've been moving so fast. I'll definitely do that next time. And as I consciously grow this channel to spread AI knowledge as far as I can, I'll try and stick to subjects at the intersection of my deepest interests and the interests of all Wizards.

[@purelogic4533]

robert vo my view is Siraj is doing a great job sharing directions one could take to explore AI techniques on various problems. Stock predictions is a worthwhile space as there can be immediate economic benefits if you have a winning algorithm. You will have to research and learn about how to make stock trading investor ready if u want to take that path.

[@ifoundthistoday]

each video is a 12 month rabbit hole ... I would rather he choose one or two topics a explore them

[@woolfel]

Since I used to work in boston's financial district, you can't "effectively" use public stock API to do forecasting. The latency is too high. There's a reason why businesses use FIX protocol and connect to paid services. If you look at recent reports about hedge funds, they've been going down and don't perform as well as S&P500. The HFT guys still make lots of money, but that's about speed and less about sophisticated algorithms/models.

[@willykitheka7618]

Goodness! The intrinsic value of the information you share is just mind blowing! Am enjoying every minute of it! Great job and thanks a lot!

[@I_Know_First]

Thank you for sharing our content, You can find more on our youtube channel!

[@appliedstatistics2043]

thank you, Siraj for your learning resources, it really helps me out regarding my thesis

[@dewasahu7714]

👍👍well explained video

[@kingsopaan]

Warm Greetings Siraj , Thanks for pushing me forward . I am awe struck each time i tune on the channel . Man god bless you . Peace . Regards

[@UsmanAhmed-sq9bl]

Superb Siraj 💞. I searching for this and you did it.

[@sgatea74]

What are the features feeding in the mentioned policy - aka the financial performance indicators ? A historical stock trend is not a values predictor by itself. How does the company strategy influences the long term investment decision - how is this information captured and used?

[@zaednovideo]

You developed this into a whole video? You have no idea how thankful I am! Bro you are the absolute best! I have been thinking of this possibility to use the latest in RL for trading for a long time although I wasn't sure where to start and how to approach it.

[@OO-ie1pe]

Mr. Raval, how can you use this for multiple stocks in different categories at the same time? Instead of just the google ticker symbol.

[@BonBourbon]

Bzzzz bzzzz.... - Bogdanov ? - Yes ? - He did it. - He bought ? - He went all in. - .... Dump it.

[@arcticspacefox864]

Always such an excited siraj 😂 love it cant stop watching

[@jialianglow]

How do you train the deep neural network, what are you optimising for? It's supervised i assume? I know the RL part is optimising the decision making. But how is the neural net part trained. What are we training it to recognise?

[@jialianglow]

For thing like DQN and stuff like that. I've never understood the neural network part of RL with a neural net

[@shamoonmohammad5230]

in your github code there is no code for visualization ,only output is printed on terminal can you help in this

[@ishantyagi1948]

Isn't the reward function initially unknown (and to be determined by the model) in the case of Q-learning(model-free) as opposed to the DP approach. The github link specifies the reward function as reward = max(data[t] - bought_price, 0) beforehand. What am I missing here?

[@zaednovideo]

Siraj, you can make a video showing how to give other types of info as input to your bot - along with stock market data you would dig into news, articles of traders and analysts, as well as financial statements and any other company data you can find. Although that would be a bit of a tall order as it requires tons of parsing.

[@oracletrading3000]

+1

[@zhongzhongclock]

I'm doing the same work in the last year. I've one question about the RL for stock trading. How to define a training session(cycle)? E.g. I've 10 years stock prices data about 100 stocks, then how can I define a training cycle and divide the training&validation dataset? Currently I'm using the first 9 years data as training dataset with all 100 stocks, and the last 1 year as the validation dataset, but if so, the training data's time is too short, and it's very easy to have overfitting problem. If I divide the 100 stocks into several different groups, and train them seperately, then their trained models couldn't be used mutually, it means more training data couldn't provide higher prediction accuracy, because we only could have more stocks' data, but couldn't have longer stock history data. Currently I could find a suitable answer on this question.

[@LiZhubjtu]

钟钟您好，By Q learning, you can only obtain the optimal action(buy, sell, or hold). How can you get the price?

[@Ur.Podcast_Buddy]

How to learn reinforcement learning from starting?

[@alexrosellverges8345]

Awesome, more on videos on Reinforcement Learning applied to real problems!!

[@rameshkannan4862]

Wow..! You made it so simple ..! Love your content..!

[@asadullahfarooqi254]

where was you looking at 0:50 ?

[@waynefilkins8394]

These videos never go anywhere. He always gets to the end and the results "aren't that good but can be improved". I thought it was going to show you a trading strategy? Not just a chart with some plots. He just goes on and on through all this super-complex shit, then it doesn't work anyway lmao. Oh well I guess it's like they say: if you can't do, teach. He has all these people who don't know anything about machine learning like "omg he's so smart, what a genius" but he's just putting out these same style videos on new topics with no real positive end result. Reminds me of tv shows that end every episode in a cliffhanger, except he shows us the result of the cliffhanger, without actually making us watch the next episode, and it's always a disappointment. Actually if he actually reads this, that's a damn good idea I just brought up..you should hold the results and put them in the beginning of your next video. I bet you get like 50% increased views that way. "So for today's video, first of all here's the results of the last video, it was another fucking failure...but oh it can be improved with other strategies i'm sure of it! Okay onto today's project which i'm sure will end with very positive results!" fucking genius.

[@paulogaleano5941]

Hi, I training the model with real historical data, but, when I try evaluate current year with the model it returns. python evaluate.py YPFD.BA model_eP1000 Using TensorFlow backend. -------------------------------- YPFD.BA Total Profit: $0.00 -------------------------------- Can you help me to understand? Action always is 2

[@madytyoot9888]

See github.com/edwardhdlu/q-trader/issues/1

[@shambhavimishra3136]

In your GitHub link the code is very nicely explained but how we plot graphs, what area the parameters you are setting in x-axis and y-axis. please let me know sir, how you predict the model. Please let me know sir.

[@evanskiplagat7303]

hi Siraj, model requires xrange which is not defined. I used def xrange(x): return iter(range(x)) in evaluate.py,function.py and train.py files then def xrange(x,l): return iter(range(x,l)) in agent.py file

[@sibonelongobese8639]

Thank you, Siraj. Keep up the good work.

[@albertomartel6508]

This was already the side project I have in mind. Trading bot with RL. The video on this topic was overdue for a long time

[@zaednovideo]

Alberto Martel same here

[@waynefilkins8394]

Is there a link to the code used for that tensorflow prediction?

[@saburihindaria9409]

hey could you explain how did you create graphs from your github code?!

[@ZobasArk]

Rocking it now thanks to you. It's starting to gradually sink in. Had to pause your video about 100 times and write your entire transcript and read it back about 5 times, but man.... my brain took a leap. I'm already thinking of ways to improve the model. I might be able to do a Siraj Rap before you know it as well, get into a rap battle:) Just kidding, I'm 56 and learning this for the first time. My dendrites are firmly planted so the elasticity to learning has to be rerouted through repetition or brain implant technique. So the more I think about this I believe the best route to any model is to keep it processing in your head until you've got it figured out 100% and then make the model to match your own thinking. I was thinking of creating a virtual clone of myself and the way I process which is far different than the average bear. I could do a gradient descent as an activation function, but then I started to think that it could become more than that, and started looking at nano processes and developed a new way of processing information. This took it to the next level, then I built my first nanochip from scratch. I couldn't see it at first, but it hit me. We are here at this crossroads of seeing nano materials that could be programmed with Artificial Intelligence. Where is this going? I'm right back to starting up your video again, more to learn would love to partner with you.

[@M4gicT0]

what. the. fuck.

[@sonofdarvin]

Good Guy Siraj, wants to make us rich.

[@eav300M]

You are amazing!! Thank you!! 🙏😀📈📉🏆

[@ai4finance_byfintech]

FinRL: A Deep Reinforcement Learning Library for Automated Trading in Quantitative Finance

[@guitarheroprince123]

I once tried RL on Bitcoin Price. AI told me to buy and never sell because it keeps going up ._.

[@cernos7230]

well it's currently down, I suggest you buy now or later since it is still going down

[@arsalan2780]

Can you please make a detail video on this topic.. it seems like a rapid fire round :D ..

[@NyroSlice]

you are awesome man! thank you so much

[@juanpabloweber9029]

Total noob here. Are RL methods better than LSTM's? Is it possible to do a combination for better predictions? I would be happy to just get a prediction for the next trading day in Forex or any instrument and trade in that direction intraday.

[@alfonshomac]

There's an NYU course on this at Coursera right now.

[@MLwithAlva]

Exciting!! Might try to implement this 🤔

[@HessianISR]

Is the Q function part an excerpt from an article? Could you link it please? :)

[@iamhamzaabdullah]

How to predict using ML estimated time of arrival of trains?

[@shivangmathur8776]

Does this use a time series model?

[@hk2780]

Do you know any paper related with the reinforce learning which work as stock investigation?

[@TQwando]

Awesome video! I watched most of your videos and was wondering why there wasn't one on Distributed ML. I strongly believe one of the keys parts of getting to AGI is to use a distributed system. For example look at how our body/brain operate, it basically distributes tasks and then connects them in several ways in order to achieve a common goal. Perhaps do a example comparison like Horovod (from Uber) VS Distributed TensorFlow (from Google) (or more fully integrated solutions like dmtk (Microsoft) vs Spark (Apache))

[@anatolyalekseev101]

I was very glad to hear your first words about that predictive models are not all what's needed for successful trading, that trading policy is no less important than accurate predictions. However I was disappointing to see your final chart where you draw a comparison of .. just predictions. Not performance metrics. Of course nobody can beat the market in 9 minutes ))

[@rizkyyogaoktora949]

I use tf_agent for this technique. it's easy to implement

[@Themusicaligera]

Please help! I cloned the project in order to do a test, I am using Intellij IDEA Ultimate, how can I train and test the project??

[@oracletrading3000]

Learn python first

[@MakerClubforALL]

Can anyone help me with the code error? Traceback (most recent call last): File "train.py", line 1, in from agent.agent import Agent ImportError: No module named agent.agent

[@xtr33me]

Not sure if you got this resolved, but I got around this by using python 3 rather than 2

[@GermanCoder42]

try to put every klass into same file

[@oracletrading3000]

Install gym requirements

[@maryammoghimi3579]

Hello, thank you for your useful videos. I would like to know about XCS in stock market. Could you please help where I can find this code?

[@diegoantoniorosariopalomin4977]

Do you know how to use ai to explain why stock has / will change its value ?

[@diegoantoniorosariopalomin4977]

Can one texniques developed for ai interpretability used for that purpouse ?

[@dummydummie5235]

Hai.. can anyone help me....I'm trying to predict stock price using this code....But I'm getting value only 0....And I'm not getting any graph printed...

[@OO-ie1pe]

neva mind just had to watch your video several times

[@Foxxxxx96]

can you make one using chat GPT?

[@akramsystems]

Omg this guyyyyyy such a great guy !

[@AbhishekKumar-mq1tt]

Thank you for this awesome video

[@Gottii92]

@siraj raval how do i create my own gym? literally every tutorial i find uses gym.make but how can i apply reinforcement or q learning on my own environment? keep up the good work

[@SimonLimygoogleplus]

The code represent is better than coding on the note book. It is easier to watch the code. Thanks.

[@LiZhubjtu]

By Q learning, you can only obtain the optimal action(buy, sell, or hold). How can you get the price? U did not tell.

[@shubhamagarwal5933]

Great. It would be awesome if you could teach us something about your specialization i.e robotics..

[@oracletrading3000]

Informative project 👍 But can we use the close price to affect the action? Maybe we can use more information rather than close price (open, high, low, close) I've modified the environment to match binary options but it seems the agent can not find the result even after 10.000 episodes

[@SurajKumar-rs5xh]

siraj bro you rock

[@zupachai]

Thank for your sharing!

[@carlelg5001]

Why python2?

[@wolfisraging]

U r one and only

[@Funny9689]

Can you please do a video on deep reinforcement learning for continuous action spaces? Thanks.

[@jokeith2798]

interesting.... what if we could apply to us humans itself? I mean you get things done this way right?

[@enkimarduk7028]

Mine was the 1000th thumb up, ah ah ah!

[@saiskanda]

Siraj can you do a video about Language learning and translation? And what languages do you speak? Xiaomi made a mini translator that doesn’t need data. But can instantly translate between 14 languages. Found it very interesting. I heard they are using Microsoft developed algo for it

[@JohnKlopp]

Learning AI is fun!

[@ironbike]

It could be amazing if you are creating a video about AI and Social Marketing Influencers. As the social nets are growing, and the number of followers, there is a huge market for adds that very few people know the real ROI, because there are very few real followers behind the big accounts. Can you show and discover the revenue of influencer marketing and share your knowledge about this field? You can compare the big influencers against the small influencers and predict who makes better sales after advertising a product via a post/picture/video. Thanks for this video of trading.

[@saiksredmi6196]

Hey Siraj, hope you are doing well. Thanks for your youtube channel and the stuff you put. I am starting off with learning Python. Can you suggest me one best book or web content for Algorithms and Data structures in Python?

[@ravindulandekumbura6811]

hey it was very interesting and it helped me a lot in one of my academic project. Could you please give me the where you took this dataset?

[@SciTechTalk]

I would very much like to see a video on “Self supervised Learning” (Yann LeCun) !!

[@eminem2]

Man how is this guy still on youtube

[@Acid31337]

finally you got it

[@M.G.R...]

*Sir pls make a video on Reinforcement Learning for Binary Option and Forex*

[@oracletrading3000]

I'm doing it, but still don't get the best solution from the agent

[@ShakTMT]

great

[@sbhutani304]

He just speaks and acts... No match of what is shown and what is spoken...dislike

[@SirajRaval]

hmm will improve thanks

[@johnvonhorn2942]

Quagmire values had me in hysterics. My neural network did NOT predict that; straight out of left field.

[@VISHALSHARMA-qb7ue]

You are awesome bro

[@3verythingisbroken]

Hello Siraj, it’s World.

[@OO-ie1pe]

thank you once again

[@RJ-or8bw]

First

[@p.j.carpio9585]

Thanks !!

[@wdai03]

This is pretty cool and all but you seem to be doing a lot of finance stuff lately. Having worked in the financial industry before I really think it would be much better to focus on how ai can be used to make the world better cause benefits finance brings to the world is really really really marginal. True that hedge funds can make a lot of money but they are already applying crazy techniques and sometimes returns aren't even stellar. We already have a plethora of phds chasing money whilst contributing nothing to the world, we don't need more

[@kalebbruwer]

People are greedy by nature. The types of videos he publishes most won't change that. Siraj is just all over the place and covers basically everything, trading just happens to be what he's focusing on now.

[@pluto8404]

Efficient financial markets are extremely beneficial. Every successful nation, has well established financial markets. We still might be burning whale oil if not for stock markets and jp morgan. Tesla may not have taken off. Stock markets move money to successful innovative companies and away from old failing ones. It promotes creative destruction.

[@justsomerandomguy933]

Cool! I am gonna be rich! :D

[@eeee8677]

you rock

[@phcpovoa]

Siraj, can you also post the same code without usage of Python classes? I find very hard to understand it, although I try very hard... PS: thanks for sharing your knowledge. Your videos are great!