Bellman Equation - Explained!

Let's talk about the most consequential equation in reinforcement learning: The bellman equation.
ABOUT ME
⭕ Subscribe: kzbin.info...
📚 Medium Blog: / dataemporium
💻 Github: github.com/ajhalthor
👔 LinkedIn: / ajay-halthor-477974bb
RESOURCES
[1] Reinforcement Learning book: incompleteideas.net/book/RLboo...
[2] Paradigms of ML: idapgroup.com/blog/types-of-m...
[3] Model Free vs Model Based RL: spinningup.openai.com/en/late...
PLAYLISTS FROM MY CHANNEL
⭕ Reinforcement Learning: • Reinforcement Learning...
Natural Language Processing: • Natural Language Proce...
⭕ Transformers from Scratch: • Natural Language Proce...
⭕ ChatGPT Playlist: • ChatGPT
⭕ Convolutional Neural Networks: • Convolution Neural Net...
⭕ The Math You Should Know : • The Math You Should Know
⭕ Probability Theory for Machine Learning: • Probability Theory for...
⭕ Coding Machine Learning: • Code Machine Learning
MATH COURSES (7 day free trial)
📕 Mathematics for Machine Learning: imp.i384100.net/MathML
📕 Calculus: imp.i384100.net/Calculus
📕 Statistics for Data Science: imp.i384100.net/AdvancedStati...
📕 Bayesian Statistics: imp.i384100.net/BayesianStati...
📕 Linear Algebra: imp.i384100.net/LinearAlgebra
📕 Probability: imp.i384100.net/Probability
OTHER RELATED COURSES (7 day free trial)
📕 ⭐ Deep Learning Specialization: imp.i384100.net/Deep-Learning
📕 Python for Everybody: imp.i384100.net/python
📕 MLOps Course: imp.i384100.net/MLOps
📕 Natural Language Processing (NLP): imp.i384100.net/NLP
📕 Machine Learning in Production: imp.i384100.net/MLProduction
📕 Data Science Specialization: imp.i384100.net/DataScience
📕 Tensorflow: imp.i384100.net/Tensorflow

[@gauravshinde8767]

KZbin algo, please make the relevance score of this video to 10/10. This video is too good to be ignored

[@CodeEmporium]

Thank you! Now if only the KZbin gods listen

[@jsp991204]

Thanks alot!!😀

[@vanilan3585]

you just make video. what am i about to study😃

[@slitihela1860]

can you prepare a video for Double Q-Learning Network and Dueling Double Q-Learning Network please

[@rinibhasin17]

Confused :(

[@bhaveshachhada7242]

I was confused. You made me more confused. This doesn't explain the intuition.

[@amiralioghli8622]

Hi sir, Please turn your series direction on implementing Transformers on Time Series data Please we are waiting.

[@vasarmilan]

I never heard anyone using Transformers for time series, doesn't sound to be a good idea

[@amiralioghli8622]

@vasarmilan Hi, sir. There has been a lot of research done on implementing transformers in time series. Please do a search on Google, please. However, there are no videos available on KZbin for a step-by-step guide on transformers in time series, only for educational purposes. If someone creates a playlist and uploads a video, it will be the first one on the entire KZbin platform as well as solve a lot of students problem like me.

[@vasarmilan]

​@@amiralioghli8622 I did a Google search now, I see in the last 1-2 years there has been an increased research interest. However, all the papers I see are very much "primer"s that ask the question if there will ever be truly efficient timeseries transformers. While I can see the value in some specific cases, like ones similar to speech (very high dimensionality and discrete, relatively low numbered timesteps), for "textbook" timeseries problems (eg. when you have a single or low numbered timeseries with many steps), there is really no point in trying to apply Transformers, as they are really meant to work with high dimensions. And I never encountered a practical situations so far when a (numerical) timeseries was like that.

[@CodeEmporium]

While I have mentioned in the past that transformers can be used for time series data, it isn’t standard practice. So if you are blocked on a project, I would recommend looking at either traditional methods (like ARIMA) or Machine Learning methods (like building a regressor) for this. I have a video couple of videos on “Time series forecasting with machine learning” that you can look up. Hope this helps for now :)

[@amiralioghli8622]

@@CodeEmporium thank you sir from your replying I did that 🙏