Kalman Filter - VISUALLY EXPLAINED!

[@shaswatgupta6590]

Sir, this is beautifully explained. Loved the clarity. The sequence of building blocks used to teach was so intuitive. Thank you!

[@KapilSachdeva]

🙏

[@dominiquedeveaux361]

+

[@100rabhkr_India]

Hands down. The best lecture on Kalman Filter!!!!!

[@KapilSachdeva]

🙏

[@user-or7ji5hv8y]

So much grief could have been saved if your explanation was available in my university days. Really clearly explained.

[@abhinav_mzn]

Your presentation makes it very simple to understand...please keep making videos like this, there is a shortage of content like this on KZbin.

[@KapilSachdeva]

🙏

[@KapilSachdeva]

*Key insights/takeaways that I had wished to convey using this tutorial* _Failure to convey those insights properly and intuitively is a function of my limitations_ * Appreciate the difference between Model & Algorithm. Our goal is *state estimation* using the State Space model. There *could be many algorithms* that could operate to find the unknown parameters of our model. * Start first by understanding the model and then worry about the algorithms to use. Appreciate *why states form a Markov chain* and *why we can consider the observations as independent* of each other. * Instead of starting with Kalman Filter equations and/or trying to justify its components (Gain, matrix, etc), a more intuitive & scalable approach is to first understand the model, then *understand how and why the Bayes Rule would help with state estimation* * Kalman Filter is an *efficient algorithm* to find the parameters of the State Space model provided the model is *constrained/limited* to have "Gaussian" Random variables for state and observations and linear functions for transition & observation functions. * Efficiency of Kalman Filter is a consequence of the fact that when the components (likelihood and prior) in Bayes Rule are Gaussian then we do not need to compute the integral numerically (the normalization constant). We have an *analytical solution i.e. a closed-form solution* or perhaps in simple English, we can say we have a formula. * As mentioned above if every component in Bayes Rule is Gaussian it is helpful for computation. However, if your transition & observation functions are *non-linear* then the output (or the next state in our case) would not be Gaussian and then Bayes Rule will haunt us. This is *why linear functions are required*. In a way, appreciate that the requirement of linear function has more to do with the requirement to use Gaussian Random variables in Bayes Rule. * Many algorithms in statistics follow this 2-step pattern of "predict" and "update" and so does Bayesian Filtering (... and by extension Kalman Filtering) * Prediction at the very first step in the state-space model is your "intelligent" guess. In the subsequent steps, it's even more intelligent as it has included some observations from the previous time steps * Is it really fair to have assumptions of Gaussian Random Variables and linear functions for transition & observation? * Appreciate the delicate dance/balance between the accuracy of inference and the computational cost. If your product is extremely accurate but computationally so heavy that it does not provide desired user experience it is of no use to anyone! * Control theory people use *x* for state and *z* for observations. Statisticians and ML folks use *z* for state and *x* for observations. And people who suffer because of all this notational inconsistency are us!

[@rahulbirari401]

Just Mind-blowing loved the way you explained concepts and slowly built on it, that's how inventions and human mind works and mathematics is just a tool to realize/record complex ideas. Other lectures directly jump into maths without explaining the idea.

[@KapilSachdeva]

🙏

[@pectenmaximus231]

I rarely comment on these kinds of videos but this really was a great exposition, well done.

[@zlatasmolyaninova1140]

this is the most clear explanation among all i have read/watched! thank you very much

[@arunabhbora5610]

Best video I have ever watched in AI domain, Thank you Sir for the precisely detailed concept.

[@KapilSachdeva]

🙏

[@JagadesanGanesan]

What a clear explanation ! A gem.

[@ruslansergeev4061]

This is an absolute internet gem !

[@KapilSachdeva]

🙏

[@GabrielOduori]

I am only learning bayesian filters and I am yet to get any other material that explains it like you did in this video. A huge part of the work I recently embarked on! Thank you

[@KapilSachdeva]

🙏

[@lakdafari]

Outstanding explanation. Thank you so much for making this so clear.

[@KapilSachdeva]

🙏

[@Omeomeom]

you ate this video up no lie your presentation and scope level was perfect thank you for this amazing information

[@KapilSachdeva]

🙏

[@michaelwirtzfeld7847]

Thank you. Your framing of the linear Kalman filter is very well done. I look forward to your future videos of the Kalman filter under non-linear conditions.

[@KapilSachdeva]

🙏

[@ayushichaturvedi3854]

Wish I came across this video wayyy back! Such a saviour, thanks Kapil sir!

[@KapilSachdeva]

🙏

[@violinplayer7201]

Best explanation on kalman filter! Thanks so much!

[@KapilSachdeva]

🙏

[@abcdefghijklmnop754]

This ground-up approach is excellent 🙂Thank you for explaining...

[@NurettinOzcelik]

Very good explanation of Kalman filter, thanks for your time and work for that video.

[@Charmander36023]

Thank you for your work, you are a very talented and valuable teacher

[@yaoismyhero]

Kapil, your videos are amazing, including this one! Your teaching and presentation skills are top-notch.

[@KapilSachdeva]

🙏

[@amiyabanerjee2443]

Loved your way of explanation. Amazing! Keep adding more such educational videos.

[@KapilSachdeva]

🙏

[@annawilson3824]

Kapil, this was nothing short of a brilliant video, 11/10!

[@KapilSachdeva]

🙏

[@pranabislam7875]

This lecture was incredibly clear and informative wow 10/10

[@KapilSachdeva]

🙏

[@mmattb]

Dumb question: Why are the transition (A) and emission (H) matrices time indexed (referring to the slide at ~26:30)? Is that intended to imply that a Kalman Filter can be applied to a nonstationary process? Second dumb question: why is the time index on H k-1 instead of k? The conditional probability distribution on the right side of the slide shows k as the time index, which makes sense to me, but I don't understand why k-1 is shown on the left side. Anyway, thanks a bunch for the video. I love you build up the basics first!

[@KapilSachdeva]

Both of your questions are valid (and definitely not dumb)- See below resource: arxiv.org/pdf/1910.03558.pdf Look at Section 8 (the last section). Also, pay attention to the symbols/notations (x vs z) (a) yes it can be applied to non stationary process and some literature puts the time step as you will see in above paper (b) I made the typo 😒it should be H_k and not H_{k-1}

[@mmattb]

Ah okay, that clears it up. Thanks a bunch Kapil.

[@KapilSachdeva]

🙏

[@gvpathi]

Clear and information flow is natural.

[@KapilSachdeva]

🙏

[@aruntakhur]

Beautifully Explained. Really seeking some other topics to be covered by you like Deep unsupervised learning (Pieter Abbeel) in your way :).

[@AI_ML_DL_LLM]

A superb video, thanks. a simple numerical example at the end would be great.

[@KapilSachdeva]

There is a link to the notebook in the description of the video.

[@alexandervocaet566]

Your video is gold, good sir.

[@KapilSachdeva]

🙏

[@xuyang2776]

Thanks a lot ! This video is easiest to understand that I have ever seen. But I have a question. If the parameters A(k-1) in transition function and H(k-1) in the emission function are unknown, how to handle?

[@ritishmaram3326]

Excellent explanation sir, truly motivating and inspiring sir. Thank you so much !

[@KapilSachdeva]

🙏

[@rossijuan9548]

Excellent explanation, many thanks!

[@KapilSachdeva]

🙏

[@gentle-g]

Nice explain!! Thank you prof.

[@KapilSachdeva]

🙏

[@gastonparra2971]

Excellent presentation; the video is so interesting.

[@ssshukla26]

I just completed your video. Nice one. Subscribed.

[@KapilSachdeva]

🙏

[@user-or7ji5hv8y]

A possible suggestion. I think a Tensorflow probability example of this would be really nice.

[@raulpalmaa]

Thanks for this tutorial.

[@KapilSachdeva]

🙏

[@MLDawn]

I really wish you would put an equally amazing tutorial on Kalman Bucy filters (continuous state space models). Thanks

[@alnar447]

Excellent. At around 13 min mark the measurements could potentially indicate any state with certain probability? Emission function can take measure ment to any given state? So multiple arrows potentially

[@KapilSachdeva]

Not sure if I understand the question. Could you please elaborate?

[@luizemediato6987]

Very nice explanation. Thank you very much.

[@KapilSachdeva]

🙏

[@FGundogan]

An application would help great, many thx Kapil!

[@KapilSachdeva]

🙏

[@SaurabhKataria10]

Thanks so much for this. Terrific!

[@KapilSachdeva]

🙏

[@Vidit2011998]

Professors in Germany couldn't teach a shit about this. Meanwhile you explained wonderfully!

[@KapilSachdeva]

🙏

[@MLDawn]

I really wished you had also explained the meaning of Kalman gain at 28:45.

[@vi5hnupradeep]

Thankyou so much sir ! You make things look so simple 💯

[@KapilSachdeva]

Thanks Vishnu for your appreciation 🙏 . It makes me happy to know that this is helpful.

[@deepikabhoominathan4969]

Explained neat and clean...thanks a lot 👏 keep explaining the topics in artificial intelligence sir

[@KapilSachdeva]

🙏

[@roy1660]

Please upload video on particle filters and feedback particle filters

[@KapilSachdeva]

Sorry for the late reply; will make some tutorials on them as they are indeed very interesting. Working on some other tutorials that would be prerequisites for particle filters.

[@roy1660]

@@KapilSachdeva Thanx for the consideration.

[@KapilSachdeva]

🙏

[@100rabhkr_India]

Time 28:58, Can you please explain what will be the value of m_k in the 1st equation of Update? How is the calculated and updated? Is it a constant?

[@KapilSachdeva]

m_k is the observation (eg the value that your sensor will give) at step k.

[@sun1908]

Thank you.. very nicely explained.

[@KapilSachdeva]

🙏

[@Manishkumar-ww4gm]

Very Nice explanation Sir.Than you!

[@KapilSachdeva]

🙏

[@user-or7ji5hv8y]

Do you have examples of where such Kalman filtering can be useful.

[@KapilSachdeva]

I am assuming that you are asking about why Bayesian Perspective of Kalman Filter is important & useful. Kalman Filter, the algorithm, assumes "Linear Gaussian" State Space Model whereas most of the real world phenomenon do not have linear transition & observation functions and assumption of Gaussian distribution for State & Measurement Random variables is not always appropriate. Note - In the above paragraph I have distinguished between "algorithm" (i.e. Kalman Filter) and "model" (i.e. State Space). It is important to appreciate this difference. Seeing Kalman Filter as a special case of Bayes' Filter helps you appreciate that you can apply other algorithms to find your latent/state variables. Essentially, you now have a "general framework" of Bayesian "predict" and "update" equations where distributions could be other than Gaussian provided you have a way (algorithm) to solve these equations. For e.g. you could use Variational Inference or MCMC methods to approximate the posterior state. And in the modern machine learning realm, you can think of the transition & observation functions being modeled by neural networks. See for example this paper - arxiv.org/abs/1609.09869 (Structured Inference Networks for Nonlinear State Space Models) As you can appreciate that this way of thinking in terms of a general framework helps you find solutions for more sophisticated problems where Kalman Filter's requirements (i.e. linear functions & gaussian random variables) are not met. Hope this makes sense; let me know if you need more clarifications.

[@naifalkhunaizi4372]

Art at its best

[@KapilSachdeva]

🙏🙏

[@tuna5287]

The best! Thank you sir

[@szb120]

Well explained, thank you!

[@KapilSachdeva]

🙏

[@100rabhkr_India]

Sir, do we have lectures on particle filter. An solved example for Kalman Filter will be a "cherry on the cake"

[@KapilSachdeva]

Thanks Saurabh. Nothing on particle filter yet but I have thought about making one. Hopefully soon 🙏 Look in the description of the video. You will find the link to the example.

[@kentoshintani3020]

Which presentation software are you using? Microsoft ppt?

[@KapilSachdeva]

Yes MS Powerpoint!

[@jameschen2308]

Top notch!

[@KapilSachdeva]

🙏

[@pratikanand2200]

you really explained the concept very well but I think that if you could give an example first then understanding this, was fun. Wikipedia has a good example of this.

[@KapilSachdeva]

Thanks Pratik. Point noted. Good feedback; though it does not compensate for a concrete example I could have provided, there is a link to a notebook in the description. It shows a modern implementation of KF using tensorflow probability. Check it out if interested. 🙏

[@adarshsdinopark8251]

Can you please share the link for particle filter

[@KapilSachdeva]

Haven’t made one yet :(

[@Bbdu75yg]

Amazing !

[@vijaymaraviya9443]

Thank you 👌🏼❤️

[@KapilSachdeva]

🙏

[@aishahalbarakati7942]

Would you please share the paper of the derivation that you mentioned at the end of the video??

[@KapilSachdeva]

Here it is - arxiv.org/pdf/1910.03558.pdf Look at Section 8 (the last section). Also pay attention to the symbols/notations (x vs z).

[@aishahalbarakati7942]

@@KapilSachdeva Thank you

[@KapilSachdeva]

🙏

[@partha95123]

Wow!

[@KapilSachdeva]

🙏

[@jameschen2308]

Are you a professor?

[@KapilSachdeva]

No I am not :) ... but any concern?

[@jameschen2308]

@@KapilSachdeva no concern, sir, just very appreciative

[@KapilSachdeva]

@@jameschen2308 thanks. Very kind of you.

[@PrashantBatule]

A good explanation, much appreciative of the content! But I think there is no extraordinary "visualization" here compared to a standard textbook. I would suggest a more intuitive explanation through an example, which is nowhere to be found on the internet.

[@KapilSachdeva]

🙏 thanks for the feedback. Yes I have been thinking about re-explaining it with an example. Regarding “extraordinary visualization”. This could be subjective and I understand that it is not a 3blue1brown level visualization 😊 but it does not mean it is not good enough. As a matter of fact some of the “extraordinary visualizations” could hide the necessary underlying conceptual details. They impress people but not necessarily help them understand the details.

[@user-wr4yl7tx3w]

Be great to see a python coding example.

[@KapilSachdeva]

There is a link to an example notebook in the description

[@raj-nq8ke]

Great.