He died playing basketball a week ago

That's a very astute point; the standard linear kernel used in DMD (e.g. 13:08 and 15:30) is bilinear although more generic kernels such as Gaussian and polynomial are not!

In that case, you can use linear PCA as opposed to kernel PCA (which is approximately what we're doing here, except without orthogonalisation). Indeed, you can combine linear PCA and kernel PCA if the state dimension is large in both the original and kernel spaces. Let me know if that doesn't answer your question!

@@peterj.baddoo3813 Thank you. It will be nice if you could share a reference where these cases have been dealt.

@@souvikdas7773 Here's the original Kernel Recursive Least Squares paper that explicates the connection between dictionary learning and kernel PCA: ieeexplore.ieee.org/document/1315946 The Wikipedia page on PCA is quite good, including the subsection on nonlinear PCA: en.wikipedia.org/wiki/Principal_component_analysis Also, our paper is here: arxiv.org/abs/2106.01510

@@peterj.baddoo3813 Thanks again.

nu represents the sparsity of the dictionary so larger nu implies a sparser dictionary. If you want a generalisable and fast model then larger nu is better, but if it's too large then the model can be inaccurate. Another view of nu is that it functions as a regulariser for the model, so increasing nu can also prevent overfitting. The algorithm is fast enough that you can try a few different nu's and pick the best one; at present, there is not an optimal way to choose nu a priori.

@@peterj.baddoo3813 thanks that makes sense. Are there any problems that can come with using the L2^2 norm as a distance metric in this context? I can see why you've used to get a direct solution, but could $\pi_t$ ever be sparse or something like that?

@@imicoolno1 Yes, great questions. One philosophical issue is that the L2 norm doesn't have a clear physical interpretation in the feature space induced by the kernel. In the original feature space, L2^2 usually corresponds to a measure of energy. So other norms may be more meaningful in certain applications; you could certainly adapt this work to look for a sparse $\pi_t$, but I don't know if the same updating equations will work.

@@peterj.baddoo3813 thank you very much Peter! Really fascinating work 🙂

Absolutely, this is the idea behind the famous "Random Fourier Features"! people.eecs.berkeley.edu/~brecht/papers/07.rah.rec.nips.pdf

@@peterj.baddoo3813 Thanks!

Great questions! 1) It will depend on your aims but, as with many of these methods, more data is usually better. We find that a quantitative description of the spectrum needs nonlinear transients whereas a qualitative reconstruction doesn't need much data. Of course, the rank of the data is more important than the number of samples, so samples from different nonlinear regimes can be helpful. We are also working on a physics-informed version that requires far fewer samples than usual. 2) I have not tried the method yet for SDEs but I hope to in the future!

Thanks for the question! There are a couple of ways to model this. One is to incorporate an unknown control variable into the model as we describe in appendix C. For a non-autonomous system you could include time as an explicit function of the kernel. On the other hand, if the transition matrix of the (nonlinear) system is varying in time then you could use the online version of the algorithm (described in appendix B) with an exponential weighting factor or windowing.

@@peterj.baddoo3813 thanks for your valuabe respond. I will follow up on this paper.

@@peterj.baddoo3813 Hi, Peter, Thanks for your reply. I've carefully read appendix C. Is the control force should be pre-known input, like DMDc. My question is the situation of an unknown control force. Hope to hear from you.

Thanks for the question, that sounds like a challenging scenario but it could be worth a try with LANDO! Sometimes the kernel representation can uncover a latent space that cannot be represented with finite-dimensional features. This can allow more efficient model identification, which could be relevant in your case.

@@peterj.baddoo3813 Thank you for your answer.

Thanks for your comment, Kouider! The code will be published open access here in the coming days: github.com/baddoo/LANDO

@@peterj.baddoo3813 Thanks @Peter. Waiting for more videos such this

Hi Sebastian, thanks for the questions! 1) We are currently testing the method on data from channel flow simulations to learn the full Navier-Stokes equations! There is scope to include the effects of multiple scales in kernel design. 2) We discuss the sensitivity to noise in appendix E of the arXiv paper (arxiv.org/abs/2106.01510). Some problems might require smoothing the data before applying LANDO (e.g. via total-variation regularised differentiation).

@@peterj.baddoo3813 Thanks a lot for the answers! Your work is really interesting. About the multiscale in kernel design, are multiple scales included by the different magnitudes of the weights for each kernel? I have an additional question, but it's about the general framework of data driven PDE/ODE identification. Do you know if these methods have been applied to delay ODEs?

@@sebastiangutierrez6424 Sure, you can include this both through the choice of weights and the type of functions included in the kernel. Similar methods have been applied to delay differential equations, but only in the linear case e.g. www.sciencedirect.com/science/article/pii/S2405896318309832

@@peterj.baddoo3813 Thanks a lot !

Hi Hugo, this was recorded using a "lightboard studio" e.g. www.lightboard.info/. You can see many great lightboard presentations on Steve Brunton's channel: kzbin.info