AI/ML+Physics Part 4: Crafting a Loss Function [Physics Informed Machine Learning]

Рет қаралды 25,702

Күн бұрын

This video discusses the fourth stage of the machine learning process: (4) designing a loss function to assess the performance of the model. There are opportunities to incorporate physics into this stage of the process, such as adding regularization terms to promote sparsity or extra loss functions to ensure that a partial differential equation is satisfied, as in PINNs.
This video was produced at the University of Washington, and we acknowledge funding support from the Boeing Company
%%% CHAPTERS %%%
00:00 Intro
00:55 Case Study: Fluid Velocity & Navier-Stokes
05:56 Case Study: Incompressible Flows & Poisson
07:46 Case Study: Lagrangian Neural Networks & Euler-Lagrange
09:38 Sparse Loss and the L1 Norm
12:51 Case Study: SINDy + AutoEncoder
15:41 SINDy and Loss Regularization
17:59 Parsimonious Modeling
20:16 Equivariant Loss
21:59 Outro

Пікірлер: 29

@CascaGrossaSuprema 2 ай бұрын

Dear Brunton, your classes are wonderful. Perfect didactic and incredibly easy to convey complex information in a simple and practical way. Congratulations!

@krystiannowakowski7980 2 ай бұрын

That's gold, I always can't wait for the next video, wish I could jump forward into future and just watch them all at once!

@pgrudzien1221 2 ай бұрын

Thank you Steve, I was waiting to see that video. I'm excited for the series, great work

2 ай бұрын

same here haha

@stephenfischer1868 2 ай бұрын

Thank you for your excellent panning and knowledge base. You are a learned machine!

@mostafasayahkarajy508 2 ай бұрын

Thank you, Prof. I have seen the videos you cite here and this new video was an effectie review for me to understand them. Your lectures are parsimonious! I promise only people with mechanical engineering background have the ability and tendency to understand and describe complex math parsimoniously ;)

@kshirsagarabhayshrikrishna8665 2 ай бұрын

Hi @steve could you please add the papers you mentioned in the description?

@andrewm4894 2 ай бұрын

Loving this series!

@drskelebone 2 ай бұрын

I guess the fluid dynamics case actually does have a case where the Lagrangian constraint could result in observable errors between model and reality (where training is done from t0 to t1, and then fit from t1 to tf).

@grapix1184 2 ай бұрын

Great video, this is the good stuff! Can't wait for the next one!

@brady1123 2 ай бұрын

Another great video. It would be interesting to do a video about Physics Informed ML in the context of Sutton's Bitter Lesson, since this seems to be a case where adding extra knowledge into the architecture/loss of the network actually beats out more generalist approaches. This is probably due to the lack of training data in physics/engineering domains, but maybe building in physics knowledge helps in the large data regime as well.

@nicholasjaramillo9561 2 ай бұрын

Thank you so much, Steve!!!!

@johnlocke6563 2 ай бұрын

I am interested in the concept of equivariance and invariance related to neural network interpretability. Usually to satisfy the physical constraints given by symmetry we build neural networks that are equivariant, why don't we build neural networks that are invariant instead? In this way, it is not only the output of the network that satisfies the laws of physics, but it is the network itself, with its parameters. Basically instead of choosing via sgd optimisation any parameters in the parameter landscape, can we constrain these parameters in a physically relevant sub manifold? My idea would then be to build neural networks analogous to physical systems, where the parameters of the whole network have an analogue in a physical theory and not just those in the autoencoder bottleneck. An application of these neural networks could be in the field of topological quantum field theory but in general in any lattice gauge theory, where the neural network itself becomes a piece of graphene that spreads the input current over the output boundary state. It can also be a potts model, a spin glass or a penrose spin network, which recognises physics because it is built in analogy with the physical model. Perhaps putting such a strong constraint on the parameter space would be counterproductive, making the neural network lose its ability to generalise. But this is a very interesting topic.

@ArbaouiBillel 2 ай бұрын

super amazing, I'm waiting part 5

@cirobrosa 27 күн бұрын

Brilliant

@BreakingMathPod 2 ай бұрын

Are there models of loss functions/ optimization functions that can switch tactics (or swap out functions completely) depending on what stage of training a machine learning model gets is on? I was studying transformers/ self-attention architecture and it made curious if “self attention” could be used specifically on the loss or the optimizing functions to either tighten up focus on a more specific goal, or broaden it depending on where the training is at. Does that make sense? This is the “Multiple loss or optimizing functions that are activated at different times using different triggers” approach. The self-attention method of deciding when to tune or modify (or replace) an optimizing function- I think would be spectacular to demonstrate! It is the key architecture used in OpenAi’s SORA, Chat GPT 4 (and 3 and 2), and many other successful machine learning tools. Also- open question: in what ways can LLM’s or image classifiers / generators be utilized in physics intended machine meaning? Could SORA figure some physics on its own simply via studying video footage as well as the ability to identify, label, and categorize objects in a video ***and *** learn and generalize how these objects change over time? (Like objects falling). I know a lot depends on the training data (watching leaves fall vs watching rocks fall… That’s my big question!!

@cleisonarmandomanriqueagui9176 Ай бұрын

Hi Steve. I was recently diving into ML and AI applications for CFD but I realized its better to focus on robust solvers . I don't know much about this but in other video you show the adaptive wavelet method for CFD and this seem to handle or solve a bunch of CFD problems whereas ML can tackle a specific problem. Am I right ? btw I am a huge fan of your videos . Thanks .

@khawar0o7 2 ай бұрын

Please list this video. I couldn't find this video on your channel directly.

@datagigs5478 2 ай бұрын

Dear Brunton, your teaching is excellent. My path of learning is book oriented. Could you recommend me a book/paper where I can learn physics informed neural networks so that i can apply this in my nuclear engineering field like reactor design parameters, radiation transport, neutron transport etc.