12 - Planning and control

[@youtugeo]

I did a bit of digging into control theory fundamentals, in order to be able to get a bird eye view of the field and understand where DL and RL techniques fit. Here are my initial thoughts for anyone interested.. Feel free to correct me where I'm wrong. Control theory in a nutshell In general, control is an optimization problem and thus the term optimal control. It refers to the process of finding the optimal parameters of the controller (for example the matrix K in typical linear systems) via an optimization process where we minimize a cost w.r.t to the parameters of the controller, constrained by the dynamics of the system. Control problems can be classified in 2 large categories. Control of linear systems (or nonlinear systems that can be linearized around a state) and highly nonlinear or completely unknown systems. Classical linear theory of control works for linear systems, systems where the dynamics of the system can be described with linear differential equations, for example x-dot is a linear function of x. It also works for non-linear systems that we can be linearized under certain conditions (around a state). For all the other cases where the system is highly nonlinear and can’t be linearized, or if it is completely unknown, we apply other methods that fall under the umbrella of data-driven control (or ML control). In these cases ML (or DL) can be used to 1. Create a data driven model of the world for example approximating the unknown function f with ML techniques, or with DL models. 2. Learn controllers, for example learn a DL model that generates the proper control actions and 3. To learn good sensors and actuators, for example where to place the sensors and the actuators. Data driven control is an active area of research. In linear systems (linear quadratic regulator, Kalman filter, LQG), we have a linear optimization problem which gives us a closed form solution. (As I understood it, "closed form" means that we can find a set of optimal parameters for the controller in an one off optimization process. The controller can then operate with these parameters). But in nonlinear systems the optimization becomes really nasty, and we can’t find those closed form solutions. Instead, we are forced to do optimization on the fly like in the model predictive control approach, where we continuously do an optimization for a window ahead in time (for a few timesteps) as we operate the controller. The car control case shown by Yann Lecun is a data driven control approach, specifically predictive model control. Deep RL is another data driven control approach (where the cost is given at the end of a long sequence of actions instead of in every action so model predictive control can’t be applied). But these methods have limitations. Lecun says that we don’t yet have good planning techniques because we don’t know how to plan complex sequence of actions or how to create hierarchical representations of action plans. And here is an excellent series on control theory (I only watched the first videos but it is very clear that the instructor is amazing) Linear control theory 1. kzbin.info/www/bejne/hpqanWujgs-MjKc Data driven control 2. kzbin.info/www/bejne/pabPfYVma9Gdlac I hope it helps!

[@alfcnz]

Yup, you got the big picture. And Steven is indeed amazing.

[@dinhnguyen2923]

great video . Btw can I get this notebook ?

[@alfcnz]

Sure. Everything is available on the class website! 😀😀😀

[@normrubin5275]

are the notebooks on line some place?

[@alfcnz]

You're right, uploading now. Bear 🐻 with me 😅

[@alfcnz]

Alright, the notebook is on GitHub now. Have a look and let me know if anything is missing. Thank you for the feedback! 😇😇😇

[@НиколайНовичков-е1э]

Hello, Alfredo. Great video! As i understand you splitted material in video "The Truck Backer-Upper" on three parts, am i right?

[@alfcnz]

Not quite, I've added a lecture on optimal control. Next two videos are the truck backer-upper and our research paper.

[@liongkhaijiet5014]

I am very curious whether the live students can see your after effect animation or it is only on youtube?

[@alfcnz]

Overlays are made at video editing time. I could think of having them during the live session, but we're going back to in-person teaching, so it won't happen.

[@dwv_0x]

great video. are you gonna make a lot of these control related videos in the future?

[@alfcnz]

Yup, two more are coming. 🤓🤓🤓

[@francescoricigliano1819]

Ciao Alfredo! Innanzitutto grazie mille per tutti questi FANTASTICI video. Ho appena finito di seguire le lezioni universitarie e cercavo proprio del materiale aggiuntivo, ma poi ho trovato questa serie che è fatta molto meglio del mio corso! In dei commenti di altri video ho letto che pensavi di aprire un patreon per rispondere alle domande, ed è una cosa che sarei felicissimo di supportare. Questo soprattutto perché pure io avrei qualche domanda sugli argomenti, ma mi sento un po' in colpa a rubarti del tempo. Quindi diciamo che sto cercando di risolverle da solo, e se proprio non riesco, magari apro un topic sul canale di reddit o chiedo qualche consiglio nel discord. Però avrei una domanda molto veloce, cioè di quante lezioni si compone tutto il corso? Così per farmi un'idea delle lezioni mancanti da modificare/caricare. Ti chiedo questo perché sono indeciso se vedermi le lezioni dell'anno scorso per rivedere gli argomenti che non mi sono molto chiari, o vedere gli argomenti che non sono ancora stati caricati. Però sapendo che gli argomenti sono stati un pochino modificati (EBM), ho paura di fare la frittata. Pure io vengo da un background molto diverso, ing. meccanica, e a volte mi trovo un pochino spaesato, quindi ti volevo chiedere se consigliavi qualche ulteriore corso online (coursera, udemy e altro) da seguire per apprendere meglio tutti questi argomenti. Grazie mille ancora di tutto, soprattutto per la simpatia e leggerezza che trasmetti. E' veramente un piacere seguire le tue lezioni

[@alfcnz]

Ci sono 30 video in totale. Siamo al 50% online, al momento. Cercherò di aggiungere quelli di Yann a due a due ogni settimana. Ci sono poi ancora due miei. Non credo ti servano altre lezioni, ma se vuoi materiale sul apprendimento automatico, qui c'è il corso di un mio collega davidrosenberg.github.io/ml2016

[@francescoricigliano1819]

@@alfcnz grazie mille :D

[@alfcnz]

Prego 😇😇😇

[@erikdali983]

🎉🎉🎉

[@alfcnz]

😁😁😁