Kolmogorov-Arnold Networks: MLP vs KAN, Math, B-Splines, Universal Approximation Theorem

[@josephamess1713]

The fact this video is free is incredible

[@umarjamilai]

You're welcome 🤗

[@goldentime11]

Thanks Umar for such a wonderful tutorial! I've been eyeing this paper for a while!

[@edsonjr6972]

Your videos are literally the only ones with 1hr+ I would ever watch on KZbin. Keep going mate, extremely high quality content 👏🏽👏🏽

[@boredcrow7285]

Ive learnt so much from your videos, Ive been following you for about a year now starting from the diffusion implementation and the quality of content you post in here is insane. Thank you so much Jamil, people like you are the reason why AI community is so open and free for anybody to learn and explore.

[@AdmMusicc]

You're on a mission to make the best and friendliest content to consume deep learning algorithms and I am all in for it.

[@nokts3823]

Thanks a lot for making this accessible for people outside the field, for which reading and understanding these papers is quite tough. Thanks to you I'm able to stay slightly more up to date with the crazy quick developments in ML!

[@MrNathanShow]

The intro of a basic linked up linear layers was so well done and really makes this introduction friendly!

[@kashingchoi564]

Thank you for bringing me into the world of neural network. Your videos always make difficult topics become easier by interconnecting relevant concepts that greatly enhance the understanding to follow your mindset. I hope I can learn more knowledge from you and apply them into my life goal some day.

[@BooleanDisorder]

I love that this research area develops fast enough that we need dedicated channels to explain new developments.

[@franciscote-lortie8680]

Incredibly clear explanations, the flow of the video is also really smooth. It’s almost like you’re telling a story. Please keep making content!!

[@mohamedalansary2542]

Clearly explained and very valuable content as always Umar. Thank you!

[@ChadieRahimian]

Thanks for the amazing explanation!

[@manumaminta6131]

Your videos help me (a grad student) really understand difficult, often abstract concepts. Thank you so much... I'll always support your stuff!

[@xl0xl0xl0]

Wow this was a super clear an on-point explanation. Thank you, Umar.

[@odysy5179]

Fantastic explanation!

[@AlpcanAras]

This is life changing, in my opinion. Thank you for the efforts on the videos!

[@luigigiordanoorsini5980]

Ho appena letto la piccola bio del tuo canale, spero di non essere offensivo dicendo che adesso capisco perché il tuo ottimo inglese mi sembrasse comunque molto familiare. Ad ogni modo ti ringrazio enormemente per il tuo contributo hai spiegato tutta la teoria in un modo, a mio avviso, estremamente chiaro e soprattutto coinvolgente. Ti prego continua così, di nuovo un enorme grazie e complimenti per il tuo contributo alla scienza

[@umarjamilai]

Grazie a te per aver visitato il mio canale! Spero di pubblicare più spesso, anche se per fare contenuti di qualità ci vogliono settimane di studio e preparazione. In ogni caso, spero di rivederti presto! Buon weekend

[@luigigiordanoorsini5980]

@@umarjamilai Avevi già guadagnato un iscritto adesso hai guadagnato un fan. Ahahahahah

[@bensimonjoules4402]

Amazing content, thanks! I'm very excited about the continual learning properties of these networks.

[@MirjanOffice]

Hello Umar, this video is my best birthday gift I have ever received, thanks a lot :)

[@JONK4635]

Extremely clear explanation and content here! Very helpful. I am happy that you came from PoliMI as well :) keep it up!

[@Adityagupta-vk9um]

i don't comment on YT but man oh man, this man is love. Too good of an explanation.

[@seelowst]

Having a such good teacher is so adorable, i wish i could be your students.

[@umarjamilai]

哪里哪里啊，谢谢你的赞成！

[@seelowst]

@@umarjamilai 太棒了，您还会中文👍

[@umarjamilai]

@@seelowst 我就是刚刚从中国来的，在中国主了4年了，现在回欧洲了。

[@seelowst]

@@umarjamilai 我从没离开过我的城市，我希望像您一样👍

[@stacks_7060]

One of the best math videos I’ve watched on KZbin

[@brandonheaton6197]

Best explanations of splines i have seen. Legit 100%

[@zaevi6855]

crazy that it took me an hr video to understand that its the (control points) being trained on the spline graph vs weights with MLPs and CNNs, thank you!

[@johanvandermerwe7687]

I saw this paper on papers with code, and thought to myself I wonder if Umar Jamil will cover this. Thanks for your effort and videos!

[@MuhammadrizoMarufjonov-os5fv]

Thanks for including prerequisites

[@anirudh514]

Thanks for the crystal clear explaination!!

[@balachanders6350]

Great explanation and underrated also waiting for "Implementation of KAN from scratch" video

[@cavidanabdullayev4533]

It is a amazing resource for KANs. Thank you so much 🙂

[@andreanegreanu8750]

Very clear, well explained, top notch!

[@emiyake]

Thanks!

[@vanerk_]

Great, as always, thank you sir!

[@alfredmanto5487]

Thanks

[@MuhammadMuzzamil-ki4he]

Thank you for such great and detailed explanation.

[@MasoudAminzadeh]

It was fantastic. continue my friend.

[@harveyp.1949]

Awesome explanation!!!

[@enricovompa1876]

Thank you for making this video!

[@mychan-lu5iv]

Amazing! Thank you very much for this.

[@anmolmittal9]

This is really great! Power to you!!🚀

[@ansonlau7040]

Thankyou Jamil, what a cool video

[@howardmeng256]

Amazing video! Thanks a lot !

[@lethnisoff]

Your explanations are the best, thank you so much😘🤗

[@ScottzPlaylists]

High quality explanations.. Thanks.

[@paolobarbieri7483]

Thank you for what you do, you are amazing.

[@kmalhotra3096]

Hats off, what an awesome video!!!

[@artaasadi9497]

that is very useful, informative and interesting! Thanks a lot!

[@ozgunsungar9370]

awesome, easy to follow even person dont know anything :)

[@vaadewoyin]

Cant wait to watch this, saved! Will comment again when i actually watch it..😅

[@DiegoSilva-dv9uf]

Valeu!

[@filippobargagna]

Thank you so so much for this amazing content.

[@ezl100]

thanks Umar. Very nice explanation. Just 2 questions : 1 - Does it mean we can specify different knots per edge? 2 - I am not understanding how the backpropagation will work. Let's say we calculate the gradient from h1. It will update phi 1,1 and phi 1,2 but how the learning process will impact the knots to the desired value?

[@jeunjetta]

I think KAN will be the catalist of a significant tipping point in science. I want to apply this to power system grids and replace existing dynamic models with ones made from PMU data using KAN

[@zzduo-w2p]

Thank you for your excellent explainations 🤩🤩🤩🤩

[@binfos7434]

Amazing! Just wanted to ask if I should expect an implementation of this concept on this channel?

[@GUANGYUANPIAO]

awesome explanation

[@prathamshah2058]

Thank-you so much for explaining the paper, it is so easy to understand now, btw can you also make a hands on video with the kan package developed by mit which is based off pytorch.

[@hajaani6417]

You’re fantastic, mate.

[@samadeepsengupta]

Great Content !!

[@subhamkundu5043]

Hey @Umar, great content as always. Looking forward to a KAN implementation video from scratch. Also I think in 31:01 there is a minor language mistake. I think it will be for using a quadratic Bspline curve rather than quadratic Bezier curve

[@Lilina3456]

You are amazing, thank you!

[@wolfie6175]

Good video, quality content.

[@arupsankarroy8722]

Sir, you are great..💙💙

[@AD-zj7ck]

Thanks for the amazing video. Can you make video explaining and proving the universal approximation theorem?

[@ashithen1833]

Much Thanks for this video

[@RiteshBhalerao-wn9eo]

Amazingg explanation !

[@imanghotbi4651]

Is the explicit form of the obtained functions accessible after training the model and performing L-1 regularization? Is there a repository and code for it already?

[@RomanLi-y9c]

This is awesome!

[@coolkaran1234]

You are savior, without you mortals like me would be lost in the darkness!!!

[@sergiorego6321]

Phenomenal! Thank you :)

[@mohamedessam3154]

Thanks for the video. For the first feature x0,1 we have 5 features for the same input x0,1 how the output is going to be different although they used the same input, grid size, degree and knot vector?

[@yuningliu6300]

at 2:21 you mentioned the documentation. where can I find it ?

[@karanjakhar]

Nice explanation. Thank you. Please can you implement it also like you did for other videos.

[@pabloe1802]

An implementation video will be awesome

[@JuliusSmith]

Excellent video, thanks! At the end, I _really_ wanted to see an illustration of the relatively "non-local" adaptation of MLP weights. Can that be found somewhere?

[@danielegiunchi9741]

brilliant video!

[@willpattie581]

One thing I didn’t catch: how are the functions tuned? If each function consists of points in space and we move around the points to move the B spline, how do we decide to move the points? Doesn’t seem like backprop would work in the same way.

[@umarjamilai]

The same way we move weights for MLPs: we calculate the gradient of the loss function w.r.t the parameters of these learnable functions and change them in the opposite direction of the gradient. This is how you reduce the loss. We are still doing backpropagation, so nothing changed on that front compared to MLPs.

[@dhackmt]

i loved it sir .

[@ntej7927]

Excellent.

[@p4ros960]

bruh so good. Keep it up!

[@faiqkhan7545]

Umar bhai you the great

[@girandoconandrea]

Ciao Umar. Innanzitutto grazie mille del tuo lavoro, sei una fonte di conoscenza infinita per come esponi gli argomenti. Ho seguito interamente questo video ed ho dei dubbi. All'inizio, quando introduci le b-splines si parla di control point in quanto punti che vengono dati come input e per i quali viene creata una curva che passa vicina ad essi secondo la base function. Successivamente, quando viene introdotto il network, si dice che ad essere trainate sono le funzioni ed in particolare i control points. Cosa vuol dire questo? I control points non sono gli input che diamo al modello e quindi i nostri dati che vogliamo approssimare ad una funzione? Sarei grato se mi chiarissi questo concetto. Grazie mille e buon lavoro :)

[@umarjamilai]

L'unico parametro che definisci è il numero di control point (che ne determina la granularità, ovvero quanto "precisa" deve essere l'interpolazione). Compito di una rete neurale è "apprendere" i parametri di una funzione complessa per ridurre una funzione di costo (loss function). Quali sono i parametri che si allenano? La posizione dei control point, non il loro numero, che invece è deciso a priori. È come quando cerchi di interpolare dei punti usando un polinomio: prima scegli il grado del polinomio (quante potenze della X), poi usando un qualche algoritmo "alleni" i coefficienti di ciascuna potenza. Spero ora sia più chiaro

[@shubhamrandive7684]

Great explanation. What app do you use to create slides ?

[@umarjamilai]

PowerPoint + a lot a lot a lot a lot a lot of patience.

[@satviknaren9681]

Please do post more ! please do more videos !

[@グワ氏]

There are continuous but indiferable points in the spline, right? What are you going to do?

[@daleanfer7449]

刚好期盼这个!

[@umarjamilai]

期待你的评价😇

[@daleanfer7449]

❤很好的内容，有考虑做inverse rl的内容吗❤

[@Kishan31468]

Thanks man. Next xLSTM please.

[@fatemeshams9758]

awesome👍

[@routerfordium]

Thank you for the great video! Can you (or anyone) help understand why you need to introduce the basis functions b(x) in the residual activation functions?

[@ikramaharchi1042]

thank You so much

[@plutophy1242]

this video is so amazing！！！！！！！

[@fouziaanjums6475]

Hi, can you please make a video on multimodal LLMs, fine tuning it for custom dataset...

[@umarjamilai]

Check my latest video!

[@akramsalim9706]

awesome bro.

[@AkhoNdlodaka]

THANK YOU

[@rohitjindal124]

Sir I have been a huge fan of your videos and have watched all of them . I am currently in my second year BTech and really passionate about learning ml sir if possible can work under you I don’t want any certificate or anything just want to see observe and learn

[@RudraPratapDhara]

Could you please next explain multi modal llms, techniques like Llava, llava plus, llava next?

[@Patrick-wn6uj]

I waiting for that day too

[@umarjamilai]

Check my latest video!

[@RudraPratapDhara]

@@umarjamilai Yeah checking out, your are as usual the G.O.A.T

[@christopherc168]

But what about wavelt Kolmogorov Arnold networks ?

[@ai__76]

amazing

[@bzzzzz1736]

thank you

[@MrAloha]

Wow! 🙏