How to Read Math in Deep Learning Paper?

[@Dom-zy1qy]

Honesty just work through a math book, you will be able to read math in the ML domain pretty easy. I read "Mathematics for Machine Learning", it was a struggle for me, but taught me a lot of very useful skills.

[@deeplearningexplained]

Nice, will give it a read thanks for the recommendation!

[@amunif_]

I really wish there were courses in CS master’s degrees, teaching how to decipher the math in AI research papers

[@deeplearningexplained]

Yeah, that would have been helpful haha. However I had a cool course during my PhD in which every week we were reviewing a computational neuroscience paper. It was cool because we were digging into the code + we had a lecture with a similar style I adopted in my channel. It really helped see behind the veil of the papers method section!

[@alikin]

@@deeplearningexplained link?

[@jerahmeelsangil247]

Yeah, in Italy by the completion of bachelor degree in Computer Engineering you are in condition to read, understand and in some cases write papers atleast published in ARXIV

[@IngramSnake]

@@jerahmeelsangil247that’s cool but do they actually teach you how to do each of those things before requiring you to understand them? I feel this is much of the problem: there’s an element of show-off that keeps things closed off. 😂 this video is amazing ❤

[@dylanhafner7608]

You shouldn’t be trying to “decipher” it…. You should actually know math

[@Samuel19356]

Saw a random screenshot about this video on twitter, so glad I came to watch, thanks for the insights!

[@deeplearningexplained]

Glad it was useful Samuel!

[@maahipatel5827]

⁰⁰0​@@deeplearningexplained

[@Antoniothescientist]

As an AI/ML practitioner with no proper math education, I find this video very helpful for understanding the complexity of the algorithms and ensuring proper implementation for my use case. Would love to see more content like this!

[@deeplearningexplained]

Glad it was helpful, do let me know if you have topic requests!

[@AlexsandroPessoa-m9x]

Sincerely, after taking Calculus 3, 4, and Numerical in college, it feels like a trauma that will last for the rest of my life. Every time I use a gradient, I remember having to calculate it using only my paper.

[@deeplearningexplained]

Haha, at least you have a deep understanding of the material which is essential to understand topics that build upon it. Worth the PTSD!

[@sepro5135]

I dont do CS or ML, but am getting into advanced Lattice Bolzmann stuff for Fluid Simulation, where a lot more basic understanding Is required than here (obviously, as the video is intended for a wider audience) but it was great to see the steps I usually take being actually formalized and first reasoning myself through this particular problem, before watching you do it, which worked pretty good. Overall great video, especially for ‚beginners‘

[@massithegame8753]

Very good explanation! You’re the Jon Snow of mathematics.😅

[@deeplearningexplained]

Haha thanks for the kind words!

[@dub161]

In IISc, any student from any dept. are not allowed to touch any course from Intelligent Systems pool/AI dept. unless you are done with Linear Algebra, Stochastic Models/Random Process and Optimization and Analysis course. No AI for you unless you are cracked in Math.

[@deeplearningexplained]

Those are great pre-requisite for undergraduate or graduate AI courses 👍

[@Chadpritai]

are you at IISc dub161?

[@rahulmalik1083]

Thanks to my exposure to advanced macroecon, the formulas don't seem crazy.

[@deeplearningexplained]

The more exposed to math you are the easier it gets for sure!

[@eldarzakirov5571]

As an alternative to paper, I may suggest to use eink tablet with screen 13.3" and with drawing support, e.g. Boox Max Lumi, which alliws to draw directly on pdf.

[@deeplearningexplained]

Ah good idea, I also heard great things from ReMarkable!

[@seanoconnor1984]

Fast transforms have a matrix equivalent that you can view as a neural network weight matrix. And then have parametric activation functions as the adjustable part. 1000 times faster neural networks.

[@deeplearningexplained]

Cool, thanks for sharing!

[@seanoconnor1984]

@@deeplearningexplained 'K. The fast Walsh Hadamard transform is a good choice.

[@kshitijdesai2402]

Very insightful!

[@deeplearningexplained]

Glad it was! :)

[@rastko614]

This is gold.

[@deeplearningexplained]

Glad it was useful :)!

[@martinsteinmayer3557]

Great video man! Very well explained!

[@deeplearningexplained]

Thanks, glad it was useful! Let me know if you have request for the next tutorial.

[@Fleniken]

1. What's your sketching software that you put screenshots into? 2. How did you know to look in the "Adam" paper for the missing formula, and how did you find it? 3. What papers should I prioritize reading if I want to become a research engineer. And should I try tinker with the papers concepts to try put out my own blogs/mini-papers to demonstrate on workshops / to potential employers?

[@MahmoudSayed-hg8rb]

The SW is microsoft whiteboard Also im interested in his answer to your 3rd question so plz someone @ me if he replies

[@deeplearningexplained]

For sure CC: @MahmoudSayed-hg8rb 1. It's TLDRAW, it's free over here: www.tldraw.com/ 2. I know to look into the Adam paper because they mentioned that phrase in the article "Note that only the last expression differs from vanilla Adam". So I followed the reference for Adam and then in the paper I followed the flow until I hit the algorithm section (as you saw it was pseudo-code, not formulas). 3. My two cents is to start out with the classical architecture or core discovery of the last decade in the specific field you are interested. Read them and reproduce the result gradually. These are great to start out because they already have been implemented in different ways in bigger software package (Pytorch, Tensorflow). So you won't feel too much alone. Once you are getting the hang of it you can start reading and tinker with more recent result. I would suggest setting up a Github Repo for these reproduction and work on them gradually. No need to reproduce 100% of all the result in a paper, but by gradually working through the most important one you will start to get a hang of how the authors were thinking while getting there result. Plus you will have a set of nice project to walkthrough with potential employers!

[@MahmoudSayed-hg8rb]

@@deeplearningexplained Thanks for @ing me and thanks for your answer 🫡🫡

[@MahmoudSayed-hg8rb]

@@deeplearningexplained Thanks for @ing me and thanks for the answer, really appreciated.

[@conradgnad]

that was great, you are legend! thank you so much!

[@deeplearningexplained]

I’m glad it helped!

[@sitrakaforler8696]

woooow was a cool video man ! keep it up

[@deeplearningexplained]

Glad it was useful!

[@lukamoz]

Ok as coding/math enthusiast who is looking into ML, i have understanding of calc 1,2,3, some stats and L.algebra how long it takes you guys to read paper like this (30 pages) from top to bottom? and implement it. I know on KAGGLE there are torunaments and they tend to use reserach papers for solutions.

[@deeplearningexplained]

Implementing the idea behind QHM and QHAdam is very fast, less than 1h. This paper is also very straightforward since it’s well written. Reproducing all the result in this paper though can take more time (to set up the experiments). But generally, reproducing a paper main result can take anywhere from a few hours to a full month depending on the complexity and how much I know about that subfield.

[@aaryanbhagat4852]

great explanation

[@deeplearningexplained]

I'm glad it was useful!

[@leolacic9442]

It's finaly whole explanation. Can I to go on the relax?

[@deeplearningexplained]

Yes you can 👍

[@pratiknapit8494]

what about harder stuff when they talk about data manifolds or proving the convergence of stochastic gradient descent?? That stuff is way too difficult unless you have taken graduate math courses after 4 years of undergrad maths

[@deeplearningexplained]

Great question! What you need to do is start in reverse, go from the intuition and figure out the path from the primitives. A manifold is named like this for a reason that make intuitive sense for people understanding the mechanism behind. What I would do is first look at the path from primitives to result from many people/educators. Then I would make sure I understand what we are starting with. Finally I would take a step by step approach just like we did in this video and go fetch the information I’m lacking externally. Knowing the math well sure help speed the process up, but you can still figure out complex topic like that no matter your level.

[@pratiknapit8494]

@@deeplearningexplained thank you!

[@amirs6472]

great, thanks for video

[@deeplearningexplained]

Glad you enjoyed!

[@shahulrahman2516]

I have completed LA, Probability and ML course. Have not done DL. I want to learn transformers and LLM's to conduct research upon it. Can you give me some directions.

[@deeplearningexplained]

Andrej Karpathy content is absolutely awesome to get started right in transformers and LLM: kzbin.info/www/bejne/sJvOc4B8nbOdlckfeature=shared

[@whoami6866]

You know nothing, Jon Snow... but deep learning?

[@deeplearningexplained]

Hahaha

[@jakeaustria5445]

Thank You❤

[@deeplearningexplained]

You’re welcome!

[@jaidansvevo439]

What application do you use to list the main aspects of the paper?

[@deeplearningexplained]

It's TLDRAW: www.tldraw.com/ ! Great app and it's free!

[@jaidansvevo439]

@@deeplearningexplained Thanks. Appreciate it!

[@fatihmustafatuglu236]

which sketch program is this?

[@deeplearningexplained]

It’s TLDRAW, it’s free and really solid!

[@luka.software]

Do you have any paper recommendations for someone that is just getting started with DL?

[@deeplearningexplained]

What background knowledge do you have and what aspect of deep learning interest you most?

[@luka.software]

@@deeplearningexplained I have a bachelors in software engineering and a bit of experience using SAM. Don’t have a specific interest but the image generation models like MJ are cool to me.

[@deeplearningexplained]

Okay neat, if you have a general interest then I would recommend the Deep Learning book. It’s not a paper per se, but it’s written with a similar flow as research paper and has pretty good references to the literature. It’s accessible though, so start with that and whenever you see a result that catch your attention dive into the base research it reference. This way you get both the benefit of context and with the depth of deep learning research. Hope it helps!

[@luka.software]

@@deeplearningexplained thanks i’ll check it out!

[@sinan_islam]

Reading math notation is a huge obstacle for me.

[@deeplearningexplained]

Did the tips in the video help a bit?

[@sinan_islam]

The video helped alot. Thank you!

[@walterobi535]

What app do you use to split the pdf in sections, and dragging them around. Noob haha

[@deeplearningexplained]

I take screenshot of them and then I paste them in the TLDRAW free app!

[@walterobi535]

@@deeplearningexplained thanks.

[@drprince8766]

Any Udemy course.:D

[@deeplearningexplained]

On which topic?

[@Chadpritai]

nice yapcine

[@deeplearningexplained]

:)

[@sidharthcs2110]

All i saw was some woodoo magic

[@deeplearningexplained]

Haha, which part felt like this?

[@EkShunya]

can u please please setup a discord community? 🙏🏾 if u need help on this lemme know

[@deeplearningexplained]

Hey there, it's already setup over here :) 📌 discord.com/invite/QpkxRbQBpf

[@EkShunya]

@@deeplearningexplained thank you :)

[@achunaryan3418]

Learn deep learning math.

[@deeplearningexplained]

Nothing beat strong fundamentals that’s for sure!

[@cedricharris-v2r]

Coo)l

[@deeplearningexplained]

:)

[@RalphVikchain]

I didn't get anything!

[@deeplearningexplained]

Oh no, where did I lost you?

[@dffhhfhdifh]

simply go learn maths calculus statistics differential and then jump into ai.why you in AI when math statistics is basics of AI.

[@BryWMac]

Well I think most ML practitioners have just a CS degree which typically don’t require beyond calc 3 and Lin alg 1. It’s just the way it is :/

[@kar-s6716]

Wtf was that thumbnail

[@deeplearningexplained]

Did my best haha

[@DuckGia]

Broken maths. Solely to tackle a trivial optimization but lacks fundamental analysis.

[@deeplearningexplained]

Hey, thanks for the feedback! How would you have ran this tutorial differently?

[@imaspacecreature]

No bullshit, I think in this "math" at times, but cannot even begin drawing a formula. 😂

[@deeplearningexplained]

Try to code it instead, it's much easier in my opinion than using formula to start out.

[@imaspacecreature]

@@deeplearningexplained Yes! This is it. I've actually come up with some pretty interesting algorithms. I think in algorithm, it's strange, but yes I prefer to code it, then I can make a formula for what is happening. Perhaps it's kind of like sheet music. Also I'm working on a Neurosymbolic PHP only model, it's doing pretty well so far.

[@redwingbeast1396]

​@@imaspacecreature thinking in algos. please teach me how to!!

[@imaspacecreature]

@@redwingbeast1396 a while ago when I was younger my cousin took me to Kyoto, while there he decided to give me a pop quiz. He asked "In Japan, what is the tallest mountain in Japan?". I immediately knew I had access to it, but couldn't draw the memory in that instant. I in that moment, thought to my self "what if I rearrange my memories and sort by letter?", I did and upon reaching "F", "Mount Fuji" popped up in my mind.