What Linear Algebra Is - Topic 1 of Machine Learning Foundations

In this first video of my Machine Learning Foundations series, I introduce the basics of Linear Algebra and how Linear Algebra relates to Machine Learning, as well as providing a brief lesson on the origins and applications of modern algebra.
There are eight subjects covered comprehensively in the ML Foundations series and this video is from the first subject, "Intro to Linear Algebra". More detail about the series and all of the associated open-source code is available at github.com/jonkrohn/ML-foundations
The next video in the series is here: • Plotting a System of L...
The playlist for the entire series is here: • Linear Algebra for Mac...
This course is a distillation of my decade-long experience working as a machine learning and deep learning scientist, including lecturing at New York University and Columbia University, and offering my deep learning curriculum at the New York City Data Science Academy. Information about my other courses and content is at jonkrohn.com
Dr. Jon Krohn is Chief Data Scientist at untapt, and the #1 Bestselling author of Deep Learning Illustrated, an interactive introduction to artificial neural networks. To keep up with the latest from Jon, sign up for his newsletter at jonkrohn.com, follow him on Twitter @JonKrohnLearns, and on LinkedIn at linkedin.com/in/jonkrohn

[@purpledragon9413]

Thank you so much for taking the time to make these clear and easy to understand videos. I was feeling hopeless with my ML courses but your videos brought light to my life!

[@JonKrohnLearns]

Thank you for your kind words! I'm thrilled to hear the videos are helpful in your ML journey. Keep pushing forward! - Jon

[@CloudNativeJanitor]

you have a new follower, thank you for keeping them short and informative

[@DatA_Sc1enc3]

i loved algebra more because you made it easy for us to understand. thank you

[@VanceRex]

Thanks for including the Origins of Algebra in this lesson. It was a nice interlude and is trivia gold.

[@mitchellthompson9642]

Direct ties to machine learning and deep learning highly appreciated!

[@JonKrohnLearns]

You bet, Mitchell! Having now drafted 87% of the content for the 30+ hours of videos in this ML Foundations series, I can confidently assure you that you will find ML and DL ties heavily throughout the entire series :)

[@JS-pc9ls]

Excellent quality video, looking forward to the next one!

[@JonKrohnLearns]

Next three videos will be published tomorrow!

[@itsothie1196]

I'm a software engineer who wants to be a true data scientist(not a mediocre) so I want to do first things first. I have tried to consume more math content for ML but mostly all of it was un-understandable until I found this one. Love the way you break things down. I pray I don't get confused in future videos

[@Mr.MadStick]

how is going for now? I just started this course too

[@JonKrohnLearns]

I appreciate your kind words, Othie! Transitioning from software engineering to data science can indeed require a deeper dive into certain mathematical concepts. My goal is always to make complex ideas as intuitive and hands-on as possible. Stick with it, and don’t hesitate to reach out if you ever find yourself feeling stuck or confused. Here’s to your journey into data science!

[@itsothie1196]

@@Mr.MadStick I am following and enjoying the stuff. I've learned though that u have to chronologically follow the videos. The moment u skip things then it starts to confuse

[@itsothie1196]

@@JonKrohnLearns Thank u for the generousity. I'm working on a project but I still have alot of questions. Pray I get a chance to be mentored by people like you

[@Rochak.kahaniya3]

​@@itsothie1196 hey can you list videos you watches previously so i can avoid them

[@rajasekharp1176]

Thanks a lot for sharing this video. Your videos helped me to understand Algebra concepts, as a beginner. Appreciate your efforts! Now I am a subscriber ( student ) of your channel.

[@vatsal_gamit]

This is what I was looking for!! Keep doing your amazing work!! Thank You so much :)

[@JonKrohnLearns]

Yay! So glad you found it, Vatsal! More new content to come soon :)

[@chandankushwaha8932]

wow its amazing. you explained the whole algebra so easily . I am having too much difficulty in understanding mathematics for machine learning from other sources. But after finding your series its like a cakewalk for me. Thank you so much for creating this series. would like to meet you someday

[@JonKrohnLearns]

Awesome, Chandan! So great to hear that I've been able to make learning math for ML a cakewalk -- makes my day to hear that! Yes, would be great to meet in person. I speak regularly at conferences, which are an ideal opportunity to meet. You can sign up for my email newsletter on jonkrohn.com to be sure not to miss an upcoming live appearance!

[@chandankushwaha8932]

@@JonKrohnLearns thank you

[@mucool328]

Thank you 😊 🙏🏻 Jon. Looking at the Linear Algebra equations reminded me of my Undergrad Math courses. I wish someone back then related these equations more towards real life examples(like you did) rather than letting us Mug up the formulas and passing the courses. I liked the simple and straightforward Cop chasing a robber example and also Formula breakdown with House pricing example. Although i was not able to comprehend 100% of your video. I think i still got about 70-80%, i guess have some homework to do ;). I agree that we need to look at history behind certain Topics, but i also wish if there was some content out there that explained the motivation behind the creation of these formulas and break them down step by step. I know these video is not done with that intention, but i still thought to share my thoughts.

[@JonKrohnLearns]

This is perfect, Mukul. Some of the content in this first video was a little tricky -- a teaser of what's to come in the forthcoming videos, which should break everything down in detail!

[@rajeshpatil4756]

Awesome video! Thanks Dr. Krohn.

[@JonKrohnLearns]

You're most welcome, Rajesh :D

[@yahya89able]

Uploading your precious vids on youtube is a great idea thanks a lot

[@JonKrohnLearns]

You bet! I hope you'll be happy to hear that I published the next three videos in the ML Foundations series today: kzbin.info/aero/PLRDl2inPrWQW1QSWhBU0ki-jq_uElkh2a

[@fahadreda3060]

Great video Mr.Jon , Thanks for sharing your knowledge with us , really loved the video

[@JonKrohnLearns]

So glad to hear it, Fahad, and you're most welcome! A few minutes ago, I published three new videos in the series: kzbin.info/aero/PLRDl2inPrWQW1QSWhBU0ki-jq_uElkh2a

[@elp09bm1]

Nice to know. you found this series one year earlier than me. Did manage to complete it. your thoughts how to stay motivated to complete the course, if you have done so. Any collaboration is welcome. I have Ph.D. from Sheffield University, UK. I am a learner and teacher of Machine Learning.

[@prasadnayak9075]

Great Video. Superb Knowledge. Thanks Mr. Jon Krohn for sharing your knowledge😊

[@JonKrohnLearns]

My pleasure!

[@godlyradmehr2004]

I'm really thankful about this tutorial, and honestly i was excited When you mentioned Khwarizmi, because im Persian.❤❤❤

[@ahsanzafar4921]

Amazing content, finished calculus yesterday started algebra today, Really amazing. Surely, took him a long time to make these videos and we are fnishing it in just few days. Hope Probability winds up before I finish algebra😂. Thank you Jon for such an amazing and easily comprehendable approach

[@JonKrohnLearns]

Congrats on finishing the Calculus course, Ahsan, and thank you for the encouraging words! I'm so excited to get started on recording and releasing more videos from the Probability course :)

[@DataScientist268]

Thank you sir, this is very much appreciated ❤.

[@alpeshbharodiya9677]

i loved it its to easy the way you explain it

[@mucool328]

Also I am looking forward to more of these lectures Dr. Krohn , No Pressure 😎🙏🏻.

[@JonKrohnLearns]

Tomorrow :)

[@KervinVasquez]

Amazing explanation!

[@tankingblitz]

Just what I was looking for

[@shaiknayeem3277]

Excellent course . Well understood ❤️🥰

[@My_lazy_Perspective]

I really appreciate your work...

[@nguyenngocly1484]

Don't forget ReLU is a switch. The electricty in your house is an AC sine wave. Turn on a light and the output of the switch is f(x)=x. The same sine wave as the input. Off f(x)=0. ReLU is a switch with a binary predicate deciding if it is on or off. A ReLU neural network is a switched composition of dot products. If the switch states are known then there exists a linear mapping from the input vector to the output vector.

[@heydyvex]

john thank you for being so good at teaching :))

[@JonKrohnLearns]

You're most welcome, Amir :)

[@pinklady7184]

This is interesting. I have just subscribed. Thanks for tutorials.

[@JonKrohnLearns]

You're welcome! I'm recording more content presently and am planning to release another batch by the end of the year :)

[@millennial_post]

Hi Jon, Thank you for the explanation. I have a doubt here, in this eq y=a+bx1+cx2+.......+mxm, Why 'a' is only considered as avg. house price value or can we take the mode value or it is only the avg value we should take during the model building.?

[@netanelkaye3014]

Is there a resource for doing problems so that we can solidify this info? maybe a textbook or a pdf?

[@alunicat]

what an incredible video

[@palakkapahi3272]

Hi sir, just wanted to ask that do you learn python commands for deep learning and machine learning models, or use it directly from the documentations and amend them as per your requirement?

[@openyard]

The graph at 4:35 could have been more comprehensible if the x and y axis were ticked and marked (numbered), then the determination of the slope would have been explained by simply showing change in y over change in x. I am thankful for your series on ML foundations, this seems to be by far the best training course on ML that I have come across after years of searching.

[@joelausten]

do you think that these 8 subjects are very important for machine learning beginners

[@helpplz6021]

Great video Mr.Jon But I a facing complexity regarding the curve where the curve of Robber should go 5 mintues ahead as you mentioned that graph 5 mins before Could you please make it clear?

[@padmakatkozu9]

Can anyone explain tensor flow example, I didn't understood. From 13:14 sec

[@B2RNinfoshorts]

Thanks for this ml math love you so much ❤️❤

[@ngkaruku]

I was brought in math using a cane however you sir have made an appealing sense to math i like it

[@JonKrohnLearns]

Thank you, Robert! Encouraging to hear this :)

[@nguyenngocly1484]

Don't forget you can put the cart before the horse. And use fixed dot products (weighted sums) and adjustable activation functions to make neural networks. In fact it is not clear who is putting the horse before the cart and who is putting the cart before the horse😲 since fixed dot products can be done quickly with fast transforms. Eg. Fast Transform (fixed filter bank) neural networks.

[@AbhigyanDey-nn2wu]

In 9:55, you talked about the 'a' variable. I didnt get that part.

[@harjeet932]

Hi, In the topic1 of Machine learning foundations - What Linear Algebra is, when you had shown the example2 , where we have to find out the values of multiple unknowns, and then the tensor flow code in jupyter notebook, i felt lost for a couple of minutes in this video. My question is that, do you think I should continue to watch this series further? Or before continuing in this series, I should probably first learn more about tensor flow library? Thanks,

[@ehsansaesh]

Big Love and respect from Iran (Persia)

[@ZaidShery]

{ Intro to Linear Algebra Algebra can be used to solve systems of linear equations Solving a system of linear equations to find time and distance. The more features, the more accurately you can predict house prices. Linear algebra can be used to represent high dimensional convolutional objects in machine vision models. Algebra has a long history, dating back to ancient civilizations. Linear algebra is foundational in machine learning and other fields. like deep learning } thanks you

[@jamalkhaled-fitvampire8129]

can we get the course slides?

[@faqeerhasnain]

Thank You for this

[@JonKrohnLearns]

You're most welcome, Faqeer :)

[@bilalhassankhan221]

I really looking for this content...

[@JonKrohnLearns]

I'm glad you found it, Bilal! Hope you enjoy it :)

[@Informalschool]

Thank you Sir

[@mdyounusahamed6668]

Enjoyed it.

[@JonKrohnLearns]

Super :D

[@nightskymusic5163]

Jon sir love from India keep going

[@JonKrohnLearns]

You bet, sir! Will be releasing new videos as soon as I can :)

[@joelausten]

​ @JonKrohnLearns Hello, I just wanted to start a journey from 0 to an AI machine learning engineer, could you give the best course that I could take?

[@Krishna_5143]

tnk sirr

[@nguyenngocly1484]

Don't forget the variance equation for linear combinations of random variables applies to the dot product in most amusing ways.

[@tomjordan3686]

Great video Jon. Thanks for creating it and making it available. One question though. Shouldn't the equations in the sheriff problem be these? Eq 1: d = 2.5 (t + 5) Eq 2: d = 3 t Solving for t = 25 min Check of the answer: At the 5 minute point the sheriff is 12.5 km behind (2.5 km/min * 5 min). At a closure rate of 0.5 km/min, he will catch in 12.5/0.5 = 25 minutes.

[@JonKrohnLearns]

Glad you enjoyed the video, Tom :) I haven't had a moment to work back through the math in detail. At a glance, however: Do your equations provide the time elapsed from when the sheriff starts moving (25 minutes) whereas mine provide the time elapsed from when the bank robber starts moving (30 minutes)?

[@tomjordan3686]

​@@JonKrohnLearns Valid point. The question was "How long does it take the sheriff to catch the robber?". So, I assumed that meant from the time the sheriff starts moving (i.e., after the "5-minute head start"). Interestingly, both produce the same answer to the next question "What distance will they have traveled at that point?". Mine: d = 3t = 3 * 25 = 75 km (where t is from when the sheriff starts moving) Your's: d = 2.5t = 2.5 * 30 = 75 km (where t is from when the robber starts moving)

[@tomjordan3686]

@@JonKrohnLearns Good point. Based on the original question "How long does it take the sheriff to catch the robber?", I assumed it was the time from when the sheriff started moving (after the "5-minute head start"). Interestingly, both produce the same answer to the next question, "What distance will they have traveled at that point?" Mine: d = 3t = 3 * 25 = 75 km (where t is from when sheriff starts moving) Your's" d = 2.5t = 2.5 * 30 = 75 km (where t is from when robber starts moving)

[@JonKrohnLearns]

@@tomjordan3686 Nice yeah. That all makes sense. Whether we start the timer when the bank robber starts moving or the sheriff starts moving, the sheriff will intercept the robber at the same spot on the road. I suppose when I was asking myself "How long does it take the sheriff to catch the robber?" I was imagining starting the timer when the robber starts driving away from the bank but I can see where you're coming from. I'll ask the question less ambiguously when I put it in my forthcoming book so thank you for the feedback :)

[@raymondkyruana118]

I was so confused at first haha but then I assumed both answers could be right depending on the framing of the question haha glad someone else thought the same!!!!

[@Gamer-rs9mf]

I don't have computer or laptop but I m learning graphic and web design from an institute I want to increase my skill by the knowledge of mi maths so I can think about it and presue it so plz tell is there any need of laptop or computer to complete it because now I can't afford it but after 7 8 months I can so plz tell me do I need laptop to complete it

[@AbhigyanDey-nn2wu]

Can you share the slides?

[@KylianMbappe07303]

5:12 how come sherrifs point is in 5min, shouldnt it be the robber

[@sohambhadra7939]

Hello I am a Post Graduate in Robotics Engineer and a freshman. I am trying to enter into the field of DS and ML and crack interviews for some top tier organisations. I have approximately 8-9 months in hand for proper preparation. Please guide me what and how should I achieve my goal

[@Minecrafterr]

I am starting with machine learning as a hobbyist and this is the first course, after that I would take the calculus and Probability. Thanks sir for such informative content. Love from India 💛 I have a question I don't have much knowledge about calculus but is it very important for me to learn it before moving into the Machine Learning part.

[@JonKrohnLearns]

Hi there, glad to be helping you get started on your journey! I think you'll find my "Calculus for ML" playlist to be a great place to begin your ML journey because it quickly moves from the most essential calculus concepts to applying it so that you can understand how ML works in practice.

[@Minecrafterr]

@@JonKrohnLearns Thanks sir for clarification of my doubt

[@ishaanms9194]

Every well explained sir... Before learning this topic, is it mandatory to know python programming

[@JonKrohnLearns]

Nice! Glad you enjoyed this Ishaan :) Yes! Python programming is a prerequisite. All of them are provided here: github.com/jonkrohn/ML-foundations#prerequisites

[@anasrabe6799]

could you provide the slides, please ?

[@JonKrohnLearns]

Yep, Anas! You can find them by looking for my linear algebra lectures on jonkrohn.com/talks

[@jeevamchivate4473]

i didnt understand the use of the y-intercept

[@elp09bm1]

Hi Jon, Thanks for this excellent Series. I wish I should have found it in 2009 when I started my PhD but anyway that is how life is. Is it possible to get your slides so that I could teach my students from the same slide. Yes it goes without saying with due credit to Dr. Jon Krohn.

[@JonKrohnLearns]

Sure! Please send me an email from your academic email address. My contact details are on jonkrohn.com/contact

[@elp09bm1]

@@JonKrohnLearns Thanks for your response. I have tweeted you my academic email id.

[@elp09bm1]

@@JonKrohnLearns Sorry I couldn't find your email address there but only twitter and linked contact unless missing something. Thanks

[@JonKrohnLearns]

@@elp09bm1 Email address is definitely there!

[@elp09bm1]

@@JonKrohnLearns Thanks got it, I sent you mail from my academic address. I hope I could pick it at 52 years of age. I could be your oldest student :-)

[@rafiqamar9606]

Jon Krohn, it takes the sheriff 25 minutes to catch the bank robber, but you calculated 30 minutes.

[@joelausten]

yep I also calculated 25 minutes that makes me confused

[@joelausten]

Oh i finally got the concept, the sheriff catch duration is 25 minutes because the sheriff is 5 minutes late. The robber has a duration of 30 minutes, since the robber started running away first, and was 5 minutes head start.

[@parinitapatil993]

hello sir , I have started learning machine learning is this playlist enough ?

[@JonKrohnLearns]

Hi Parinita, per my "ML Foundations" GitHub repo README (see here: github.com/jonkrohn/ML-foundations), this playlist is not an introduction to ML. Rather, it is an introduction to the subjects (e.g., linear algebra) that underlie a strong ability to understand ML. At the end of my "ML Foundations" curriculum, I leave you with resources to dig further into the ML topics that interest you most.

[@karanrathod5690]

Sir.. please share notes.. PDF..😊

[@Ali-2812]

Hello, is it possible for you to share slides?

[@JonKrohnLearns]

You bet, Shujat! You can find them on jonkrohn.com/talks. Search the page for "Linear Algebra".

[@Ali-2812]

@@JonKrohnLearns You're awesome

[@JonKrohnLearns]

@@Ali-2812 awwww, thanks, Shujat! Hope you keep enjoying the videos :)

[@AlwaysSlise]

Isnt algerba also geometric?

[@theyashjani]

i want to become ML engineer can i refer this playlist. and please guide me how to become a ML engineer

[@JonKrohnLearns]

Yep! Learning the math covered in this playlist is a critical step to becoming an ML Engineer for sure.

[@NavelsUrith]

I am new to machine learning i really don't know what kind of stuff i have to learn can anyone help me where to start it and what i have to learn i have no idea about it what programming language do i have to learn i am good in java .Mr jon krohn or anyone can help me out in this.

[@JonKrohnLearns]

Hi there! For ML, you pretty much need to learn Python. My "Linear Algebra for ML" playlist covers some of the fundamentals. For Python concepts that don't immediately make sense to you, you can copy/paste them into ChatGPT (I suggest using the GPT-4 model) and it will automatically fill in the gaps for you.

[@NavelsUrith]

@@JonKrohnLearns Really appreciate your help Mr. Jon krohn

[@JonKrohnLearns]

@@NavelsUrith anytime!

[@firstflight1]

Sir can a 10 class student do this course

[@JonKrohnLearns]

Hi there! What is a "10 class student"? I haven't heard that term before.

[@Mr.insane6992]

i am 14 year old and learning ML, for Mechatronics is it worth to make and train my own sensors?

[@joelausten]

Helo do you think that these 8 subjects are very important for machine learning?

[@joelausten]

Im a beginner and I want to lear about machine learning to be a machine learning engineer

[@My_lazy_Perspective]

hey jon, This is Prathap. your first example about sharif and robber, there is miscalculation. the time taken to reach Robber is 25 min and distance traveled is same 75 km. explanation is 1. at start sharif is at 0 km with 0 min and robber is 12.5 km away 2. speed/min is 3 for sharif and 2.5 for robber. 3X25=75 km, 2.5X25=62.5 km but robber left 5 min early so at 0 th time for sharif, robber is 12.5 km away 12.5+62.5 = 75 km. sharif took 25 min only. correct me if i am wrong

[@raymondkyruana118]

Bro I got the same time of 25 mins as well! IDK how but somehow I think both are correct??? His assumption is that the cop is starting with a -5 to the cops t value whereas I did my math with a +5 to the robbers t value. Same distance but different times I guess. IDK why that confuses me but it does

[@raymondkyruana118]

Oh wait he explained it in another question. if you search for the number 25 on the comments section you'll find it but basically it's because Jon is solving for t when the robber starts moving, but you and I solving for when the cop starts moving and then giving the robber the extra 5 mins (12.5km distance) at the end. They are both correct depending on how you interpreted the question haha

[@My_lazy_Perspective]

@@raymondkyruana118 ya that's was correct from Robber perspective. Robbery done at 30min back.

[@dhruvraj9548]

I don't know much about python, should I watch it??

[@JonKrohnLearns]

Great question. We start off by explaining the fundamental ML-relevant concepts using fairly simple Python but then it does start to get more complex quickly. One option would be to learn the necessary Python while you go, e.g., by consulting ChatGPT whenever you don't understand something (the GPT-4 algorithm would be particularly adept at this).

[@dhruvraj9548]

@@JonKrohnLearns Thank you

[@mohamedbahgat4883]

where this slide !!

[@JonKrohnLearns]

Hi Mohamed! Head to jonkrohn.com/talks and search for "linear algebra" on the page

[@warriorsimplicity1]

Wait.... That was just an intro to the course?

[@JonKrohnLearns]

Yep, buckle up!! If you haven't found it already, my entire Linear Algebra course is in this playlist: kzbin.info/aero/PLRDl2inPrWQW1QSWhBU0ki-jq_uElkh2a

[@rockit9913]

13 September

[@nguyenngocly1484]

Don't forget that you can evolve neural networks with sparse mutations. That's okay because dot products are statistical objects, summary measures. If a neuron over-expresses itself to improve the output of one forward dot product it will cause many other forward dot products to have increased error. The over-expression will be fought back during training. Hence you are not exploring the full vector function composition space, only a far simpler subspace of statistical solutions that are accessible to weak optimisers. How weak is a good question. BP is very weak if you ask me. Anyway if you have multiple GPUs you put the full neural model and part of the training data on each. Send each GPU the same short list of mutations. Each GPU sends back the cost of its part of the training data. Then each GPU is sent the same accept or reject the mutations packet. Very little data has to move around during training. The type of mutation is quite important. You can try to find the Continuous Gray Code Optimization pdf around the town👓

[@pinklady7184]

You must be an interesting teacher who makes pupils' brains swell within a short time.

[@nguyenngocly1484]

@@pinklady7184 'K.

[@shobhitgrhwl]

this guy gives me the vibe he is a program teaching us programing. He gives mark zuckerberg vibes.

[@rayanebradjou4396]

I'm new to linear algebra for ml so I want just to know do c is good for ml or do I should use c or cpp or python and why ❤

[@thegrind9628]

python anyday man..cuz inbuilt libraries

[@darkphinkwastaken3362]

is this going to teach me linear Algebra?

[@JonKrohnLearns]

100%!

[@darkphinkwastaken3362]

@@JonKrohnLearns do you need to know calculus for this course?

[@JonKrohnLearns]

@@darkphinkwastaken3362 nope! Calculus is the follow-up course, intended for after Linear Algebra. It's available here in case you're curious for later: kzbin.info/aero/PLRDl2inPrWQVu2OvnTvtkRpJ-wz-URMJx

[@pesgamer00]

Is this playlist enough for linear algebra for machine learning. I'm tired of watching KZbin 😢 Please reply..........

[@sumansaurabh7772]

Your teeth is too perfect to real

[@JonKrohnLearns]

hahaha thank you! Sometimes I think about getting veneers so feedback like this is helpful to steer me away from that invasive, permanent dental work :D

[@jefflokanata]

I see puppy , I click .