How Support Vector Machines work / How to open a black box

Part of the End-to-End Machine Learning School course library at e2eml.school
A gentle introduction to how support vector machines work. Also, a guided tour through the process of teaching yourself anything.
Slides: github.com/brohrer/public-hosting/raw/master/Brandon-Rohrer-Black-Box-light.pptx
Code for examples: github.com/brohrer/public-hosting/blob/master/svm_kernel_trick.py
Blog and post: brohrer.github.io/open_the_black_box.html
Follow me for announcements and update: _brohrer_
This was a dry run for Metis' Demystify Data Science Conference. Check out the other awesome talks: www.thisismetis.com/demystifying-data-science

[@knuth42]

Easily the best illustration of the Kernel Trick I've ever seen, but the real utility of this video is found in encouraging everyone to become an autodidact -- particularly in an era where the majority of human knowledge is available for free. Well done.

[@BrandonRohrer]

Thank you knuth42. Your encouragement means a great deal to me.

[@sakthigeek2458]

Great work on all your videos!! Keep enlightening the world :)

[@harirao12345]

Why is it called Support Vector Machine?

[@merce9728]

What software do you use for the data visualisation?

[@vishnuvarthan8304]

@@harirao12345 vectors near the decision boundary acts as support vectors so it is called svm

[@paoloangeles]

This is how theories and models should be explained, extremely clear.

[@govou3000]

Einstein said it himself, if you can't explain it to a six year old, you don't understand it yourself

[@balajig2129]

I had gone through multiple videos. But this explanation is the best for SVM. Thanks for sharing

[@oyardimci]

As you said, I watched several videos to understand clearly, and at the end, I hit your video, now I feel so confident about this topic now. Thanks.

[@BrandonRohrer]

I love it! Thanks :)

[@vishnukunchur]

This is easily the single greatest instructional video I have ever seen.

[@sanjivsoni2389]

You'll only know when you open the box. Great stuff Brandon, one of the best explanations that I have ever hear of.

[@3rdman99]

Finally videos about ML explained by somebody whose English I can understand.

[@jasonk334ever]

great explanation of not only how SVM's work at a basic level, but also great insight into the approach you take to learn anything

[@MrMaipeople]

The best kernel trick explained EVER to me. Thank you so much for your effort.

[@michaeldrolet7386]

WOW, amazing explanation. This is just what I was looking for! I can tell you put a lot of effort into making this as digestible as possible

[@yasinilulea]

A youtube video summarising other youtube videos in a plain language, amazing! :)

[@timnoordewier8089]

Thank you for putting this video together! Your explanation was both informative and motivating for an aspiring data scientist!

[@BrandonRohrer]

Thanks for the note Tim! I'm very happy to hear it.

[@garrett6064]

I stumbled upon your channel in much the same way, looking for a good explanation of LSTM. Your explanation was good enough for me to want to see what else I can learn from you. I think I've spent the two hours now learning about BECCA and the robot house. You've done great work and I appreciate you taking the time to share it with us.

[@glowish1993]

Searched for a SVM video, found a video to explain on how to learn better and open a black box well. I see this as an absolute win!

[@RoryFrenn]

I love people who explain things like you.

[@hexa3688]

Great video :) The explanation of the kernel trick was very helpful. I think it would have been even better if you talked a bit more about the math, but most people are maybe only interested in the intuition and methodology introduced in the video, so it makes sense to avoid making things too complicated.

[@moharbab2987]

Very nice video and very educational. I didn't feel the time running while I was watching. Thank you.

[@bluelight9508]

Lots of awesome ML videos with great explanations - clearer than most anywhere I've seen. I hope you keep them coming. :)

[@mayankpj]

Really nice advice Brandon, sometimes emphasizing the process of self learning helps one to "do it the right way" and not rush through things. I feel that one core ethic that one needs to adhere to while self learning is patience. So Thanks for putting in a lot of effort and being patient enough with the difficult concepts that you brought out so beautifully!

[@dipanwitanath8053]

Still the best explanation for Kernel Methods. Thank you for this

[@BrandonRohrer]

Thank you :)

[@levon9]

Fantastic video, and a great visual demonstration of the Kernel Trick - thank you for sharing. Also love your description of your process of learning this, it is very similar to what I do, and your advice at the end of the video is right on target as well.

[@BrandonRohrer]

Thank you! I'm glad you got a lot out of it.

[@arung80]

What a nice way to express SVM’s ! Please keep up the great work for the community

[@DeepakSadulla]

A very good toy example coupled with great/intuitive visualizations make this one of the best explanations :) Thanks!!

[@vaporizer08]

One of the clearest explanation on youtube. Thanks

[@Larock-wu1uu]

Very well explained! This makes SVMs extremely intuitive.

[@llbrunollllbll9347]

Great! Please, keep on showing us how to dig for the details. Really good your comparison between "city driver" vs. "racer driver" .

[@karthin3839]

The Best visualization of the Kernel Trick that I've seen ever....

[@KayYesYouTuber]

Simply beautiful. Thank you very much for taking the time to do the video.

[@Chillos100]

Wow!! To you sir, my hat goes off!! Thank you so much for this wisdom & ease of explanation.. Many blessings to you in the future..

[@BrandonRohrer]

Thank you so much!

[@ritikajain8859]

Great and Amazing video! I have seen many videos but was not able to understand the basics of svm specially the kernel thing but ur video has a clear and simple explanation of it and helped me to understand the concept easily. Thank u so much! :)

[@ahmed7408]

Brandon you are awesome! Thank you so much for explaining it in such a clear and simple way.

[@suryarp]

Instructional, Informative, Insightful and Inspirational! Good work!!

[@TamerEid1]

you are the best video describes SVM all over the internet ....after seeing many others ...you did the best ...Great Job Dude

[@wembly8225]

By far the best explanation of the kernel trick ever!! god bless you

[@BrandonRohrer]

Thanks wembly!

[@philipabraham5600]

Brilliant Explanation on SVMs. Thank you.

[@senthilcaesar]

This video is super helpful to understand SVM . Simple explanation are always best

[@husohomeful]

Thanks for also sharing your learning process. What a gem!

[@joanbrod4561]

Brendon, again another well thought out video presentation. By presenting something one is really forced to understand it clearly oneself, as indeed you do. Please continue.

[@thegreatskywalker]

This has to be the best explanation of this topic. Period.

[@geraldgibson6055]

Good job communicating the Key abstracted concepts.

[@igorrizhyi5572]

i expected for some formulas to reinforce my knowledges with numbers and math tricks, but it is also good without them. big thank you for your efforts dude!

[@anandsingh1011]

Amazing explanation..!! I can't believe anyone can explain SVM in so simple way. I think if i explain it to my 6 year old son he will also understand it.

[@Peter-bg1ku]

Great explanation of the kernel trick!!

[@eleakokkonen6093]

Great stuff! Will definitely check more of your enlightening videos. Big Thanks!

[@kozzuli]

Another amazing video Brandon, you rock!!

[@terakiei]

Thank you for the video! Think I'm going to open a lot of boxes from now on.

[@iliachigogidze9929]

You have talent! Best explanatory video EVER.

[@af121x]

Great! Clear Explanation with extremely nice visualization

[@omerkaragoz]

That's the best explanation I heard. Thank you.

[@saketthavanani312]

Great Brandon. That was amazing please keep posting more!!!

[@adnanhashem98]

I hope you find the below annotated summary of the explained method of "opening the box" helpful😊 In the process of going through the steps of the method, think about the following questions*: q1: What is the "box" design for? (e.g. What is the purpose of SVM?) q2: What is the "box" used for? (e.g. What is SVM used for?) q3: How to visualize the key concepts? (e.g. How to visualize SVM kernel trick?) q4: How the underlying Math works? I'd like to think of the below "steps" as strategies that I can select from and mix together (depending on the box I'm trying to open). Steps: 1. Read the original source that explains the "box" (e.g. scikit-learn docs).** 2. Read good Tutorial(s). 3. Watch good KZbin videos. 4. Read some good (blog) posts. 5. Explain the "box" to yourself and try to draw illustrations of the key concepts. 6. Choose a toy example (i.e. simple example that preserved the fundamental features of the "box".). 7. Explain it to a 12 year old (to avoid using jargon and to get to the essence of the "box"). 8. Understand the weaknesses of the box. (e.g. What conditions make SVM a poor method of choice?) So, that's it! This is how you open a box 🙂 Footnotes: * Of course some of the questions are not applicable to some "boxes" :) ** Be aware that this step might not be accessible to beginners.

[@pratik6447]

Wow! Amazing Explanation!! The best ever visualization and explanation. (y)

[@literallyshane4306]

Excellent! Very well explained, thank you

[@mortezaabdipour5584]

Thank you for the magnificent presentation

[@viktoriiauntilova7695]

Amazing way of explanation ! Love it !

[@chaitanyajeesrivastava5333]

Great explanation Brendon. Amazing effort by you

[@sofluzik]

Awesome Brandon ...just apt and precisely what the math conveys

[@andreas7278]

I learn most of my things as an autodidact and reach very high levels by doing so. It feels very rewarding, you can use your own tempo, choose the right methods and whatever you worked out stays in your brain. However, it's probably not the fastest way in general compared to having great tutors/masters/teachers at your disposal I might add. I actually just did a similar thing, read a lot about svm, the kernel trick, played around with it in python, visualized it, watched videos etc ... at least it stays in memory if you do it that way

[@emmanueloluga9770]

How long did it take you

[@TJ-zs2sv]

Thanks a lot for such good work, and to give wonderful intuition . :)

[@ladiharish7843]

thanks for making it an unforgettable concept.

[@khannawaz2600]

THank you so much for explaining it so well .

[@buh357]

simple and beautiful explanation, thank you:)

[@nkdms.2031]

SuperAmazing GENERAL intuition!

[@trimbandaru1767]

Thank you. It just amazes me

[@iamsiddhantsahu]

Superb video, loved it!

[@user-jq3um6ez2h]

Thank you , it's really lively.

[@foobar1672]

Good explanation. Thank you!

[@theDgrader]

great video, excellent work.

[@yoyoyoyo7813]

This explanation is god level!!

[@GO-wr3de]

Such a nice explanation!

[@chaitanyajeesrivastava5333]

Amazing explanation Brendon. You rock !!

[@ChrisKnorowski]

Great explanation! The only part that may be slightly confusing was the use of colors in 1d plot. Someone may ask, why not just reorder the colors so that they are linearly separable.

[@bktsys]

Good to understand SVM

[@navneetmohit1]

I always had an inclination to teach and I always believed that if I know something very well I can teach it to anyone. But the tool that you used to visualize in this video is mind blowing. Would you be able to share what application you use for this illustration? Thanks a TON!!! I mean it.

[@dz_ai_school4491]

Awesome Awesome Awesome explanation, Thank you 10000 times for the effort

[@BrandonRohrer]

You are very kind :) Thank you.

[@simonstrandgaard5503]

Great explanation

[@tanmaynagori4977]

You sir are awesome!!

[@K6G8M9P0]

Thank you for the video!

[@ahmadest61]

Excellent. Can you please upload an introduction and working of MRF too?

[@__lee873]

excellent work

[@ankit3j]

That was awesome dude

[@kunalkalwankar7304]

Best svm explaination

[@BrandonRohrer]

Thanks Kunal :)

[@karannchew2534]

Summary for my revision: Separate different groups based on the distance between closest points from each group. Separate (classify) based on equal distance of gap. If not possible, then based on combination of gap distance and amount of wrong classification. Wrong classification can be quantified based on soft-wrong data (wrong but within the margin). very-wrong (way off the margin and the center separation line) If still not possible (i.e. the data are not linearly separatable) then bend and warp the data (i.e. apply the kernel function to the data), until it is possible to apply the above steps.

[@arjunpukale3310]

Please make such videos on other machine learning algos

[@nomadml9716]

Brilliant explanation. Are you using the Richard Feynmann method of breaking down problems? It would seem so possibly or maybe you just approach subject matter to be understood in this way. Regardless you have me as a new sub. Thanks for the video.

[@Tyokok]

was expecting to see you connect visualization with one simple math sample to illustrate how kernel bend the data representation. you have such kind of video? thanks!

[@rohanchadha3506]

You Rock Brandon :)

[@mojojo_san]

Thank you, this helped me a lot as to understanding how the kernel trick works. But I'm still wondering, how to best explained the radial kernel trick when we're trying to predict a continuous variable (Support Vector Regression). Does anyone have a suggestion ?

[@COOLGODBOSS]

bam nice explanation, impossible to not understand.............now my only question is if this is all SVMs do or is there more about them?

[@siddharthGupta632]

Amazing

[@truliapro7112]

What software do you use for the data visualization? That is really great!

[@minhphuongtran9461]

You are great! Lots of thanks :p

[@JeanPaulUS]

Excellent explanation. If you don't understand SVM from here you will probably never understand it

[@user-gn5uw3nu5y]

Thank you very much

[@butterflytwilight]

Extremely well explained, very helpful... @Brandon a request - can you also post a video tutorial with the Math behind the SVM?

[@imedkhabbouchi2161]

Thank you for this very simple and intuitive description that demystified the SVM. I have one question about the presentation how could you make the 3D plots rotating using mouse cursor as seen in minute 10:00 for example?

[@BrandonRohrer]

Thanks! This is a Matplotlib interactive 3D plot. It lets you drag the point of view with your mouse. Here's a deeper dive if you need it: stackoverflow.com/questions/49981313/rotate-interactively-a-3d-plot-in-python-matplotlib-jupyter-notebook

[@patite3103]

Your video is simply amazing! Would it be possible to make a video applying it to a concrete example in Python?

[@hongpengjin9435]

so good

[@catnudro]

Brandon - what software/package are you using for the 3D viz?

[@AllanDaemon]

The code he used is linked on the video description. You can open it and look it yourself. It's a very simple and straightforward code. He used the Python Library matplotlib, and I bet he used together with Jupyter notebooks to visualize in the browser.

[@anirbanchakraborty4706]

Amazing. Can you suggest anything for ELM? Would be much obliging.

[@SiddharthSubramaniansiddhu]

Awesome