Multi-Armed Bandit : Data Science Concepts

[@jc_777]

enough exploration for good youtube lecture on ml. i should keep exploit this guy. 0 regret guaranteed :)

[@Sad-mm8tm]

I hope you will continue making videos forever. Your explanations are the best I've ever seen anywhere + the wide choice of topics gives me food for thought when dealing with my own optimization problems.

[@ritvikmath]

Thank you :) I'm happy to help

[@VahidOnTheMove]

If he makes videos forever, we'll get zero regrets.

[@Xaka-waka]

​@@ritvikmathdon't let this channel die man

[@marcelobeckmann9552]

Your explanations, didactics, and dynamism are amazing, way better than several university professors. Well done!

[@faadi4536]

What an amazing explanation. I am taking a Machine Learning Course and he tried to explain the concept using Bandits but couldn't quite really grasp it in detail. I understood what we are trying to figure out but wasn't quite their yet. You have made it so much easier. Kudos to You Brother.

[@bilalbayrakdar7100]

Bro I completed my CS degree with your help and now I got accepted for master and you are still here to help. You are a true man, thx mate

[@savasozturk00]

After watching 5 videos, finally I found the best lecture teller for this topic. The examples are great, Thanks.

[@111dogger]

This is the best explanation I have come across so far for the Upper Bound Confidence concept. Thank you!

[@itssosh]

It would be great if you made a whole playlist where you explain the statistics for machine learning by explaining the formulas in an intuitive way like you do (you make me understand them all). For example, explain the various distributions and their meaning, statistical tests (p-value), etc. Thank you so much for the work you do and the knowledge you share!

[@abdulsami5843]

A thing I absolutely like is how palatable you make these concepts, not too mathematical/theoratical and not overly simplified, just the right balance ( € - greedy is set right 😉)

[@Dr.RegulaSrilakshmi]

U r just awesome ,any person who doesn't have any knowledge of Reinforcement learning can understand,Keep up the spirit...cheers

[@pranavlal2289]

Best explanation of MAB ever. Thanks

[@AnasHawasli]

Thank you so much for this simple explanation It was impossible for me to understand this concept without your video NOT EVERYONE SPENT HIS LIFE IN A CASINO I am not familiar with this armed bandit trash Here is a sub!

[@malice112]

What a great and easy to understand explanation of MAB - thank you for this!!!!

[@shahnazmalik6553]

Your teaching method is highly appreciated. Please make lectures on statistics and machine learning algorithms

[@softerseltzer]

Love your videos, the quality just keeps going up! PS. the name of the slot machine is "One-armed bandit", because of the long arm-like lever that you pull to play.

[@irishryano]

....And the bandit bc it’s the WORST odds in every casino

[@spicytuna08]

i guess the slot machine is a bandit cause it keeps robbing money from the players.

[@SURBHIGUPTA-o4w]

Thanks Ritvik! this is the best explanation I have come across so far!

[@spicytuna08]

we need to a person like you to democratize these important concepts cannot express how grateful i am to understand these important concepts which i have struggled in the past.

[@heteromodal]

Great video, and it's really nice listening to you! Thank you :)

[@maxencelaisne4141]

Thank you so much, I passed my exam thanks to your explanation :)

[@ritvikmath]

Glad it helped!

[@hameddadgour]

I just realized that I need to explore more to maximize my happiness. Thank you Multi-Amed Bandit :)

[@adanulabidin]

What an amazing explanation! Thank you so much. Keep making such videos.

[@llescarini]

Subscribed since few days, your videos are more than excellent! Amazing skill for teaching, thanks a lot.

[@ritvikmath]

Awesome, thank you!

[@CaioCarneloz]

The way you explain is stunning, what a awesome lesson.

[@jonathanarias2729]

Why 330 is the response in the explotation example? Should t be; 3000-2396=604??

[@khaalidmcmillan9260]

Well said, needed a refresher after not seeing this for a while and this nailed it. Hopefully you've gone into more advanced topics like MAB reinforcement learning

[@bobo0612]

Hi! Thank you for your video. I have a question at 6:28. Why the roh is not simply 3000 - 2396?

[@senyksia]

2396 was the happiness for that specific case, where restaurant #2 was chosen to exploit. 330 is the (approximate) average regret for every case. So 3000 - 2396 would be correct if you were only talking about that unique case.

[@myoobies]

@@senyksia Hey, what do you mean by average regret for every case? I'm still having trouble wrapping my head around this step. Thanks!

[@madmax2442]

@Bolin WU I know it's 8 months already but I wanted to know whether you got the answer or not. I also have the same doubt.

[@vahidsohrabi94]

I'm grateful to you because of this great tutorial.

[@gabrieldart9943]

This is so cool! Thanks for your clear explanation.

[@nassehk]

I am new to your channel. You have a talent in teaching my friend. I enjoy your content a lot. Thanks.

[@ritvikmath]

Thanks!

[@traiancoza5214]

Perfectly explained. Genius.

[@raphaeldayan]

Amazing explanation, very clear, thank you Sr

[@anaydongre1226]

Thanks so much for explaining this in detail !!

[@ritvikmath]

You are so welcome!

[@nintishia]

Very clear explanation. Thanks for this video.

[@sahanar8612]

Great Explanation!. Thank you 😊

[@jinpark9871]

Thanks, your work is really awesome.

[@ritvikmath]

Thank you too!

[@soundcollective2240]

Thanks, it was quite useful, heading to your Thompson Sampling video :)

[@DarkNinja-24]

Wow, great example and amazing explanation!

[@A.n.a.n.d.k.r.]

Awesome cool technique just got hooked to this

[@jroseme]

This was a useful supplement to my read of Reinforcement Learning by Sutton & Barto. Thanks.

[@ritvikmath]

Glad it was helpful!

[@krittaprottangkittikun7740]

This is so clear to me. Thank you for making this video!

[@rikki146]

i cannot thank you enough for makin this excellent vid!

[@SDKIM0211]

Love your videos. To understand the average regret value for exploitation, which extra material should we refer to? Why not 604?

[@prasadbbd]

love this video and explanation is very clear

[@kunalkasodekar8562]

Perfect Explanation!

[@nastya831]

thanks man, this is truly helpful! 6 min at 2x and I got it all

[@ritvikmath]

Great to hear!

[@hypebeastuchiha9229]

My exam is in 2 days and I'm so close to graduating with the highest grades. Thanks for your help!

[@warreninganji7881]

crystal clear explanation worth a subscription for more👌

Awesome! Thank you! You helped me a lot!

[@dr.kingschultz]

You are very good! Please explore more this topic. Also include the code and explain it

[@aryankr]

Thank you for a great explanation!!

[@michaelvogt7787]

multi-armed bandit is a misnomer really... it should be multi-one-armed-bandit problem. slot machines were called one-armed bandits because they have a single arm that is pulled, and the odds of winning are stacked against the player making them bandits. the goal is not so much about finding out which to play, which would become more apparent given enough plays, but instead to determine which mix of N plays to spread out across the group, settling in on the best mix to achieve exploration in balance against exploiting the best returning bandit. i am a career research scientist pioneering in this field for 40 years... i am always reviewing videos to back-share with students and learners and YOURS have Returned the greatest value for my Exploration, and I will be Exploiting YOURs by sharing them the most with my students. its the best compliment i can think of. cheers. dr vogt ;- )

[@amirnouripour5501]

Thanks a lot. Very insightful!

[@NoNTr1v1aL]

Amazing video!

[@yongnaguo8772]

Thanks! Very good explanation!

[@stanislavezhevski2877]

Great explanation, can you leave a link to the code, which you used in simulations ?

[@ritvikmath]

Thanks! I have a follow up video on Multi-Armed Bandit coming out next week and the code will be linked in the description of that video. Stay tuned!

[@fridmamedov270]

Simple and accurate. That is it. Thanks!!!

[@abogadorobot6094]

WOW! That's was brilliant! Thank you!

[@TheMuser]

I have explored and finally decided that I am going to exploit you! *Subscribed*

[@dr.nalinfonseka7072]

Excellent explanation!

[@Status_Bleach]

Thanks for the vid boss. How exactly did you calculate the average rewards for the Exploit Only and Epsilon-Greedy strategies though?

[@debashishbhattacharjee1112]

Hello Ritvik This was a very helpful video. You have explained a concept so simply. Hope you continue making such informative videos. Best wishes.

[@ritvikmath]

Thanks so much!

[@francisliubin]

Thanks for the great explanation. What is the essential difference between contextual bandit (CB) problem vs multi-arm bandit (MB) problem? How does the difference impact the strategy?

[@velocfudarks8488]

Thanks a lot! Really good representation!

[@rifatamanna7895]

It was awesome technique 👍👍 thanks

[@ritvikmath]

thanks for your words!

[@michaelvogt7787]

Nicely done.

[@zahrashekarchi6139]

Thanks a lot for this video! Just one thing I would like to find out here is where we store the result of our learning? like some policy or parameter to be updated?

[@victorkreitton2268]

What ML books do you recommend or use?

[@rutgervanbasten2159]

really nice job! thank you

[@vijayjayaraman5990]

Very helpful. How is the regret 300 in the second case? Shouldn't it be 3000 - 2396 = 604?

[@davidkopfer3259]

Very nice explanation, thanks!

[@ritvikmath]

Glad it was helpful!

[@seowmingwei9426]

Well explained! Thank you!

[@welidbenchouche]

This is more than enough for me

[@TheFobJang]

Would you say exploit only strategy is the same as the eplore-then-commit strategy (also know as explore-then-exploit)?

[@bassamry]

very clear and simple explaination!

[@ritvikmath]

Glad it was helpful!

[@jams0101]

awesome video ! thanks so much

[@annahuo6694]

Great videos ! Thanks for your clarification. It's much clearer for me now. But I just wonder how you calculate the 330 regret in the case of exploitation only ?

[@ritvikmath]

Good question. You can get that number by considering all possible cases of visiting each restaurant on the first three days. Something like, consider the probability that of the first three days of visits, what is the probability that restaurant 1 is best, vs. probability restaurant 2 is best, etc. You can do this via pencil and paper but I'd recommend writing a simple computer simulation instead.

[@annahuo6694]

@@ritvikmath Thank you for this prompt response. I think I get the idea from the epsilon greedy formula (option number 3 in the example). Thank you a lot, your video is really helpful :)

[@alirezasamadi5804]

You explained so good

[@josemuarnapoleon]

Nice explanation!

[@georgiak7877]

This is amazing !

[@snehotoshbanerjee1938]

Best explanation!!

[@ritvikmath]

Glad you think so!

[@avadheshkumar1488]

excellent explanation!!! thanks

[@sbn0671]

Well explained!

[@shahulrahman2516]

Great video

[@yitongchen75]

Cool explanation. Can you also talk about Upper Confidence Bound Algorithm relating to this?

[@ritvikmath]

Good timing! I have a video scheduled about UCB for Multi-Armed Bandit. It will come out in about a week :)

[@뇌공학박박사]

Best example ever!!!

[@TheMuser]

Nicely explained!

[@yannelfersi3510]

can you share the calculation for the regret in case of exploitation only?

[@wenzhang5879]

Could you explain the difference between the MAB problem and the ranking and selection problem? Thanks

[@PhilipKirkbride]

Related to regret, we never really know the true distributions (since we can only infer from taking samples). Would you basically just use your estimated distributions at the end of the 300 days as the basis for calculating regret?

[@joyo2122]

can you explain why you need a random number generator?

[@tariqrashid6748]

Great explanation

[@manabsaha5336]

Sir, video on softmax approach.

[@quanghoang3801]

Thanks! I really wish the RLBook authors could explain the k-armed bandit problem as clearly as you do, their writing is really confusing.

[@sampadmohanty8573]

I knew everything from the start. Ate at the same place for 299 days and got pretty bored. So watched youtube and found this video. Now I am stuck at this same restaurant on the 300th day to minimize my regret. Such a paradox. Just kidding. Amazing explanation and example.

[@newwaylw]

Regret for your exploit only strategy should be 3000-2396=~604 no?

[@thinkanime1]

Really good video

[@ritvikmath]

Thanks!

[@qqabt24816]

I love this vid! It would be great if you could also do more videos on online learning and regret minimization 😆😆😆

[@shantanurouth6383]

I could not understand how it turned out to be 330, could you explain please?

[@sunIess]

Assuming a finite horizon (known beforehand), aren't you (in expectation) better off doing all the exploration before starting to exploit?

[@ritvikmath]

You've just made a very good point. One strategy I did not note is an epsilon-greedy strategy where the probability of explore in the beginning is very high and then it goes to 0 over time. This would likely be a good idea.

[@Trucmuch]

Slot machines were not called bandit but one-arm bandit (they "stole" your money and the bulky box with one lever on its side kind of looked like a one-arm man. So the name of this problem is kind of a pun, a slot machine with more than one levers you can pull (here three) is a multi-armed bandit. ;-)

[@ritvikmath]

Wow I did not know that, thanks !!