How to Build a Superhuman Poker AI using CFR | Creating a Poker Bot Part 2

[@KylieYYing]

What are your thoughts on superhuman AIs?? P.S. Check out Taskade, a super awesome platform for managing collaboration and organizing projects. It's simple to use and free! www.taskade.com Be sure to subscribe for more fun content like this :)

[@govinej]

可以

[@ygjt76v0-----]

I love you❤️

[@nicolozaniolobedankt]

Go solve cancer not poker

[@IshaqIbrahim3]

Just discovered this channel. I wish I discovered it a long time ago. lol!

[@atgaming5318]

one of the most underrated computer science youtuber

[@skeletonlord2126]

This is not a real tutorial, this is an ad for a task management software.

[@utuber1752]

mmmmm.... normally I'd agree, but I think it's ok this time.

[@thecaptainubi]

i can't describe in mere words how wonderful you are and how much you've impacted my life. i thank you deeply.

[@KylieYYing]

🥺🥺🥺 Wow, thank you for watching!!

[@thecaptainubi]

@@KylieYYing omg you actually saw and replied to me :) please continue making awesome content.

[@LeeSpaziano]

Awesome video @KylieYYing. I was searching for some insight on clusters for CFR, and your take was perfect. I think you also need to abstract other aspects of the game state such as total pot, chip stacks, pot odds, and most importantly, player statistics. I think the self-play aspect of it makes it impossible to learn to play against various player types (i.e. TAG, Fish, Nit, etc). Therefore my multiplayer simulation will use two MCCFRs and four expert systems, each modeled an avatar of a player caricature. Let me know if you'd like to collaborate IRL.

[@KeithGalli]

Damn that was a fire intro

[@zchamdawala]

Was eagerly waiting for this. Thanks for uploading 🔥👍. You rock 😀

[@KylieYYing]

Thank you for your support 🥰

[@jonesbts6393]

I love your content, please keep making more if you are able!

[@Luckypantera27]

Wow Kylie, you are so talented! love all your videos!!!

[@faheemhussain3176]

A very good video. Im on my to learn programming and a bot. I assume you arent short of money, knowing the jazz. But youve presented ideas well. Regret is a nice tabgent in perspective. Something just for weight, apart from logic. Good story telling. Wosh you the best. 🎉

[@flightisallright]

You should get an online casino to sponsor you ;)

[@misclic2408]

13:00 that was heads up = not a tournament format = tournaments nodes probably is 10 in the tenth more never the less all theese r extremely interestin ,,,please keep on

[@trtlphnx]

Damn!!! I Love Your Incredibly Cogent Insight... Damn; I even Think I Love You!!!

[@rubyshekhar2869]

Same here

[@TypicallyThomas]

I've never seen a more invasively sponsored video. Also I never trust free products that can afford sponsorship deals. If they pay to give it away for free, you're the product

[@TeeTeeTeeL]

Can you make a bot that’s parallel to the hand your in in real time ? Something like a real time assistant???

[@andredias5061]

That's all the steps used in training right? I'm trying to understand the Sub-game solving part........

[@SoundsOfTrance-rg8xo]

Okay, but how do I actually build it?

[@whatnow1194]

how good is learning from from videos and do code and create my own mess and again while trying to get rid of own mess ending up creating more mess out of it and it works fine but not perfect?

[@jetshot6133]

we have a big project with Nvidia and aws to create python application using machine learning

[@SynedocheFernbank]

Great video 🎉

[@heyman620]

I really liked the video - but I think we should not consider CFR as reinforcement learning since CFR aims to minimize regret (not to maximize reward). It looks like it's a small difference, but in practice, it's the difference between trying to possibly exploit our opponent (maximize reward) and minimizing exploitability (CFR) - which also means CFR is unable to be adaptive if I get it correctly. RE RL for poker, I think Noam Brown has some new related work. Though I am not an expert for RL or CFRs as I do mostly NLP... What do you think?

[@roboltamy]

Reinforcement learning doesn't have as much to do with the objective as it does with the way that the model is trained towards it. Reinforcement learning is where you train the model by having it attempt whatever task it is meant to do (in this case playing poker), then based on the outcome relative to its objective (whether that is minimizing regret or maximizing some other reward) it is optimized. All the regret is, is a different objective, not an alternative style of training. Sorry to reply to a year-old comment and hopefully my explanation is clear enough.

[@heyman620]

​@@roboltamy Well, I agree with most of what you say, and what I wrote is not accurate, but nowadays I know much more about RL and still, CFR is clearly not an RL algorithm. Anyway, Barto might disagree here (he says RL is much broader) but the standard approach would be to not call CFR RL, if you do it you might as well call MCTS RL and so on, but it's incorrect. Again, looking at counterfactual outcomes to maximize your outcome is not the way MDPs work, in MDP, by definition, you don't search in history. But nice comment (nice to realize I know a little more after 1 year).

[@OleguitoSwagbucks]

Does it work with more than 1 opponents?

[@rizwanahmad9921]

When there will be the next part?

[@anpan4318]

Hey Kylie I'm down if you want to build this together. I've been reading CPRG Lab's paper since 2014. Plus I can pay for the infra bill.

[@stonertrading]

let's go to vegas and try it out. +)

[@luhao4337]

Is this only for two players games tho?

[@sebastiansahlman4754]

Hi! I have a brief programming question about Python, and I wanted to ask it on the most recent video so you would see it. I am new to Python and wondering how to separate the conditions of an "if" statement. Here is an example: w = 0 x = 1 y = 2 z = 3 if w == 0 and x == 1 or y == 2 and z == 3: print('Pass') See how this could be interpreted multiple ways? One way is: if (w is 0 and x is 1) or (y is 2 and z is 3) Another way is: if (w is 0) and (x is 1 or y is 2) and (z is 3) for the first one, w = 0 x = 2 y = 2 z = 3 would not pass the test and print "pass", because w is 0, but x is not 1 for the second one, however, w = 0 x = 2 y = 2 z = 3 would pass the test, because (w is 0), and (x is 1 or y is 2) and (z is 3) How do you separate these conditions in Python? Any help is appreciated. Thanks

[@yoyo-yx1qx]

Bro you are confusing yourself for no reason just understand how "and' "or" operators work by using "and" like if x ==1 and y==6 print("ho") you are telling the program that it should only print when both are true if one is not true it will not print however if you say ,if x == 2 or y ==1 print("bruh") you are telling the program print bruh if one of these two is true. after you understand how both this operators work you can play with them anyway you want,i prefer using one operator per 1 if statement in my code.depending on the situation i don't like mixing "and" with "or" in one if statement if it is not required, it creates bugs in my programs.i hope that helped you to understand better.

[@khytron06]

If you're using 'and' and 'or' in one line of code. it seems to prefer the 'and' first, and here's how Python interpretes your statement: if (w == 0 and x == 1) or (y == 2 and z == 3): print('pass') notice how the 'and' compared to 'or' is like the multiplication/division and additiion/subtraction. w = 0 x = 2 y = 2 z = 3 would pass the test, bcz y == 2 and z == 3.

[@seungsooim2183]

this is not interpreted multiple ways. when you code that conditional statement python requires you to bracket the 'and' together and so the or is between the two conjuncts. nothing arbitrary about it.

[@Ms123kill]

But the real question is did you make money on online poker. Asking for a friend I might make one but I heard they have been taking down such outlier performers.

[@Fine_Mouche]

6:40 : this algo seem very result oriented, but i heard it's bad to be result oriented in poker. So why it's good for the AI ?

[@misclic2408]

u thinking about it very simple = its exactly the oposite = it calculates all possible expected values

[@Luckypantera27]

@@misclic2408 Hi- how can I contact u? I have a question that I am sure you will be able to answer 🙏🏻 Thanks

[@Junmarvi]

New Follower here..:). Keep it up..

[@urbleeding]

you are super smart

[@MsYann34]

nice video

[@meMe-qi1tp]

Bro this could have been the biggest brain video if you decided for the AI to take into account other people’s regrets and use that against them. I have not heard a poker AI developer said they developed their AI that way (or at least they didn’t reveal that)

[@KylieYYing]

Actually, that is totally possible! Basically in poker, there's two streets of strategies - game theory optimal (GTO) and exploitative. GTO tried to play at Nash equilibrium (like this CFR bot) because in theory it's unbeatable. However, how close you can get to Nash equilibrium depends on your state/action abstractions and how long you train the bot for. Exploitative play though does exactly what you mentioned. You try to find your opponents' weaknesses (such as their regret) and use that against them. However, in theory, this will lose to a bot playing at Nash equilibrium (since that's what's considered optimal, and a bot playing at Nash equilibrium does not lose in the long run).

[@meMe-qi1tp]

@@KylieYYing queen sht but I’m talking about the boy playing the pros only so exploitive might still work idk? Would be fun to see pro poker players like my man Daniel got trolled tho

[@guilhermegdeavogtmannsberg3776]

@@KylieYYing there are more conditions that the bot work,not only GTO,Nash etc, Pluribus $350 usd, can do millions, but not at many poker rooms, because they ban

[@utuber1752]

You say a higher regret means take it more often. Surely it's the other way around....

[@Nedwin]

❤️❤️❤️

[@Eliel10268]

Is she Korean descend?

[@ygjt76v0-----]

😘😘

[@tobiaskarl4939]

I would prefer less bla bla but more source code. And not in lame python but in C.