How Does OpenAI Five Really Work And Its Impefections

[@lorenzoblz799]

I liked the video overall, just wanted to add a note about ML. ML is not about DOTA as it was not about chess, Doom, Atari, Go or other games. They are not trying to build the ultimate bot as an end in itself, OpenAI or DeepMind are not Valve or EA: they have no special interest in videogames. It just happens that videogames are a good way to test different ML solutions as they present a nice mix of short/long term problems to solve in a complex dynamic environment. Solutions that you CAN port to solve real world problems. ML today is about computer vision, language understanding, self driving cars, medical diagnosis, real world interaction and a lot of other applications. Games are just a commonly used "benchmark". Any research project needs money so I think it's perfectly reasonable to exploit any visibility they can get with their work. Just one last note: you say that if the AI only plays against itself it will never be able to "predict" what humans will do. This sounds reasonable but AlphaGoZero was able to beat humans by training in the same way (with even less human input, actually). So I think it may be possible for an AI to learn strategies through self-play that work well against humans with DOTA2 too. We humans rely a lot on prediction but this does not necessarily mean that this "type of reasoning" is mandatory or even the best (or not achievable in differen ways). Thanks for the video, coming from the ML world I think the video was well made.

[@GaryMcKinnonUFO]

Good points well put Lorenzo. I think we'll never develop true AI since we can't fabricate consciousness, but that's a loooooong discussion ;+}

[@tazz1226]

ML is also used in the model design for a car for making it hard and light the machine will try thousands of models and test it, and discard what's not good, just like dota, they can be effective for solving new challenges us the human face, like learning how an alien species works or something strange to the human, whatever task you assign them they will work hard like no human can, yes in this case of dota they miss some things, but the technology is really new, for sure they will change how bots in videogame works, but it will take them years to set the rules properly. IN DOTA: if you want to have fun playing against them they will have to set rules like "challenge humans", something like that, and yeah would be a good option if you can matchmake against dota ai, and they could collect info even if its really shit at the start, if that could give you rewards for the battle pass or something like that would exponentially grow ML for games, i think they need to fine tune the rules, they are always thinking about the" what did we do when they did this" , more than "what its better to do now", and that takes creativity and its not possible to a machine to do, they always have to evaluate something from the past, would be good if they could in a given match moment, play that moment in a fraction of a second see the outcome, play again that moment, infinite times, and then make a choice in the real game.

[@philliplochlan1134]

I guess im randomly asking but does any of you know a method to log back into an Instagram account? I somehow forgot the login password. I love any tricks you can offer me.

[@maxjacoby3048]

@Phillip Lochlan instablaster =)

[@philliplochlan1134]

@Max Jacoby I really appreciate your reply. I found the site through google and im trying it out atm. Looks like it's gonna take quite some time so I will reply here later when my account password hopefully is recovered.

[@heltok]

13:05 "Why even bother with machine learning" - Could you propose any alternative to ML? Maybe that's the reason so bother with ML, because that it is the only way we know how to solve the problem as of today.

[@codeincomplete]

Yeah, before that point the video was not bad at all but, yeah, at that point the youtuber fails to understand the directions ML is going or what even the purpose of the project is or what even the capabilities of ML is - for example, games with hidden state are direct targets of ML even today (technically for a long time now). Nitpick, ML is an extremely broad category - there are no "alternatives" to ML - there are alternatives to the deep reinforcement learning technique that open ai is using but why wouldn't you use the technique that is having the most amount of growth and potential.

[@huevonesunltd]

While i don't agree with him, i think what he meant is "why bother with machine learning if it's going to do the same thing as an AI made with set rules would" his point is that while the whole idea of a machine learning AI is that it can solve problems by itself without any set rules or scripted decisions but instead ends up being as limited as one that does, then why going for all the trouble if it's going to end up the same way? He's talking about how right now the AI is limited to a pool of heroes since it's machine learning it would require even greater time and processing power if it could choose any hero. me myself i wonder how the hell are they going to overcome that problem.

[@chardonnay5767]

What a ridiculous comment when you question the point of machine learning. Just look at the pace of improvement rather than the currently available product. At this rate of development we’ll see artificial intelligence that humans can only think of as a godly existence during our lifetimes. Just because it hasn’t yet translated to prowess in a video game doesn’t mean that it’s pointless.

[@BuceGar]

Good video. The other main problem with this type of AI is that, as more heroes and items are added to the game, the total number of interactions rises exponentially. That is to say, at a certain point, you won't be able to use raw computational power to account for every possible interaction. I different type of AI would need to be developed for the real game, but might be possible.

[@banut2116]

"you won't be able to use raw computational power to account for every possible interaction" This is exactly what separates AI from a normal computer. AI doesn't go through every possible outcome to find the ultimate best move. Instead they find a local optimum that minimize a given loss function (which can be anything, but in this case is probably survival, maximize hero kills etc.). In other words the AI settles for a sub optimal solution in favor of speed and flexibility. This is what separates AI from just basic raw code. So to your point the rise in number of interactions will not necessarily affect AI performance as long as the AI has been adequately trained on it.

[@georgeterme]

Can you please give us some references to the concept of cooperative ai and team spirit on AI? a paper, an explanation, anything

[@Pohka]

I think this blog post summarizes most of it, there is more posts on that website too openai.com/blog/openai-five/

[@Toby_a_Pa]

How can. We play pro bot(AI)

[@davidwang4461]

Hi Pohka, in 2:30 why killing has a negative weight?

[@Pohka]

That must have been a typo on OpenAI's github page. The full list of rewards is here gist.github.com/dfarhi/66ec9d760ae0c49a5c492c9fae93984a

[@atticusbeachy3707]

+David They get gold and experience from killing enemy heroes, so the total score gained from a kill will still be positive. The researchers are just reducing the reward for some reason.

[@williamross6477]

Maybe I'm reading too far into it, but you seem to suggest that one of the biggest flaws in machine learning is it's inability to adapt to things it hasn't yet experienced. A core aspect of machine learning is for computer systems to predict and identify things that they have not directly observed, based on inferences from patterns they have observed. A system as complex as DOTA requires an extremely large data set. I think it's important to keep in mind that this is essentially the first iteration of bleeding edge technology. Image recognition AI started out very rough, but can now perform at near human levels, of not better. You make a very good point that, in order to advance much further it will need extensive play time against real players, exposing it to new patterns that it has not yet incorporated into its data set. Unfortunately this can only be done in real time, making training much slower. Fortunately, much of the time spent training against itself was spent experimenting with and slowly figuring out basic functionality (I'll bet the first few thousand years of play time would be incredibly tedious to watch ;) ). With that out of the way, each game against real players can make a significant impact on the AI's future abilities.

[@UnixLinuxWaffles]

Farty minutes long

[@xenonxi7564]

how the hell are they trained for 900 years in a single day??? 900 years in a day?? 900 years = 328717.979 days 1 day = 1 day 328717.979 =/= 1 wtf bruv!!

[@qudizzle1]

I am assuming they run a modified version of the game at a sped up rate. It normally practices vs itself and not vs humans. Also it can probably also play multiple games simultaneously if it were to train against humans

[@huevonesunltd]

its simple, they ran it in super speed or various computers playing games, that's why it requires a lot of processing power and lot of money too.

[@junyan2604]

they are all one,it means one game 5 v 5 60 mins is 600 mins of trainning , and it can even play multiple games speed up at same time.

[@michaeltolibas4276]

default bots in dota has 0ms response time once they see you, they go crazy

[@Razta_S]

But the OG VS OPEN AI 5 matches tho

[@Shafin15]

Imperfections?

[@davidlangley9287]

I presume that you are not even an engineer, the way you analyse OpenAi is not objective. here's a really good video kzbin.info/www/bejne/eqvdd4avr9tkppI from an AI specialist, it's a bit technical though.

[@_Baleful]

Very much disagree with your negative framing of the situation. Obviously you showed respect and you weren't bashing the developers, but "it's like playing chess with half the pieces" is a really bad take. The adjustments they made to the game are more analogous to playing chess without En Passant.

[@TonmoyBorah7]

900 years of training? What, they training since jesus christ?

[@wilconbarro3469]

900 years worth of training, techniques like speedup, multiple simultaneous game, more computers/server plus each game the time is multiplied by number of ai playing

[@hyouzanren1846]

if this use in military(robot armies!) they can simulate how to wipe out humanity with the most efficiency billions of time!