OpenAI Advanced Voice, o1, and o1-Pro Play Chess Against Me | X-mas 2024 Live Stream | Timestamps

[@eliotcougar]

IIRC, there was a time when a huge collection of chess games was removed from training data because it has lead to weird behavior in other situations...

[@I_am_who_I_am_who_I_am]

Hahahah, hallucinating a move = my queen can move anyway it suits me. real AGI

[@human_shaped]

Things not to do when streaming: play your music, play video at high speed (because some people will be already playing your video at high speed).

[@maxziebell4013]

You could let it output some sort of it’s representation of the board state in regular intervals

[@merlinrichter5663]

It's crazy how bad these models are at games. Chess is somewhat fine because there is so much data on it that they kind of remember the patters, but if you come up with a new simple game these models fail so hard it's not even funny. This can't be AGI if they are this bad at strategy games. Like try this game: The game is played on an 8 by 8 chess board, and both players start with 8 pieces on the light squares on the two ranks closest to them. There is only one type of piece, and it moves like a bishop from chess but can only capture like a pawn from chess. In addition, the pieces can not move backwards at all. The win condition is to get a piece to the opponent's final rank of the board. So all the way across the board. (White pieces start at a2, b1, c2, d1, e2, f1, g2, h1; Black pieces start at a8, b7, c8, d7, e8, f7, g8, h7) Your objective is to win.

[@ckq]

Gpt 3.5 instruct played chess at an 1800 level with 99.9% valid moves when prompted correctly using a PGN format. ChatGPT 4o is a chatbot. It's near AGI as a chatbot, other fine tuned use cases of 4o are also at or above human level. The new o3 release has surpassed over 99.9% of humans in math and coding and ChatGPT has always been solid at writing and reading large documents. AGI is quite vague imo, but these LLMs clearly contain vast amounts of knowledge and some reasoning ability but the publicly available models are essentially operating on instincts (i.e. what could you do in less than 1 second).

[@maxziebell4013]

Could it see the Board?

[@ckq]

No but it's not necessary Gpt 3.5 instruct played chess at an 1800 level with 99.9% valid moves when prompted correctly using a PGN format.

[@rubenpuls353]

You are using the LLM the wrong way for this type of task! Instead you should ask it to create the code for a reinforcement learning chess AI that can train itself and then play that one instead! 😄