Scrabble's Human vs AI Showdown

[@Cats83747]

Wow that loti sequence is insane, the AI vision to see that your opponent could draw Qats and score a lot, and then blocking it while still allowing you to score with it if you draw it with Qadi!! I could never in a million years lol

[@Annihilator_5024]

qats more like cats!

[@penumbra-scrabble]

and Will explained it so clearly. Fascinating to follow.

[@stylishlyheartless979]

The fact that the scrabble Ai was that good that long ago is impressive

[@chammy2812]

It should be noted that a software in 2007 was not using Machine Learning. It was an algorithm designed by humans to produce this result. AI in this sense no different than NPC’s in video games responding to inputs in a specific well defined way. If you are familiar with Chess Engines, this is much closer to something like Stockfish than Leela. Stockfish evaluates positions based on human understanding of pieces and looks down the future for potential board states. Leela was programmed to only know what moves are legal and by playing itself over and over again determined it’s own evaluation function to maximize winning.

[@stylishlyheartless979]

@@chammy2812 i guess that’s true, another thing thats interesting is it’s noticeable that some of the moves that the scrabble bots make seem much more ‘human’ than some of the odd moves you get from a chess engine. But that’s more likely down to the differences in the games. But I guess chess is ‘solved’ and in a perfect game always results in a draw while scrabble has many other factors that stop that from being the case.

[@chammy2812]

@@stylishlyheartless979 chess is definitely not yet solved. The main difference you are discussing is information. Chess is a full information game (there is no information your opponent has that you do not) whereas this obviously isn’t the case in scrabble. In a hidden information game like scrabble you are playing with probabilities of what your opponent has and what you will draw, because of this everything is much more fuzzy. Something that may be a fantastic move 95% of the time but may be the worst move if the opponent has a certain set of tiles that will lose you the game instantly. This added complexity drastically changes the problem to find an “optimal” play. As sometimes the optimal play may put you in a lost position.

[@stylishlyheartless979]

@@chammy2812 i meant closer to being solved, since as you pointed out, there’s nothing in chess that you know that your opponent can’t know

[@Blizz4rd]

Not all software is AI.

[@AlexDings]

People should know that we are still a long way from optimal play by engines. There's a ton of concepts that haven't been implemented yet, at least not in a dedicated fashion. The engines are great tools as they are and of course LOTI is a fantastic move, but we haven't even seen truly advanced Scrabble "AI" yet. If there are any machine learning experts in Will's audience who want to apply their skills to Scrabble, that's still an unexplored area!

[@pgck]

Interesting that 2006 was around 10 years after computers reached elite levels at chess and 10 years before they reached elite levels at Go.

[@irakyl]

It's great to see how it takes all its opponent's and its own potential future moves in mind while making its plays, except it doesn't come to those conclusions with thought or language. You know Will you're sort of the Elvis Presley of blackjack.

[@wanderer15]

I look forward to these comparisons all week now

[@dentonyoung4314]

A spectacular final game to cap things off. Even if Quackle is now outdated (I don't know if improvements have been made to keep it current or not), it'll always have that moment.

[@bvoyelr]

You missed an excellent opportunity to shout out Mack Meller's grandmaster vs AI series on his channel! He's playing against best bot in one series (and it's beaten him something like 40-60 in the past), and he's also got a blitz scrabble series going against HastyBot. It's pretty interesting to consider that the best AI beats the best grandmasters at pretty close to that 3-out-of-5 spread even today. Once you are at the level where you pretty much play optimally, the rest is up to the tile gods (and keeping your cool under time pressure)

[@axcertypo]

Mack isn't the best player in the world, and there has yet to be an engine that is proven to be better than the best of the best. We're not yet at the point of Deep Blue vs Kasparov, or AlphaGo, for Scrabble. I think we could use a new human vs computer showdown to really test whether the newest engine technology can really best humanity. If there was enough incentive to maintain the highest possible level of play, top GMs would probably improve their calibre to face off against AI.

[@ohtani2024]

BestBot does not come to Quackle's level. It occasionally made dumb move while Quackle rarely made suboptimal move

[@axcertypo]

@@ohtani2024 BestBot uses Macondo and Macondo is better than Quackle. And it's still improving. Not sure where you're getting your information, but it is inaccurate.

[@wanderer15]

I could've added a bigger plug for sure (he does make the briefest cameo at the very very end of the video, but only people who know him already would recognize it).

[@ohtani2024]

@@axcertypo not sure but what i am sure is that quackle does so much better in leave valuations. Macondo overvalues vowels a lot

[@kb27787]

Still, if the human player could've sniff out the "qadi" plan and blocked over there... Quackle might just end up Q-stuck instead.... I would've considered such options over "loo"--since we drew the last u, we are not so afraid of drawing the q, but if the bot draws it, it's likely insta-win since there is no way to fit qat or qi anywhere on this board anymore.

[@axcertypo]

David indeed said that he didn't think too hard about the position, and if he had just taken the time to consider what LOTI was doing (rather than assuming it was 100% an attempt to draw a bingo in the spot LOO blocked), he would have sniffed it out.

[@EvanBerofsky]

loved that tourney. it felt daunting to play the bots, so i was happy enough to beat maven (while getting crushed by quackle). if only i would've won one more, i might've had a shot to reach the final (dave was gibsonized in rd16) and have the honor of getting swept by an inanimate machine.

[@semnejlepsi8983]

seeing someone who participated in a tournament that took place days before i was born comment on a video about said tournament almost 18 years later is a little surreal

[@EvanBerofsky]

​@@semnejlepsi8983don't worry. you'll get used to that feeling when you're older. :)

[@robinier]

Awesome video as always. Also if it helps: hemin [HEE-min] (rhyme "demon") - same prefix as /hemoglobin/. Your pronunciation is articulate, confident and assertive-it's never distracting in your videos, and I like hearing you say the words. Really just an "if it helps" from someone who is way too nerdy about words. Looking forward to the next video!

[@wanderer15]

Of course, it has to do with blood so I should've realized that! Thank you

[@Jkfgjfgjfkjg]

In Scrabble, the words have no definitions and no pronunciations.

[@yellowdicesam]

I was just rewatching this and wondered, what would happen if David had played a phoney in the finals? Wouldn’t Quackle always have the edge on being able to challenge it, or would the protocol be different with the fact that volunteers were operating the AI

[@pokejoey1]

Another great video! Would love to see a video covering the history of AI/Computer Scrabble!

[@craiglarimer1173]

Fabulous video. That Loti was was incredible !

[@MisterFeline]

Question Will: do we know how good Grandmaster Zoey from Scrabble Go is compared to the other bots out there? I've managed to best her a few times on that difficulty, but I don't have much else to compare it to. Is the bot working with every valid word and 100% optimal strategy on that difficulty, or does Scrabble Go simply not have a bot that goes that high?

[@axcertypo]

Grandmaster Zoey simply plays the highest scoring move every turn. Nothing close to top level Scrabble computer strategy.

[@MisterFeline]

@axcertypo That makes a lot of sense in hindsight, considering the questionable endgame decisions I've seen her make, and the fact that she never tile swaps even when clearly saddled with a terrible rack. The QADI setup exhibited in this video seems like a level far beyond what Zoey would ever pull

[@wanderer15]

Yes, GM Zoey is all about scoring the most it can on the current turn. You'll see a lot of moves where it burns the blank for just a small number of extra points, which is rarely a good idea.

[@davidmoore1253]

How much progress have engines made since then? 2006 is a very long time ago in software terms, yet as I understand it the engines still aren't much better than the best humans?

[@wanderer15]

These projects are mostly a labor of love by players from within the tournament community (or really talented people who happen to take an interest in Scrabble AI), so Quackle has been the gold standard for quite a while now. Woogles.io's BestBot has made some solid improvements, though, and another engine named Elise had some cool features like "heat maps" predicting next play locations that haven't been duplicated.

[@Simon-T.]

This has got me thinking about the opposite of an AI against a top player, me. I'm a decent to average home scrabble player and a decent to average home chess player. I imagine that if I played a million games against a top chess player I'd never win but if I played enough games and the tiles that came out were so stacked in my favour, is there a chance one time out of a million I could beat a top scrabble player? Or would their word knowledge and word play be enough to make up whatever advantage I had everytime and I would literally need them to draw only vowels all game and them to then quit for me to ever win? Great video as always.

[@jahonasnapkins]

you’d likely win a lot more than one in a million! even with very moderate skill you could beat the most skilled opponent on any given day

[@wanderer15]

My mom, a solid home player but not someone who would ever study the dictionary, beat Quackle once after about 200 tries. I think many casual players would be surprised how moderate the effort would be required to get to that “every once in a blue moon” win rate even against super elite Scrabble opponents.

[@Simon-T.]

Ohh, that's exciting!

[@Joslinmeister]

Shoutout to Mack Meller's Human vs AI series!

[@plebcrabslayer]

*ROAD TO 50K!!! LET'S GO!!!* 🥳 3:41 I wouldn't assume that he might have another S _just_ because he played one; I would simply think he played the S to hit the DWS-nine points (plus opening up the S-hook opportunity for his opponent) versus 20 points (and no setup)? Seems straightforward to me (a non-expert). Why is the expert player analysis that his playing that S makes it likelier that he might have another S? Genuinely curious. Edit: 6:26 I suppose the chance of David's having OATIEST is so low as to warrant not blocking that particular TWS (and given the difficulty of landing the top-right TWS, which it actually could at that moment). Edit 2: 9:20 The even more impressive thing is the translation from those thoughts-about endgames, pre-endgames, setups, etc.-into a programming language. Just the ingenuity to turn ideas into if-then-else statements (and all the other statements required) is an astonishing feat; a triumph of our beautifully evolved primate brains. 🧠⚡🙌

[@wanderer15]

For me, the reason I’d be wary after AHOLDS is that (Y)ODH was available from the Y saving the S for the same score. Normally, an S is too valuable to spend if you have a move that scores even close, let alone equally. However, YODH is also a tough word to spot, and even a strong human player could miss it from time to time. Quackle went ahead and took the aggressive approach anyway and it surely paid off.

[@plebcrabslayer]

@@wanderer15 Ohhh! I see. Thank you! :D

[@TheRealGregariousGreg]

I want to watch Nigel Richards play Countdown

[@Superb17C]

9:47 "Frivoleraugoury"?! What video is that from?

[@wanderer15]

A stream of me playing Scrabble for Game Boy on emulator and realizing it doesn't challenge every phony word even on master level!

[@astanix]

That was a great series of games for sure!

[@BunniBuu]

It would be interesting to see if computers could analyze which player got "luckier" in a game. I'd love to see the statistics of who the the "luckiest grandmaster" is, for example

[@roelliesafari9988]

dropped to my knees and cried when Quackle won. Humanity has lost

[@glowstonelovepad9294]

Loti is what you would call a "Nigelic" play.

[@PelumiOTE2B]

Brilliant as always

[@alifnafili5153]

holy crap i am a scrabble word

[@ryanjohnson3615]

This has "John Henry" vibes.

[@headlessnotahorseman]

Has a scrabble AI ever deliberately picked an incorrect word that might not be challenged?

[@wanderer15]

Quackle isn't programmed to do that. However, Brian Sheppard had prepared Maven to be able to play "plausible" phonies, and it even got away with one against one of the stronger players! It wasn't enough for Maven to outperform Quackle, though.

[@Jkfgjfgjfkjg]

@@wanderer15 What was the phony?

[@jaolaugh]

@@Jkfgjfgjfkjg POTAGER (acceptable in CSW but not in TWL, which was the dictionary used for this tournament). Maven's opponent was David Boys.

[@jaolaugh]

Source for the above was just my own memory of hearing it from both Brian Sheppard and David Boys while at the tournament, but I found David mentioning it in a post to the old CGP mailing list. excerpted below.... And believe it or not, I also lost my game to Maven mostly because I did not challenge its Sowpods word POTAGER# with three in the bag. Brian S told me that Maven was juiced to play phonies in this tournament, but I just laughed that off. He who laughs first, laughs worst. So that's pretty lucky, really. I make three critical dictionary mistakes (not counting the 2 or 3 # bingos I got away with) and it only cost me two games, only one of which mattered. As for not blocking QADI, that was just plain word blindness and overeagerness. Once I saw that QAID did not play I eased up, and thought I had it in the bag. [For some reason I was 100% sure that Quackle had the Q] I suppose it's only fitting that the deciding factor in the final was an only too human error! Too bad it wasn't some unique flash of human insight to win ... -Dave

[@jaolaugh]

The other phony-adjacent anecdote I have from that tournament is my dad unsuccessfully challenged one of Maven's plays. AFAIK potager# was the only phony played by Maven in Toronto.

[@gsas3012]

So whats the current best bot today then? I'm curious if Quackle was over thrown

[@wanderer15]

Woogles.io’s BestBot has made some solid improvements over Quackle - it’s just now starting to become more widely available to play.

[@lildmckay]

Did the commentator have access to live analysis or did she see that qats/qadi sequence too?

[@wanderer15]

I think all of the attendees of the tournament were watching the finals together and collaborating on ideas for candidate moves. Organizer Tony Leah told me that at one point, the need to block QATS was suggested, and shortly after, the possibility of instead trying to draw QADI was floated as well, and Robin synthesized those two ideas.

[@jellomochas]

GINZO was expurgated

[@oskain]

At 5:40, was vintaged at G6 not playable here?

[@Jkfgjfgjfkjg]

They were probably playing with the TWL dictionary.

[@onyonable]

Now let's see alphazero vs quackle

[@Floriaka]

can't tell me that AI opponents in games like this don't get tilted cause it's ALWAYS when they are losing and on the brink of being destroyed in these games.. they will perform better by at least 45% and use more CPU and RAM to overclock itself mildly to make more impressive yet bold call outs. I should know, I play against AI opponents in games similar to this all the time

[@FaliggetFTW]

Cool video! Have the tools improved with more recent iterations since 2006? Is the top tool still called Quackle?

[@galoomba5559]

Macondo is a new engine in development, and afaik it's marginally better than Quackle

[@poke7661]

hi galoomba

[@Loffs]

​@poke7661 hi poke

[@literally_why8999]

I think Quackle is still better than Macondo, for the sole reason that Quackle has a proper interface and a proper "new game" function while Macondo does not.

[@wanderer15]

Woogles.io's BestBot, which is powered by the Macondo Scrabble engine, represents a small but meaningful improvement over Quackle. Another engine called Elise pioneered some features like predictive "heat maps" of upcoming plays on the board which haven't been duplicated by any other Scrabble AI.

[@minaballerina]

omg just played ziti in a game and then it shows up in this video

[@EltWilder]

There’s a lot of words.

[@ciscoortega9789]

And this was back in 2006? Good grief! I'm scared

[@louisng114]

Video idea: Quackle vs ChatGPT

[@alimbis]

chatgpt is not good at playing games

[@louisng114]

@@alimbis But it is able to cheat

[@Cats83747]

That ChatGPT vs Grok scrabble game revolutionized the game fr

[@alimbis]

@@louisng114 ah yes you are correct. i think chatgpt would actually have a chance as long as quackle can't challenge

[@qqw743]

At 3:13 we see the word "JEWS" at center. This is not playable in my cheapo app, and is obviously a word that would generate controversy. What is the definition in the prevailing dictionary at the tournament, and does anyone else think this ought not to be in the lexicon? Game dictionaries have in recent years eliminated many identity terms that might give offense, quite rightly in my view. How is this still a thing?

[@wanderer15]

It’s an issue we’re still grappling with. This is a game from 2006. Since then, words with primarily discriminatory definitions have been purged from the official lexicons (this happened in 2020-2021). That being said, there are pockets of players who have resisted these changes and insist on playing with unexpurgated word lists. It’s a controversial topic, but I’ve been considering making a video about it for some time.

[@Jkfgjfgjfkjg]

@@wanderer15 You could easily make a 3 hour documentary about it.

[@qqw743]

@@wanderer15 I'm sure all the angles have been considered, I have nothing to add. I'm just curious how it shakes out.

[@evandonovan9239]

But Jews is a regular ethnic term, not a racial slur?

[@qqw743]

@@evandonovan9239 No, as an ethnic term it would certainly be disallowed as it would be a proper noun. As a verb it has historically meant something like "To cheat" or "to bargain down sneakily" and is widely considered offensive.

[@socksygen]

Hamburger

[@EchoHeo]

waow

[@AOOA926]

At 8 am sharp

[@alimbis]

4:59 isn't it a hebrew letter, not an arabic one?

[@Willwantstobeawesome]

Hebrew is alef, Arabic is alif

[@alimbis]

@@Willwantstobeawesome ah that makes sense. i thought since hebrew often has several correct transliterations in english it might be something like that. like ch and kh

[@AOOA926]

6 minutes ago!!