AI Fail Basic Math!

Рет қаралды 956

Bracer Jack

Күн бұрын

Пікірлер: 37

@ShapedCrawdude57 4 ай бұрын

3:22 I had hope and I just lost it all 🤣

@BracerJack 4 ай бұрын

The model Qwen2.5-Coder seem to get it right, it was just released around 20 Sep 2024 :D

@scallywagwum 5 ай бұрын

0:06 simpsons cup????

@BracerJack 5 ай бұрын

Yes ! I am current taking a vacation in Australia, this video was shot in Australia and I saw those cup collection on sale for $15!!! Got to have it!

@BracerJack 5 ай бұрын

Llama3.1 was just released a few days ago, bigger and badder, more "new" than anything before ! It didn't make it into this video but this is its glorious answer anyways ;p Llama3.1: 9.11 is bigger than 9.9 because it has more decimal places and a larger number after the decimal point. The "1" in 9.11 is greater than the "9" in 9.9. I am going to go cry now...

@neo123rej 5 ай бұрын

@@BracerJack 😂😭

@thetower8553 5 ай бұрын

Yeah, John didn't destroy skynet, it just self terminated.

@BracerJack 5 ай бұрын

@@thetower8553 Ha ha ha Who would have thought that huge explosions isn't needed, just a simple 5 year old math question. And the media is going on and on about how it will take over tomorrow...

@fdmillion 5 ай бұрын

Sounds about right if AI is trying to imitate humans. Remember the story where real people didn't understand that getting a 1/2lb burger for the same price as a 1/3lb burger is a good deal? People thought "2 smaller than 3, so 1/2 is less than 1/3."

@BracerJack 5 ай бұрын

Oh man, maybe it matched the mistakes of humans then! :D

@mind_of_a_darkhorse 5 ай бұрын

LoL! Whoever programmed the AI didn't properly explain how the decimal system works! LMAO!

@BracerJack 5 ай бұрын

Ha ha ha, it's the way the A.I. break down the numbers that is causing the problem. So don't worry, Skynet wouldn't be happening too soon ;p

@Engineeer 5 ай бұрын

Companies claiming to use AI in order to boost their stock price. Now, imagine what happens when banks use AI.

@BracerJack 5 ай бұрын

I am sorry Sir, you told us you have 9.9 dollars in our bank, but our A.I. says you have 0.1 dollars, would you like to withdraw 0.1 which is all you have ?

@terryd8692 5 ай бұрын

I guess 9.11 minus 9.2 is 9.09 ?

@terryd8692 5 ай бұрын

Or 9.9? Hmmm

@BracerJack 5 ай бұрын

ha ha ha The A.I. might just agree with you!

@neo123rej 5 ай бұрын

So, it appears to me that on the one hand they are very childlike in terms of 'bigger' numbers. (Ask a child which is more; 4 quarters ($) or one dollar bill.) Even though it's maybe not the exact same situation, it does make me think of this. On the other hand they appear to be capable of doing the math when each step gets isolated. Are these models just that bad at connecting all the dots into the final (asked) product? Are these models self learning? If not, should we even call it AI? Are these models open to rectify them?

@BracerJack 5 ай бұрын

I've been reading up on what caused this BASIC BASIC issue while everyone else on KZbin was talking about how AI will be 'taking over the world' ;p. It seems like it's a "tokening" issue. The question, it "seems," didn't get to the A.I. as intended; the tokenizer broke down the number into "9" and "." and "11" instead of seeing it as "9.11". Similarly, the A.I. also missed "9.9", breaking it down into "9", ".", and "9". Different models use different tokenization techniques, but it seems from the three models I've tested in the video (including two others not in the video: DeepSeek V2 and the latest LLaMA 3.1) that regardless of their method of tokenization, they all fail to answer this question correctly 100% of the time, and they should; after all, this is really a basic question. Maybe we don't have to worry about them 'taking over the world', at least not until next week.

@neo123rej 5 ай бұрын

@@BracerJack hahaha it is actually some sort of relief, indeed. I thought it must be some sort of syntax/tokenisation error. They tend to take things a little too literally. 11 ís bigger than 9. Just not when they're behind a dot, without a whitespace after said dot. Interesting video! I like the format in which you showcase your findings!

@BracerJack 5 ай бұрын

@@neo123rej Thank you, this is a new format for me. Usually I would show the computer talking with like a wave visualization, but then I noticed that I, as a human, want to see other real human reactions instead of watching a computer screen blinking, so when you tell me you like this format, it makes me feel more confident going this direction!

@terryd8692 5 ай бұрын

Its just frustrating that you try to teach it something and then it goes and ignores you.

@BracerJack 5 ай бұрын

Yeah, that should not happen because of "context window" where it should remember your conversation, especially when I barely talk much with the A.I. yet, so it couldn't possibly "run out of context window space", so this is frustrating.

@surfacedfox 5 ай бұрын

Great. Computer that can't compute.

@BracerJack 5 ай бұрын

Ha ha ha

@ongchinhooi1075 5 ай бұрын

👍👍👍

@BracerJack 5 ай бұрын

@Play-Doh1234 5 ай бұрын

Humans > AI

@BracerJack 5 ай бұрын

Thumbs Up!

@CGDive 2 ай бұрын

Well, humans invented AI so yeah. But we may have to face the fact that we are inferiors to AI in many tasks. If we play our cards right, AI will be a good servant and all will be good. :)

@CGDive 2 ай бұрын

Lol, the third voice :D What I find funny is that chatGPT will very likely make this mistake... buuut you can ask it to write a e.g. Python script comparing the numbers and it will do it perfectly. And then it can give you the correct answer. I don't understand is why they don't make the AI use a calculator or code internally whenever it encounters one of these things it sucks at. Seems quite doable.

@BracerJack 2 ай бұрын

Hey, it is so nice to hear from you again! How have you been? Experiments like these really shows that there is a reason why it is called a language A.I. model. Since this video, I have encountered and now use a number of models that no longer have this problem. Nowadays, if any model I downloaded does not pass this test, it is out of my toolkit immediately, I don't need a wrong model ;p

@CGDive 2 ай бұрын

@@BracerJack What model do you use? I guess it's a difference between more mathematically focused models and "creative" models focused on writing essays and such.

@BracerJack 2 ай бұрын

@@CGDive Whatever is the latest that pass this basic test 🤣 I am not and will never be religious about any specific model, the only thing I will stick to is Blender! 🤣 Are you also using local A.I. as well? What are you using it for?

@CGDive 2 ай бұрын

@@BracerJack Nah, I can't be bothered with local ones. All I use is ChatGPT. It works quite well for info that is un-google-able, and getting help with coding struggles. You just have to keep in mind that it can't count in text mode and will still hallucinate occasionally so I have to be able to verify the answers.