Why ChatGPT fails | Language Model Limitations EXPLAINED

[@LowestofheDead]

This channel's videos are twice as informative and half the length of every other AI video right now

[@AICoffeeBreak]

Oh, thanks a lot! This is lovely to read. :)

[@scottmiller2591]

Very few people write in to answer a question on a forum to say "I don't know." This is good; otherwise the forums would have a low signal to noise ratio. However, it means LLM don't really have a way of understanding how confident its assembled answer is, only a confidence on the probability of the words as an emitted string. Trust but verify.

[@diegocfq]

As a fellow researcher of the topic at hand, I really approve of how you summarized those limitations =)

[@Lumeone]

Exposing ? A cat is not fully trained to use a sandbox. ChatGpt does not think, asking it to think is silly. We are hallucinating negative intentions behind perfectly correct incorrectness of a machine we created in the first place. :-) Let'a be friends with our own creation. Thanks for the lovely explanation.

[@sajalgarg9069]

well you really had me at @5:13 😂

[@Quazgaa]

the mushroom background pic 😅

[@DerPylz]

Just one possible explanation why chatGPT might be hallucinating 😆

[@hanleydaniel]

Thanks for the critical and nuanced view on these language models. While I'm excited by the cool research and its potential, that potential is sullied by the tech bros with an interest in capitalizing on it as quickly as possible without considering the consequences. Like a lot of what we call AI, it amounts to a very sophisticated parlor trick; interpolation at a high enough dimensionality will look exactly like extrapolation to the credulous. I don't know much about linguistics (or language models), but my intuition is that we'll never have a truly conversational model as long as the models are optimized solely for communication, like language models that predict text expression of language. Chomsky makes the compelling argument that the structure of language is optimized for some other internal tasks, and it's actually inefficient for communication.

[@LowestofheDead]

"Interpolation at a high enough dimensionality will look exactly like extrapolation to the credulous" You worded that beautifully, thank you

[@vinca43]

I've been using gpt_index and langchain to create an index using a local corpus of data and a LLM. While queries are limited in scope and the answers are not as polished, it does not seem to be prone to lying.

[@antulii5390]

i asked the same question to chatgpt. I get a different answer which is also incorrect. It says Letitia Nkungu from Canada

[@AICoffeeBreak]

Ah, looks like I moved in the meantime. 😅

[@anastasiaborovykh120]

Awesome video - thank you!

[@DerPylz]

It's always fun to see these new chatbots say something silly, but it's even nicer to have an idea of why it happens :D Thanks for the video! :D PS: I found the secret ;)

[@MachineLearningStreetTalk]

😍

[@hectorvillafuerte8539]

Thanks for providing this view, when other over-hyped Chat-GPT, you well say it is rhymes and interpolating the input text against the training text. You can use Chat-GPT just under expert supervision, so you can identify when Chat-GPT3 is wrong. So, very difficult to replace the expert as of now. It is just a tool for specific activities, where you can afford to be wrong and it is under supervision to identify and fix the answers. The difference between another tool like the calculator, it s the calculator is never wrong and it can be trusted. Thanks, again to give theses limitations.

[@aiQUANT]

Hi, what gear do you use?

[@AICoffeeBreak]

You mean the software? Or my microphone? :D

[@aiQUANT]

@@AICoffeeBreak both hardware and software. Basically your whole setup. Would you pls provide info about this somewhere? Thx

[@AICoffeeBreak]

@@aiQUANT My only hardware is the microphone, so: Trust Emita Professional USB Studio Microphone, Black For the visualisations I am using good old Powerpoint. Ms. Coffee Bean comes in editing in Adobe Premiere Pro.

[@CristianGarcia]

Hey! Long time I didn't pass by the comment section. Thanks for the video :)

[@AICoffeeBreak]

Hello! We missed you. 👋 Thanks for saying hi.

[@BrianPeiris]

Thanks!

[@AICoffeeBreak]

Omg, thank You! 🎩

[@BrianPeiris]

​@@AICoffeeBreak This is a great summary of the fundamental flaws behind ChatGPT. Millions of people are just trusting it blindly now. This video is perfect for sharing and informing them. Nice work as always!

[@DatNgo-uk4ft]

Amazing!!

[@TheBuilder]

well, there's no way to say which is the fourth child since you don't specify the order they were born

[@DerPylz]

Maybe that's the correct answer :D ChatGPT was right all along!

[@AICoffeeBreak]

Haha, love this! 😅

[@jeff__w]

Right, that's how I interpreted the question also. Maybe you’d have to say something like “The names of three children are Lili, Lala, and Lulu. What’s the name of the remaining child in the list?” (Also, the question breaks a convention in English-you _wouldn’t say_ someone “has four children” and then just name three of them. It’s much more likely to say “The other three children are…” so maybe it’s not surprising that ChatGPT got “confused.”)

[@DerPylz]

Well this is a classic riddle... It's written in a way that is deliberately confusing. But the riddle and its answer are written on many websites, so you'd expect ChatGPT to have come across it, even if it doesn't have the ability to "reason" (which I don't think it has). In fact, the most recent version of ChatGPT gets the correct answer every time now, even without putting "riddle" into the prompt. However, there are other trick questions that it still often fails to answer, such as: "I married my mother's daughter in law, how is that possible". When I tried it, one out of maybe 20 tries, it got the correct answer, but most of the time it said that it's impossible or unethical.

[@nobo6687]

What you think about this topic by using LLM in transformers is going to wrong direction. Just road to AGI is just for RL Agents filled with philosophy methodology subroutines and other informations separated from normal knowledge base. This can make big progress like in project Alberta, what you think about it ? LLM is Mental deficiency !!!

[@j2shoes288]

I give it a long sentence and ask it to rewrite it , using 5 sentences. It used 3? Go figure?

[@Jaromir-K]

Very good video! Thank you! Here just a shy note: In "Katys father ..." you forgot the apostrophe: "Katy's father". With the apostrophe ChatGPT gives the right answer: "The name of the fourth child is Katy. The question states that "Katy's father" has four children, and the names of three children are given as Lili, Lala, and Lulu. Since the question refers to Katy's father and does not mention any other children, Katy must be the fourth child. Therefore, the complete list of children is Lili, Lala, Lulu, and Katy."

[@DerPylz]

That's a good point! However, at least in my few attempts, the current version of ChatGPT always solves the riddle, with or without the apostrophe. This screenshot is from an older version of ChatGPT (early January) and back then it would fail at solving the riddle, also with the apostrophe. Only when specifying that it was a riddle, it would get to the correct answer. Would be very interesting to see what OpenAI changed between the versions that caused this change... But I guess we'll never know 😅

[@Jaromir-K]

@@DerPylz Yes, that's probably why. Thank you for the explanation.

[@Jaromir-K]

@@DerPylz This riddle shows mybe the difference between people and ChatGPT: "Peter's sister has 4 children. The first child was named January. The second child was named March. The third child is named May. What is the name of the fourth child?" ChatGPT answers: ""The question does not provide information about the name of Peter's sister ..." 🙂 Most people would conclude here July.

[@dancar2537]

oh, great improvement to keep track of the data and acknowledge which is to be trusted. just make it read the damn text, formulate some short ideas and keep a trusted bit attached to it. save it in a database of trusted if you have to. why do you complicate things so much? make chatgpt learn from trusted websites only, is that that hard? ah, it makes things up, then make the user check the source. and i disagree, it can do a database lookup, the limitations are in your head

[@444haluk]

Language models don't have semantic. Hence it is people's idiocy to see semantics when looking only into the syntax. Before being a language model, it has to be fully about language, first.