ChatGPT can't do math...

[@TomRocksMaths]

🌏 Get NordVPN 2Y plan + 4 months extra ➼ nordvpn.com/tomrocksmaths It’s risk-free with Nord’s 30-day money-back guarantee! ✌

[@lio1234234]

These models don't do any background reasoning (essentially thinking before answering). Definitely recommend trying out o1-mini which does do this. Currently o1-mini does better at maths than o1-preview, but o1-preview has better general knowledge reasoning. o1 when it's finally released should be just downright better than o1-mini at everything including maths. Highly recommend trying some of these out on that model :)

[@IsZomg]

This uses ChatGPT 3 which is outdated. The latest free tier model is ChatGPT 4o and the top model is o1. Both of these are much better at math that ChatGPT 3 which is TWO YEARS OLD now.

[@nofilkhan6743]

Chatgpt doing black magic instead of geometry.

[@asiamies9153]

It sees the world differently

[@delhatton]

@@asiamies9153 it doesn't see the world at all

[@alexandermcclure6185]

@@delhatton that's still different from how humans see the world. 🙄

[@obiwanpez]

“Narn, flëmadoch, F’Tadn ygsorath, loqgawtygsdryr!”

[@obiwanpez]

Seems to be a Deep Language learning model…

[@narutochan620]

ChatGPT invoked the Illuminati on the Geometry question 😂

[@bekabex8643]

the geometry drawing it produced had me gasping for air 🤣

[@Eagle3302PL]

The problem is that chatgpt or any llm, they are not applying formal logic or arithmetic to a problem, instead they regurgitate a solution they tokenized from their training set, and try to morph the solution and the answer in the context of the question being asked. Therefore, just like a cheater, it can often give a correct result confidently because it has memorised that exact question, sometimes it can even substitute values into the result to appear to have calculated it, but in the end it's all smoke and mirrors. It didn't do the math, it didin't think through the problem, that's why llm's crumble when never before seen questions get asked, because an llm has no understanding, only memorisation. Also llms crumble when irrelevant information is fed alongside the question, because the irrelevant information impacts the search space that's being looked at, so accuracy of recall is reduced. LLM's do not think, they do not process information logically, rather they process input and throw out the most likely output, and use some value substitution in the result to appear to be answering your exact question. LLM's cannot do mathematics, at best they can spit out likely solutions to to your questions where similar or those exact questions and their solutions have been fed to them in their training set. An LLM knows everything and understands nothing.

[@mattschoolfield4776]

I wish everyone understood this.

[@Nnm26]

Try o1 brother

[@mattschoolfield4776]

@@Eagle3302PL it's even in the name Large Language Model. I don't get how anyone thinks they have any understanding

[@IsZomg]

New o1 model can 'show its work' and reason in multiple steps. If you think LLMs won't beat humans at math soon you are mistaken.

[@CoalOres]

They _might_ process information logically, we actually don't know. Since they generate it word by word (or token by token), after enough training it might have learned some forms of logic because it turns out those are very good at predicting the next token in logical proofs. Logic is useful for many different proofs, just memorizing the answer is only useful for a single one (i.e. it would be trained out pretty quickly); this doesn't guarantee it knows logic, but it makes it plausible. It is a common misconception that these programs work by searching the dataset, 3Blue1Brown has an excellent video series I would recommend that shows just how complex its underlying mechanics actually are.

[@shoryaprakash8945]

I once asked chatGPT to prove that π is irrational. It gave back the proof of √2 problem, discuss squaring the circle problem and in final conclusion wrote hence π is irrational.

[@RFC3514]

Wow, it independently (re)discovered the Chewbacca defence!

[@yagodarkmoon]

Question 3 the geometry one ends up much better when you give it the graph with the instructions. I tried it and got a much better result. To do this I used the snipping tool to make an image of both the question and the graph. Then I saved it to desktop as screenshot.jpg and dragged that into the ChatGPT window. It read them both fine.

[@Pedro-op6zj]

after using snipping tool you can directly Ctrl C + Ctrl V in chat gpt

[@toshiv-y1l]

20:42 power of a point is a basic geometry theorem...

[@gergoturan4033]

I've only watched up to the first question so far, but I came up with a different solution that's interesting enough to mention. Another way to think of the problem is dividing the characters into 2 subsets, one of them is the characters that were typed 1 late and the other is all the others that weren't. If all the characters are different, these 2 sets give enough information to reconstruct any possible spellings. Therefore, we just need to count all the ways to make these subsets. We know that in an n character long word the last character can never be 1 late. So we only have n-1 letters left to work with. [n-1 choose k] will give us a k sized subset. To get all possible subsets, we need to sum up for every case of k. [sum(k = 0..n-1)(n-1 choose k)] This is the n-1st row of Pascal's triangle. We know that the sum of the n-1st row of it is 2^(n-1). The word "OLYMPIADS" has 9 letters, therefore the answer is 2^8 which is 256.

[@bigbluespike5645]

I asked the o-1 preview the geometric question and it approached the problem very analytically - by setting up a coordiante system, finding the points X,Y and Z by solving equation systems for lines and the circle and finally showing BZ is Perpendicular to AC using vectors and dot product of BZ⋅AC. I can't fully evaluate whether it's perfect, but I still think its solution was way better.

[@bornach]

@@bigbluespike5645 How does it do on the other problems that ChatGPT made a mess of?

[@bigbluespike5645]

@bornach I didn't test yet, but i'll update you when i do

[@JavairiaAqdas]

we can add shape through the attachment icon right in the left corner of the prompt box, just take a Screenshot figure and put forward like this.

[@abdulllllahhh]

On an unrelated note, I remember sitting this BMO paper last year and struggling but enjoying it. I recently started uni in Canada and have been training for putnam, and now I’m looking back at these questions both cringing and being proud at how much I’ve grown in just a year, how I’ve gone from finding these questions tough, to now being able to solve them without much struggle. This is what I love about maths, how I can always continue with just some practice. P.s, great video Tom, really enjoyed watching it.

[@DandruffDave-hh7kr]

Thanks for coming to my school (I was one of the year 10s), the presentation was very interesting!

[@tymmiara5967]

It becomes obvious that the language model is essentially a separate module to the image generator. I bet even if the solution had been flawlessly found, the drawing of a diagram would be completely bonkers

[@Hankyone]

Cool video and all but are you aware of o1-mini and o1-preview???

[@TomRocksMaths]

yes of course. the plan here was to use the free version as it is what most people will have access to, so I wanted to warn them to be careful when using it.

[@IsZomg]

@@TomRocksMaths 4o is the best 'free' model, not ChatGPT 3

[@9madness9]

What to know if you could test with Stephen Wolfram add in! To see how good the addin makes chatgpt at maths

[@devilsolution9781]

​@@9madness9are there plugins???

[@IsZomg]

@@TomRocksMaths ChatGPT 3 is TWO YEARS OLD now lol you didn't do your research.

[@Twi_543]

When I did this practice paper I got the same thing as u for question 2 about how the difference either increases or stays the same at each point, so if it is 1 at 2024 then it must be 1 at 1 bc the each term is an integer but I was confused when looking at the mark scheme so wasnt sure it was right. Thanks for explaining the mark scheme it helped me understand it better😁👍

[@KoHaN7]

Hi Tom, I really like the video! 😀If you want to see a good performance in logic and reasoning from GPT, using GPT o1-preview seesms to be the best at the moment. It would be interesting to repeat the same with that more advanced model. It thinks before answering which allowes it to check its own answeres before saying the first thing that comes to mind

[@TomRocksMaths]

ooooo this is exactly the kind of thing I was thinking it needs!

[@Justashortcomment]

Why didn’t you use OpenAI’s new model o1, which is designed for these types of problems? Would be interesting to see the performance of o1-preview with these.

[@MorallyGrayRabbit]

25:43 Obviously it just used the power of a point thereom

[@dmytryk7887]

In Q1 there seems to be an error in chatgpt's explanation. For example, it says "D" must be in position 7, 8 or 9 but "DOLYMPIAS" is a valid misspelling...every letter is one late, except for D (early) and S (correct).

[@SgtSupaman]

Yeah, its mistaken assumption that a letter must be within one of its original location (in either direction) actually limits the number of possible permutations to 55. So, it definitely didn't properly pair up its explanation with its answer.

[@coopergates9680]

You caught it first. I'm surprised GPT could pull out the correct number while misunderstanding the terms along the way.

[@Jordan-gt6gd]

Hi Dr, Can you do a lecture series on any math course you like,similar to the ones you did with calculus and linear algebra.

[@cheesyeasy1238]

0:24 maybe i'm too panicky but the mere mention of the MAT sends a shiver down my spine... hoping for a non-disaster tomorrow 🙏

[@lupusreginabeta3318]

1. The Prompt is definitely upgradable 😂 2. You should use the new preview model o1 it is quite a lot better than 4o

[@jesseburstrom5920]

First you did not use o1-preview would be more interesting, also 0-shot is something humans do not be limited 0-shot which means I have to give my first thought in university exam, no you have typically 1+h per question. So do test with o1 and give natural critique not leading might give better results. Like just try to convince it to be better to look into it's own arguments. It would be simple to do, better results hmmm no answer there from me but yes when they become better when? Great channel Tom!

[@thecon-artist8548]

Hi Dr Tom! I am a fan from Singapore and I would like to inform you about the Singapore A level, which is known to be harder than the IB HL maths paper. I think that you would probably enjoy doing that paper

[@ramunasstulga8264]

Nah jee advanced is easier than IB HL, lil bro 💀

[@thecon-artist8548]

@@ramunasstulga8264 If you are so retarded that youre unable to even do both paper before making a valid criticism you shouldn't even comment. I find it baffling someone like you is even watching this video.

[@hondamirkadirov5588]

Chatgpt got really creative in geometry🤣

[@gtziavelis]

19:35 LOL, the diagram drawing looks like equal parts 1) M.C.Escher, 2) Indian Head test pattern from the early days of television, 3) steampunk, 4) Vitruvian Man. It's all sorts of incorrect, its confidence is a barrel of laughs, but it's lovely to look at and fun to contemplate how ChatGPT may have come up with that. My favorite part is the top center A with the additional 'side shield' A, and honorable mention to how the matchsticks of the equilateral triangle have three-dimensional depth and shadows.

[@kaisoonlo]

Try using GPT o1 preview. Unlike GPT 4o, it excels at STEM questions due to its "advance reasoning"

[@jppereyra]

Our jobs are safe, ChatGPT can’t do maths at all.

[@Justashortcomment]

Hey Tom, Thanks for the video. BUT! ;) OpenAI will release the full o1 “reasoning model” soon. Currently we only have access to the preview. It would be fantastic to see a professional mathematician evaluate its performance, ideally with a problem set that isn’t on the internet or in books or has only been put on the internet recently.

[@Neuromancerism]

Yes, you can copy the diagram. Thats no issue at all. You can just copy and paste an image into chtgpt (or click the image button) as long as you have access to full 4o, after a few prompts until you pay accordingly itll downgrade to 3 though.

[@nicoleweigelt3938]

Looked for something like this after I got frustrated it was getting algebra and calculus wrong 😅 Thanks for the vid!

[@CosmicAerospace]

You can input images onto the prompt by copy pasting a screenshot or 🥇 lacing an attachment onto the prompt :)

[@djwilliams8]

I found it works a lot better when you upload a photo of a question. Just press a screenshot or snippet tool and paste.

[@TheMemesofDestruction]

I have found the WolframGPT is better at Maths than the standard ChatGPT. That said both often require additional prompting to achieve desired results. Then again it could just be human error on the prompter side. Cheers! ^.^

[@loadstone5149]

Tom is not locked in. Every uni maths student knows if you take a picture of the question it will always give you the right answer

[@micharijdes9867]

Facts, but for some reason it has a really hard time with topology

[@patrickyao]

Hey Dr Crawford - thank you for your video and insight. It seems that you are using the basic GPT4 model to solve these BMO questions. There is a different model ChatGPT provides called the o1-preview, which is specifically designed for complex and advanced reasoning and solving difficult mathematical questions like this. If you use the o1-preview model, it would take way longer time (sometimes even more than a minute) before giving you a response, and it thinks in a way deeper way than the model you have used here. With that model, I've tried feeding it questions 5 and 6 on the BMO1 paper, and it could solve them perfectly. Therefore I would encourage you to try again with that specific model. I do believe that you have to have ChatGPT subscription to access that model, but I think that they are going to release a free version of that model. Anyways, thanks you so much! P.S. It would have been better if you simply uploaded a screenshot of the question as diagrams could have been included, and ChatGPT would be able to read the question from the image (probably better than it being retyped with a different syntax)

[@GodexCigas]

Try using GPT-o1-preview - It uses advanced reasoning.

[@Exzyll]

Yeah I was gonna say that he will be shocked

[@samarpradhan3985]

Me who also can’t do math: “Maybe I am ChatGPT”

[@Rubrickety]

I think Numberphile did a video on the Power of the Point Theorem and the counterintuitive properties of the Perpenuncle.

[@dan-florinchereches4892]

The second problem reminds me of euclids alogithm and most notably the chinese usage of such method. If you got 2 vessels of volunes a And b the lowest volume which you can measure is the greatest common divisor of a and b. By using this logic and the fact that any ai and ai-1 are some linear combinations of a0 and a1 it folowsthat gdc(ai,ai-1)=gcd(a0,a1) henceif they are consecutive they both have gcd of 1.

[@obiwanpez]

19:50 - “Wull there’s yer prablem!”

[@rostcraft]

Power of a point is actually real and while I’m usually bad in geometry at olympiads, some of my friends used it several times.

[@deinauge7894]

ok. to use this at the point Z you need two lines through Z which cut a circle in 1 or 2 points. Say this circle is centered atB with radius BA. You can conclude: ZX*ZY = ZB*ZW (W is the point where ZB coincides with the circle) Since ZW=ZB-BA we get ZX*ZY = ZB*ZB-ZB*BA. This looks almost like what chatGPT wrote. I'd give it a pass 😂

[@francoislanctot2423]

You should try o1 Preview, which is supposed to be very good at logic and reasoning.

[@am01am]

The latest matrix calculation equations hasn't been added in to the chatgpt. It's using old matrix calculations. But I guess it takes a bit to actually move from one to another science. It is difficult for humans to explain to humans, even more tricky maybe to explain a computer.

[@Smashachu]

You didin't use the newest model 1o, which is significantly better in every way at mathematics.

[@komraa]

That image had me dying for 2 minutes straight😂😂

[@ootakamoku]

Would have been much more interesting with o1 preview model instead of 4o

[@TheDwarvenForge05]

ChatGPT has, on multiple occasions, told me that odd numbers were even and vice versa

[@snehithgaliveeti3293]

Tom can you try the TMUA entrance exam paper 1 and 2

[@nightskorpion1336]

Yesss I've been asking this too

[@massiveastronomer1066]

I have this test coming up on the 20th, these questions are brutal.

[@SayanMitraepicstuff]

You did not use the latest o1 series of models. I was trying to search for where you mention which model you were using - couldn’t find an exact response and you have cropped the part where it mentions the model and also haven’t shown the footage of the answer generation - which would give away the model you were testing. O1 can not generate images - which was the give away. Do the same tests with o1-preview.

[@blengi]

yeah this all moot if not o1 which is openAI's first reasoning model, all the others LLMs are just level 1 chatbots by openAI def

[@OzoneTheLynx]

I tried getting Gemini to draw its 'solution' to 3) and it responde with the link to the solutions XD.

[@Henrix1998]

The newlines might confuse it slightly

[@Anokosciant]

power of points is a niche set of tricks for olympiads

[@jursamaj]

On the unreliable typist: I feel ChatGPT mischaracterized the possible positions of letters (or I'm drastically misunderstanding the rules. In steps 1 ^2, it said 'S' can only be in the last 2 positions. But 'SOLYMPIAD' appears to fit the rules ('S' is way early, and each other letter is 1 late). It may have gotten the right answer, but it's argument was flawed. On the polygon: Step 1 is false. Convex with equal sides does *not* imply the vertices lie on a circle. A rhombus is convex and all its sides are equal, but the vertices are *not* on a circle. This alone invalidates all the rest of the proof, which relies on the circle. Also, in step 4 part 'n=5', the 3 diagonals do *not* form an equilateral triangle. Nor would it "ensure … a regular polygon" if they did. The important thing to remember is that LLM "AI" isn't *reasoning* at all. It's just stringing a series of tokens together based on how often it has seen those words strung together before, plus a bit of randomness.

[@dominiquelaurain6427]

@20:00 : as an euclidean geometry addict...I like the diagram a lot ;-) "Power of a point" is of course a not accurate definition. I know the "power of a point with respect to a circle" only. "Please draw a sheep". I tried some months ago to get a generated picture but no way. They must be taught the Compass and Ruler techniques.

[@vasiledumitrescu9555]

I use it to study some theoretical stuff, it’s good at explaining theorems and definitions and producing good examples. It can even prove things pretty well, because it’s not actually doing the proof but just taking it from its database and pasting it to you. Of course it makes mistakes now and then, but they’re so dumb they’re easy to catch. And by “using it” i mean: as i’m studying from my notes or books i ask from time to time chatgpt things in order to understand the mind bogglingly abstract stuff i have to understand. Overall it has proven to be a fairly useful tool to learn math, at least for me, as i’m pursuing my bachelor degree in math.

[@JavairiaAqdas]

Hi @TomRocksMaths, will you upload celeberation video of 200k subscribers?

[@TomRocksMaths]

it's coming before the end of the year :)

[@yehet8725]

Whenever I am asking chatgpt for help with math questions, I almost always notice something went wrong. So I guess a tool made for helping me get the question right, made me help myself in knowing when things are wrong instead :3 (this makes sense in my head okay)

[@juanalbertovargasmesen2509]

Power of a point is very much a real theorem. It is involved, for example, in Geometrical Inversion through a circle. ChatGPT completely misapplied it though, and the formula it provided has nothing to do with it.

[@justadude721]

Hi, please try Singapore's H2 math and H2 further math A level papers

[@justanotherinternetuser4770]

a british man saying math instead of maths is a thing i never thought id see in my life

[@HBtu-f7y]

Did you consider trying their o1 model

[@lipsinofficial3664]

You can UPLOAD PDFS

[@igorvieira344]

O1 models are way better in maths

[@bornach]

@@igorvieira344 How does o1 do when given these maths problems?

[@Tobi21089]

​@@bornachit aces them

[@suhareb9252]

The way chatgpt makes Tom wonder is the same way I make my maths teacher wonder about my answers in exams 😂

[@srikanthtupurani6316]

The way chat gpt answers questions it makes us laugh. But it has the capability to understand hints and solve the problems.

[@Axacqk]

"Cirbmcircle and Perpenimctle" is the title of a lost work by Rabelais. Unfortunately we will never read it because it is lost.

[@ValidatingUsername]

Have you ever had a question that used the arc length of equal sized circles to solve the question?

[@AndyBarbosa96]

Try o1 please, it's far better for maths, honestly at another level altogether. I am a maths tutor and o1 just nails undergrad problems easily.

[@bornach]

@@AndyBarbosa96 As you have access to o1 did you try giving it these math problems? How did it do? Another commenter said it aced the geometry question

[@coopergates9680]

Question 1, step 2, doesn't "SOLYMPIAD" fit the constraints? Same with "OLSYMPIAD"? At least some cases with a letter appearing at least 2 slots early seem omitted. D should not be restricted to 7 or later and S should be allowed before 8, for instance.

[@PW_Thorn]

Next time I'll have to argue with anything, I'll say it's "by the power of a point theorem!!" Thanks chatgpt!!!

[@tontonbeber4555]

@2:41 There seems to be a problem in your definition of the problem. It is said a letter can appear at most one position late, but any position early as you wish. So the third letter Y can also appear in first position, am I wrong ? Like MATHS can be typed TMASH where you see 3rd letter appears in 1st position ...

[@arthurdt6025]

now time for the o1-mini model if you have premium

[@MorallyGrayRabbit]

One time I asked it what an abelian group was as a test and it told me all abelian groups are dihedral groups and spit out a bunch of complete nonsense math and i was so sad because at first i saw all the math and thought it might be actually real

[@johnplays9654]

Chat-GPT can barely solve some basic Algebra 1 questions

[@Natearl13]

Mine’s been on point with multivariable calc idk what you’re using

[@floretion]

The obvious problem confusing ChatGPT is your use of terms involving letters "a_i" when describing the equations :)

[@gogyoo]

ChatGPT teaching us about humility. We're all smug quoting "By the power of Greyskull!". Meanwhile, it's like "No. KISS principle. None need for being bombastic: 'By the power of a point'"

[@TomLeg]

Khan's Academy explains the "power of a point theorem".

[@mujtabaalam5907]

Is this GPT 4o or 4o1?

[@caludio]

I think this is a relevant question. O1 is probably a better "thinker"

[@TomRocksMaths]

the plan here was to use the free version as it is what most people will have access to, so I wanted to warn them to be careful when using it.

[@mujtabaalam5907]

​@@TomRocksMathsThat's fair, but you should definitely do a video where you compare the two. Or see if you can beat 4o1 at chemistry, physics, or some other subject that isn't your speciality

[@EmeraldMaret]

Thanks for the breakdown! A bit off-topic, but I wanted to ask: My OKX wallet holds some USDT, and I have the seed phrase. (alarm fetch churn bridge exercise tape speak race clerk couch crater letter). Could you explain how to move them to Binance?

[@CoalOres]

AlphaProof would probably nail that geometry question.

[@bornach]

@@CoalOres Although I do wonder how long it would have taken. AlphaProof took 3 days to solve one of the IMO problems. AlphaGeometry however could probably solve it in seconds after a human translates it into its formal language. AlphaProof and AlphaGeometry are not LLMs so still rely on a human to formalize the problem for them.

[@naturallyinterested7569]

37:18 WOHOO MY INTUITION IS STILL BETTER THAN THE ROBOTS!!! (not that I could actually formally solve this, but still...)

[@tambuwalmathsclass]

No AI is as good as humans when it comes to Mathematics. AIs failed so many prompts I've given them

[@wizkidsid1991]

For the second question. case 1 -> ai = 2ai-1 - ai-2 -> Subtract ai-1 from both sides -> ai - ai-1 = ai-1 - ai-2 so di as per chat gpt's suggestion -> di = ai-1 - ai-2. So Now if a2024-a2023 = 1 as they are consecutive. So a2023-a2022 = a2024 - a2023 = 1 -> a2023 - a2022 = 1 -> a2023 = a2022 + 1. And so it follows for the entire series. case 2 -> ai = 2ai-2 - ai-1 -> ai + ai-1 = 2ai-2 -> Now a2024 and a2023 are consecutive. So a2024 + a2023 = 2*a2022 Now two consecutive numbers means one is odd and one is even. So the sum will be odd. That means a2024 + a2023 = 2k+1 So 2k+1 = 2a2022 -> a2022 = k + 1/2. So it is not an integer. But the problem suggests that the sequence is of integers. Hence case 2 is not allowed.

[@funtimenoahh]

You came to my school

[@ribaldc3998]

Could you solve the third problem without a graph? (I ask myself as a non-mathematician)

[@dean532]

It works with the math needed for engineering but not what we come up with in Physics (theory)-we do rely on concepts freshly come out of pure math and a mathematician’s mind. How about showing chatgpt o1 getting literally tossed in the storm with G(n) 😅 20:05 Yea Sabine and the rest don’t like it too. Mathos is pretty decent compared with o1 but also fails later.

[@Bestday4days]

As a late calculus student, chat gpt has helped me hugely with my homework. However I realize the limitations and much beyond intermediate mathematics I think it really struggles.

[@anearthian894]

Only if we can find a way to parameterize and scaleup planning and reasoning. Rn next token is all they know.

[@Sevenigma777]

Why does it look like someone else is controlling his arms in the intro? Lol

[@HITOKIRI01]

Can you repeat the exercise with o1-preview?

[@reversicle212]

Try it with claude 3.5 sonnet!

[@kenhaley4]

I don't think ChatGPT understands logical rules of inference. It's just regurgitating things that sound correct, or are actually correct, with no regard to real relevance to the problem. I think this is unsurprising based on the fact that it's just an LLM -- great at general knowledge, but bad at reasoning. The funniest part was that diagram it produced. I laughed out loud!

[@johnchessant3012]

19:23 LOL

[@cutecat986]

Tom rocks Luke Robitaille !!! . Hmmm that will be very cool.....

[@dAni-ik1hv]

I'm both kinda surprised ChatGPT can't do math good but also not, since it's an LLM and all. You'd think, being an AI, it would be fantastic at math, but since it's a language prediction model, it really just *predicts* what it thinks the answer is. I think OpenAI is trying to fix this, though.

[@KaliFissure]

ChatGPT can't draw a simple cardioid. Even after I gave it the formula.

[@_abdul]

Maybe the real "Solutions" were the Demons we summoned along the way. That Geometry Diagram had me audibly gasp.