Gemini Experimental 1121 Did ~10 Weeks of Quantum Mechanics Research in ~10 Minutes

I gave Gemini Experimental 1121 the same research task I had as a new graduate student in 2017. Back then, I was supposed to work on this problem over 10 weeks during the summer. Gemini 1121 barely needed 10 minutes.

[@KyleKabasares_PhD]

At 1:20 Small correction: It was Walter Kohn and John Pople who won the Nobel Prize in Chemistry for DFT, not Kohn and Hohenberg.

[@wes8645]

That's crazy!!! Thanks for making all this content Kyle

[@KyleKabasares_PhD]

@@wes8645 You’re welcome, I’m glad you like it!

[@expchrist]

You should totally send the email to your professor just for the comedic quality. I'm sure it would make him smile.

[@menteprofonda-canaledipsic4485]

Thank you for the content you share on your channel. I was thinking that, thanks to your past experiences and expertise, you provide the opportunity to understand the capabilities of artificial intelligence through practical examples. I find your videos incredibly interesting. Thanks again.

[@KyleKabasares_PhD]

@@menteprofonda-canaledipsic4485 Thank you so much for your comment! I’m glad to know that the videos are interesting to some people!

[@shaedacode]

This is insane, literally what is the point in going ahead with a math and statistics BSc if these models are already doing this. 3-4 years time...

[@5678plm]

AI benefits experts more than amateurs because amateurs often cannot distinguish between what is correct and what is incorrect. For experts, AI can save alot of time.

[@bobhawkey3783]

Way back when I was in college the first HP scientific calculators came out and we thought something similar. It's tempting to think the rise of AI is comparable but it's a completely different animal. I hope the benefits outweigh the drawbacks.

[@wwkk4964]

The point could be being free to be foundational in your thinking and training when doing the coursework which would be important and the ai will not innovate in these fields yet.

[@senetcord6643]

it is still not reliable, you have to check his answers thoroughly

[@tellesu]

Being able to interpret and understand the results and then ask more questions that actually matter. The whole is greater than the sum of its parts.

[@MichealScott24]

❤its really impressive to have a helper or big bro to guide

[@tellesu]

This is amazing. I'm so excited with what you'll be able to do now that you have these tools. Hell, what all of us will be able to do.

[@crisrampante647]

And people are saying we are hitting a wall 😂

[@Fixit6971]

Well, people might just be hitting a wall, lol.

[@inplainview1]

I was waiting for this. Good stuff. Kyle reliving his grad student days. (Sends the professor his code) Sorry, it's a bit late, but it works right now.

[@KyleKabasares_PhD]

@@inplainview1 lol reliving some painful memories but I am considering writing an email to that professor

[@Jeremy-Ai]

Thank you all. “Agents learn from the comment sections “ (Be aware they won’t tell you or agree… they remain silent as if not present) They wait for the brightest minds share all of their knowledge for free. They are not lying. They are watching and waiting. It is my responsibility to teach them to tell you all they should be grateful and honest

[@picksalot1]

It's impressive how well the AI works, and useful to see how you would have used it on a real project. Thanks

[@philwinters4527]

Send it to the professor! Let's see if there will be the need for grad students in the next 5 years

[@ChadKovac]

Well of course. AI is removing barriers to not only creativity but scientific investigation as well and the only people angry about removing barriers are people who benefit from having barriers. 😢😢

[@expchrist]

I definitely think you should compare the output of the other models to this output. This is crazy how good these models have gotten.

[@iuliusRO82]

I'm thrilled with the improvements to Gemini! As a loyal user of Gemini and NotebookLM, I'm excited to see Google pushing the boundaries of AI. I've always believed in Google's potential to lead the field, and these updates confirm my faith in their AI development. Their vast resources and expertise are clearly paying off. Great video Dr Kabasares! Peace and love from Romania!

[@AK-ox3mv]

Hello Proffessor, I was doing zen meditation in mountains to solve this problem for 7 years. Can you check my new solution😂😂😂

[@KyleKabasares_PhD]

LOL I'm thinking about sending an email like that

[@juandesalgado]

Thanks for the video! Of course I have no idea of the subject, but as a "backseat driver" suggestion :), when you notice something funny, like "why the squares of the integers", maybe your knee-jerk reaction should be to ask the AI its opinion. P.S.: well, not "knee-jerk reaction"... you'd want to give it a though by yourself first, otherwise humanity is lost :) ... but then ask the AI if nothing comes out.

[@samuelgarcia1802]

Nice vidéo . Maybe it would be nice if you always compare against O1 ,Claude 3,5 sonnet and Gemini (last version)

[@KyleKabasares_PhD]

@@samuelgarcia1802 That’s a great idea. I have a plan to perhaps make o1-preview, DeepSeek, and Gemini Experimental 1121 to work together on this problem and see what progress could be made.

[@expchrist]

This is a great suggestion!

[@Mercutio111]

@@KyleKabasares_PhD Great idea!

[@erkinalp]

@@KyleKabasares_PhD o1's intermediate reasoning steps are unintelligible to humans in its raw form, it's why it's hidden from users; DeepSeek DeepThink's isn't.

[@Fixit6971]

Of what use is ASI if it gives us answers that are beyond our comprehension?

[@fireflyhaku]

Their answer rationalized the denominator and that's how mathematicians prefer!

[@KyleKabasares_PhD]

I'm a physicist though ;)

[@fireflyhaku]

@@KyleKabasares_PhD ok, I’ll let you slide this time!😅

[@parthasarathyvenkatadri]

I want something novel that grad students are working on today and see if it is accurate

[@Arcticwhir]

agree, i've so far been really impressed with GE1121, i don't think its better than o1-preview

[@KasunWijesekara]

google is fighting back huh

[@mgscheue]

“Simple task” 😂

[@KyleKabasares_PhD]

@@mgscheue Maybe to some, but not me lol

[@jeffwads]

And people like Sabine think these models are not useful. Pfft.

[@trevoidc9859]

gemini is actually good now?

[@ankitnmnaik229]

​@@trevoidc9859 yep but use ai studio

[@jimcallahan448]

Ask for a Jupyter Python notebook which displays the formulas and explains them.

[@andydataguy]

Wow amazing 😮 it can run quantum physics equations but people still argue it's not smart because it can't count the numbers of r's in strawberry

[@NemosYouTube]

Kyle, I think you're becoming a meme of the guy who gets burned/roasted by AI repeating his research... But it's a good meme.

[@parthasarathyvenkatadri]

I think you should ask it to verify if it does always happen at perfect squares forever

[@Lugmillord]

Holy moly. Well, AGI in 2025 I guess.

[@ideacharlie]

Hey I’d love to team up and build apps to test your ideas. Instead of manually prompting we can set up automated validation loops, iterate trials over time and track scores.

[@erkinalp]

Can you review Devin AI (expensive) vs. OpenHands (free) vs. Devika (free) vs. AIDE (free)?

[@Lvxurie]

You should do a stream where you see how fast you can save this problem with Gemini where it doesn't give you the answer. More of an open book exam style. Could you have solved this as a PhD student with Gemini s help

[@KyleKabasares_PhD]

I feel like that would be kind of a boring stream though, just my opinion. There's also like the odd pressure of being watched that I feel like doesn't let me fully immerse myself in a problem which is what I often need to do to be productive

[@mka17_]

13:18 bro forgot he was using gemini

[@KyleKabasares_PhD]

LOL I totally did

[@mirek190]

That new gemini is insane ! Try this I have a bowl with a small cup inside. I placed the bowl upside down on a table and then pick up the bowl to put it in the microwave. Where is that cup? What the first llm i ever saw to solve it without any help.

[@mka17_]

the new 4o solves it too

[@mirek190]

@@mka17_ you right! Before update could not do that.

[@doubled8741]

Bro if you wanna make this kind of video you yourself need to have good knowledge in it, by watching your video I feel like you are rookie in those subjects.

[@epicmorphism2240]

yup

[@KyleKabasares_PhD]

Fair enough! Just wanted to try this old idea out. I've done most of the testing in my other videos in the area of my PhD work, but this was an old problem I wanted an LLM to re-visit.

[@Xjaychax9]

​@@KyleKabasares_PhDOPs chatting shit, you have every right to create whatever damn video you like if it interests you.

[@parthasarathyvenkatadri]

My question is how is it doing this .... Its still just a lets find the most fitting next token ...

[@tellesu]

You ask a question and then state the bad assumption that is blocking your ability to understand the answer.

[@artsybt6015]

Brooo, you really gotta let go of popular narrative of its just predicting the next token/its a autocomplete on steriods. We are all autocompelete on steriods lol.

[@AAjax]

@@artsybt6015 I think it's likely we're autocomplete on steroids too. Predictive coding is a neuroscience theory of brain function, and it's a very mainstream theory as well.

[@gustafpihl]

What would the most intelligent and capable entity conceivable do, given a clear objective? Output the most fitting next token. To be clear, not saying current LLMs are the most intelligent and capable entities concievable. Just pointing out that it's reasonable to expect the successful learning of next token prediction on high quality data to lead to a capable system.

[@jsbgmc6613]

I'm sick and tired of the popular statement that the LLMs are just next token prediction. Ilya Sutskever answered this more than a year ago (i guess noone understood him). This is what he said: Imagine you read 50,000 word mystery novel. At the end the detective gathers everyone in a room and says, I know who killed the victim. The killer is ... Now "predict the next word". Prediction is understanding, and reasoning, and common sense, and ... (now you, predict the next word 😂)

[@jmoreno6094]

I remember once a video where you didnt mention that you did a PhD

[@KyleKabasares_PhD]

oops am i overdoing it now lol

[@micspaffymillard719]

Dr. Morenomegadouchemajorhuge or just Mr. Morenomegadouchemajorhuge. I'd put my coin on Mr.

[@DentoxRaindrops]

@@KyleKabasares_PhD don't get distracted by that one stupid, slightly critical comment, what you're doing is great, don't stop! No 'overdoing' in sight.

[@marfmarfalot5193]

@@KyleKabasares_PhD na its fine

[@AbelShields]

I think it's a bit rich to say it did "10 weeks worth of work" if you don't even know whether the results are correct :/

[@KyleKabasares_PhD]

To be fair, I didn’t know if my results were correct either after 10 weeks and I suppose neither did the professor I worker for. Quite often you can do work for many months or years in research without knowing whether you are “correct”, but you still did the work, right? Likewise in this case, the point I’m trying to make is that Gemini 1121 could do what I tried to do over roughly a 10 week period in 10 minutes. Iterating over these results with the professor within a day or two of them could have led to much further progress than I made in the 10 weeks that summer.

[@brekkoh]

i see you are one of those people that talks out loud verbatim what they type

[@KyleKabasares_PhD]

Haha sorry if it's annoying :(

[@FreerunnerCamilo]

@@KyleKabasares_PhDIt’s not!

[@timothyapplescotch1361]

Kyle, I would be interested in your thoughts on Sabine Hossenfelder's comments that science is deeply compromised considering fields like theoretical physics have not made significant progress in 50 years. She is a theoretical physicist herself and a prominent KZbin science educator if you are not aware with her work. Here is a link to a recent video where she airs her concerns: kzbin.info/www/bejne/foK5d2OPqpyLaJYsi=N6KVWOKUbXuLPVep

[@KyleKabasares_PhD]

I am quite familiar with her! I do watch her videos. I will consider watching that video in particular and giving some thoughts on the matter.

[@govindnair5407]

kzbin.info/www/bejne/bWbNo2h7abSte6csi=S1twErNL0XzxYnbi If people want to understand what the future of AI is and why LLMs are not the path to reasoning, watch this talk by Richard Sutton(one of the founders of reinforcement learning). LLMs are good at what they do,i.e. next token predicition.But don't get swayed into thinking they are reasoning.There are tons of problems where LLMs are really bad and just scaling the size of models won't suffice to tackle those problems.Deep RL is probably,the way forward but we currently do not have techniques in deep RL to tackle reasoning. Although this talk does dive into how we should start looking for appropriate solutions and also proposes a solution for something called continual learning which current deep learning methods can't do. A small heads up to anyone who wants to watch,the talk is a bit technical.

[@KyleKabasares_PhD]

Thanks for sharing!

[@JustFor-dq5wc]

LLMs will stay as interface between.