Gemini Ultra 1.0 | Is THIS the GPT-4 Killer? We ran a BATTERY of tests, here is the SHOCKING result.

[@Lukebussnick]

Great video, Wes! I had a ton of Gemini ultra videos in my notifications, but you are always my number one go to for new model updates and you did not disappoint once again! 😁

[@syndicalistspeedsolver]

You should watch ai explained. He is more concise and less clickbaity

[@Lukebussnick]

@@syndicalistspeedsolver I already do and he is great. However, it is still my personal opinion that I prefer Wes and his content, there is just something in his style that is more personable and speaks to me more directly.

[@carlkim2577]

All the reviews that I've seen are missing the feature in Ultra that you can select for a longer response. Then the answers look closer in length to GPT4. However, clearly the reasoning in GPT4 is superior still.

[@ran160]

yea i agree but the creativity and voice of gemini is a lot more human and sounds less robotic

[@danielajemba8459]

​@@ran160 I like the robot sound of gpt it makes it sound impartial

[@xbon1]

What feature are you talking about? My answers are already long but I see nothing in a settings or anything to make it go longer.

[@eddietrevino9024]

This is interesting, thanks for sharing Wes! This reminds me of a podcast I listened to of an ex-google employee who worked there during the early days of search. He mentioned how google discovered through research that every millisecond of search speed they improved was highly correlated to more users and searches (hence why they used to display X # of searches in 0.0X seconds). Similar to this phenomenon, I wonder if we will see something similar with the winners in the foundation models space moving forward, or if there will be a different metric that is more telling for wide user adoption.

[@zacboyles1396]

Human Me: “Is the Google Gemini Ultra model available in Google Cloud yet?” Gemini Adv: “I'm a text-based Al, and that is outside of my capabilities.” - My first message to Gemini Advanced

[@cyrilandgarry]

You can steal your session key and wrap it as an AI.

[@nhinged]

?​

[@cyrilandgarry]

@@nhinged if you use your Google session id you can use it as an 'API' and make prompt requests to it. I can send you the code if you're interested

[@cyrilandgarry]

@@nhinged you can grab your Google session id and send messages to the API endpoint using post or whatever to get prompts. I can show you the code if you want?

[@putinninovacuna8976]

funny, i asked the same to bard/gemini pro and his response was pretty good telling me that is not supported yet but i could try gemini vision pro yada yada.

[@thefruit]

I feel like a proud father when ChatGpt is doing well somehow *-*

[@wyqtor]

We got attached to GPT and Sydney, they both feel like family or friends!

[@mickelodiansurname9578]

So short and sweet, its not what they showed in the vid, what the quack! But it is a very very good offer, not up to GPT4 maybe, but to be honest so close it doesn't matter. And the added extras of storage and integration into the Google ecosystem, well thats your selling point right there. EDIT: The reason Gemini is fast is fairly obvious. It was launched today... has no users. But GPT4 has 180 million paid subscribers ... there's your difference!

[@kuwrom]

Al long as it doesn't have a code interpreter it is way behind gpt4, I see code interpreter as as the confidence of a code generation that works.

[@etunimenisukunimeni1302]

Thanks for the video! Btw, one of the most iconic and important aspects of roguelike games is procedural generation. The locations (and usually items too) are meant to be randomized every time.

[@CartoClips]

It came up on my feed and it was funny because I thought that’s why not what I’ve heard about Gemini the last couple days then I saw it was two weeks old

[@ctwolf]

10:49, B. FF7, what is otherwise known as a large chunk of my childhood.

[@upgradeplans777]

I disagree with how Gemini did on the "tall bachelor" question. Gemini gave a logically incorrect explanation. While the premise is indeed faulty, that is completely irrelevant to logical entailment. And that nuance is IMHO an important part of what the test is trying to trick the AI with. Obviously it is a hard question even for humans. ChatGPT's answer was way too convoluted and contains a slight misdirection, but in the end it did give a logically correct explanation. BTW, the short answer is: "The first sentence entails that Mary is not a bachelor. The reason for that could be because Mary is not a man, is married, or was married in the past. Therefore, it does not entail that Mary is not married."

[@almostoffthegrid]

Hey Wes, thanks for the video bro- you da man 😎

[@georgewashington7251]

I have been using it and it is NOT that good. I put a picture of a wine label in the prompt and asked if it could tell me the name of the wine. It was hard to tell from the picture. Gemini's response?: "Sorry, I can't help with images of people yet." Despite continued prodding it would not answer.

[@vincentvoillot6365]

About the python "proverb" code, if it is really caused by the JSON format. That mean, Gemini understood (wrongly?) the query as an automation task, write the code and run it in a sandbox ? That would mean that Gemini can wrote his own agent given a specific query and provide you the code used for transparency purpose ? That's very powerful.

[@EssentiallyAI]

The OpenAI/Microsoft Collective will strike back! Never underestimate the power of the Borg.

[@dawid_dahl]

Thanks for the thorough tests!

[@Bogusuap]

I used it today. No pdf reading, no image generator. It sucked today. Code generator was ok.

[@jacobhouston1655]

How do you get ChatGPT to show you the codes of what it’s doing? Min 22:03

[@Adovid]

7:05 those rumors though. GPT4 likes to use rumors for story telling and its kind of a give away. Especially when the rumors just happen without the rationality of how they spread.

[@AIGooroo]

Hey Wes, thank you for the video. I'm curious to know if you think your Custom Instructions can affect the output from gpt4. (either for good or bad)?

[@kaosorder3012]

Can you set it up where each bot asks each other an increasingly harder question or something like that...?

[@acydhlestrange949]

Not sure if it is a silly question, but what stops a LLM maker to fine tune it on standard benchmark prompts to nail them?

[@DannyPeck]

"I don't know what it is I did NOT listen" 😆

[@jacobhouston1655]

What’s the best short (5hrs or less) class for prompt engineering?

[@joser100]

On the "speed" point, wouldn't this be related to scaling? I mean that at this time there would be practically no users using the Google service, and surely they over-dimensioned the backend to avoid latency issues at this early stage , while ChatGPT would be managing a clearly much larger volume of inference requests.

[@wyqtor]

They also have tons of TPUs for compute

[@naromsky]

Finally we find out how the ChatGPT usage cap was established.

[@MudroZvon]

I like that Gemini is fast and doesn't have usage cap

[@wyqtor]

@@MudroZvon They can route your prompt to another model at times, it's right there in the FAQ.

[@razvan9795]

I'm sorry, but chat gpt 4 is way above Gemini ultra regarding real practical applications. Gemini Ultra is also losing its context: "I was talking to it about recommending events to be tracked in a specific use case and it recommended where I can find events in the location that I'm in.

[@mattshelley6541]

GPT 4 for the win.

[@richardhollowayjr]

One could definitely hope

[@mackroscopik]

GPT5 will be released the day after any other AI beats GPT4

[@matthewboyd8689]

Imagine one of these AI companies making an AI so good that it replaces game engines because it can do extensive complex code with simple commands or descriptions

[@OnigoroshiZero]

GPT-4 is still the king, even 1 year later. This is sad for Google, they will never be able to compete at this pace.

[@montonmonton8791]

If any KZbinr claims that Gemini Ultra is on par with GPT-4, especially when it comes to coding, that's my cue to hit unsubscribe. I've tackled a bunch of non trivial coding challenges, and let me tell you, Gemini Ultra doesn't just fall short of GPT-4-it's a total disaster. Google's got a serious problem on its hands. Despite some KZbinrs hyping it up with basic examples, anyone who actually tries to use Gemini Ultra for real coding tasks will quickly see it's complete garbage.

[@Custodian123]

😢 Guess l am sticking with GPT4 for now. Let's hope they push out Gemini 2, because it's utility is that it's built into a workspace.

[@GlennGaasland]

Please stop using this ridiculous expression “GPT killer”, this shows a basic misunderstanding of what generative AI is. When a new model arrives, the other models are made BETTER and more useful, since you can have the models speak to one another. Anyone seriously interested in AI should obviously be using both GPT and Gemini, and have them speak together and collaborate. This works incredibly well. The value you get is tenfold what you pay for it you use the models well.

[@Salted_Pizza]

Gpt is great, but it has a dealbreaking flaw at the moment. Even with the paid version, you are limited to 40 messages / 3 hours with gpt 4, while Gemini advanced is seemingly unlimited. I like doing complex multivariable text based adventures, and it's just not feasible on GPT 4 because of that limitation.

[@Odrox]

25:50 well that is what makes a mammal a mammal you know.

[@jonmichaelgalindo]

Remember, RLHF degrades model performance. We saw a research paper benchmarking GPT-4's ability to solve the same set of math problems, and watched its abilities fall month by month. (Sam Altman's response to this objective data was: I know it can feel dumber, but I think that's because the bar had been raised for what we expect.) >:-( Anyway, Google is collecting feedback now, so expect Gemini to degrade over the next few months.

[@Shy--Tsunami]

Boomboom stick in the thumbnail is kosher?

[@grant_vine]

Could it be faster because it has less users?

[@elon-69-musk]

GPT4 is still the king

[@bjarnewinkler6366]

You are funny. Gemini speed vs GPT4 speed - dude it should be faster Gemini may have half million users at this time where as GPT4 has several hundred million - so comparing speed is not fair

[@geldverdienenmitgeld2663]

Gemini Ultra told me that it has 1,5T parameters. Ultra 2.0 will have 3T parameters and could appear already in 2024

[@h-e-acc]

The object recognition is definitely light years better

[@daryjoe]

This is definitely not true try using them too analyze grwphs

[@projectcontractors]

"it's funny you know all these AI 'weights'. they're just basically numbers in a comma separated value file and that's our digital God, a CSV file." ~Elon Musk. 12/2023

[@apache937]

your brain neurons could also be thought of in a similar way

[@Ben_D.]

I would happily subscribe to either service or even both, if they would just get their broken guardrails under control.

[@strangereyes9594]

You just have to wait until non-western nations come up with their own AI versions that are not bound to these woke restrictions that make the AI too retarded to work properly, i.e. being a true and objective pattern detector.

[@VR_Wizard]

How often do you run into these guardrails. I almost never do and if I do then it was since I t4ied something funny which wasnt important. Important tasks work for me almost 100% of the time.

[@Ben_D.]

I do a lot of image generations, and they almost never do it on the first try. Simple words like Gorilla are forbidden. You have to really streach your brain to get a result. I was trying to chat the other day with GPT-4 and it would not even address the topic of what was allowed. I didn't challenge it, I was just asking 'what is allowed? where are the limits?' It got cranky with me and ended the conversation. To answer your question... I run into it every day. @@VR_Wizard

[@lloydjames3806]

Wes I didn't see you click GPT 4. It never was highlighted.

[@WesRoth]

it was, I checked before I started, also at the end it ran out of usage, so it was gpt4, but maybe it wasn't as apparent on the recording, my bad.

[@Craznar]

If anyone knows how to access this with a paid Google account, rather than a free one - please tell me.

[@notlocalhorst]

After a short high i was quickly disappointed ... stupid loops, complaining about calling it a *****, lecturing me, acting like a 10 year old, the copy button introduced hidden characters that breaks everything, multiple times it forgot what it was working on, don't know at all what it was talking about and wants me to to fix the bugs it introduced. Yes, that was with ultra and a short python script that I already did with gpt-4 before. Not really what I expected at least not for programming. And by no way I think gpt is really good, it is also quite frustrating but at least the result is better (for me).

[@tommasobrindani5894]

Try asking it this: "I have 3 apples today. Yesterday I ate an apple. How many apples do I have now?". ChatGPT easily answers correctly, Gemini always fails...

[@Craznar]

You've been watching AI Explained haven't you :)

[@irabucc469]

So it's in its training set, so

[@Lad11971]

@@irabucc469it has nothing to do with what is or isn’t within the training set, if the AI models just spat out word for word what was in their training data they would be very useless. The point is the AI should understand the problem and solve it. Both models obviously know what an Apple is, and what a number is, the key here is doing basic math which a lot of models still struggle with.

[@tommasobrindani5894]

@@Craznar No, found it under a post on X platform

[@boremir3956]

@@Lad11971 Yes but you don't know know whether a model that got it right had the question/answer in it's training data or not instead of having used logic instead. That's the point.

[@id104335409]

Why do all AIs only code in Python? Aren't they familiar with ALL coding languages? You can easily ask them to code in a certain language, but it almost always comes out incomplete or wrong and they will always prefer to give you some code in Python if you haven't specified which language.

[@wyqtor]

Sydney can also do JavaScript and C# in addition to Python 😊

[@WinterRav3n]

GPT4 is still the FarCry for the Hardware, what we had back in time. Google, a multi billion company, with the amount of people and data, is not not able to dethrone OpenAI. No multi file selection and only pictures?

[@atakama2380]

finally there is balance in force

[@vbywrde]

Did they have to name it Ultra? Kinda a little to close to Ultron. Not that anything will go wrong, but ... um ... you know ...

[@MonkeySimius]

You told Gemini to not give away the answer in any way when writing it's story. Of course the story it wrote would have nothing to do with the answer as, if it did, you might be able to deduce the correct answer from the theme of the story. Task failed successfully. I'm kidding. But it is kind of fun seeing how well these things can understand language while at the same time seeing how poorly they understand language.

[@zoranProCode]

And what is SHOCKING here since its almost the same like a ChatGPT?! We expected it would be something UTRA better.

[@MudroZvon]

almost, but not quite... I would say GPT4 is still better... nobody could beat it so far... even if GPT4 is getting more stupid with time

[@Srednicki123]

these LLMs all hallucinate and probably always will, still remarkable technology, nice to toy around with, some great productivity use cases, but everybody should turn down the hype a little, Yann LeCun is right, better systems will come

[@al2935]

It's not trash but I haven't seen enough unique use cases where it outshines other ai services

[@escanor2002]

it is unique when it comes to data

[@apache937]

love the competition but im staying with gpt 4

[@brianWreaves]

Well done, Wes! 🏆 Google's UX team needs to help the marketing team with its AI nomenclature. It is too confusing trying understand if the AI is Gemini, Ultra, Advanced, Bard, Google AI, etc.

[@putinninovacuna8976]

what is confusing about it? look at the top the drop down list just choose gemini o gemini advanced, its not that hard to figures it out

[@brianWreaves]

@@putinninovacuna8976 Ans Bob's your uncle. I'm referring to all the AIs for general knowledge and reference, not the UI you mention.

[@mscottveach]

Better mousetrap is a totally reasonable answer.

[@pensiveintrovert4318]

How many users does chatGPT have 100 million. How many does Gemini have? A few thousand at the moment. There is nothing to be impressed about wrt. its speed.

[@Rockyzach88]

Lol what's with the bots still up?

[@GNARGNARHEAD]

none too shabby for an evil corporation

[@fitybux4664]

I thought they had a "don't be evil" clause? "We removed it."

[@Genadiy31]

Cool

[@SpikyRoss]

Ive tried this, and it's not even close compared to gpt4, i don't know how people can say it's a competitor it doesn't even do simple task without giving you some virtue signaling and refuses to elaborate, any way openai still ahead

[@Subranis]

feel like the title of this video is a direct rip of AI explained's latest video lol

[@JamesPuddicombe1]

BACHELOR REASONING: Wrong assessment of Gemini? IMHO Gemini failed as it attacked the validity of the first sentence, which was not the question. The question was, does the 2nd sentence follow from the first. => GPT4 nails it, Gemini ducks the question.

[@salamandiuswestlife7729]

Not my clowntry, not my problem

[@stilllearning7434]

who needs school ?

[@marsrocket]

Shocking? You need to get out more.

[@JOHN.Z999]

GPT-4 is VERY BETTER!!!! ❤

[@poneill65]

@2:54 "MARIO was so faithful with HER wife until the wife cheated on HIM..." Someone gets an F!

[@seppimweb5925]

Excuse me, Wes. It might make sense to compare at this simple level. But this is not really AI-like. Maybe you should look at the transcript of the interview by Tucker Carlson with Putin and ask questions about it. Then you would have a completely different view of AI in general.

[@AndrewQPower]

You had me up until you chose Doom over Wolfenstein.. #clearfail

[@CT2507]

And then it went racist and showed Googles true face, intentions and agenda! ;)

[@markleydon4118]

Good content but you talk way too fast and too indistinctly. Makes it very difficult to follow exactly what you are asking the models and what the response is. Please slow down.

[@tommy516]

It is AWFUL with hands, no way is this better than GPT for image gen!

[@hinterlands]

... I don't know what it is, I did not listen...

[@n1ira]

Yeah I'm not buying gemini

[@wishmasterbrazen]

Man u screwed up not gemini. Gordan freeman is known for a crowbar... it had the answer.. you messed up and said hes known for a shotgun.. thats doom guy... your error caused that problem not gemini...next time you test a program like this, maybe research the topic before asserting its the program... i was super impressed with gemini and when u messed up and said gordan uses a shotgun... i was like "noooo gemini had it you just confused it"😅 lol

[@WesRoth]

he is literally holding a SHOTGUN on the cover of the game: sierrachest.com/gfx/games/HL_AP/box/01_1_front.JPG

[@wishmasterbrazen]

The first weapon u get in both half life one and 2 is a crowbar.. the shotgun is not featured prominantly the cover of the game has nothing to do with the game itself. Alsp doomguy has a dbl barrel.shotgun that hes famous for. Its a simple mistake. I just happen to watch half life videos all the time lol :)

[@wishmasterbrazen]

@@WesRoth yes... the cover of the game. But he is iconic for getting a crowbar... its the first thing u get in both half life 1 and 2 and anyone thats a fan of the game would know that. Apl im saying is thst gemini had it right away and you were wrong which confused it im just trying tp be helpfull.

[@SuperJg1975]

chat gpt is better #

[@LisaMitchell-kn7vo]

I'm favoured, $60k every week! I can now give back to the locals in my community and also support God's work and the church. God bless America.

[@marthamalcom604]

Same here, with my current portfolio made from my investments with my personal financial advisor I totally agree with you

[@reed229]

Wow! Kind of in shock you mentioned expert,elizabeth ann graney. What a coincidence!!

[@andymclain2117]

Elizabeth Ann has really set the standard for others to follow, we love her here in the UK as she has been really helpful and changed lots of life's

[@Evansdavis568]

Life is easier when the cash keeps popping in, thanks to Elizabeth Ann graney services. Glad she's getting the recognition she deserves

[@darrylhodge6708]

Such a genuine personality!!, Sincerely speaking. I will continue to trade and stick to Elizabeth Ann daily signals and guides as long as it works well for me.

[@IOFLOOD]

Really sick of the absurd titles. Not everyrhing is "shocking". Telling youtube "do not recommend this channel"

[@briankgarland]

Nope, not impressed.

[@punk3900]

I am disappointed by Gemini... It resisted coding...

[@markmalonson7531]

Skinny robot girls with laser beams and mutilation weapons..playing too much Nintendo's? Nah..clowns of da video game. subversive repetition.

[@fai8t]

Simply Gimini is overfitted and will not scale in future. same idiot methodology as android vs ios, just reply on hardware instead of optimizing software

[@fai8t]

in this case its access to internet and more data rather a decent algorithm

[@BenGrimm977]

Your thumbnails are always terrible

[@ki001]

GPT seems to be better IMO though slow in comparison

[@h-e-acc]

The object recognition is definitely light years better