Google GEMINI 1.5 Capabilities SHOCKED everyone! 1,000,000 Token Context, MoE | GPT-4 in trouble?!

[@ryzikx]

it's SHOCKING how much i am SHOCKED!

[@hynesie11]

Shocked I tell you. Well not so shocked.

[@Manicksplaining]

SHOCKED 😮

[@Manicksplaining]

Remember shocktarts candy?

[@somebody-anonymous]

I am so SHOCKED that I CLICKED the video and without even watching it I ENGAGED by writing this comment

[@StephenRayner]

Shocking!!!!!

[@OmicronChannel]

My 80-year-old grandma called me; she is deeply shocked by the 1,000,000-token context window!

[@SigFigNewton]

My two year old’s jaw hit the floor

[@reza2kn]

Cool grandma bro!

[@TaskSwitcherify]

Plot twist: it was my voice-cloned GrannyAI that called you.

[@dockdiscus3693]

This time you get a pass, the industry is actually shocked

[@reza2kn]

right?! this is all I ask for.. use it when it's asked for!

[@ismaelplaca244]

As a member of the industry I am SHOCKED

[@arshtat]

my face right now: 😮

[@hotstreak4life]

I have been shocked, changed, and stunned so many times that I am left emotionless!!

[@AIDidmedia]

😂

[@MGeeify]

I know this is almost like a meme now but these developments ARE shocking lol. I can't wrap my head around how quickly these models are evolving and with every new model the pace of development gets even steeper. I really don't know where we are heading as a species right now but I'm really starting to believe that reality as we know it is about to change way sooner than we might think..

[@bigcauc7530]

It will get really ugly for us and then hopefully there will be a real golden age of prosperity and freedom to explore, even outside of this planet. I don't think there is an upward limit on technology and what it can be made to do. There is just an exponential curve for complexity and understanding. So AI is a pivotal breakthrough to allow us to continue growing. A second industrial revolution.

[@MGeeify]

@@bigcauc7530 100%, very well said

[@Odrox]

SHOCKED everyone!

[@alertbri]

Meanwhile, AI video from OpenAI just CHANGED THE WORLD! What a time to be alive! Lol

[@Sajuuk]

Sora? It's amazing isn't it?

[@Shy--Tsunami]

​@@Sajuukyeah. Absolutely crazy

[@HoD999x]

hold on to your papers

[@StephenRayner]

Shocked!!!

[@mshonle]

Hellllo community! … And happy prompting!

[@apdurden]

Nice job Wes! But get ready to make another video. "Sora" text-to-video model from OpenAI just SHOCKED the industry like 10 mins ago

[@ali.saghabashi]

Lumiere>>>>>>Sora

[@COW879]

Wes Roth used SHOCK! It's super effective!

[@jeltoninc.8542]

Underrated comment 🏆

[@ryzikx]

industry SHOCKED!

[@djlongdistance]

Every comment that jokes about the shocked headline has essentially given the video a massive boost on the KZbin engagement algorithm. At this point it would be foolish not to use the word, or some might even say …… Shocking.

[@mhrzsafaeian7976]

I used to block any content creator that used the phrase "SHOCKED." Please don't do that, you are a smart, authentic content creator, you don't need these click bait phrases to get more audiences. Anyways, can't wait to see you explain how openai's sora "SHOCKED THE PLANET"

[@tracy419]

Sounds like maybe you grew up a bit and can now stop dwelling on the reality that a business is going to do marketing things to get new customers and that it's actually ok, so long as the content is good enough to keep them.

[@kylemorris5338]

I'm glad they got the message that the demos need to include the time each prompt takes, even if they needed that lesson beaten into them.

[@m0ose0909]

AIs ability to comb through video to find minute details is amazing - but also consider how this is going to impact the surveillance capabilities of the world. Strange future approaching.

[@KolTregaskes]

28:35 Oh thank you, Wes. I'm so happy you have highlighted AI Explained's video on the issues with the MMLU benchmark. This needs highlighting more.

[@galailliz]

This ladies and gentlemen, is the real SHOCKER!

[@KoroushRP]

Im gonna have a heart attack if i keep being shocked like this

[@ZacheryGlass]

We will SHOCK you back to life

[@mydudesima]

ONE MILLION TOKENS??? I'm very shocked by what Google is doing, this is insane. I can't believe this is happening so fast.

[@brianpalmer967]

As a man who has searched for many needles in many hay stacks, I'm glad to hear this has been a priority of developers.

[@artscollab]

This is my “go to” channel for AI. I use it to keep up while in a graduate AI program. Keep it up !

[@zerobot_tech]

I just seen lightning strike and couldn’t help but notice that it SHOCKED the industry 👀

[@SonOfTamriel]

Hope you'll cover OpenAI Sora today, its unreal!

[@s991514]

Shocking even

[@afti03]

I used Gemini yesterday to help me change a title from "Study in the UK" to something more attractive. It proceeded to write almost a paper on the emotional side of people when reading certain words, ultimately giving me titles for each of the emotional categories I might want to explore (e.g., optimistic, intriguing, adventurous). Then, to my surprise, it actually generated three different types of responses with different tones! I read everything it came up with and was absolutely shocked. For comparison, I gave the same prompt to ChatGPT and GPT-4, and they both provided short answers with just a single title, "Unlock your career: Uk studies await." Interestingly, all three models - Gemini, ChatGPT, and GPT-4 - started with this exact title, with slight variations: Gemini's was "Unlock your career: Study in the uk," while ChatGPT offered "Unlock your career: Uk Studies." What truly impressed me was that Gemini offered at least 50 good titles to choose from.

[@alanjenkins1508]

I wonder if the most useful thing AI will provide us is an expert that can correct lies and misinformation instantly because it knows everything. This will also be a threat to the powerful who have achieved their positions through disinformation and ignorance.

[@collectivelogic]

Who reads a whole dictionary? "Eminem I guess" 😂

[@lthedoperabbitl9258]

about to be SHOCKED AGAIN WITH A SORA VIDEO

[@beardordie5308]

Okay at this point he's trolling all of us KZbin commenters

[@2099EK]

Yup, he inserts "shocked" like 3 times in the video.

[@Wanderer2035]

All the AI KZbinrs are lol

[@r34ct4]

Yeah I'm liking it. Leaning into the absurd@@Wanderer2035

[@J3R3MI6]

But this is actually mind blowing 💎

[@dertythegrower]

Increases the algo... simple and effective for younger people who try to always fit into the crowd

[@bloopbleepnothinghere]

Tokens counts are increasing at insane speeds. 10M was unthinkable last year. Hell, an hour ago.

[@zerobot_tech]

Shocking how the industry gets SHOCKED on a daily basis 👀

[@asastudios7782]

Okay I'm actually SHOCKED this time!

[@SkilledTadpole]

I, for one, was SHOCKED to hear this

[@r34ct4]

These videos have rendered me literally convulsing on the floor drooling out the side of my mouth from the sheer amount of repeated SHOCK dispensed intoward my body

[@StephenGriffin1]

Shocked! Awed!

[@andrewwalker8985]

@wes I don’t know how I’d keep up without your work. We appreciate you

[@HowToAiNow]

Great as always Wes. A little comment. Even though move 37 was really creative and showed "imagination" or creativity from a machine for the first time in history, the 4th game was won by Lee Sedol because of another famous move at that same game, the 78th by Sedol.

[@Phrenor]

Dam I love your content, I really hope you get more views and subscriptions, you deserve it!!!

[@1sava]

Wes, your pronunciation of “Les Misérables” was actually really good!

[@filip.jakubowski]

Thanks @Wes Roth! Good summary! Good job @Google.

[@zerobot_tech]

Our industry must be incredibly conductive with all this SHOCKING going on 👀

[@KolTregaskes]

Well it's nice to see at least one KZbinr talking about this today. Everyone else is talking about OpenAI's Sora model. 😉

[@MonkeySimius]

The other KZbinrs were so shocked they can't even comment on it.

[@kait3n10]

To get a good "needle in a haystack" performance for smaller models like Chat with RTX, can't we just implement a "search and extract" step to put the relevant paragraphs at the start/end of the context? PDF readers can already search through text really fast.

[@StamatisSkiadas]

Why do I use midjourney and ChatGPT but not Gemini??

[@drawnhere]

It's shocking how shocked they are !!! But don't forget, they're also stunned! And that is stunning!!!!

[@sonydee33]

you can best believe the opensource they used for this was the same openai used for sora. they both dropped their demo the same day after a breakthrough 2 days before 💯 we're moving so fast.. I love it

[@JimWellsIsGreat]

After Google's last semi-faked demo and report, I have my doubts about real world results.

[@solaawodiya7360]

Thanks Wes. But at this point with Google, I'll believe it when I see it because they've hyped so many things but under delivered with blunders at the end. For now I'm more keen on seeing OpenAI Sora text to video model that was teased.

[@sheanmassey]

It's shocking how terrible it is. Google lost the AI game years ago

[@cyanophage4351]

The official transcript has "it's one small step for a man, one giant leap for mankind". Gemini got it wrong and left the "A" out. So it didn't actually pull from the document and just pulled from memory instead

[@drakethesnek6429]

Will this version hate wypipo as much as the last one?

[@MisterDivineAdVenture]

19:45 "MOE" (Mixture of Experts) conjecture for GPT3 to 4 metamorph, consistent with Sutskever's original paper (2013). (I was thinking about this from ma different vector around the same time if not earlier. But only conjectured.)

[@spoonikle]

Using an obscure language to push the model towards deeper understanding was a brilliant move.

[@King-a-ling]

Yes they were able to double platform all of the negatives which in turned created positives. This was genius. Now if they can update the multimodals and make them congruent to the gemini architect, then it will enhance the performance.

[@Sajuuk]

Oh teh noes! I'm SHOCKED again! 😂

[@andreinikiforov2671]

Unfortunately, it won't be released to the public until 2027, when GPT-7 rules the world...

[@DannyPeck]

So why would I pay for ultra?

[@LostMekka]

didnt lee sedol win game number 4 against alpha go? if i recall correctly, it was a best of 5, alpha go won the first 3 and so won the match, but they played the other 2 games regardless

[@GSK4EVER]

Berman had a “shocking” survey recently. Glad we are all having fun here.

[@TaskSwitcherify]

I'm regularly shocked by how bad Bard/Gemini is. From being uninformed to being confused, it shocks me all the time.

[@malimal4972]

Jeremy Shockey, Gemini Shocking!!

[@Yewbzee]

Do they actually survey the industry with some kind of poll to assess if the people who make up the industry are actually shocked? What’s the % threshold to claim “industry is shocked”?

[@fullmentalalchemist3922]

Actually is SHOCKING! Assuming google isn't BSing us again.

[@LesPaulguitar134]

Does anyone have information on number of parameters and number of parameters per expert? I only know the number that 1.0 have

[@tricktap8502]

Love that you kept the gag running from all the people hating 🤣

[@colorado_plays]

Yesterday was quite an industry shaker. With Google and OpenAI both having huge announcements, who was the winner??? I think it is UC Berkeley! They released an Open model 2 days ago with a 1M context window (video and text), with a solution to the current quadratic memory requirements for context windows. I think this was the forcing function for both Google and OpenAI having surprise announcements yesterday.

[@danielballard3364]

Google for "Sepp Hochreiter" and his AI model (xLSTM algorithm).. it only has linear memory requirements per context windows based on the way his AI model works.. if he had the funds and support to upscale his model like others, his model would clearly prove superior to most others

[@reza2kn]

@23:50 BAHAHAHAHA 😂😂😂😂Glad you got the message!

[@Nightfallとばり]

SUPER SHOCKED

[@milkywaydev593]

Wes Roth's pan 23:53 the whole industry!!!

[@TreeLuvBurdpu]

Goody-2 looks hilarious. Do they sell shoes?

[@PianoMastR64]

"one small step for [a] man" Everyone gets this quote wrong, but I would expect an AGI to always get stuff like this right. We'll get there eventually, but it's the little mistakes like this that add up

[@dukemagus]

This industry has an electric wiring problem.

[@ReflectionOcean]

Understand how Google's Gemini 1.5 enhances performance with a new mixture of experts architecture. 0:43 Explore the revolutionary experimental feature of Gemini 1.5 Pro allowing a context window of up to 1 million tokens for a select group of developers. 2:24 Recognize Google's pioneering work and adoption of mixture of experts (MoE) technique in AI. 3:14 Grasp the implications of near-perfect needle recall in Gemini 1.5 Pro across all modalities and its potential massive impact. 4:07 Realize the significant improvements Gemini 1.5 Pro provides over previous models, especially in recalling information from large datasets. 13:02 Learn how to access Gemini 1.5 Pro for testing through AI Studio or enterprise contacts for developers and enterprise customers, respectively. 14:22 Acknowledge the importance of competitive advancements in AI, spurred by models like Gemini 1.5 Pro, in driving innovation across the field. 32:58

[@ktb1381]

Remember how shocked we all were a few months ago when they announced how great Gemini ultra was going to be?

[@Classicalpianosongs]

Unfortunately no matter how big the context window is the quality of output is reduced depending on how many words the model tries to squeeze the answer into. When chatgpt increased to 128k context the answers didnt improve responses much because the output is still about 2500 words. Lets say u are a researcher trying to collect information from sources and put 10k tokens in and ask it to extract all the points with no summaries, u still lose 70percent of the points and thats not including the fallibility of the models in expressing them accurately because it cant fit them all in its likely 1000 word response it will give you. We do need increased maximum outputs for performance increase else performance wont be increased unless we ask for very specific things

[@bloopbleepnothinghere]

I work in software. I am definitely shocked, I feel a bit sick if I am honest.

[@cfkelty]

Judging by the comments it looks like “shocked” is about be cancelled…shocking. Never the less, good video, Wes

[@memegazer]

They should integrate this into a youtube feature to automate timestamping

[@cacogenicist]

There's a lot of cool shit they could do with KZbin and their multimodal models.

[@J3R3MI6]

@@cacogenicist KZbin + Gemini would be perfect.

[@memegazer]

@@cacogenicist True...imagine yt studio with AI enhanced features out of the box instead of using extra steps to try and leverage AI for SEO like clippy for yt studio but with actually useful tips Same goes for anyplatform really

[@stevewall7044]

This kept me up til 2am yesternight.

[@nouxcloete3129]

This is good news for us, meaning now OpenAI will be pushed to release an equivalent new set of features, GPT5?😅

[@ravenragnar]

Who is responsible for the decisions made by AI systems, and how can we ensure accountability? How can we ensure that AI benefits all of society and not just a select few? How will the rise of AI impact employment, and what strategies can we implement to mitigate job losses? Thanks for your time.

[@madshebs]

I am also SHOCKED over the SHOCKING amount of SHOCKS Wes exposes us to

[@PaulBrunt]

I think this might be bigger then most people think, I've seen people reporting that it works inanely well at 1M tokens and even 10M tokens. But why have they not reported the point size this thing actually starts to break?

[@Antares0210]

SHOCK!

[@jeffeast7983]

NICE TITLE: SHOCKINGLY SHOCKED!!! SHOCK SHOCK SHOCK I'M SO SHOCKED!

[@pureconcept-q7p]

I AM SHOCKED THE INDUSTRY IS SHOCKED WES IS SHOCKED

[@chrisbo3493]

Well until 128k for all does not run smooth, I hold my paper.

[@robertmariano]

While, I appreciate your video, I find it very technical. I liked the video a few days ago about robots being trained in virtual environment for a simulated 100 years and then using that knowledge to operate in a real environment.

[@psmontte]

Wes, for the love of God resist the temptation to jump on the "SHOCK", "GAME OVER" bandwagon. Do not panic, if you are not growing as fast as you were growing a few weeks back.

[@quentinkumba6746]

I did research into statistical models of natural language analysis as a PhD back in 1986-9 in the UK. Today is the first day I’ve had that uncanny valley feeling about it. It’s getting so powerful and this is just baby steps, it’s scary weird.

[@skavenqblight]

I’m always skeptical of Google after their prior antics. I’ll believe them when they pull an Altman and take queries in realtime from Twitter or a live audience

[@styx1272]

'Surely' all they've done is embed a search engine within the AI model and then let the AI model use the search engine to find the question ?

[@Startupfy]

do we trust Google demos again? that was fast

[@r34ct4]

In all seriousness though, this looks absolutely insane. This is legit scary. Problem solving on 800,000 tokens of code...

[@couldntfindafreename]

We'll see how fake is that long context with practical workload. Anything sub-quadratic is a compromise. For example if you have to tell the model explicitly what is important to remember, that's cheating. Look out for "fake demos" as well :)

[@inspectorcrud]

I thought that Lee Sedol won 1 of the games against AlphaGo?

[@BrianSmithology]

Gemini refused to let me give it a nickname. ChatGPT don't give a dern. I'll hang with friendly over stuffy AI. Keep shockin' and rockin', Wessy!

[@Mark-ny3bs]

*SHOCKED*

[@kawalier1]

It's choking and chunking. Mixtral was brought by G?

[@gue2212]

Wow, the next shock! We are clearly in the exponentially-growing AI shock phase. LOL

[@freedom_aint_free]

We will reach a point in the near future, were the future will be instantaneous, when AI be capable of recursively self improvement, the only limit will be the environment.

[@zoranProCode]

Does Gemini have API ?!

[@helge666]

I asked it that and it said that there is no official API yet.

[@liberty-matrix]

"The woke employees at Google are so broken that they trained an AI to make racially diverse Nazis." ~Tim Pool