Project Astra: Our vision for the future of AI assistants

Introducing Project Astra. We created a demo in which a tester interacts with a prototype of AI agents supported by our multimodal foundation model, Gemini.
There are two continuous takes: one with the prototype running on a Google Pixel phone and another on a prototype glasses device.
The agent takes in a constant stream of audio and video input. It can reason about its environment in real time and interact with the tester in a conversation about what it is seeing.
Learn more about Project Astra: goo.gle/3wAUwFh
#GoogleIO2024
Watch the full Google I/O 2024 keynote: www.youtube.co...
To watch this keynote with American Sign Language (ASL) interpretation, please click here: www.youtube.co...
#GoogleIO
Subscribe to our Channel: / google
Find us on X: / google
Watch us on TikTok: / google
Follow us on Instagram: / google
Join us on Facebook: / google

[@Google]

Thrilled to be able to share with everyone how exciting Project Astra really is. Drop a ✋if you're ready to give it a go, and learn more about it here: goo.gle/3wAUwFh

[@MauriceVanrumbeke]

me too

[@r.m8146]

[@juanwononeyuan]

is this an actual demonstration, or is this to show what ur working towards in the future?

[@Jean_Charles_Hubert]

What will happen if we give this to very young children? What's your position on young children being educated by Google AI instead of their parents?

[@phazei]

✋

[@kancharlasuryateja]

Google glasses : as you can see, I’m not dead

[@BarelyAverageDude]

Yibambe!

[@majidurahman7106]

@@BarelyAverageDude beha

[@The_Mining_Pickaxe]

My death was... greatly exaggerated.

[@fernandolunadev]

Google glasses were born 10 years before the Machine Learning and speech synthesis developments catch the today's capacity. The good part of it is that they have the MKT hardware ready to refrit and sell it. They just need to give them a little twist to nuts and bolts!

[@godwinarchibong8204]

😂

[@youtube_gaming]

google glasses are back babyyy 😎

[@_tr11]

I also thought about that lol.

[@syrus3k]

Google glass you mean

[@_tr11]

@@syrus3k it's the same thing lol :)

[@DontReadMyProfilePicture566]

Don't read my name!

[@montavio]

It should have never left

[@monicaperez2843]

This would be a Godsend for Blind and Low Vision users!

[@Themightylivingsword]

Yea

[@muntahmahfuzsrestho.]

You're right.

[@olegtarasovrodionov]

Yea, but it takes Advanced subscription.

[@Blaze6108]

Yeah, at the end of the day it seems to me that the most sensible application for all this AI stuff is 1. portable secretary, and 2. medical device for the perceptively impaired. Although for 2. it would be pretty cool to eventually just develop bionic eyes and finish the job on hearing aids.

[@fuzziestlumpkin]

Not that you are necessarily wrong, but at this point that sentence has become an I meme lol.

[@curtishaynes8425]

Open AI's GPT 4o and now this. I'm loving this AI competition

[@Nicdehouwer]

Are you?

[@shenxi]

@@Nicdehouwer I'm loving it too! AI is awesome!

[@縣]

This isnt finished this is a "vision for the future" aka what they want to be able to do not what they are able to do

[@ZaynomR]

@@縣 i mean they did say they did it in real time so i think it is a real product. they def picked and chose what to show for the demo but still.

[@縣]

@@ZaynomR where did they say that?

[@brandallslibraryofwonder]

What a time to be alive

[@muntahmahfuzsrestho.]

We don't know the future of AI.

[@user-yv7dt5je1l]

Skynet is already here.

[@Sped55]

two minute papers reference?

[@slxdy]

The reference is crazy lol

[@StefanReich]

I don't know. There are better times. MUCH better

[@5amyak_]

Google Glasses making a strong comeback indeed

[@sumanmondal3992]

Every time I bet with 4ra, I know I'm getting the fairest odds out there 💯🔍

[@premiumdomains]

That was an incredible demo! The glasses especially blew me away.

[@nazid]

Google assistant + Google lens + Gemini + Glasses = Perfect combination

[@Google]

What excited you the most?

[@Ison2882]

Gotta go with glasses

[@sharminsultana9206]

@@Google being able to automate tasks on my phone with my voice.

[@bishoybasm553]

We hope that these features will be available in Egypt. We will always be the last people to receive these amazing features. I am a big fan of the basic Gemini system. Thank you. ❤

[@DrJacobGoodin]

Google Glass has RE-entered the chat😎

[@NithinJune]

ok i’ll admit it. That thumbnail made me click 😅

[@TravisLee33]

This is a big leap forward. Congratulations!

[@idontexist-satoshi]

You sure about that? Google are notorious for faking videos. Look at last time they showed us a video of 'real-time' things. It was all fake. For a company that have no 'motive' they sure do seem to be trying their hardest to try to impress people and sway over OpenAI's fans.

[@Google]

We're always learning and improving - thanks for being on the journey.

[@kiilee5963]

@@Google :D

[@Banana_Fusion]

@@Google Even Google is going for parasocial relationships!? /j

[@idontexist-satoshi]

@@Google What a way to tell everyone I'm right and you faked the whole thing.

[@teabag_exe]

The ability to read and understand schematics is a gamechanger Also being able to understand abstract toys' breed and drawings sounds so cool

[@corylong5808]

I’m skeptical it did more than read the words Server and DB and ignored the rest, as if those were the only two words on the board it would surely have the same answer, so this particular video isn’t really testing its ability to understand schematics.

[@mechafractal]

The ability for a dog and a soft toy to form a band is a game changer. AI can do anything these days.

[@Google]

So glad you think so!

[@Murv]

@@corylong5808 I mean adding a cache is the most easy answer ever

[@joshuabruno]

@@Google creep

[@adamdorsey3060]

4o seems to be faster with responses and the voice seems to be more human like

[@khonager]

but they don't have glasses YET

[@madhavraghu]

yea 4o seems to have a much better understanding of tone and inflection which makes it really powerful when it comes to speech. This just feels like a simple TTS of a chatbot

[@Zaphod_]

The fact that it remembered the glasses was pretty impressive though. Not sure if 4o could do that

[@lievais]

​@@madhavraghuI just think they turned up the dail on informal word choice and happy intention. It's almost to optimistic... This just sounds practical, to the point. Just another choice of how to dail in your AI personality. I bet Google could also make it react as your best buddy.

[@PramodhRachuri]

​@@Zaphod_well 4o remembered that someone walked behind the Host. Made funny gestures and left. I think it can remember the glasses

[@MrYesidfbr]

chatgpt 4o is better

[@negochristian1]

I saw a white man coding and even a white dog! Is this allowed in Google?

[@AI-Consultant]

Don't worry, Google is an equal opportunity employer. They'll accept any dog, regardless of its fur color

[@LarsLarsen77]

@@AI-Consultant Except white, that's racist.

[@mechafractal]

“our vision for the future” means they don’t have anything that does that. We had that vision in sci-fi decades ago. Put it in our hands and I’ll get excited about it.

[@saukash]

Google trying to catch up to the GPT-4o hype train

[@djbombba]

Waiting for Apple response 👀

[@serkan5951]

@@djbombba Meet Apple Car 🤡

[@ondrazposukie]

lol Astra product and its announcement was not made in a single day

[@chawlagrv]

what a noob comment

[@yak-machining]

​@@ondrazposukie but a quick response from google because of the Livestream yesterday is possible

[@abdulr2084]

Your vision is already realized by yesterday's OAI announcement

[@almor2445]

Gpt4o is so much better but the presentation from Google was next lever boring!

[@ankitnmnaik229]

Basically both were working on same things.

[@Sped55]

this looks a bit better than GPT4o but it is probably better for some things and worse for others.

[@chawlagrv]

It will become better at a faster rate though with the huge android user base

[@Blaze6108]

It's good that they're competing. Big tech is already monopolistic enough as it is.

[@eshanprashar9361]

This is MINDBOGGLING! What a time to be alive! Quick question: I noticed that the mic was on mute when the presenter asked "what is that part of the speaker called" at 0:22. How did the AI pick up that command? Also, more demos of system design use cases please!!

[@Mezzocrip]

Chat gpt 4.o is far far better

[@sandenium]

Thank you Mr troll 😄

[@mraniketrai]

🗿

[@sanjaypatel6746]

Why, don't people see what game openai is playing they just lunch oneday before it and as it is first you are amazed and now Google one seemed just fast and open ai one game changing they know this strategy from start

[@faagunman]

regarding this feature; in what sense?

[@Mezzocrip]

@@sandenium 😅

[@Avm371998]

Damn this would be so useful for people that keep losing their stuff

[@Google]

Good catch, we're super excited about that!

[@RedOneM]

Remember, when something is free, you're the product. You will hand over your data (video recording, like the glasses in this clip) willingly. I rather wait for open source local models to become this advanced.

[@LarsLarsen77]

The problem is you need chips the size of a small pizza that dump 25KW of heat to train these things. Do you have one of those?

[@RedOneM]

@@LarsLarsen77 No, you don't need one of those in the future. Harvard Mark II mass was 23000kg and that thing did barely any calculations by today's standard. AI models will become much tinier when we learn to innovate (removing deadweight parameters as example), tech always aims for efficiency. I can already today run a small llama3 model locally which is not that far off from GPT4.

[@kingki1953]

I think it's ok if we are the product. Atleast we have freedom to use it or not. Like most social media in internet. We control the tech or the tech control us, it's our choice.

[@dougchampion8084]

youtube is free dude, so is google search, so is Gmail, so is the internet.

[@saukash]

Google tried, but GPT-4o is still better

[@crimsonlion100]

Okay, you got me Google. I'm genuinely impressed. This would be revolutionary for people with visual impairments.

[@sharif47]

Damn, just when Open AI announced their version of it... While GPT-4o sounds more natural, this one seems more versatile. Hope this one can sound more natural as well.

[@YamilGonzales]

The link on the description doesn't work.

[@Potato_PlayzYT2024]

Type the URL into google or google chrome

[@zt9233]

I just felt a disturbance in the force. It was as if the sounds of 200 million jobs all cried at our at once. And the en vanished.

[@sandeepsrinivas7]

You can tell that they cooked up some random BS in just a day to show the investors that they got it too. OpenAI is dominating Google.

[@mclandeg1]

FREE PALESTINE 🇵🇸

[@muntahmahfuzsrestho.]

Mind blowing. 🤯 I want these glasses. 😊

[@user-Jfoweisawsome]

Me too

[@bombombalu]

To do what exactly?

[@Hysus]

​@@bombombalu Can experience iron man's live.

[@vectoralphaSec]

@@bombombalu Anything you want.

[@muntahmahfuzsrestho.]

@@bombombalu Also, you can use this in exams as regular glasses. 😆

[@average_rite]

This looks like ancient technology bcuz of open ai's gpt 4o

[@chawlagrv]

noob

[@KetaFPV]

Is this more deceptive editing like the last one you did?

[@jaimelubrani4435]

Very cool and very unnerving. I can't wait for the ability to give an AI a stranger's photos and ask the AI to pinpoint where the photo was taken. Or to casually walk passed someone at their computer and ask the AI to memorize all the keystrokes they press when they're logging into their bank account. But the limerick was cool :) Very cool technology

[@user-yv7dt5je1l]

Still waiting for the google glasses, thanks.

[@AI-Consultant]

from 2012? lol

[@joekavalauskas8767]

They are the rayban meta glasses

[@ေက်ာ္ကုိကုိကုိကုိ]

​@@joekavalauskas87671:24

[@vectoralphaSec]

Using those glasses on while I'm reading a technical math or programming book or online and asking questions about what we are seeing would be a game changer. Imagine trying to learn a new math equation or algorithm and asking the glasses to explain it while looking at it. Learning would be dramatically improved.

[@akjhdkjahskjdlf]

This would really be amazing to have!

[@TheNewton]

being able to unpack parts of a formula/symbol in real time would be something

[@dwat3r]

Learning would be dramatically impaired. A person learns when it hurts, when it's hard to get it.

[@denissetiawan3645]

@@dwat3r a person learns what benefit him/her most.

[@broski120]

@@dwat3ridiot

[@nivedithapatil98]

Wow, Google! Project Astra is seriously impressive. The future of AI assistants is looking bright. Imagine a world where AI can seamlessly understand us and the world around us. This could be a game-changer for education, accessibility, etc. #GoogleIO #TheFutureIsHere

[@dorly39]

I do think the google glass was ahead of its time and now is the ideal time to rerelease it especially after Apple and Facebook sudden interest in this market. Greetings from 🇭🇹

[@NickZaden]

Well played here! Excited to see more of this.

[@Google]

Ready to make magic happen together! ✨

[@JohnnyB.]

Wow, that truly is amazing! Well done Google. The way it remembered where your glasses were while you were just scanning over the room is insanely impressive!!

[@Google]

We're excited for you to try it!

[@JohnnyB.]

@@Google Can't wait!!

[@fii_89639]

The real world application of the Needle-in-a-Haystack memory test

[@DabidarZ]

I mean it's cool and all but all OpenAI has to do is make glasses and GPT-4o Video/Voice Chat sounds SOOO much better, I'd say OpenAI still beat them on this.

[@chelovechik]

THIS IS EXACTLY HOW I IMAGINED A PERFECT AI ASSISTANT!!! This does precisely what I would want it to do.

[@Google]

Glad to hear you're excited!

[@mark_shagal]

@@GoogleI’m glad you’re answering comments here, huge respect to the community managers and the one who made this decision!

[@DreamingConcepts]

@@mark_shagal you know its a AI, right?

[@mark_shagal]

@@DreamingConcepts you're sure it's not a community manager?

[@DreamingConcepts]

@@mark_shagal yes, unless there's hundreds of community managers to answer all comments on all videos 24/7

[@SamuraiSandi]

Google dropping Astra a day After OpenAI dropped Gpt 4o is like the Drake and Kendrick Lamar beef all over again

[@utshodas001]

If it's not fake and staged then a great product.

[@BonBonBonni]

Wow, it's amazing how quickly the AI has progressed! I don't think anyone anticipated it becoming this good so soon

[@Google]

Your enthusiasm means a lot - thanks for being on the journey with us.

[@Data-qj7mo]

Why do I get the feeling that it's Gemini responding to comments? 😂

[@supermj767]

no way i could keep working with that lady doing this right next to me.

[@samwight]

The last Gemini demo was faked so there's no reason to believe this wasn't also faked lol

[@jacobyoung2045]

My thoughts exactly 😂

[@lynxnexux]

There have been numerous comments, but no one has asked the right questions. I doubt the Google experts will respond, but let's ask the pertinent questions anyway: 1. Is it ready to use? 2. Is there a free version available, and if so, what is it called? 3. Does the free version work on iOS? 4. What are the costs for the paid versions? Thx in advance.

[@NickKhoOfficial]

What glasses are she wearing? I want a pair!

[@Drone256]

They can't do it live like OpenAI ? Ouch. We all remember the last time Google gave us pre-generated content.

[@SilentTrek]

This has the potential to help so many people and could save lives. So many possibilities, can't wait to see what the world will look like in even a few years!

[@Google]

We're always learning and improving - thanks for being on the journey!

[@haggle196]

Available to you this decade !

[@jacobfavreault4067]

"My death was... Greatly exaggerated" - Google Glass

[@ConcietedMuchXD]

Without a doubt one of the best commercials of the year.

[@Google]

We're glad you're loving it!

[@dudleydooright]

How exciting! Can you imagine this kind of intelligence on every street corner, planted in robots walking around?! Who needs freedom? The government will be able to see our every move and know where we are any time we are in public!

[@enage.]

GPT-4o 🤨

[@GaminylGames]

So it's like ChatGPT's update but 10 times more boring and less intuitive. Got it

[@RicardoCantor]

Google is on fire!!! Enjoying my Pixel Phone and Waiting for those glasses now!!!!

[@IndigoGPT]

When will we get access to this?!?! I wpuld love to try it!

[@Dika-fe5zh]

So much potential! 👏

[@Google]

We can't wait for you to try it for yourself!

[@Denso481]

Google: "hey look, this is our answer to GPT-4o, we're totally not falling behind, and please forget about the black nazis and very kings fiasco"

[@Amine382]

Are are those Google glasses?

[@rahulgoel968]

As a digital marketer i am scared beacuse ai take the jobs of seo in furure. I pray that ai will not come in india because it takes lots of jobs of india. My suggestion is that changing in the search engine is not a good idea. We should launch our ai app and if people are using ai features then they use the google ai app rather than changing in the search engine beacuse they are very effecting the website ranking or specially seo😢😢😢😢

[@Gallowglass7]

Hats off to the teams! Exciting times, indeed.

[@Google]

Thanks for being a part of this with us!

[@grafando]

I wonder if this would enable regular people to do mechanic work? I'm sick of paying the mechanic thousands of dollars for a repair where the part was a couple hundred.

[@Themightylivingsword]

This is so futuristic

[@Google]

Thrilled you're as excited as we are!

[@weeb3277]

with my $300K salary at google where can I buy a house to avoid minorities and high crime rates but also don't want to travel in public transit for the same reasons

[@harbirsingh7266]

This will now finally complete all those DIY Iron Man helmets!

[@krin_c747]

people are so much overwhelmed by GPT4o > that they are not cherring the announcements 😂 like they are completing the boring assignment again 😝

[@klebrigetasten]

its not even out and already behind the curve really bad considering the resources google has not even a live demo... tells you all you need to know search also still sucks google is a failing business dependent on advertising revenue

[@LarsLarsen77]

Google offered me a job once, turning them down was the best decision I've ever made. I'd rather chew glass.

[@fraserdab]

AI is getting fun

[@qrowing]

Google saw OpenAI drop GPT4o, and were like "..Oh. Well we better get moving" lmao. You love to see competition fostering innovation!

[@adolphgracius9996]

they had the keynote already booked and open ai dropped the day before, they were nervous tho😂

[@clerothsun3933]

This is it. The stupidest comment on this video.

[@HerezCheez]

Google seems so insecure after gtp4o is released

[@koredebashir1747]

Looks dope!

[@Google]

Glad to hear you're excited about that one!

[@MariaReglaLARAMIRANDA-nr8oo]

Gracias por tratar de ayudarme, pero viviendo en Cuba y siendo del pueblo de apie éso es imposible, nosotros no sabemos nada de los avances mundiales en ninguna categoría¡!.

[@francobattista9581]

This explains why OpenIA made their realease

[@altierebtc]

Remember guys its still only 2024 imagine 2034

[@itsthesola10]

oh look, the rabbit r1 killer's already here

[@U00v]

The r1 is pointless, they spend their money on the marketing

[@Science-vt4vg]

Hey Google, I have always been a great admirer of your technology since my childhood. Now I am a graduate student in data science. Please could you provide me with free access to your Gemini advanced products ?

[@aniketgowardhan1456]

Amazing 😍

[@Google]

We're just as excited as you are ✨

[@AyeItsMike]

I love Google, but their ai is the most frustrating clueless thing I've ever seen. One thing these developers aren't taking into consideration is not all people are going to be asking simple questions. I'm betting you if you ask it about computer components or car details you'll just give you a brief description of what you asked about and not help you with specifics I uploaded a video on my channel about ai memes and Google's AI was the most difficult

[@AyeItsMike]

kzbin.info/www/bejne/q6Spe56Dabt4gKMsi=HCj0OWwjbTc78SDE

[@Bryanwest2571]

You have to release this Text To Speech voice now! We are tired of only one option since it was launched as bard.

[@simonmage8629]

PLOT TWIST, all comments are answered by Astra with the Google account 😎

[@TheBestLemonOutThere]

OMG HI GOOGLE❤

[@ishan9088]

This is just the beginnig of AI revolution and we can't even think whats coming in future .

[@goldnutter412]

Personalized matters🤔🤫😘

[@kuldeepshihn.vaghela7620]

I'm a partially blind man. I was looking for pixel phone but now I request you to please make glasses so I can wear be self dependent . Plz plz plz❤❤

[@anselmorosa]

HUGE! if true

[@awkward8773]

False

[@williammodisette8690]

I'd buy this. The glasses paired with a phone looks both useful and entertaining.

[@ĐàoLê-p1n]

9 0:22 0:22 0:23

[@mistikalcanavarlarparlamen3265]

The race is on!!!! Gpt 4 o or Astra?

[@Madeintheshade65]

GPT-4o it sounded more natural

[@Derpsider]

gpt4o clear winner. Just see the 'be my eyes' video from OpenAI, and what user experience they offered.

[@dougchampion8084]

Stop thinking like that. Water or air? Do you listen to one band, like one TV show, use one app?

[@pneumonoultramicroscopicsi4065]

​@@dougchampion8084 agreed

[@mistikalcanavarlarparlamen3265]

@@dougchampion8084just think of it as a capitalistic rivalry.

[@thesyntheticstudio]

And so it begins, the epic quest to zap away pesky gadgets like AI Pin and Rabit!

[@Jules-hn6un]

It’s like Google forgot why Google Glass failed dismally

[@Cam-ie3cl]

ai and better internet is a big change

[@yzz9833]

These aren’t even the same product lol, the glasses just have a camera.

[@Jules-hn6un]

@@yzz9833 lmao, they failed because they had a camera and icked everyone out. Adding AI doesn’t change that

[@faisalron]

Really excited about this! The Great Sage in Tensura could come to reality for sure!

[@crimjobbers]

Amazing like only Google could

[@RealmyTheMan]

openai did this yesterday what are you talking about

[@sarpolaris1985]

I'm now curios if it's the google team who's responding to these comments or is it AI?

[@MattGathu]

Did Google just pull a Kendrick Lamar on OpenAI? Dope!

[@maevwat]

openai probably had insiders and decided to announce gpt4o before google io

[@okman9684]

Google got an awakening after open Ai event 😁

[@jesuschrist194]

This is awesome if it works as presented, I'd love to try it myself and find out if it can actually piece together information from contextual clues and infer information from evidence that isn't conclusive. This is TRULY the future, holy schnitzels. Please be real!! And thank you if it is :) I'm ready to move on!!!