Project Astra: Our vision for the future of AI assistants

Introducing Project Astra. We created a demo in which a tester interacts with a prototype of AI agents supported by our multimodal foundation model, Gemini.
There are two continuous takes: one with the prototype running on a Google Pixel phone and another on a prototype glasses device.
The agent takes in a constant stream of audio and video input. It can reason about its environment in real time and interact with the tester in a conversation about what it is seeing.
Learn more about Project Astra: goo.gle/3wAUwFh
#GoogleIO2024
Watch the full Google I/O 2024 keynote: www.youtube.co...
To watch this keynote with American Sign Language (ASL) interpretation, please click here: www.youtube.co...
#GoogleIO
Subscribe to our Channel: / google
Find us on X: / google
Watch us on TikTok: / google
Follow us on Instagram: / google
Join us on Facebook: / google

[@Google]

Thrilled to be able to share with everyone how exciting Project Astra really is. Drop a ✋if you're ready to give it a go, and learn more about it here: goo.gle/3wAUwFh

[@MauriceVanrumbeke]

me too

[@r.m8146]

[@juanwononeyuan]

is this an actual demonstration, or is this to show what ur working towards in the future?

[@Jean_Charles_Hubert]

What will happen if we give this to very young children? What's your position on young children being educated by Google AI instead of their parents?

[@phazei]

✋

[@monicaperez2843]

This would be a Godsend for Blind and Low Vision users!

[@Themightylivingsword]

Yea

[@muntahmahfuzsrestho.]

You're right.

[@olegtarasovrodionov]

Yea, but it takes Advanced subscription.

[@Blaze6108]

Yeah, at the end of the day it seems to me that the most sensible application for all this AI stuff is 1. portable secretary, and 2. medical device for the perceptively impaired. Although for 2. it would be pretty cool to eventually just develop bionic eyes and finish the job on hearing aids.

[@fuzziestlumpkin]

Not that you are necessarily wrong, but at this point that sentence has become an I meme lol.

[@youtube_gaming]

google glasses are back babyyy 😎

[@_tr11]

I also thought about that lol.

[@syrus3k]

Google glass you mean

[@_tr11]

@@syrus3k it's the same thing lol :)

[@DontReadMyProfilePicture566]

Don't read my name!

[@montavio]

It should have never left

[@kancharlasuryateja]

Google glasses : as you can see, I’m not dead

[@BarelyAverageDude]

Yibambe!

[@majidurahman7106]

@@BarelyAverageDude beha

[@The_Mining_Pickaxe]

My death was... greatly exaggerated.

[@fernandolunadev]

Google glasses were born 10 years before the Machine Learning and speech synthesis developments catch the today's capacity. The good part of it is that they have the MKT hardware ready to refrit and sell it. They just need to give them a little twist to nuts and bolts!

[@godwinarchibong8204]

😂

[@brandallslibraryofwonder]

What a time to be alive

[@muntahmahfuzsrestho.]

We don't know the future of AI.

[@user-yv7dt5je1l]

Skynet is already here.

[@Sped55]

two minute papers reference?

[@slxdy]

The reference is crazy lol

[@StefanReich]

I don't know. There are better times. MUCH better

[@curtishaynes8425]

Open AI's GPT 4o and now this. I'm loving this AI competition

[@Nicdehouwer]

Are you?

[@shenxi]

@@Nicdehouwer I'm loving it too! AI is awesome!

[@縣]

This isnt finished this is a "vision for the future" aka what they want to be able to do not what they are able to do

[@ZaynomR]

@@縣 i mean they did say they did it in real time so i think it is a real product. they def picked and chose what to show for the demo but still.

[@縣]

@@ZaynomR where did they say that?

[@TravisLee33]

This is a big leap forward. Congratulations!

[@idontexist-satoshi]

You sure about that? Google are notorious for faking videos. Look at last time they showed us a video of 'real-time' things. It was all fake. For a company that have no 'motive' they sure do seem to be trying their hardest to try to impress people and sway over OpenAI's fans.

[@Google]

We're always learning and improving - thanks for being on the journey.

[@kiilee5963]

@@Google :D

[@Banana_Fusion]

@@Google Even Google is going for parasocial relationships!? /j

[@idontexist-satoshi]

@@Google What a way to tell everyone I'm right and you faked the whole thing.

[@teabag_exe]

The ability to read and understand schematics is a gamechanger Also being able to understand abstract toys' breed and drawings sounds so cool

[@corylong5808]

I’m skeptical it did more than read the words Server and DB and ignored the rest, as if those were the only two words on the board it would surely have the same answer, so this particular video isn’t really testing its ability to understand schematics.

[@mechafractal]

The ability for a dog and a soft toy to form a band is a game changer. AI can do anything these days.

[@Google]

So glad you think so!

[@Murv]

@@corylong5808 I mean adding a cache is the most easy answer ever

[@joshuabruno]

@@Google creep

[@5amyak_]

Google Glasses making a strong comeback indeed

[@premiumdomains]

That was an incredible demo! The glasses especially blew me away.

[@DrJacobGoodin]

Google Glass has RE-entered the chat😎

[@crimsonlion100]

Okay, you got me Google. I'm genuinely impressed. This would be revolutionary for people with visual impairments.

[@ConcietedMuchXD]

Without a doubt one of the best commercials of the year.

[@Google]

We're glad you're loving it!

[@nazid]

Google assistant + Google lens + Gemini + Glasses = Perfect combination

[@Google]

What excited you the most?

[@Ison2882]

Gotta go with glasses

[@sharminsultana9206]

@@Google being able to automate tasks on my phone with my voice.

[@muntahmahfuzsrestho.]

Mind blowing. 🤯 I want these glasses. 😊

[@user-Jfoweisawsome]

Me too

[@bombombalu]

To do what exactly?

[@Hysus]

​@@bombombalu Can experience iron man's live.

[@vectoralphaSec]

@@bombombalu Anything you want.

[@muntahmahfuzsrestho.]

@@bombombalu Also, you can use this in exams as regular glasses. 😆

[@SilentTrek]

This has the potential to help so many people and could save lives. So many possibilities, can't wait to see what the world will look like in even a few years!

[@Google]

We're always learning and improving - thanks for being on the journey!

[@bishoybasm553]

We hope that these features will be available in Egypt. We will always be the last people to receive these amazing features. I am a big fan of the basic Gemini system. Thank you. ❤

[@JohnnyB.]

Wow, that truly is amazing! Well done Google. The way it remembered where your glasses were while you were just scanning over the room is insanely impressive!!

[@Google]

We're excited for you to try it!

[@JohnnyB.]

@@Google Can't wait!!

[@fii_89639]

The real world application of the Needle-in-a-Haystack memory test

[@ekanshsudha]

I really feel happy to see that everyday when you wake up you have something to look up to...and it is so fascinating that it makes me feel worth living for.

[@vectoralphaSec]

Using those glasses on while I'm reading a technical math or programming book or online and asking questions about what we are seeing would be a game changer. Imagine trying to learn a new math equation or algorithm and asking the glasses to explain it while looking at it. Learning would be dramatically improved.

[@akjhdkjahskjdlf]

This would really be amazing to have!

[@TheNewton]

being able to unpack parts of a formula/symbol in real time would be something

[@dwat3r]

Learning would be dramatically impaired. A person learns when it hurts, when it's hard to get it.

[@denissetiawan3645]

@@dwat3r a person learns what benefit him/her most.

[@broski120]

@@dwat3ridiot

[@beancanicholson2241]

This is amazing! love to see how far things have come from glass

[@Gallowglass7]

Hats off to the teams! Exciting times, indeed.

[@Google]

Thanks for being a part of this with us!

[@user-pl9mh3bs1r]

This is incredible ❤😮

[@RedOneM]

Remember, when something is free, you're the product. You will hand over your data (video recording, like the glasses in this clip) willingly. I rather wait for open source local models to become this advanced.

[@LarsLarsen77]

The problem is you need chips the size of a small pizza that dump 25KW of heat to train these things. Do you have one of those?

[@RedOneM]

@@LarsLarsen77 No, you don't need one of those in the future. Harvard Mark II mass was 23000kg and that thing did barely any calculations by today's standard. AI models will become much tinier when we learn to innovate (removing deadweight parameters as example), tech always aims for efficiency. I can already today run a small llama3 model locally which is not that far off from GPT4.

[@kingki1953]

I think it's ok if we are the product. Atleast we have freedom to use it or not. Like most social media in internet. We control the tech or the tech control us, it's our choice.

[@dougchampion8084]

youtube is free dude, so is google search, so is Gmail, so is the internet.

[@sumanmondal3992]

Every time I bet with 4ra, I know I'm getting the fairest odds out there 💯🔍

[@dorly39]

I do think the google glass was ahead of its time and now is the ideal time to rerelease it especially after Apple and Facebook sudden interest in this market. Greetings from 🇭🇹

[@YamilGonzales]

The link on the description doesn't work.

[@Potato_PlayzYT2024]

Type the URL into google or google chrome

[@nivedithapatil98]

Wow, Google! Project Astra is seriously impressive. The future of AI assistants is looking bright. Imagine a world where AI can seamlessly understand us and the world around us. This could be a game-changer for education, accessibility, etc. #GoogleIO #TheFutureIsHere

[@BonBonBonni]

Wow, it's amazing how quickly the AI has progressed! I don't think anyone anticipated it becoming this good so soon

[@Google]

Your enthusiasm means a lot - thanks for being on the journey with us.

[@Data-qj7mo]

Why do I get the feeling that it's Gemini responding to comments? 😂

[@kokonutmaster]

You can obviously tell this is not a real demo, but it's an amazing concept. Can't wait!

[@vimalcurio]

It still a concept for Google ?and there openai has already created the Jarvis

[@Avm371998]

Damn this would be so useful for people that keep losing their stuff

[@Google]

Good catch, we're super excited about that!

[@GaneshKumar-ku3zd]

Love the fair play at 4RBT, they really make sure everyone has a shot to win 🎰👀

[@NickZaden]

Well played here! Excited to see more of this.

[@Google]

Ready to make magic happen together! ✨

[@Dika-fe5zh]

So much potential! 👏

[@Google]

We can't wait for you to try it for yourself!

[@chelovechik]

THIS IS EXACTLY HOW I IMAGINED A PERFECT AI ASSISTANT!!! This does precisely what I would want it to do.

[@Google]

Glad to hear you're excited!

[@mark_shagal]

@@GoogleI’m glad you’re answering comments here, huge respect to the community managers and the one who made this decision!

[@DreamingConcepts]

@@mark_shagal you know its a AI, right?

[@mark_shagal]

@@DreamingConcepts you're sure it's not a community manager?

[@DreamingConcepts]

@@mark_shagal yes, unless there's hundreds of community managers to answer all comments on all videos 24/7

[@LawGrant]

Gemini Vs Gpt 4.o.... You both are fabulous ❤❤❤ Love and great respect to all Great minds working behind this magic ✨✨

[@haggle196]

Available to you this decade !

[@jesuschrist194]

This is awesome if it works as presented, I'd love to try it myself and find out if it can actually piece together information from contextual clues and infer information from evidence that isn't conclusive. This is TRULY the future, holy schnitzels. Please be real!! And thank you if it is :) I'm ready to move on!!!

[@RicardoCantor]

Google is on fire!!! Enjoying my Pixel Phone and Waiting for those glasses now!!!!

[@eshanprashar9361]

This is MINDBOGGLING! What a time to be alive! Quick question: I noticed that the mic was on mute when the presenter asked "what is that part of the speaker called" at 0:22. How did the AI pick up that command? Also, more demos of system design use cases please!!

[@user-yv7dt5je1l]

Still waiting for the google glasses, thanks.

[@AI-Consultant]

from 2012? lol

[@joekavalauskas8767]

They are the rayban meta glasses

[@jaimelubrani4435]

Very cool and very unnerving. I can't wait for the ability to give an AI a stranger's photos and ask the AI to pinpoint where the photo was taken. Or to casually walk passed someone at their computer and ask the AI to memorize all the keystrokes they press when they're logging into their bank account. But the limerick was cool :) Very cool technology

[@abdulr2084]

Your vision is already realized by yesterday's OAI announcement

[@almor2445]

Gpt4o is so much better but the presentation from Google was next lever boring!

[@ankitnmnaik229]

Basically both were working on same things.

[@Sped55]

this looks a bit better than GPT4o but it is probably better for some things and worse for others.

[@chawlagrv]

It will become better at a faster rate though with the huge android user base

[@Blaze6108]

It's good that they're competing. Big tech is already monopolistic enough as it is.

[@winstonhorn9683]

Just yes…. Hope and excitement is back in tech! I’m rooting for all the companies, present and future. Amazing work all!11

[@Google]

Couldn't agree more 🙌

[@mechafractal]

“our vision for the future” means they don’t have anything that does that. We had that vision in sci-fi decades ago. Put it in our hands and I’ll get excited about it.

[@sharif47]

Damn, just when Open AI announced their version of it... While GPT-4o sounds more natural, this one seems more versatile. Hope this one can sound more natural as well.

[@koredebashir1747]

Looks dope!

[@Google]

Glad to hear you're excited about that one!

[@shashankyadav5703]

Future looks super cool

[@Google]

The Gemini era is here ✨

[@adamdorsey3060]

4o seems to be faster with responses and the voice seems to be more human like

[@khonager]

but they don't have glasses YET

[@madhavraghu]

yea 4o seems to have a much better understanding of tone and inflection which makes it really powerful when it comes to speech. This just feels like a simple TTS of a chatbot

[@Zaphod_]

The fact that it remembered the glasses was pretty impressive though. Not sure if 4o could do that

[@lievais]

​@@madhavraghuI just think they turned up the dail on informal word choice and happy intention. It's almost to optimistic... This just sounds practical, to the point. Just another choice of how to dail in your AI personality. I bet Google could also make it react as your best buddy.

[@PramodhRachuri]

​@@Zaphod_well 4o remembered that someone walked behind the Host. Made funny gestures and left. I think it can remember the glasses

[@faisalron]

Really excited about this! The Great Sage in Tensura could come to reality for sure!

[@Themightylivingsword]

This is so futuristic

[@Google]

Thrilled you're as excited as we are!

[@Nasser-bp6qf]

big leap forward. Congrats

[@Google]

Thanks for being a part of this with us!

[@aniketgowardhan1456]

Amazing 😍

[@Google]

We're just as excited as you are ✨

[@fraserdab]

AI is getting fun

[@qrowing]

Google saw OpenAI drop GPT4o, and were like "..Oh. Well we better get moving" lmao. You love to see competition fostering innovation!

[@adolphgracius9996]

they had the keynote already booked and open ai dropped the day before, they were nervous tho😂

[@clerothsun3933]

This is it. The stupidest comment on this video.

[@gavinphotography]

Incredible 😮

[@Google]

Ready for you to give it a go!

[@francobattista9581]

This explains why OpenIA made their realease

[@jodu]

This will be just incredible for a visually impaired person i can't imagine the good this will bring and the glasses are back!!

[@NickKhoOfficial]

What glasses are she wearing? I want a pair!

[@IndigoGPT]

When will we get access to this?!?! I wpuld love to try it!

[@average_rite]

This looks like ancient technology bcuz of open ai's gpt 4o

[@chawlagrv]

noob

[@BR-hi6yt]

Amazing! Great demo.

[@Google]

Thanks for tuning in!

[@NithinJune]

ok i’ll admit it. That thumbnail made me click 😅

[@Bryanwest2571]

You have to release this Text To Speech voice now! We are tired of only one option since it was launched as bard.

[@Science-vt4vg]

Hey Google, I have always been a great admirer of your technology since my childhood. Now I am a graduate student in data science. Please could you provide me with free access to your Gemini advanced products ?

[@anselmorosa]

HUGE! if true

[@awkward8773]

False

[@vectoralphaSec]

Google Glasses Lets GOOOOO!!!!

[@klebrigetasten]

its not even out and already behind the curve really bad considering the resources google has not even a live demo... tells you all you need to know search also still sucks google is a failing business dependent on advertising revenue

[@LarsLarsen77]

Google offered me a job once, turning them down was the best decision I've ever made. I'd rather chew glass.

[@brazil9691]

Hello Google I am excited about this, when you Laugh it ? 🎉

[@saukash]

Google trying to catch up to the GPT-4o hype train

[@djbombba]

Waiting for Apple response 👀

[@serkan5951]

@@djbombba Meet Apple Car 🤡

[@ondrazposukie]

lol Astra product and its announcement was not made in a single day

[@chawlagrv]

what a noob comment

[@yak-machining]

​@@ondrazposukie but a quick response from google because of the Livestream yesterday is possible

[@drcanoro]

Google Glasses is the star of the show! They are getting rebuilt and they are coming back!

[@KetaFPV]

Is this more deceptive editing like the last one you did?

[@Amine382]

Are are those Google glasses?

[@samakshty]

THE FUTURE IS SO BRIGHT 😌

[@Google]

Thrilled you're as excited as we are!

[@MrYesidfbr]

chatgpt 4o is better

[@chetanghadge8505]

Fair winnings are not a myth at 4ra, experienced it first hand last week 🌟👐

[@mistikalcanavarlarparlamen3265]

The race is on!!!! Gpt 4 o or Astra?

[@Madeintheshade65]

GPT-4o it sounded more natural

[@Derpsider]

gpt4o clear winner. Just see the 'be my eyes' video from OpenAI, and what user experience they offered.

[@dougchampion8084]

Stop thinking like that. Water or air? Do you listen to one band, like one TV show, use one app?

[@pneumonoultramicroscopicsi4065]

​@@dougchampion8084 agreed

[@mistikalcanavarlarparlamen3265]

@@dougchampion8084just think of it as a capitalistic rivalry.

[@Cuxxie]

This is beyond imagination. Goodluck On your journey Google!

[@Google]

Innovation is a team effort! Thank you for being part of it.

[@negochristian1]

I saw a white man coding and even a white dog! Is this allowed in Google?

[@AI-Consultant]

Don't worry, Google is an equal opportunity employer. They'll accept any dog, regardless of its fur color

[@LarsLarsen77]

@@AI-Consultant Except white, that's racist.

[@SeanOHanlon]

In a word: Fascinating

[@utshodas001]

If it's not fake and staged then a great product.

[@Adwrells]

I am stunned, its so awesome and useful in daily life. This would make task much more easy.

[@Google]

So ready to hear what you think

[@Adwrells]

@@Google this would be helpful for kids or elderly people and of course us fr

[@harbirsingh7266]

This will now finally complete all those DIY Iron Man helmets!

[@pick001]

The invention of the internet lead us to this and is going to change the future, I hope this is for a better future congrats!!

[@crimjobbers]

Amazing like only Google could

[@RealmyTheMan]

openai did this yesterday what are you talking about

[@massimobuonaiuto8753]

great guys!

[@mclandeg1]

FREE PALESTINE 🇵🇸

[@alphariss]

this will demolish every smartphone on the market, if they include this on the next Pixel phone, hands down.

[@user-yv7dt5je1l]

They will still need the phone to operate.

[@dwat3r]

I never use assistant

[@itsthesola10]

oh look, the rabbit r1 killer's already here

[@U00v]

The r1 is pointless, they spend their money on the marketing

[@chillax1631]

oohh the ai is becoming amazing day by day exited about this new google glasses

[@Mezzocrip]

Chat gpt 4.o is far far better

[@sandenium]

Thank you Mr troll 😄

[@mraniketrai]

🗿

[@sanjaypatel6746]

Why, don't people see what game openai is playing they just lunch oneday before it and as it is first you are amazed and now Google one seemed just fast and open ai one game changing they know this strategy from start

[@faagunman]

regarding this feature; in what sense?

[@Mezzocrip]

@@sandenium 😅

[@Pancakes037]

Wow... This is so cool. Im hoping AI will lead us to find more internet security and solutions against cyber threats

[@goldnutter412]

Personalized matters🤔🤫😘

[@digital-nomad]

Truly the biggest news in AI vision this week

[@TSUKASAYUGIISMYHUSBAND]

OMG HI GOOGLE❤

[@samwight]

The last Gemini demo was faked so there's no reason to believe this wasn't also faked lol

[@jacobyoung2045]

My thoughts exactly 😂

[@vaphualfpv]

I am so thrilled for this 🔥

[@sandeepsrinivas7]

You can tell that they cooked up some random BS in just a day to show the investors that they got it too. OpenAI is dominating Google.

[@jacobfavreault4067]

"My death was... Greatly exaggerated" - Google Glass

[@Jules-hn6un]

It’s like Google forgot why Google Glass failed dismally

[@Cam-ie3cl]

ai and better internet is a big change

[@yzz9833]

These aren’t even the same product lol, the glasses just have a camera.

[@Jules-hn6un]

@@yzz9833 lmao, they failed because they had a camera and icked everyone out. Adding AI doesn’t change that