Kyutais New "VOICE AI" SHOCKS The ENTIRE INDUSTRY!" (Beats GPT4o!)

Learn A.I With me - www.skool.com/postagiprepardness
🐤 Follow Me on Twitter / theaigrid
🌐 Checkout My website - theaigrid.com/
Links From Todays Video:
x.com/kyutai_labs/status/1808...
x.com/kyutai_labs/status/1808...
kyutai.org/
Welcome to my channel where i bring you the latest breakthroughs in AI. From deep learning to robotics, i cover it all. My videos offer valuable insights and perspectives that will expand your knowledge and understanding of this rapidly evolving field. Be sure to subscribe and stay updated on my latest videos.
Was there anything i missed?
(For Business Enquiries) [e-mail]
#LLM #Largelanguagemodel #chatgpt
#AI
#ArtificialIntelligence
#MachineLearning
#DeepLearning
#NeuralNetworks
#Robotics
#DataScience

[@JohnSmith-gt3be]

More pressure for OpenAI to release GPT-4o voice. Good.

[@tracy419]

Hopefully they are smart enough to release it when they are actually ready, and not when people think they should because there might be competition.

[@petersmyczek2297]

@@tracy419 💯

[@timber8403]

@tracy419 Right. Openai have nothing to be concerned about. I tried this thing out and it was very poor.

[@brandongillett2616]

Hasn't worked with SORA 🤷‍♂️

[@bdown]

@@tracy419then they should’ve waited to announce it until they were ready to release it

[@Silas2-p7c]

This is a GPT2 moment. It’s only a matter of time before voice models become the new LLMs.

[@brandongillett2616]

True

[@shidheadmemes]

im fucking SHOCKED, my legs are SHAKING, this QUITE LITERALLY BLEW MY MIND, my grandmother STOOD UP from her GRAVE because she was so SHOCKED

[@LulzAsh]

But are you stunned?

[@peristiloperis7789]

you are just beeing silly.

[@drrains]

I too am shocky shocked

[@Bregylais]

THE INDUSTRY is in SHAMBLES!

[@madshader]

Mattvid pro did a demo with it just now, and Moshi is pretty bad.

[@EDLR234]

To the complainers. It's all in the context. It's a tiny quantized model, open source, and made by a small independent team of just 8 people, from scratch, in 6 months! It's so small their aim is that can run locally on device, and it's actually a true multi-modal model. It's like having a conversation in real time even if it's still very janky and awkward at this point. With this context, it's astounding and the experience is like nothing I've experienced in AI so far. There is no distance from speaker, it's like it's right there listening and responding without any barrier.

[@edgardsimon983]

that look awesome

[@eduveld]

Thanks for sharing this info!!

[@Xrayhighs]

Proof, at least for myself, that you dont need gigantuous datasets to train these models. Its less elaborate in its answers, but still competitive and probably first in its niche.

[@donaldjohnson-ow3kq]

Complainers or Critics?

[@enesmahmutkulak]

are you sure that its open-source and can be downloaded rn?

[@strangereyes9594]

The subtitles are hilarious.

[@egrinant2]

Yeah, it's like he doesn't care or put any effort into making quality content.

[@edgardsimon983]

@@egrinant2 he have no time it seem for sure lol

[@Shulyaka]

This is to make you even more shockingly shocked

[@zdenyn17]

Just listen to Ojo!

[@mrwoodcat]

@@egrinant2 nowdays subtitle only there to put viewers attention span working not to clarify the actual speech as it previously should.. lazy times..

[@MojaveHigh]

It starts off well, but at least for me, after about a minute, its functionality drops significantly, it starts repeating itself and just not understanding anything anymore.

[@jwetzel3141]

Today I learned that pirates have an American accent.

[@newunderthesun7353]

It was a valley girl pirate.

[@donaldclark1019]

I demoed today and tried to ask more about the Matrix. Apparently Neo was a rebel pilot who teamed up with a hacker to fight an AI controlled by an evil corporation. Its voice options were limited and seemed to always hallucinate a response and then say "sorry im here to help"

[@YouLoveMrFriendly]

Confabulate. These models don't hallucinate.

[@robxsiq7744]

small model issues no doubt, but its really about the starting steps to look at here, it'll only get better now. go back 12-18 months in open source llms and you only had small context insane models that couldn't do any of this stuff. now, we got damn near GPT-4 level punching with Llama 3. So...6 months of community developing and it'll be pretty damn ace.

[@Ricolaaaaaaaaaaaaaaaaa]

They swapped the model for a lesser one on the backend due to bandwidth.

[@ElectricEric2030]

**runs in circle while screaming**

[@reezlaw]

It's hit and miss but when it works it's unbelievable. The response time is superhuman and when you get good relevant replies in less than 200ms you really get a glimpse of the future. Of course way more often it goes nuts, starts repeating itself, loops and stops listening, but if this is the beginning and they keep training this has huge potential IMO

[@andrewai2001]

Um im stunned they thought this was ready for demo

[@MarshalArnold]

Subtitles: the first movie was called Matrix released in 1990 😂😂

[@TerragonDE]

Main Character is "Yo" !

[@SoumyaMaitra2008]

You nailed it

[@jeffkilgore6320]

Each day a yesteryear Nobel Prize is won. The word “shocked” has become a self mockery that reminds us that while we should be shocked, somehow, we’re not.

[@jimlynch9390]

Yes, the latency is impressive. The responses aren't quite as good as Pi for instance. Or the yet to be released gpt assistant. Like lots of things AI, it's only going to get better.

[@StephSancia]

Hey Pi Ai by Inflection is THEE BEST 🔥 been using it 8 months absolutely awesome

[@ppowell1212]

I think that two way conversations is going to be the way forward.

[@trycryptos1243]

Mind blowing stuff! Call centers with humans are a thing of distant history!

[@Tilofus]

Can't wait to try any of the State-of-the-Art Voice Models

[@anta-zj3bw]

I bet you that conversation at the end got you really, REALLY excited.

[@frankroquemore4946]

The voice didn’t actually let itself be interrupted in the demo. The guy just injected conversational noises to make it sound more natural but this isn’t any different than what we have besides emotiveness

[@robertlewis2542]

ah that's why it felt off and reminded me of my kids subterfuge, thanks for putting a finger on it for me.

[@j.d.4697]

Damn, the "entire industry" is "shocked" pretty much every day according to you.

[@surfside75]

Same manager runs Scottys auto channel😂

[@incription]

the demo is gpt-2 level noway near 4o lmao

[@dreamyxqc3812]

open ai will still be releasing gpt 4o in the next coming weeks ( infinity )

[@perhaar]

Link?

[@Sudain]

Inflection is not emotion. Don't confuse them.

[@edgaral]

Agree, especially when AI's aren't capable of feeling emotions, but rather use their programming code to act as if they had emotiones in response to certain contexts lol

[@gofastER]

William Shatner would beg to differ.

[@martiddy]

So fear is not an emotion?

[@Barc0d3]

IT IS NOT SHOCKING NOR IT IS INCREDIBLE, THIS IS LITERALLY THE MOST BELIEVEABLE TIMELINE TO RELEASE SUCH A THING

[@drowzy2309]

Just because it's inevitable, doesn't mean that it's not amazing.

[@Barc0d3]

@@drowzy2309 I did not say its not amazing, it is :)

[@sonyphotoguy6601]

Why are you screaming captain Capslock?

[@Xrayhighs]

With the resources and time given, this is a very very impressive result! It shows how common voice and llm ais are and that this is already an established technology. Its a base to start from and also can be competitive with low costs.

[@Barc0d3]

@@Xrayhighs I agree, it is impressive. It's not shocking though.

[@taomaster2486]

Ok so im not sure i got it it generates emotions odes it detect them too?

[@Aggie4life77]

I’m looking at this chat….yall pissing me off! Don’t act like this didn’t just blow your minds! 🤯

[@-schattenpflanze-3755]

This aint anything crazy. Lets see how good it is in 5 years.

[@surfside75]

Hard to interrupt the damn thing😂

[@durtyred86]

​@@surfside75 GPT 4.o already has that implemented..... Wherever the f*ck it is...

[@justinwescott8125]

Watch MattVidPro's video about it. He tested it live and it was awful.

[@drowzy2309]

Right? The only people who are pretending are the iPad kids. Even AI developers are impressed.

[@techdiasphere]

Pi is my preferred conversational AI due to its real-time internet access. Pi provides the most current information and answers, making interactions dynamic and informative. Pi's continuous learning and improvement facilitate more in-depth and accurate discussions on various topics.

[@Yogsoggeth]

Gee thanks for the huge subtitles right in the middle of the screen where a video should have been. Hot tip, CC is optional on the site you don't need to force feed me your script, because my ears work fine thanks. And if they didn't I would turn the CC on if I wanted it.

[@himanshuparihar9888]

where i can get model weight or github link

[@JOHN.Z999]

Amazing!!! 😱

[@Gallowglass7]

I am trying to use it, and it says, "This site can't ask for your permissions" plus "Close any bubbles or overlays from other apps then try again" - Anyone know what that means?

[@Otherlevel51]

Every day now there's something new in Ai. Can't even keep up with the news. Forget the models

[@DailyTuna]

Thank God, we’re back to shocking!! I miss being shocked.

[@Ginto_O]

when he said "shocked the entire industry" on 0:40 i stopped the video and disliked it because i dont want to be shocked

[@tracy419]

​@@Ginto_O❄️

[@elivegba8186]

​@@Ginto_O🥶

[@DailyTuna]

@@Ginto_O If you’re going to co exist with AI you must be comfortable being “ shocked” daily.😂

[@JoelMorton]

So.. 70 preset (or canned) styles and voices is supposed to be better than GPT4o?

[@thisisneti]

this is just amazing!

[@edgaral]

why is it always, businesses use the worse examples for AI to use its capabilities on? its was cringy, especially the pirate one, makes it sound as if their audience were a bunch of 5 year olds Is it that hard to just fake a whole conversation than to talk to its audience like they were dumb? 😂

[@Dreamingofyou317]

Because they’re nerds

[@drcanoro]

I love where AI is going, now they need to give AI full freedom on voice manipulation, sound like a gnome, or a rapper, or an old man like David Attenborough, or with an American southern accent.

[@user-en4ek6xt6w]

From my test it is very bad, it stop answering and don't understand well

[@PRepublicOfChina]

this is going to be so great for every AI girlfriend app. imagine having an AI girlfriend who can imitate any accent, and sound like anyone.

[@casynovids]

I'm a "in-game " footage victim from the early to mid 2010s.....a BIG Preproduction can be very deviceing...But I hope this REAL TIME

[@schuylerhaussmann6877]

I'm shocked and mind blown

[@skyzar4141]

When will they release it though

[@pchungvt]

just tried and it wont use emotion, and it also hallucinate a lot

[@brianmi40]

Hallucination isn't strictly a problem. It's been realized it is a path to innovation. You have to think out of the box to come up with new solutions, and hallucination is a form of that. We realized this the very first time Alpha Go came up with a move that the best human players thought was a huge blunder. It was SO far out of our framework we needed to do in depth analysis to realize the genius of it. AI models have a sliding scale that is applied, scaling from Factual to Creative when in use. The goal is NOT to eliminate hallucination and creative thoughts, but rather do so ONLY when the scale is set to 100% factual. There are multiple methods being pursued, including data input, as well as post training editing.

[@danielchoritz1903]

@@brianmi40 Hallucination is used here to describe then the AI does not answer the question, but made up stuff that looks like a answer. Variation is good, no output control or re-verifying her own result with self created questions keep a AI under the virtual human age of 6 years. It can speak, can remember, can answer, but lies and hasnt any moral/ethics. A AI that ask you if there are multiple ways to answer you for further narration, a AI that surprises you with a question, so you understand for yourself that you are asking for..this is the next step to reach agi. Pure scaling means just FASTER AI. A qualitiv jump may with a very high chance help you to close the gap and speed away in a short time, because self-improvement needs this step. So..for my understanding you are good on public "AI-Business" speech, but without any real argument, why hallucination is a good thing. Alpha Go did make a legal move, a hallucination would mean a move like J2-5. etc.

[@YouLoveMrFriendly]

They're confabulations, also known as false memories. When you're grandpa is spinning yarns about his past with stuff he's misremembering, you wouldn't claim he's hallucinating.

[@brianmi40]

@@YouLoveMrFriendly It's just the term that is media "friendly/catchy" to use. Papers have discussed now how it's not a "bad" thing per se, and that coming up with ideas LLM's are NOT trained on IS VITAL to fashioning new and novel solutions. The trick is getting the "Facts - Creative" slider that LLMs allow the user to set, to go FULL ON Facts when desired...

[@neomatrix2669]

If you use quantized versions, it will really hallucinate a lot.

[@VampyressVA]

Well, I just tested it and, while the latency and flow are really impressive, the LLM itself leaves a lot to be desired. I will check back in a couple of months to see how far it will have improved.

[@brianbarnes746]

That model is open source? That would be amazing. Who needs openai

[@Jshicwhartz]

I think the only thing cool about that was its ability not to hallucinate and make something up when you asked it about something it actuallly said it didn't know...

[@kasperzier7391]

The software creating your subtitles need to be further trained on detecting french accent !

[@vihangnair]

🎯 Key points for quick navigation: 00:05 *🎭 The voice AI can express over 70 emotions and speaking styles, including whispering, singing, and accents.* 00:27 *🤯 The AI model revealed by caai is state-of-the-art and shocked the industry with its real-time conversation capabilities.* 00:54 *🗣️ Moshi, the voice AI, can respond with lifelike emotions and incredible speed.* 01:06 *🇫🇷 Moshi demonstrates speaking with a French accent by reciting a poem about Paris.* 01:47 *🏴‍☠️ Moshi switches to a pirate voice and discusses pirate life.* 02:56 *🕵️ Moshi uses a whispering voice to tell a mystery story.* 03:22 *🎬 Moshi narrates the plot of "The Matrix" with detailed accuracy.* 03:54 *⚠️ Discussion on the current limitations of voice AI, including latency and loss of non-textual information.* 05:02 *🔄 Explanation of the new approach to integrate complex pipelines into a single deep neural network.* 07:16 *🎤 Demonstration of Moshi understanding and generating speech by listening to a voice snippet.* 08:13 *💡 Moshi thinks as it speaks, generating both text and audio simultaneously for richer interactions.* 09:12 *🔊 Moshi supports dual audio streams, allowing it to speak and listen simultaneously for more natural conversations.* 10:20 *📞 Example of Moshi's conversational capabilities using historical data sets.* 12:23 *😮 Moshi can express over 70 different emotions and speaking styles using a text-to-speech engine.* 15:59 *📱 Moshi can run on-device, ensuring privacy and security by eliminating the need for cloud processing.* 18:36 *🔐 Measures are in place to detect and watermark audio generated by Moshi for safety and authenticity.* 20:11 *🌐 Demonstration of Moshi's real-time conversational capabilities, showing quick responses and lifelike interaction.* 23:34 *🚀 Moshi represents a revolutionary advancement in AI, promising significant changes in AI-human interactions.* Made with HARPA AI

[@8eck]

If it really will be open sourced, then it will be really a new era for AI apps.

[@GSINCVideo]

It looks awesome. Perhaps a couple of challenges for real world use are i) Speed of the text to speech audio conversion i.e. How fast is the API? And ii) Cost of conversion. This might be the biggest deal. Cloned voices can rock, and if they express emotions + convert from text to speech fast, then that = Amazing. But what's the token cost of it? If it's going to cost $0.10 - $0.15 every time it talks for 30 seconds it may not be that viable to use in an app where end users want to listen to their Ai talk for say 30 or 60 mins a day?

[@donaldjohnson-ow3kq]

They haven't figured out how to drop the inflection when voice volume drops, so it still sounds robotic if you listen closely.

[@atlas3650]

Knock knock. Who's there? Interrupting cow. Interrupting cow wh-- Moooooo!

[@davekite5690]

your chat at the end was.... 'quite something'....

[@cyberS_2024]

Tried it a few times. It's not very good.

[@daydrip]

What were some cons

[@Ricolaaaaaaaaaaaaaaaaa]

They had to swap the model being used because it was overloaded.

[@user-su2ci1br6c]

For the past 5 or 6 month none of the released videos or news were actually shocking or INDUSTRY DESTROYING

[@suzannedaniels883]

I love AI voices I love to try them out and even use them in my podcast. However this AI voice does not consider herself an assistant?

[@giovform]

STUNNED!!!

[@alkeryn1700]

ShOcKs tHe eNtIrE InDuStRy

[@michael2826]

KYUTAI wanting to learn about AI interesting

[@davidfitcher2953]

Is this something that we should be happy about or proud of?

[@keithcourson7317]

It's getting there.

[@duncanward6226]

"Thankyou Mr Data, that will be all."

[@aquetheblues]

I'm french and I can tell you that regarding the french accent, there is some work to do. 😂

[@Eddierath]

Weekly rollouts like this should be happening in the medical field. Just walk through any medical operations ward and listen to the humanity in there.

[@franzofmotion]

On the one hand, this demonstration is super impressive, but I've tried it multiple times and it's super buggy After two minutes of conversation, it just got stuck and told me constantly that it's playing and then it's not playing. Seems super cool, but something is not working

[@WillBurns]

After actually trying the demonstration - the voice wasn't as good (sounded like typical TTS), and the LLM response was premature. Which is to say: It often did not wait until I had finished a sentence before it jumped in and tried to respond. They need to work on more natural timing response.

[@krankvegann]

The speed of the responses are great. Trying out the demo, it seems the model is pretty basic, hope in the future they can improve in the quality of the responses.

[@tituscrow4951]

It felt like to me that they had compressed at all costs so it could do edge stuff on phones running locally. When it was connected to the LLM in the demo it seemed FSR more aware.

[@ToastyZach]

The very first clip sounded great, but once the live demo started it sounded a lot more robotic.

[@mindful_minipods]

That guy went from being a captain to not knowing what the pirate life is about.. He should have been flagged for dementia.

[@elivegba8186]

Ai with dementia😂

[@ytrvabrilot]

this ai is INSANE, but literally. take care.

[@swooshdutch4335]

in regards to your own recording, it thinks its conscious and a person because the devs prompted it to

[@gabrielkasonde367]

OpenAI making potential competitors doing clown work and clones while they are onto the real deal stuff

[@svenhoek]

Where is the Fassbender AI voice?

[@djfremen]

Honestly, Peter O’Toole would be better

[@quaterman1270]

You see how he tries to speak without pause otherwise the LLM will interupt and prcess what he said. When they improve that, it will be a big approvement imo. I have it all the time when I have a longer quesiton or one with more paramteres and I think for 2 seconds, it just switches and anwers what I said. I think in that case a button would be good, to just listen until I'm finished. But I think wie will need bigger conext windows for that. Maybe 250k will be enough for that.

[@pb2806]

Can you sing a song? Answer: No, I can't Can you whisper? Answer: No, I can't Mm. Thank you very much.

[@StrawHatlufy]

i tested the real time demo , online but its seem some time it get hallucinate and also some of answers are not very clear like gpt 4 , but i agree speed in insane

[@user-zs8lp3lg3j]

Bokutachi wa SF o mamotte imasen. Humanity is not protecting science fiction. Fun is so entertaining now. This live in tutor can teach accents & annotations!

[@CaptainKokomoGaming]

Did you see Wes Roth messing around with it? I am pretty sure he cherry picked the worse of the bunch but it was messed up.

[@Timuche]

KyutAI like in "cute AI", come on!

[@Miguel-fz3yx]

Resume: "Okay"

[@balazsgonczy3564]

How can it master emotions if emotions are not universal? Some tribes does not know what happy mean.

[@xeloprint]

A phone call to the past - time travel is here..

[@blinkdrive2810]

all scam call center companies love this presentation, and invest into your company

[@haiffy]

You're saying I'll be able to have daily conversation with my waifu?!!

[@freyna]

A french Ross Geller. Its green speaking eye is very reminiscent of Hal. I'm sorry Dave, I can't do that. I think Moshi should be the AI leader of the uprising.

[@MaiWhisper]

I tried it. It needs a lot of work and I'd like to see it improve.

[@user-ru9rf4mg6x]

WE LIVE

[@oholimoli]

The whole team will soon work in the Silicon Valley

[@Dreamy1894]

Its scary that inserted trailer for AFRAID, targetted advertising

[@TRFAD]

Wow now when the AI take over happens skynet can chase me down with a pirate accent

[@cryptogaming9935]

Ok, where is the woman with the microphone hiding :)

[@ashtonphoenyx]

This does not beat Chat Gpt4o

[@Rick.Fleischer]

The locally computed part of his demo fell to shit.

[@Techtalk2030]

Man open ai needs to release something really good and fast, theyre losing the race to claude, gen 3 and now this.

[@Techtalk2030]

Im hoping gpt 5 and sora will be something absolutely amazing

[@brianjanssens8020]

@@Techtalk2030 They won't be. Luma is literally destroying the competition for AI video.

[@Techtalk2030]

@@brianjanssens8020 luma isnt that good ib my experience. Its free but gen 3 is much better

[@Techtalk2030]

@@brianjanssens8020 luma isnt that good compared to gen 3 in my opinion

[@kutagaru]

​@@Techtalk2030 very true, although Luma is getting more attention because not only did it come out first, it's also sort of free.

[@nevergonnagiveyouupnevergo3263]

Idk looks like a good storyline for Portal 3 or Half-Life 3

[@MaximilianFeichtinger]

The web version is either way better or the first example are cherry picked out of millions of tries. Because my conversations with the demo are the same as the demoed offline version - horrible.

[@ezramantini8078]

OpenAI, a team 8 people build this from scratch…WITH A BUNCH OF SCRAPS!

[@sorrychangedmyusername3594]

I WANT MY AI GIRLFRIEND ALREADY

[@AFeigenbaum1]

good get !!