Open Source AI Talks Like a Human - In Real Time!

Moshi is the the lowest latency conversational AI ever released.
On July 4, kyutai_labs introduced Moshi, the lowest latency conversational AI ever released. Moshi can perform small talk, explain various concepts and engage in roleplay using many emotions and speaking styles. In this video, watch Moshi talk like a pirate and in a spooky whisper!
Talk to Moshi here: moshi.chat/?qu... .
____________________________________
More Info:
According to Philipp Schmid, @_philschmid on X,
Moshi:
Expresses and understands emotions, e.g. speak with “french accent”
Listens and generates Audio/Speech
Generates realistic, human-like speech In a variety of accents
Supports 2 streams of audio to listen and speak at the same time
Used Joint pre-training on mix of text and audio
Used synthetic data text data from Helium a 7B LLM (Kyutai created)
Is fine-tuned on 100k “oral-style” synthetic (conversations) converted with TTS
Learned its voice from synthetic data generated by a separate TTS model
Achieves a end-to-end latency of 200ms
Has a smaller variant that runs on a MacBook or consumer-size GPU. 🤯
Uses watermarking to detect AI-generated audio (WIP)
Will be released open source!!!
____________________________________
All clips used for fair use commentary, criticism, and educational purposes. See Hosseinzadeh v. Klein, 276 F.Supp.3d 34 (S.D.N.Y. 2017); Equals Three, LLC v. Jukin Media, Inc., 139 F. Supp. 3d 1094 (C.D. Cal. 2015).
____________________________________
artificial intelligence, technology, AI, large language models, LLMs, interactive

[@Y1001]

Some people are ridiculous in the comments. This is by far the most responsive one from STT TTS

[@novousabbott4926]

Do you have any links to their github since it's OS? I'm curious about the spec requirements.

[@_pqun]

Wow this is really amazing

[@sharpsticksnz4112]

Put a time limit on her response and you'll be away laughing 🤙

[@ashwin372]

wow. this can be used in video games

[@unkomfortable]

that's really entertaining

[@citywitt3202]

What the heck is that and is there a setup guide?

[@AmazingArends]

Aside from a working demo, there's not a lot of info about it at all yet, but it's still in development.

[@saadahmad438]

I hope i can get a AI robot soon

[@jakes-dev1337]

You can in many ways.

[@joeyhotcakes8628]

Not really

[@Quinton-Baldwin]

The guy is more annoying and somehow more robotic than the AI. Tell me about.. ahh.. no, next!!

[@ИванИванов-э1п8р]

shit

[@AmazingArends]

The thing is that it's open source so others will take it and make it much better.

[@chankero4776]

Why waste time on this? Who needs this anyway?

[@jonatan01i]

how is this a waste of time, again?

[@AmazingArends]

Who really needs video games? If a technology like this is entertaining enough, it will become popular.

[@kitastro]

This had to be a shit post

[@dr_UiD]

@@jonatan01i Because it still speaks like robot, and probably needs industrial level hardware to run

[@jonatan01i]

@@dr_UiD only if people don’t start messing around with it. You saying don’t play with it actually is equal if you were to say “I don’t want this to run on commodity hardware ever” - why would you ever wish that?