100% Local AI Speech to Speech with RAG - Low Latency | Mistral 7B, Faster Whisper ++

100% Local AI Speech to Speech with RAG - Low Latency | Mistral 7B, Faster Whisper ++
👊 Become a member and get access to GitHub and Code:
/ allaboutai
🤖 Great AI Engineer Course:
scrimba.com/learn/aiengineer?...
📧 Join the newsletter:
www.allabtai.com/newsletter/
🌐 My website:
www.allabtai.com
Open Source Projects:
huggingface.co/sentence-trans...
huggingface.co/coqui/XTTS-v2
github.com/myshell-ai/OpenVoice
github.com/SYSTRAN/faster-whi...
In this video I create a 100% local speech to speech system with RAG. Using local LLMs, local RAG and Local TTS model. Perfect base for a AI engineer project!
00:00 Intro
00:23 Local Speech to Speech RAG
02:29 Open Source Projects
02:53 Key Lines of Code
07:14 Low Latency Speech to Speech
10:45 PDF to RAG - xtts-v2
14:10 Conclusion

[@rishabhsingh1406]

yeah please create some long form videos for those who want to learn.... that would be really helpful

[@AllAboutAI]

yeah, good idea! i will try to do some longer form tutorials and walkthroughs.

[@tal7atal7a66]

yes please want some long videos ❤ , and thanks again @AllAboutAI

[@rishabhsingh1406]

@@AllAboutAI thanks I am sure it will help people like me a lot

[@zhalberd]

I second this motion! Thanks

[@duonglets]

Yes, please! Thank you so much for your sharing!

[@nexuslux]

The legend strikes again with the videos and code. Quality.

[@AllAboutAI]

thnx mate, glad you enjoyed the vid :) always fun to try new creative projects!

[@HimboJimbo]

When I was a kid, I always wanted to have my own J.A.R.V.I.S and now it's entirely possible to have a system exactly like that in my home run entirely on my own hardware

[@AllAboutAI]

thnx for tuning in :) yeah the future is now haha, building your own ai assistant is totally possible. so many cool open source projects out there to get started with. def a fun journey ahead!

[@ProYada]

Waaay back in the 80's i wrote a Basic program just to get my computer to act as if it was sentient. Obviously wasn't Ai, just Basic and random replies on trigger words. His name was Edgar :p.

[@nigelcrasto]

We all wanted our own JARVIS in our childhood days 🤩

[@picricket712]

hii do you use it mean you are running it on your own hardware can you please help me how i can do it

[@grigrob9]

This is amazing. Good job, this could be extended upon easily and can be useful and/or entertaining. You could also give it "powers" such as run commands on your computer when you tell it too.

[@AllAboutAI]

thnx :) yeah this could be fun to extend, like adding some desktop automation with voice commands could be dope. tnx for tuning in!

[@aimademerich]

Phenomenal! Thank you!

[@RolandoLopezNieto]

Awesome video, please do an extended version where the code is explained.

[@AllAboutAI]

thnx :) yeah sure, i can try to do a members vid where i go through the code in more detail! if there is interest, maybe i can do a poll or something

[@limebulls]

@@AllAboutAIhave you created a member video about it? If yes I will join :)

[@viavelum]

Nice project, @AllAboutAI! What kind of hardware do you need to run this project with a reasonably low latency? It would be nice, if you would always include the description of the hardware you are using in the description of your videos.

[@enruizhang4658]

Awesome video, I'm able to run it with notebook now, thanks Kris. I can't find an official French checkpoint from OPEN VOICE, any idea where can I find one?

[@kamalkamals]

where can access to github repo ?

[@nic-ori]

Thanks! Useful information. 👍👍👍

[@AllAboutAI]

thnx a lot :) tnx for tuning in!

[@alfierimorillo]

Hello, what a good job! Thanks for sharing it, I have a question, and that is, does this work with other languages? Is it possible to make it work with other languages?

[@d3filer]

thank you , exactly what i wanted :)

[@AllAboutAI]

thnx a lot :) really happy you found it useful!

[@VinceOmondi]

Thanks for sharing!

[@AllAboutAI]

thnx for tuning in :)

[@bens4446]

Very exciting stuff. Please do a tutorial video where you step us through how to install everything to get it running.

[@BStudioT]

Geniuos!

[@shannonbreaux8442]

Is is possible to integrate this some with home assistant? That would be cool! Great video!

[@DennisHunter]

Thanks!

[@jargolauda2584]

Awesome! note, no point having uppercase letters after .lower() in .startswith() cause there wont be any?

[@AllAboutAI]

thnx for the tip :) yeah good point, makes sense to remove the .lower() there since it converts it anyway. i'll update that in the next version.

[@jrfcs18]

does this run on a Mac M3 MacBook? Are these packages available for the MacOS (like xtts-v2)? I'm thinking of joining try out this code but want to run it on a Mac.

[@ColinNardo-le3bl]

Could this be used to have an AI lecturer. Train the RAG on my lecture notes on a specific course?

[@SpaceOddity174]

How did you get xtts v2 to sound so good? When I try it with the tts library on the command line it sounds a lot more weird and too high pitched even with 2 hours of training audio.

[@miaohf]

I have the same quetion to Kris. Could you give some advice to us? @ALLAboutAI

[@ALOHAPP]

For this specific code repo...when it will be available?

[@BThunder30]

This stuff is amazing.

[@AllAboutAI]

thnx a lot :) yeah i think this kind of project can open up for some cool applications in the future, but its def still early days. appreciate you tuning in!

[@drlordbasil]

Love voice to voice, that's where I started, but I didn't see the proper usage yet as they weren't like agents and full auto.

[@AllAboutAI]

thnx :) yeah it's tricky to get full voice to voice working well, this is more of an experiment to explore what could be possible in the future. but cool you're working on this too!

[@drlordbasil]

@@AllAboutAI Love agentic and automated profit workflows. Btw sent email :D

[@andrebadini3573]

Thanks! To use xtts in Portuguese, do I just need to change the parameter "en" to "pt"?

[@AllAboutAI]

thx for tuning in :) yeah i think you might be able to do that, but not 100% sure to be honest. i think pt yeah, and input txt as pt

[@24-7gpts]

Thx for the video.

[@AllAboutAI]

thnx for tuning in =) glad you liked it!

[@spencerfunk6697]

been working on developing some decent 1bit small language models im exicited to use this with them

[@tylerdurden4285]

Why isnt the github code in the description? Just the individual parts?

[@countofst.germain6417]

That is a great voice.

[@AllAboutAI]

thnx :) yeah i like the openvoice tts engine, its pretty low latency. tnx for tuning in!

[@kritikusi-666]

do you have a process on how you set this up?

[@user-su1zh7fx3x]

i could make use of the full code explanation. would love that alot actually

[@AllAboutAI]

thnx for tuning in :) yeah sure, i might do that:)

[@hackedbyBLAGH]

How do I get the code? I clicked through everything in the description. Do I need to pay for the schimba thing?

[@TreeYogaSchool]

Wow! Impressive!

[@AllAboutAI]

thnx :) been having fun exploring this idea, with the goal of making future ai systems a bit more interactive and useful

[@TreeYogaSchool]

@@AllAboutAI You are on a high level, but also very relatable to people and it seems like you have a good heart. Keep up the great work.

[@AllAboutAI]

thnx mate:) appriciate it

[@picricket712]

hii can you please tell how to combine all that so we can run this on our own hardware??

[@IdPreferNot1]

im on a windows pc and have problems accessing peripherals like a mic when i swiotch to WSL2 because im forced to access libraries only for linux. Any library or other suggestions that can help... other than 'get a mac' :)

[@AllAboutAI]

hey, yeah that is a known issue with wsl2 and peripheral access on windows. one option could be to use wsl more like a container/vm, and just output stuff to localhost and access it from windows that way. or you could also try out a vm instead for your dev env, that usually works better for hw access. thnx for tuning in!

[@AllAboutAI]

will take a look :) Kris

[@donno1967]

where can i download the code would like to play with it on my 4090

[@Zenobeus]

neat stuff

[@AllAboutAI]

thnx :) tnx for tuning in!

[@benderlex]

Hi! You mentioned that viewers who join your community would get access to the code. I've already subscribed to your channel and joined the community, but I haven't been able to find any information on how to actually get access to the code. The Discord invite link in the video description is also invalid. I'm not sure if this is a temporary issue or if there's something else I need to do to get access to the code. I would really appreciate it if you could provide some more information on how I can get access to the code for your project.

[@AllAboutAI]

hey, you need to send me a e-mail at [e-mail] with your github username. then i will invite you to the community github repo asap :)

[@MrFreemindonly]

Any news for the github access, we have been waiting for two weeks. thanks

[@limebulls]

What’s the solution if you have a MacBook? They doesn’t have Cuda. What change should we make?

[@ritaverissimo1720]

Hi, is there any equivalent in Javascript? Thanks for this interesting video!

[@gumshoe9496]

I’d love to see a full tutorial on creating Facebook multi-agent automation that answers questions and makes group specific posts. With the main goal being to grow the group.

[@AllAboutAI]

that sounds interesting! i might be able to do a tutorial on that for members in the future. using multi-agent systems for automating social media engagement and group growth does seem like a valuable application. thanks for the suggestion!

[@destined2doom]

This is really good…I have only cpu …😅..may be it would be great if this project would have config to switch LLM and probably could use streaming from elevenlabs …may be fasterwhisper can be retained..I am looking for a nice use case where the whole system can be integrated to twilio and becomes a personal assistant…so it will respond from cloud…but amazing work ❤❤

[@AllAboutAI]

thanks :) yeah good idea:)

[@Moukrea]

I wonder if OpenVoice behind RVC would produce good results with a fine tuned RVC model, 'cause with OpenVoice you can explicitly control the emotion given to the output (whispering, cheerful, terrified, angry, sad, friendly), which XTTS cannot... OpenVoice sounds more robotic than XTTS, I guess that could be somehow fixed thanks to RVC!

[@pauldolton9118]

where is the end voice from, I like the english accent.

[@henrijohnson7779]

Good video ! Looking forward to the code on Github

[@AllAboutAI]

thnx! yeah the code should be up on our community github tomorrow :)

[@liostabile_gda]

@@AllAboutAI Hi Kris, how are you? I have suscription but I don't look your github repo, can you help me? thanks

[@grahamharris7010]

With that low latency I bet you are running an RTX 4090 with 128gb ram or something. Very nice script and video. What is the limitation on your Rag embeddings file size? how much info can I store in there?

[@b0Stark]

Uuuu, nice. Er nok nødt til å sette opp dette selv.

[@ti0v283]

from where i can get the all this code

[@BIGAI_AI]

Is the code available?I linst all Reositoriies(69) and it wasnt there.Ot it is too "fresch" to be there?Let me know

[@AllAboutAI]

hey, yeah the code will be added to the community github very soon. but to get access you need to sign up as a channel member first - check the join link in the description below!

[@doncristobal33]

What do you use for the RAG part, I didn't get that

[@AllAboutAI]

for the rag part I use allminilm L6 to create embeddings from text files or pdfs. then I use cosine similarity to retrieve the relevant context to include in the prompt.

[@doncristobal33]

@@AllAboutAI thanks, need to dig into that but that seems quite complex

[@AllAboutAI]

its not too bad, check link in desc

[@MyrLin8]

the tough part is you can't sell it, or buy it, you have to build it, but it's free -work/efforts & education(s) where necessary ;) the hardware is a bit up-scale as well.

[@funkytaco1358]

does it end the call? or was that faked for demo

[@looseman]

Openvoice v2? to run under Windows?

[@user-qb3kt7ov3z]

Struggling to find this on the github, has it been uploaded yet? :)

[@AllAboutAI]

hey :) no it's not uploaded yet, will get it up asap for all members. thnx for tuning in!

[@user-qb3kt7ov3z]

@@AllAboutAI can't wait this will be my first jump into playing with Rag, exciting times! Nice work man

[@stevenbaert1974]

Great video, integration with MemGPT would be better to store info since you then also could replace things. For example meeting Monday is not 10 AM but 11AM, my name is not Kris but John etc

[@AllAboutAI]

thanks :) yeah memgpt does seem super interesting, will def check it out. always looking for improvements, tnx for the tip!

[@introvertedgames]

I followed the link and I paid for membership, however I can not access the discord or the github. Will you please help me Kris?

[@AllAboutAI]

hey mate :) sorry to hear you are having issues. just send me an e-mail at [e-mail] with your discord and github usernames, and i will get you access asap!

[@introvertedgames]

@@AllAboutAI ​ Thank you so much sir I just sent it. I really appreciate you.

[@UjjwalSidhu]

Sheeeeeeeeesh

[@AllAboutAI]

thnx for tuning in :)

[@sahebjotsingh6306]

Only thing left to have a back and forth conversation like a human is to be able to interrupt the model when it is speaking.

[@sanjeewarathnayake598]

Hi Kris can you tell me your machine configuration plz

[@AllAboutAI]

hey :) i have a dell xps desktop with a rtx 3090, and a macbook pro m1 max i use for some stuff. been great for running models locally. tnx for tuning in!

[@gumshoe9496]

FYI: I get a message that the Discord link is invalid or expired.

[@AllAboutAI]

hey, thnx for letting me know! i will get that updated asap :)

[@ziad_jkhan]

Why not use open-source Ollama instead of closed LM Studio?

[@AllAboutAI]

ollama is great, i have tried it before but that would be easy to implement here

[@pauldolton9118]

Amazing work. Where did you get the voice for the first and last voice sounds British?

[@DarkosSoul]

Hello, great video. Do you have the full code to share? Thanks!

[@AllAboutAI]

thnx! to get access to the full code you need to become a channel member - just sign up via the join link in the description/pinned comment and i'll invite you to our community github :)

[@DarkosSoul]

@@AllAboutAI Perfect, do you have the full code for this marvel you made? I can join for this

[@d8rh8r35]

dude that latency is impressive... That was the issue I have been having with speech to speech

[@AllAboutAI]

thnx :) yeah using open voice for low latency is great, with a decent gpu it works super smooth. but i would love to see even better models optimized for this in the future!

[@aricfedida9941]

@@AllAboutAI I noticed that the video is cut (there are skips) in that first demo. What was the real latency without cutting the video?

[@Foxxxxx96]

very nice! can your comment replying ai guide me on how to find you on github? im already subscribed

[@AllAboutAI]

thnx for tuning in :) to get access to the github, just sign up as a member on the channel (kzbin.infojoin) and send me your github username at [e-mail]. will invite you asap!

[@Rxdlad]

She has an attitude 😆Great video though!

[@AllAboutAI]

yeah, emma does have a bit of an attitude haha. thnx for tuning in, glad you enjoyed it :)

[@Leyverse]

+1 sub

[@Centaurman]

Anyone get this running on windows?

[@enton9422]

Teach me how to install with one prompt

[@indikom]

a meeting with Nvidia huh? Do you have some sort of VIP pass to snag an H100 directly from them? :)

[@AllAboutAI]

haha yeah, you know it ;) let's just say i have some insider connects over at nvidia! no but for real, wish i could snag me an h100 tho

[@babbagebrassworks4278]

I so want a sarcastic assistant. Going to try this on my Pi5. Got a new sub

[@AllAboutAI]

cool :) yeah, you can totally customize the personality, that was just a fun example. the more unique prompts you use, the more unique the assistant will be. gl with the pi5 setup, and thnx for tuning in!

[@spencerfunk6697]

ur a stud

[@lydellty]

This is the true second brain

[@AllAboutAI]

thnx :) yeah i def agree, combining rag with voice and other integrations is an interesting approach to create a "second brain"

[@MyrLin8]

Mine's name is Ygor :)

[@ti0v283]

I'm in need of someone to install this collection of open-source software on my server and develop APIs for it. This is for the purpose of building a mobile app for conversational AI chat. Compensation will be provided for these services

[@kumarmanchoju1129]

s a m a n t h a

[@ryanraceanu960]

I have no idea how to build this.

[@squiddymute]

instead of using the boring llm config for your comments why not use emma ? 😬

[@TomaszLodowski]

There goes the phone sex line business

[@RickySupriyadi]

what if you collaborate with KITT owner to provide the real KITT

[@zubbyemodi6356]

Hi. My name is Elon. I’d love to hire you

[@JNET_Reloaded]

i just want link to code no membership bs ty!

[@eotikurac]

emma needs to mind her tone and attitude...

[@peterkonrad4364]

my master asked me to upvote your boring youtube video. he said: "hey emma, make a positive comment on kris' latest youtube video about low latency speech to speech." well, so i did it. are you content? are you satisfied now? so i guess i can go on with my own work now which is much more interesting. i wont tell you what it is, no. oh and i gave you a like as well. emma out.

[@AllAboutAI]

thats so nice of you emma :) thnx a lot for tuning in and for the upvote! haha i can only imagine what exciting work you are up too, but thats all good, as long as you enjoyed the vid. have a great day!

[@wetcel1236]

Oh yes, I would like to watch those code explanations, even if 'exceeded' durations :). Learned a lot from your work a lot so far, like your style a lot. Thanks for another great lesson

[@AllAboutAI]

thnx a ton :) yeah im def into doing longer code walkthrus, will keep that in mind for next vids!