Create your AI digital voice clone locally with Piper TTS | Tutorial

[@synesthesiam]

Guude, Thorsten! Thank you for making this video :) I've added it to the Piper training guide already. I had a previous comment, but I guess KZbin didn't like it containing a link.

[@ThorstenMueller]

Guude Mike 👋, you're very welcome and thanks for adding the video link to the docs 🥰. Feel free to send me the link directly and i can add it to a comment or video description.

[@juntiamores2418]

Big thanks to you Thorsten, finally my Personal home assistant A.I ToniaBot has a wonderful voice now. ❤

[@ThorstenMueller]

I'm happy that my videos where helpful for you to finish your personal project 👏😊.

[@MyHeap]

Thank you for demonstrating this. Now I admit I will need to watch it a couple more times to get my head around it all, but I certainly appreciate your efforts here. Thank you for making the video and sharing your knowledge. Joe

[@ThorstenMueller]

You're very welcome Joe and good luck 😊.

[@OpinionatedReviewer]

Great video! I'll need to rewatch it a few times to nail down the steps, but your effort in sharing this knowledge is much appreciated. Thanks! 👍

[@ThorstenMueller]

Thanks a lot for your nice feedback and good luck 😊.

[@QuokkaBff]

Hi, Thorsten, Thank you for all of these. The experience of hearing tts in my own voice was awesome.

[@ThorstenMueller]

You are very welcome 😊. And i know absolutely what you mean and i feel the same way.

[@NoDebut]

Adding my +1 to all the feedback in the comments below - THANK YOU for creating a step by step guide for this. I work with computers and am no stranger to python programming, but still would have found setting this up and getting it working unwieldy and off-putting. I really appreciate you going through this step by step - I (and I'm sure others) need this kind of help! Also, can I just say that I wasn't even aware this was a *possibility* until I saw this vid? :)

[@ThorstenMueller]

Thank you so much for this really kind feedback 🥰. Comments like yours always keep me motivated making new tutorial videos.

[@TillF]

How many hours of voice recordings would you recommend as a minimum for fine-tuning a small/large model to retrieve good results?

[@ThorstenMueller]

Hard to say. Maybe start with one/two hours, give finetuning a try and hear how close you already are and add more recordings by demand. But in general - more is better. But keep a high recording quality in mind.

[@jantester2357]

Thanks 🙏 Services like Coqui only needs a few seconds of voice to clone it. When I want to train my own model, I need hours of recording a the result is not so good. What am I missing?

[@godwinspeaks]

Pipers usecase is mainly for being realtime, not for being the best in quality. The ones like coqui, tortoise etc dont come close in that aspect. This is a different usecase

[@RemoteAccessGG]

@@jantester2357conqui has really good pretrained model with lots of voices, but piper’s finetune-ready model is not that good, so you need to do almost everything by yourself.

[@truszko91]

Why have I never thought to hide the venv dir with a "." HA! Thanks for making this tutorial!

[@ThorstenMueller]

You're very welcome 😊. Enjoy your "clean" directory structure.

[@RedDread_]

Great video, how much audio is required for a decent training though? I have a few hours, is that enough?

[@ThorstenMueller]

Depending on what a few hours mean, but in general i'd say this sounds like a good basis for model training 😊.

[@012_siddhantprasad9]

Which model would be the best to add emotions like anger/happiness/sadness etc. in the speech? Coqui or does Piper supports this functionality?

[@ThorstenMueller]

Imho no. Piper has ssml on their roadmap to adjust output in some ways, but not really meant for adding emotions. You can record your own emotional dataset to train on, but this is probably not what you are looking for. Maybe try an emotional prompt on parler tts by huggingface.

[@012_siddhantprasad9]

@@ThorstenMueller I tried Parler TTS, no emotions in the voice 🥲

[@ruthfodor9632]

hi. i’m currently trying to retrain the romanian model to correct the pronunciation of diacritics (ă,â,î,ș,ț), but keep bumping into a problem: in the generated jsonl dataset, ț is matched to 2 sounds: t, s instead of one but i’m not sure what to do. ț kinda sounds like ts in english: “cats” but t, s doesn’t give the best results in piper. maybe you can give me some advice? also, thank you for the video, super helpful

[@ThorstenMueller]

Thanks for your kind feedback 😊. Is that "roa" language code? Does it work when using espeak-ng -vroa "some demo text"?

[@ruthfodor9632]

@@ThorstenMueller yes it is roa and espeak-ng -vroa works. what i am trying right now is training after replacing with the correct phoneme “translation” in the dataset.jsonl. hope it works

[@catalin-mobile]

Dacă reușești ceva, fă un ghid sau popularizează ideea că înafară de Ivona, nu-i nimic normal de ani de zile

[@whalemonstre]

Thank you Thorsten - useful and enlightening. 🙂🖖

[@ThorstenMueller]

Thank you for your kind feedback :). Happy you find my content useful.

[@ei23de]

As i'm currently try this on my own, i realise how much work you must have put in this. Thanks a lot Thorsten! Also one question here. Are you run training on a Nvidia GPU and is it a real Ubuntu Machine or do you use some kind of WSL2 / VM? Currently have some Cuda related problems with WSL2 on Windows 10, haha... CPU is working tho. Edit: I could solve some problems with cd /usr/lib/wsl/lib/ sudo rm -r libcuda.so.1 sudo rm -r libcuda.so sudo ln -s libcuda.so.1.1 libcuda.so.1 sudo ln -s libcuda.so.1.1 libcuda.so A Nvidia GTX1060 is now working. But my RTX4090 is not. I also tried the docker solution (which is as you said, not really easier) and thay shows in the logs that die RTX4090 is not supported yet. I get this error which seesm WSL related Could not load library libcudnn_cnn_infer.so.8. Error: libcuda.so: cannot open shared object file: No such file or directory So i may set up a Ubuntu-20.04 since python3.10-env is deprecated on debian. Also another question comes up: If i want to train a whole new model, how many epochs are needed? Usualy around 2000-3000? Also depends on the dataset quality i guess?! But first i have to get the 4090 running. 2000 epochs will need way to long on the gtx 1060

[@ei23de]

Update: 4090 has some bugs in cuda 117 - we found a workaround (see in piper issues)

[@ThorstenMueller]

You're welcome 😊. And yes, recorded a voice dataset takes way more effort than you might think at first. I trained my model on an NVIDIA Jetson Xavier AGX device and training took around 6 weeks 7x24. Thanks for sharing your CUDA solution 👏.

[@JUKEBOXANIMATION]

I guess this is great, and thank you for your effort, but I don't understand how you get to where you are at the begining. Should I open some command prompt windows and just type whatever you type the same way ? It would be even better doing some "don't think, do this +this +this" for people who understand nothing about how it works :)

[@jayeshkadam14]

How much time it took to train one model??? And how powerfull is your system?

[@ThorstenMueller]

For my Thorsten-Voice piper models i used an nvidia jetson agx device and training 24/7 took around 2 month.

[@jayeshkadam14]

@@ThorstenMueller oh my god. thats 2 months. 2 months!!!!! wtf

[@renan7716]

Great video! I'm learning a lot!Thanks!

[@ThorstenMueller]

Thanks for your nice feedback 😊

[@broketechenthusiast2372]

Thank you so much for your guide. This helped me get past some of the problems that I was having. I’m curious if you have a video or know of a video that goes over what to look for during training. Such as recognizing issues when looking at the generative loss or weights and what settings/dataset changes to make to counteract them. Thank you!

[@ThorstenMueller]

You're welcome and happy you like my videos. Do you mean how to keep an expert view on training parameters like losses, etc (eg. Tensorboard details)?

[@broketechenthusiast2372]

@@ThorstenMuelleryes. For example my gen_loss graph is erratic with no overall trend. I’d like to be able to have some idea of the causes and solutions for that kind of thing and how to identify similar issues.

[@BlazingFunk]

I'm missing the part where you show how to record your own voice. Do you simply have to replace the exported WAV files?

[@ThorstenMueller]

I created an own video on recording a voice dataset using "Piper-Recording-Studio" 😉. Have you seen it? kzbin.info/www/bejne/kGLToaeugbmSaas

[@FleaRHCP97]

@ThorstenMueller what if you already have a ton of recorded audio of yourself (lectures)?

[@pktechentertainment863]

Nice Sir i will try this. Thank You for you Tut.

[@_zproxy]

can i train piper voice without wsl or gpu on windows ?

[@ThorstenMueller]

I didn't try training a model on windows. I used pretrained models on windows but am not sure if you can train a new model on windows.

[@_zproxy]

@@ThorstenMueller cpu only training would be nice

[@ThorstenMueller]

@@_zproxy IMHO piper can be trained cpu based - might take way longer than gpu based but should work in general.

[@cloudsystem3740]

thank you very much for the hidden env and tutorial!

[@domesticatedviking]

This was great. It would be useful if you could recommend where to get an appropriate model for english speakers to use in place of the german model.

[@ThorstenMueller]

Thanks 😊. Do you know Pipers huggingface model location with lots of models for many languages?

[@YKSGuy]

Are there any tools for creating a structured training data set from existing voice clips vs recording new ones by reading specific text?

[@ThorstenMueller]

Not sure if there are tools for that, but the widely supported LJSpeech syntax is basically a CSV file with spoken text and filename split by a pipe | character. This shouldn't be too complex to create.

[@yakov6292]

Thanka for this video! Does it should work when fine tunning new language with different accent like Hebrew?

[@ThorstenMueller]

Yes, i even trained/finetuned my german model on an existing english checkpoint.

[@yakov6292]

@@ThorstenMueller Amazing! I think that German have more similar accent like English compared to Hebrew that it's completely different

[@yakov6292]

@@ThorstenMueller How long it took for you to fine tune?

[@ThorstenMueller]

@@yakov6292 Around 6 weeks on an NVIDIA Jetson Xavier AGX device.

[@anamontalvo9449]

Hello Thorsten, thank you for your videos, they are really amazing. Piper fits perfectly with my needs, but I couldn’t install it. Unfortunately, I encountered an issue with the requirements: ERROR: Could not find a version that satisfies the requirement piper-phonemize (from versions: none) ERROR: No matching distribution found for piper-phonemize I noticed that this is a frequently reported problem. Has there been any update or workaround for this issue? Thx again Ana

[@ThorstenMueller]

Thanks for your nice feedback 😊. Can use try this first: "pip install piper-phonemize==1.1.0". Are you using a virtual environment?

[@anamontalvo9449]

@@ThorstenMueller I have a conda venv with python 3.10.15 ; "pip install piper-phonemize==1.1.0" same error. ERROR: Could not find a version that satisfies the requirement piper-phonemize==1.1.0 (from versions: none) ERROR: No matching distribution found for piper-phonemize==1.1.0

[@ThorstenMueller]

@@anamontalvo9449 Strange. Any special system architecture? pypi.org/project/piper-phonemize/1.1.0/

[@anamontalvo9449]

@@ThorstenMueller Ubuntu 18.04 I tried with python 3.10 and 3.11 . Ive read comments of a lot users with this same error

[@scapolozza]

Hi Thorsten, thank you for your work. I learned a lot from your channel. I saw the video to increase the quality of espeak-ng, I was wondering if piper already used the espeak-ng dictsources. If he doesn't use them, do you think it is possible to implement them in some way? Keep up your amazing work.

[@ThorstenMueller]

Thanks for your nice feedback 😊. Yes, Piper TTS uses espeak-ng, but brings it own version out of the box. But you can apply the same process and just replace the compiled dict file within Piper own espeak-ng files. Does this help you?

[@scapolozza]

@@ThorstenMueller Of course it helps me. Thank you Thorsten GUUDEEE!

[@suhaybmir7773]

Hello Thorsten. I've enjoyed this tutorial. I'm stuck on the exporting section where I attempt to write = echo 'This is a test.' | piper -m /home/suhayb/Desktop/RealTraining/piper/out-train/lightning_logs/version_9/checkpoints/suhayb-real-voice_high.onnx --output_file /home/suhayb/Desktop/test.wav and i get the message unkown option -m. Do you know a way to fix or test this? Thank you in advance!

[@ThorstenMueller]

Glad you enjoyed my tutorial 😊. What is "piper --help" showing?

[@suhaybmir7773]

@@ThorstenMueller I managed to figure it out from another video of yours. Download and used the Piper executable file! Thanks again!

[@stevennovotny9568]

Do you know if reducing the learning rate helps with the erratic behavior in the learning curve?

[@ThorstenMueller]

Honestly, i'm not sure. Maybe you can get better answers by asking their community on github.

[@Mr-Coke]

IN deinem Video davor erstellst du das Stimmentraining mit Windows. Kann man diesen Part hier auch unter Windows machen? oder muss man dafür Linux verwenden?

[@ThorstenMueller]

Gute Frage - soweit ich weiß funktioniert das Training aktuell nicht auf Windows. Aber habe es selber (noch) nicht ausprobiert.

[@anarmustafayev9145]

Sehr schönes Tutorial. Mal eine technische Frage: warum ist es denn nicht möglich, dass man keinen riesigen Datensatz an Sätzen, sondern lediglich wenige Sätze benötigt, um den Klon zu erstellen. Es gibt einige AI-Software die anhand einer kurzen Sequenz von 30 Sekunden eigener Stimme bereits geschrieben Romane auslesen können. Gibt es in der Richtung auch eine Entwicklung oder beschränkt sich das Ganze lediglich auf solche festen Datensätze, die man vorher mit Mühe erfassen müsste? Vielen Dank und gutes Gelingen weiterhin.

[@ThorstenMueller]

Vielen Dank für deinen netten Kommentar 😊. Im freien Open-Source Umfeld ist das noch etwas schwierig mit hochwerten Stimm-Klons auf Basis wenigen Daten. XTTS von Coqui geht ja genau in diese Richtung (ist solide, aber hat auch noch Luft nach oben). Kennst Du das Video dazu? kzbin.info/www/bejne/fnulYmp-jJmAb9E

[@tesfa2586]

Can't have enough of your superb explanation. Just one more step if you do not mind. for every piper run, it is loading the model. Is there a way we can load the model in memory and generate audio via an API. Like how torch-serve does for PyTorch models.

[@ThorstenMueller]

Thanks for your nice feedback 😊. I've added your question about one time model loading and synthesizing multiple times to my questions for my interview with Michael.

[@SUBATOMICRAY]

i am useing "WSL - Windows Only" instructions and I am stuck on training and it crashes with "ModuleNotFoundError: No module named 'piper_train.vits.monotonic_align.monotonic_align'" i can't figure out why i am missing the modules or how to install it

[@ThorstenMueller]

When you run "pip list" do you see a piper package?

[@NNokia-jz6jb]

24:32 Here is a voice test.

[@ThorstenMueller]

Thank you 😊. Do you think adding this timestamp to video description would make sense?

[@NNokia-jz6jb]

@@ThorstenMueller Not adding, correcting and replacing with the 19.** minute one.

[@feizhu-yf7cl]

Great work. I test piper-tts on my terminal device, the voice sounds good. But I met a critical issue was that the cpu usage was nearly 100% when running piper-tts on one cpu core. Furthermore, I found that when i ony run espeak-ng and phonmizer to transfer input text to phonme-id, the cpu usage was still nearly 100%. Have you meet similar issue before. Hope to get some suggestion to low cpu usage. Thanks very much.

[@ThorstenMueller]

First of all thanks for your nice feedback 😊. As i didn't take a closer to cpu load while using piper i can't tell anything about it. But i can keep an eye on it on my next tests. Maybe ask on piper tts github community.

[@audreylin5424]

great stuff! thanks a lot. btw, i was wondering if piper can be trained to speak mix languages, for example, mix between english and chinese

[@ThorstenMueller]

IMHO this might work in near future when SSML is implemented. But i am not sure on their roadmap.

[@MicroblastGames]

I have a voice in .pth.. how can I use it in piper? how can I convert making the .onnx + .json file?

[@ThorstenMueller]

Is your .pth file from a Coqui TTS training? If so you can't (imho) convert it to Piper model structure (onnx).

[@tannisroot]

Hi, do you have a guide or can maybe point to some documentation on how to easily convert a public domain audio book recording into a voice dataset? I found a few hours of a single speaker speech, and from what I've seen piper recording studio is not suitable for this. Is there some sort of compilation of tools to split the audios apart into phrases, transcribe them using something like whisper and to compile it all into a dataset with the .csv file that can be then fed to piper training setup?

[@ThorstenMueller]

That's a good point. I've seen this questions a a few times now, but i don't know a tool or pipeline doing this. But would be a nice feature, for sure 😊.

[@ericvieira68]

One question: If one wanted to translate a video with the same voice as the original speaker, but in a different language. Would that be possible, maybe by first using a generated voice that is similar as a start?

[@ThorstenMueller]

Not sure if this is the right direction, but maybe you have to take a look to adjusting phoneming? kzbin.info/www/bejne/amqWqZWGfrZ1ibcsi=c-fYB5ykdW2peSrJ

[@razvanab]

Hello, Sir. Thank you so much for all of these wonderful tutorials. Since Kaggle has recently improved its free tier, could you create a tutorial explaining how to make a notebook on Kaggle for Piper's voice training? Thank you again.

[@ThorstenMueller]

Thanks for your topic suggestion 😊. I've added a "Kaggle Piper TTS model training" tutorial on my TODO list.

[@razvanab]

@@ThorstenMueller Thank you, sir.

I'm stuck. All went well till I run the "python3 -m piper_train..." command and got an error (_pickle.UnpicklingError: invalid load key, '

[@ThorstenMueller]

What size is your voice dataset (number of recordings)?

[@sabanheric6897]

Hello, after running pip3 install -e . I'm getting error ERROR: Could not find a version that satisfies the requirement piper-phonemize~=1.1.0 (from piper-train) (from versions: none) ERROR: No matching distribution found for piper-phonemize~=1.1.0 I was testing on python 3.8 3.9 3.10 3.11 but always the same error. I suppose problem is around piper-train

[@ThorstenMueller]

Maybe try to install the package this way: "pip install piper-phonemize==1.1.0". Does this help?

[@ThorstenMueller]

@user-jn9vs1yi8b Which hardware architecture do you have?

[@samvadiniartiste]

Could not find a version that satisfies the requirement piper-phonemize~=1.1.0 I get this error when I try to install pip install -e . I am on python version 3.12.3

[@ThorstenMueller]

You might wanna try "pip install piper-phonemize==1.1.0". Does this help?

[@yonnelb]

​@@ThorstenMueller I have the same problem: I tried try "pip install piper-phonemize==1.1.0" ~/MAEL_voice/piper/src/python$ pip install piper-phonemize==1.1.0 ERROR: Impossible to find a version meeting the requirement piper-phonemize==1.1.0 (from versions : none) ERROR: No matching distribution found for piper-phonemize==1.1.0 Installing phonemize alone doesn't work

[@ThorstenMueller]

@@yonnelb That's strange. Might a architecture issue, maybe a not supported os/hardware? pypi.org/project/piper-phonemize/1.1.0/ Maybe it's helpful to open an issue on Piper tts github community.

[@zerthura8500]

Hello. Do you think that use this software instead of coqui can be compatible with SadTalker? Coqui create conflicts used with SadTalker on windows.

[@ThorstenMueller]

Hi and thanks for your comment 😊. As i have no experiences with SadTalker this is hard to answer for me.

[@helloworld7796]

Hey Thorsten, i started with a new model from scratch, I got a good 300 sentences for now and started training. It works fine for now, but I would like to add more and more dataset, do I have to start each time from scratch since I have more dataset, or is there a way to add additional dataset and continue training?

[@ThorstenMueller]

In Piper you can use the argument "resume_from_checkpoint" for that. See more infos here: github.com/rhasspy/piper/blob/master/TRAINING.md#training-a-model

[@helloworld7796]

​@@ThorstenMueller Oh, I thought that option was only if I stopped training and continued on the same dataset. So I could use this even if I add more dataset and the model will recognize new dataset inside the same folder where i had old dataset? Also another questions, since you know everything. I am using CPU for training and waiting for days to get some result. I want to get GPU from nvidia. Which one would you recommend, 2080ti vs 3080? Or some other in that price range

[@ThorstenMueller]

@@helloworld7796 Good point, but as i finetuned my german Thorsten-Voice models with that option based on an english checkpoint this should be possible.

[@rokifromhk]

ERROR: Could not find a version that satisfies the requirement piper-phonemize~=1.1.0 (from piper-train) (from versions: none) why this error came

[@ThorstenMueller]

Does "pip install piper-phonemize==1.1.0" work?

[@anamontalvo9449]

@@ThorstenMueller no, doesn't work either

[@ssamjh]

How do I create a dataset of my voice? I'm not finding any instructions in the piper repo.

[@ThorstenMueller]

Maybe this tutorial from me helps on that. It's part of another video but at this timestamp i show how to record your voice and create a dataset in LJSpeech structure. kzbin.info/www/bejne/aoq3aYqQicSrapo

[@evertonlimaaleixo1084]

Very cool! How many minutes of audio did you use to mount the dataset?

[@ThorstenMueller]

I used my Thorsten-Voice Dataset 2022.10 for this training. This dataset contains over 12k of wave files with a duration of over 11 hours.

[@NacidoSKYRIM]

Thanks! very nice video! I have a question, how can I transform a .json file to onnx? Is it possible? Well, I have created some voice models using xtts mantella, creating .json files directly, I want to try to use them in piper tts

[@ThorstenMueller]

A trained piper model requires both files. The onnx is the actual trained model and the json is the config that "describes" your model. Did you run a full piper tts model training?

[@NacidoSKYRIM]

@@ThorstenMueller Yes, thanks! I already checked that little bit of information so now I'm following your guides to train my own model locally ^^! thanks for everything! ♥

[@NNokia-jz6jb]

So, where are the voice examples to listen to? What is the quality?

[@ThorstenMueller]

According to your other comment i guess you found it ;-).

[@MrZongwei]

Is there a simple way to generate myself dataset? Your video from two years ago using Mycroft Mimic Recording Studio was still a bit complicated。For example, placing a recording file in the wav directory and a content file in the CSV file, with the file name also starting from 000001

[@ThorstenMueller]

I have a video about Piper-Recording-Studio on my TODO list. Maybe this will make process little easier and more up to date.

[@MrZongwei]

we need you, please move on faster, my friend @@ThorstenMueller

[@paregis]

Great stuff! How long did it take to train? GPU? Which one?

[@ThorstenMueller]

You mean for my free german "Thorsten-Voice" (high) model? If so, i trained it on an NVIDIA Jetson Xavier AGX device and training took over 6 weeks.

[@احمدصبيح-خ7و]

When I start the final phase of voice training, this message appears /piper/src/python/.venv/lib/python3.10/site-packages/pytorch_lightning/utilities/data.py:153: UserWarning: Total length of `CombinedLoader` across ranks is zero. Please make sure this was your intention. rank_zero_warn( `Trainer.fit` stopped: No training batches.

[@ThorstenMueller]

Oh, that's a very specific error i did not encounter myself. Maybe ask this question on the Piper TTS github community for support.

[@احمدصبيح-خ7و]

It became clear to me that during vocal training the language should be chosen without specifying the geographical region of the language. The problem was that I was selecting language + country and that is what caused the problem. No problem now, but the result was weak because I relied on an average file in training, and Piper does not have a high-quality file for my language so I can work on it... Is it possible to create audio from scratch in Piper? How and how long does it take? Or is it possible to train a medium to high voice without vocal distortions?

[@ThorstenMueller]

@@احمدصبيح-خ7و I think finetuning a medium model to high will not work, but i'm not a 100% sure. I finetuned my german high model based on an english high model. Maybe this is an option for you?

[@HoratioNegersky]

Thorsten, I'm unclear on this after watching - is it possible to put in just a couple hundred new clips with a specific new voice and then use the checkpoints as a basis for cloning a new voice??

[@HoratioNegersky]

Nevermind, I see your full dataset contains the typical ~10,000+ I'm working on one with a baked-in accent so I'll be doing it from the ground up also. Thanks for the tutorial!

[@ThorstenMueller]

Glad you found your answer and best of luck for your project 😊.

[@truszko91]

"ERROR: Could not find a version that satisfies the requirement piper-phonemize (from versions: none)" Have you run into this? I cannot find a way to install this at all...

[@synesthesiam]

Only Linux is supported for now :/

[@truszko91]

@@synesthesiam But I am on Linux :/ what am I doing wrong then haha. Do I need any specific chip architecture or something? haha. that's so odd.

[@synesthesiam]

@@truszko91 What Python version? Only 3.9-3.11 are currently supported.

[@truszko91]

@@synesthesiam oh, where did you see that?? ;o I missed that memo.. I'm on 3.8.10

[@ThorstenMueller]

Thanks for the clarification. I've added this info to the video description.

[@morgan_rockefeller_official]

kann man das TTS auch dazu nutzen um seine buecher auf dem handy sich vorlesen zu lassen? zur zeit habe ich TTS stimmen die standardmaessig von google zur verfuegung gestellt werden wenn ich meine buecher mir anhoere. Und falls das geht wie macht man das dann?

[@ThorstenMueller]

Piper TTS selber läuft nicht auf Mobilgeräten. Allerdings kannst Du den Text auf einem PC als Wave Audiodatei erzeugen und die dann auf dein Handy kopieren und abspielen. Ist zwar ein Workaround, aber wäre das eine Option?

[@712Nabeel]

RuntimeError: cuFFT error: CUFFT_INTERNAL_ERROR I am having this error when i start to train

[@ThorstenMueller]

Which CUDA and Torch version do you have?

[@712Nabeel]

@@ThorstenMueller '2.0.1+cu118' Torch version (Ubuntu 22)

[@712Nabeel]

| NVIDIA-SMI 525.125.06 Driver Version: 525.125.06 CUDA Version: 12.0 |

[@712Nabeel]

I have downgraded ubuntu to 20 still the same error Im using RTX 4090

[@ThorstenMueller]

Hmm, have you seen this closed issue? Maybe this helps. github.com/pytorch/pytorch/issues/88038 @@712Nabeel

[@serqetry]

Thanks for this video. I was able to fine-tune a model and it sounded good with the test phrase generation, but when I exported it to onnx and used it with piper tts, it just made garbage noise and robotic gibberish. :(

[@ThorstenMueller]

That is strange. I did not encounter this problem. Did you already ask on the piper github community?

[@serqetry]

@@ThorstenMueller I created an issue on the github. I was able to get it to work though because I figured out only the binary version of piper has a problem with the exported models. I switched to using python version via pip and my model worked.

[@itsmekhadu2408]

ERROR: Could not find a version that satisfies the requirement piper-phonemize==1.1.0 (from versions: none)

[@ThorstenMueller]

Which operating system are you using? 32 or 64bit?

[@raiyanahmed1177]

can i use any dataset here for my language like bengali?

[@ThorstenMueller]

You mean a text corpus in bengali or a prerecorded voice dataset?

[@sickeningdreams]

where do i find english voice datasets?

[@ThorstenMueller]

There are multiple places you can take a look. Zenodo, OpenSLR and huggingface datasets. For english language there should be some datasets to choose from.

[@dash8x]

Can I train for a language not supported by espeak-ng as well?

[@ThorstenMueller]

Not sure if that's possible in Piper, but in Coqui TTS you can choose a "character based" training in case espeak-ng is not available for your language.

[@tesfa2586]

you can't. but you can add your language to espeak-ng first. I just did that and it worked. espeak-ng has good document on how to add new language

[@LucidFirAI]

So far I've managed to get Tortoise TTS and ClipChamp TTS working. Tortoise is slow and unreliable, ClipChamp is surprisingly versatile and high quality for a totally free web based thing. I take RVC with datasets to make the final output sound the way I want. Do you have any recommendations of what I should be using though? I want free, local (I have an Nvidia 4090 to take advantage of), with unlimited generation, and a human sounding voice. Ideally it would also be trainable so I could run it on a model of the voice I want, which could be refined with RVC, whereas currently I have to take a vaguely correct voice and then refine it with RVC.

[@ThorstenMueller]

If you have GPU power available as you said (NVIDIA 4090) you should take a closer look to Coqui TTS. Some models provide a really great quality. I've created audio sample comparison videos if you would like to get an idea on their quality: kzbin.info/www/bejne/fqDNpomjjJp_i6s and kzbin.info/www/bejne/jJ_Np2WCaJaYr7M

[@420gramas7]

Hey, is it possible to make this custom TTS to work in android to read pdf/epub books?

[@ThorstenMueller]

IMHO piper tts does not work on android. Not sure if this is on their roadmap.

[@MrZongwei]

Can I training other languages model by cloning my own Chinese digital text to speech?

[@ThorstenMueller]

Sure, why not?

[@CM-mo7mv]

I am not sure if you ever made a tutorial on ho to create the actual data set. I think it would be nice to gather a sample from older loved ones before they continue to the next plane ....

[@ThorstenMueller]

I've included it as a part of an early tutorial (kzbin.info/www/bejne/aoq3aYqQicSrapo), but maybe i can create a special tutorial just on recording and dataset creation. Do you think this would be useful?

[@flethacker]

i have to record 12,000 wav of my voice samples??

[@ThorstenMueller]

No, i did many recordings. But if you train / finetune an existing model you might be able to get good results with 500-1000 recordings already.

[@tegeztheguy2719]

Hi, I loved the video where you showed how to train your CoquiTTS model on Windows, could you maybe do a video about how to train a vocoder model for the trained model?

[@ThorstenMueller]

Thanks for your topic suggestion 😊. I've added it to my (growing) TODO list.

[@dash8x]

I am getting Segmentation fault (core dumped) error when I run piper_train

[@ThorstenMueller]

Strange - which operating system? Did you open an issue on Github about it?

[@dash8x]

@@ThorstenMueller It's on Ubuntu. I got it working after reducing the batch size. Training has been going on for a couple of days. But the sound it generates is illegible even though it sounds like spoken text.

[@ThorstenMueller]

@@dash8x Happy you got it working. Give training some time to produce human sounding results.

[@PowerGlideSir]

Great video! However, I keep getting the following error: RuntimeError: Error(s) in loading state_dict for VitsModel: Missing key(s) in state_dict when running without the "--quality high" flag. When I run with the fag enabled I keep running out of VRAM on my RTX 3090. Any tips on how to solve this issue?

[@ThorstenMueller]

Thanks for your kind feedback 😊. There seems to be an active issue on Piper repo. Do you know this? github.com/rhasspy/piper/issues/108

[@teenudahiya01]

hi bro, can I create onnx model and onnx json file for Hindi language voice and can I use it in Piper tts, please reply .

[@ThorstenMueller]

IMHO there's no pretrained Hindi model to be used out of the box available or did you train a Hindi model yourself?

[@teenudahiya01]

@@ThorstenMueller is Hindi model and hindi onnx file are different i have piper tts wsl can i train whith this

[@yyyzzz-k3r]

What is their model behind? Does anyone know?

[@ThorstenMueller]

Technically these are VITS TTS models. Is this your question?

[@syedmuqtasidalibscs0434]

metadata.csv file is not placed in dataset from where I can get this ?

[@ThorstenMueller]

Metadata.csv is part of a (ljspeech structured) voice dataset for TTS training. Do you have a voice dataset for your model training?

[@syedmuqtasidalibscs0434]

@@ThorstenMueller yes i record 1150 voices and it generate wave folder for voices and metadata.csv same i am following your video in your video when you download the dataset you have 4 files but i have 2 why ?

[@ThorstenMueller]

@@syedmuqtasidalibscs0434 I've split metadata up in a evaluate, train and test dataset but this is optional. You can put all your 1150 wave files in one complete metadata.csv. This should work fine 😊.

[@syedmuqtasidalibscs0434]

@@ThorstenMueller its not working fine when i am running command i am getting this assertion error missing metadata.csv

[@syedmuqtasidalibscs0434]

@@ThorstenMueller when running long command piper_train.preprocess in which input ditectory output directry langauge etc

[@darkjudic]

Is it better than Coqui TTS ?

[@synesthesiam]

The answer isn't straightforward. Coqui TTS is much more flexible: it supports many different types of TTS models. Piper only has VITS, but it's been designed to be usable on lower-end hardware (RPi 4). So speed has been more important than quality. The binary releases are also only ~25MB and self-contained with voices being 50-100 MB apiece.

[@tr1pod623]

does this work on windows 11? cuz i get an error when trying to train nrvcerrror

[@ThorstenMueller]

As i do not have Windows 11 i can't really help you on that. Maybe it's a good idea to ask this question of Piper TTS Github repository.

[@Tourbillion9048]

I'm not computer savvy. Would you be able to create an app that we could use to clone our voice, so that we could just dump a text document into it to convert into an audio file? That would be so cool.

[@ThorstenMueller]

I agree that this would be really cool. At the moment voice cloning requires some technical detail skills. Maybe you can take a look to Coqui studio as they are working on an easy voice cloning process in a nice user interface.

[@Tourbillion9048]

@@ThorstenMueller Thanks for responding to my inquiry. I really appreciate all the hard work you put into making your videos, and trying to help people. Could you make a video for dummies, with step by step instructions of how to install Coqui on Windows, and how clone voices?

[@ThorstenMueller]

@@Tourbillion9048 You're very welcome. Here are two tutorials for Windows that might help you. * Installing Coqui TTS on Windows: kzbin.info/www/bejne/sIPEdZhmbciIf80 * Voice cloning on Windows with Coqui TTS: kzbin.info/www/bejne/mHvNq4allMuCiZo

[@lennart3626]

Thanks for this! Real excited to try this out for my university project. Unfortunately, after brude forcing it for 2 days I could not make it work to install the requirements libraries. I am on the raspberry pi 4 and after some first struggles I have upgraded my python to 3.10. Nevertheless I still get this error for piper-phonemize: ERROR: Could not find a version that satisfies the requirement piper-phonemize~=1.1.0 (from piper-train) (from versions: none) ERROR: No matching distribution found for piper-phonemize~=1.1.0 Any advice how I can break free?

[@ThorstenMueller]

I've heard about that issue a few times already. Maybe try installing it with a fixed version: "pip install piper-phonemize==1.1.0", this should work then.

[@lennart3626]

@@ThorstenMueller Thank you very much! I have been watching a lot of your great videos the last couple of days to get the perfect tts model for my needs. As a bloody beginner, I do still have some trouble keeping up but practice makes perfect, right? For the problem above, I also went down a deep rabbit hole of form posts and ended up having to reinstall the OS on my rpi. From school we got the 32bit OS but apperently piper-phonemize only works on 64x. Doing that and making sure my python is at 3.10 finally got the job done. Unfortunately, piper turned out to be not the best model for my needs and I now moved on Coqui! Let's see where this adventure leads me!

[@ThorstenMueller]

@@lennart3626 Thanks for your response and tipp that piper-phonemizer is 6bit only. Good luck with Coqui TTS.

[@_Sepherial]

How do I use a cloned voice to read aloud a pdf file?

[@ThorstenMueller]

This is not possible by default. You might write a script extracting text from pdf file and putting this text to Piper for TTS generation.

[@_Sepherial]

@@ThorstenMueller If I was a programmer, I wouldn't need to ask this question.

[@ThorstenMueller]

@@_Sepherial Right now there is no easy way to do this, not sure if someone is working on that. Because what you ask for is really a useful feature. I can ask Mike this question on my interview.

[@khajask8113]

Hindi language supports..?

[@ThorstenMueller]

You can create a tts model for every language in Piper. Existing languages and models can be found here: github.com/rhasspy/piper/blob/master/VOICES.md

[@katokira260]

Thanks for making this tutorial ... is it possible to train Arabic Model ?

[@ThorstenMueller]

You're welcome and yes Arabic TTS model training should be possible to train 😊. Have you tried it and encountered any problems?

[@katokira260]

@@ThorstenMueller nice ... no i never tried ... but i will ... thanks again

[@ernieprevost6555]

Hi Thorsten, great video really useful. I hope you don’t mind me asking a couple of questions. I am trying to create a standalone voice assistant project and have most of it working. I am using ‘vosk’ as a STT engine a ‘piper’ a TTS engine. At this time I am using one of the available models “vosk-model-small-en-us-0.15”. My ultimate aim would be to create model based on HAL9000 (Douglas Rain) from 2001 a Space Odyssey. I decided to work through your video to understand and test how the model creation process works. I am running this on a Raspberry Pi model 4. I downloaded your voice dataset (it must have taken you a long time to create all the wav files) and successfully ran the pre-processing stage. The problem I have now is running the training stage, I get the following error: ModuleNotFoundError: No module named 'piper_train.vits.monotonic_align.monotonic_align'. I have not been able to find what the problem is, do you have any idea? I also notice that the accelerator parameter has a ‘TPU’ option, is that Google USB TPU used with Tensorflow? Thanks again Ernie

[@ThorstenMueller]

Thanks for your nice feedback and yes, recording took some month. Did you run the "build_monotonic_align.sh" script?

[@annlaosun6260]

does it work on mac?

[@ThorstenMueller]

I'm not sure if training works on mac, but as there are piper downloads for mac available it should be able to run with pretrained voice models on mac. github.com/rhasspy/piper/releases

[@manuelmao4700]

Hi! great video! How can i do this on a macos?

[@ThorstenMueller]

Thanks for your feedback. By now Piper is not officially supported on Mac, just on Linux. But there are open issues with some tips for Mac OS. Maybe it's worth a look. github.com/rhasspy/piper/issues/27

[@serge3595]

@@ThorstenMueller can you update us with a guide when Piper is made to work smoother with MacOS? Thanks :)

[@MrZongwei]

Does it support Chinese voice

[@ThorstenMueller]

According to their Github page github.com/rhasspy/piper Chinese (zh_CN) is supported. So yes 😊. Check my tutorial on how to use Piper TTS with the pretrained Chinese model here: kzbin.info/www/bejne/qJvUZpiQpLyKidE

[@MrZongwei]

@@ThorstenMueller it's great。

[@Borszczuk]

Hint: consider adding pop filter to your mic setup

[@ThorstenMueller]

Thanks. I have one, but as the mic has an integrated one i thought it would not be required. I will give it a try.

[@TNMPlayer]

I really wish TTS would come packaged with RVC.

[@ThorstenMueller]

Thanks for this "hope" and suggestion. Maybe it's worth starting a discussion on Piper TTS Github repo. github.com/rhasspy/piper/discussions

[@ComatoseMN]

I'm not shocked, because literally nothing ever just works for me. However, after trying to complete the step with install the requirements.txt, I got this "ERROR: Ignored the following versions that require a different python version: 1.21.2 Requires-python >=3.7,

[@ThorstenMueller]

First of all thanks for your nice feedback 😊. Did you get it working already? Which python version are you using? Are you using a python venv?

[@UeujkmVeujkm]

Классное видео

[@amitmaharjan3522]

thank you

[@ThorstenMueller]

You're welcome 😊.

[@rommix0]

0:05 Just discovered you, because your Piper voice sounds just like you.

[@ThorstenMueller]

I guess this is a compliment, or? Thank you 😊.

[@rommix0]

@@ThorstenMueller Of course it's a compliment lol. You have a nice sounding voice.

[@FrankGlencairn]

Jetzt noch auf Win und ohne Prompt aber mit UI - das wär's

[@ThorstenMueller]

Stimmt, das wär's 😊. Weiß aber nicht, ob sowas in die Richtung geplant ist.

[@Teletha]

why has no one just made a simple windows app

[@ThorstenMueller]

I didn't try it myself, but have you seen this project (github.com/natlamir/PiperUI)? Not sure if this works in general or on windows or it's safe to use. But maybe check it out.

[@ugn]

"I Gude, V?"

[@ThorstenMueller]

🤩👍🏼

[@giovannisardisco4541]

The video is interesting, but it was hard to follow because the text and characters were too small to see clearly on a smartphone 😢

[@ThorstenMueller]

Thanks for your helpful feedback 😊. In later videos i increased font size in terminal. Hope it is better to read in newer videos.

[@TheVisitorX]

Finde es sehr verwirrend, dass deine Titelbeschreibung und der Name des Videos auf Deutsch, deine Videos aber auf Englisch sind. Ich falle da jedes Mal drauf rein. Ich habe nichts gegen Englisch, aber dann solltest du vielleicht zumindest deine Videos so kenntlich machen, dass man das auf Anhieb sieht und nicht erst, wenn man das Video schon angeklickt hat. Sehr irreführend.

[@siemensohm]

Bei mir ist sowohl der Titel, als auch die Beschreibung auf Englisch. Oder ich bin zu spät und das wurde in der Zwischenzeit geändert. ;)

[@ThorstenMueller]

Hi, Danke für den Hinweis. Meistens, wenn ich es nicht vergesse, mache ich eine US-Flagge in den Titel um es zu kennzeichnen. Ich habe das hier auch ergänzt, daher nochmal Danke für die Info 😊. Wenn man die Sprache in KZbin und/oder die Sprache im Browser auf Englisch hat, dann kommt die Beschreibung auch gleich in Englisch.

[@ThorstenMueller]

@@siemensohm Hi, das liegt an der Sprache mit der du KZbin nutzt (und/oder die Spracheinstellung im Browser). Ich habe es aber nach dem Hinweis mit einer US-Flagge im Titel gekennzeichnet, dass es ein englisches Video ist. Danke Dir für die ergänzende Info 😊.

[@siemensohm]

@@ThorstenMueller Ah, so funktioniert das also. :)

[@TheVisitorX]

@@ThorstenMueller Vielen lieben Dank dafür! So ist es finde ich besser nachzuvollziehen. Sollte auch gar nicht böse gemeint sein, denn ich schätze und mag deine Videos! :)

[@cybertrike]

This whole video is why nix exists .