I Created Another App To REVOLUTIONIZE YouTube

[@ThioJoe]

If anyone from Google Cloud or Microsoft Azure are watching this 👀 The official SSML (Speech Synthesis Markup Language) spec actually does have an attribute called "duration" for the "prosody" tag to say exactly how long the synthesized speech should be, BUT no services seem to support it. (UPDATE: Azure now does support a duration tag!) 😔 (Amazon Polly actually does but only for non-neural voices) Please add support for this tag, because it would make it WAY simpler to accomplish all this! It would eliminate the need to stretch the audio or do multiple passes. ------------Some Additional Notes for everyone else: • Update: The app now supports DeepL! • Obviously it's not anywhere near perfect, so it is probably best suited for viewers who wouldn't be able to watch the video in English at all. To avoid KZbin automatically switching to the dubbed track for people who have a very high likelihood of understanding English anyway, I might consider just doing it for languages with fewer bilingual speakers.

[@edwardgabrielgaming8524]

Like this! Please make romanian

[@𪜊]

second

[@muizzsiddique]

I assume it's because they're going to do the same as you and generate the dialogue twice.

[@lajawi.]

Even though translations are great, I hate the implementation of it, in the sense that there is no setting to enable or disable whether it automatically selects the language your app is set to. The same for video title translations.

[@pinetrax]

@@lajawi. Exactly! KZbin should either implement a switch to turn it off completely or ideally let us choose which languages we understand natively and only switches to a native language if the video is non-native.

[@strangestparticle]

as a Brazilian, watching you dubbed in Portuguese feels like I'm watching a documentary on a cable channel xD

[@MarcioHuser]

Yep, I agree

[@torpedo_]

me assustou pra poha

[@Airton2]

@@torpedo_ tbm, com essa dublagem estranha do nada kkkk

[@Ptaaruonn]

God, yeah, it feels so weird.

[@EnaBBQ]

Agreed

[@Jet_Set_Go]

You should use the API of DeepL, it is much faster and more accurate than the Google translator, in fact, I don't even speak English, all this text is completely translated with DeepL. (I say this because the Spanish audio, although understandable, is poorly translated).

[@ThioJoe]

I'll have to look into it

[@NoelleTGS2]

I came here to suggest the same thing, DeepL is much more accurate than Google!

[@muizzsiddique]

Sora the Troll even recommends DeepL when it comes to English Japanese translations.

[@1aboPLZ]

Yup, as a German student i can say DeepL is much more accurate for french and English than Google Translate

[@nagoyamaasuncion3417]

Somewhat this is what it's lacking for me because some wording use for other words is not refined to native speaker

[@kindofanmol]

As an Indian, I can confirm that the Hindi dub sounds ridiculously good. It even uses less popular words from Hindi which aren't very common in day-to-day conversations, but are an essential backbone of Hindi. Impressive stuff!

[@gamersworld4176]

the voice sounded very fast and very slow different times. it even overlapped in some times. its good but it still is very flawed.

[@lucky_lol]

Ya it uses some 'less popular words' It's the only problem I hope there's some Hinglish engine which uses words we are more familiar with!!

[@kindofanmol]

@@lucky_lol I've been using shuddh unadulterated Hindi for a while now and its such a flex tbh. So many people are losing the essence of the language.

[@Explorerladka2001]

​@@kindofanmol yes some words rarely used in daily Life,and lack of human emotions, only straits reading Hindi me bole toh- fill nahi aa raha hai, But ye kaam karega Main start kar raha affiliate me logo ke hindi videos ko English me dub karna Bahut paisa Hai

[@kindofanmol]

@@Explorerladka2001 oh dubbing as in subtitles? kidhar?

[@Fiaspo]

Your work is simply amazing! A doubt about the function of audio tracks on KZbin. Are you able to add audio tracks to old videos, or is this limited to future videos only, and can only be added at upload time?

[@gabrieItv]

slv fiaspo

[@Loneslol]

Yes, you can easily add them to videos you already uploaded. As example: Mr. Beast, he used this option in some old videos, for example: Mr Beast "Golden pizza" video were created so much before after this function. I hope it helped you :D

[@Fiaspo]

@@Loneslol Mr. Beast is a guy that KZbin could easily make an exception for modifying old videos. So my doubt about this dubbing option still exists.

[@Loneslol]

@@Fiaspo Makes Sense, but alot of other youtubers do this Fiaspo, JuckSucksAtLife did this for one of his old old videos for spanish for texting, since he were thinking of removing the JackSucksAtSpanol channel, and theres some mini youtubers that did that, i remember people with like, 5k, 60k subs doing that, and since you are a Very big youtuber, i guess you could actually do It, and Ive been studying alot about this feature, and if you want i can contact you things like how to apply and things like that.

[@Loneslol]

Add: Mr Beast were only the 192nd youtuber to have this added to their Channel, and this function is already quite old, and KZbin probably have added more features to it.

[@clickbaitpro]

He is literally *REVOLUTIONIZING* KZbin

[@ali99_82]

Its funny how he used to be a troll and disrespected in the beginning

[@KingLarbear]

@@ali99_82 and now he is so wholesome

[@ricedbroccoli]

@@ali99_82 I still remember that video where he said if you wrap your phone in tinfoil and charge it, it will charge faster

[@DnvGoodwin555]

Russians have same fuction at least one for year. ^_^.

[@detective2221]

@@KingLarbear no

[@WeegeepieYT]

Hey dude, how did you get access to the feature? Tried contacting YT via live chat but it went nowhere lol

[@jerstumc5033]

If he doesn't make a video about it, i also want to know Lol

[@darikyou3983]

Oh Hello Weegeepie!

[@KooperTheTrooper15]

What u doin' in here Weegepie?? xDD love your videos

[@vikombucha]

I want to do it too - damn KZbin srupid chat !

[@fghsgh]

6:17 Note, the feature to speed up or slow down audio is actually just a built in browser feature, not a youtube-specific thing. KZbin just uses the browser's integrated video playback feature, which supports playback rate. It also doesn't sound all that great. But Audacity can do something similar if you disable "high-quality stretching". And yes, for some kinds of audio (notably speech, but not music) it does sound better.

[@ertg3486]

yep. just about to write that

[@questocanale]

Having listened to the Italian version, I have to say, I am very impressed with the accuracy of the translation and the voice generation, up until you start talking about technical stuff, where terms like host, play, run, are translated out of context and make it more difficult to understand what you say. Still, I’m very impressed and can’t wait to see how this evolves!

[@Damianir750]

im italian too!

[@LukeSmith1984]

@@Damianir750 buonasera signori!

[@FineCurry]

I’m also listening to the Italian, and the voice is pretty concise but at times I get hung up on it

[@nonsuccesso88]

Tachipirina 500 if you take 2 it becomes 1000

[@charginginprogresss]

Le file però lol

[@lalatendupdeo4398]

From being a parody channel to revolutionizing KZbin, ThioJoe came a long way!

[@jimschips254]

When he said the new settings option I proceeded to listen to the whole thing in German. I don't speak German.

[@jannikheidemann3805]

@@jimschips254 I do speak German. The german audio track sounds robotic to a native speaker.

[@Barrinetic]

@@jannikheidemann3805same thing with english, and every other language

[@poopiecon1489]

i still sometimes have hard time believing what he says

[@Barrinetic]

@@poopiecon1489 lol

[@evannapitupulu5023]

This is revolutionary! I thought Mr. beast hired an interpreter/translator to dub his voice in some of his video for Indonesian audience.

[@Niki1A_]

I know this can be helpful for a lot of people, but I don't like that for me as a German it starts the video with German audio track and I manually have to switch to the English track, although English is one of the preferred languages of my Google account. The same problem actually happens with subtitles always showing even for languages you know, but you can turn off available subtitles always showing (of course they then also don't show for languages you don't know). I hope KZbin will add a setting for not using alternate audio tracks by default (so far there isn't one), before this feature gets widely used. If you know the original language well enough, the original will always be better than a translation (even if the translation and voice acting is done by humans). (This is because some things like puns don't always work in the other language.) I think this is a problem that many sites from the US (and other mostly monolingual countries) have, that they don't really let you set multiple preferred languages (or in Googles case don't consistently use that information). (The worst case I have seen is on WattPad, where you have to change your language settings to find books/stories in a different language.) PS: The German audio track sounds pretty good, but it has a few weird quirks: At one point there was a significant jump in voice speed to a very slow speed that sounded unnatural (maybe you should cap the slowdown to some reasonable value and just pad with silence in that case). Also the AI didn't know how to translate 'you' and sometimes went with the honorific 'Sie' and sometimes the more personal 'du' (on KZbin it should always be the personal, unless your demographic is over 50 and you want to treat them as strangers), sometimes it should have translated 'you' with 'man' when talking about general instructions (in sentences like 'So how do you create them?') and instead went with 'du'.

[@noelletheradcat]

I'm a German too, and Google just doesn't care what your preferred languages are, it will always show any translations that are in your native language: titles, descriptions subtitles, and now audio tracks too. It's been like this forever and I doubt they're going to do anytging about it even tho it's super annoying :/

[@GURENO]

I'm japanese and so superied at how natural the translation is. I never saw anything like this!

[@Adam__Lee_Jang_Chu.2024]

Yandex Browser

[@Datosdelconejo]

Si, se escucha muy natural, parece una persona real que lo tradujera

[@0xC272]

The speed up and speed down is a bit odd at times, but it's actually surprisingly good!

[@tomatann635]

最高

[@Penden]

日本語で言っちゃうけどこれは作者に脱帽するレベル 字幕だけ日本語対応してる場合はラジオ感覚でざっくり理解しながら聞くっていうのが難しかったけど、音声も他言語に対応するようになってKZbinの利便性がめちゃくちゃ向上した　感謝感謝

[@KevinStratvert]

Good stuff! Do you also see an impact on your metrics like retention and average view duration by dubbing over a video into multiple languages? I've found that by including high-quality captions, videos see higher average view duration and retention with manual captions vs. auto generated captions. I would hypothesize that the impact is even more profound when you dub over a video. There's a reason Mr. Beast is hiring a crew to dub over all his videos into multiple languages to improve the accessibility.

[@user-ir6xh2mx9d]

Wow Kevin ❤️

[@MohammedMehdiTBER]

Imagine merging this script with Magenta synthesis project where sounds will sound natural and musical instead of robotic.

[@just.nobody]

Wow, One of my favorite creator commenting on a video of another one of my favorite creators.

[@CattopyTheWeb]

@@just.nobody same. I didn't know he watches ThioJoe

[@KIDHOBBYVault]

Same

[@SejaContraste]

How did you manage to access that option? I have no idea where to submit a request. I actually have a large channel, and I'm quite interested.

[@user-ov6kg5yn3k]

Did you figure that out, i just send a feedback message in youtube studio today?

[@ex0stasis72]

I love that you so heavily add comments in your code. I know a lot of experienced devs in the industry prefer not to add comments on code that is "self-evident", but what is self evident for them isn't necessary so for me.

[@faleay2849]

Вау. Хоть и звучит русская дорожка не идеально, но это всё равно очень круто. Наверное, пока что я бы использовала английскую дорожку, так как в целом понимаю всё, что говорят, но с видео из других, не англоговорящих стран так сделать не выйдет и данное приложение будет очень полезно.

[@shikajf]

у Яндекса лучше

[@rodion_runchev]

@@shikajf нет

[@Шестиугольник]

@@shikajf у Яндекса буквально так же :/ я смотрел несколько обучающих английских видео на русском и во всех них диктор либо быстро либо медленно говорил

[@KrasBadan]

@@rodion_runchev Я думаю у яндекса работает лучше именно для перевода на русский. Я прочитал в другом комменте, что он умеет давать разным людям разные голоса (всего есть 6 мужских и 6 женских). Но автор тоже сделал реально круто, плюс работает переводит на многие языки, так что более универсально.

[@ATtiny13a-PU]

Я просто хочу функцию добавления аудиодорожек к оригинальному видео, чтобы можно было смотреть оригинал в фанатском переводе

[@costyapetrov1067]

In the Russian Yandex browser, there is a similar function on all KZbin videos, but the list of languages is limited. There are only the most popular languages. The voice in Yandex is much better and more human, and the translation quality is good. And he also gives each person in the video their own voice. If two men speak, then they will have different voices, and women will have a female voice.

[@AsdfgAsdfg12363]

Эль Примо

[@damedane5981]

Да нах он нужен

[@costyapetrov1067]

@@damedane5981 чтобы видео смотреть на иностранном языке.

[@damedane5981]

@@costyapetrov1067 ну а какой смысл? язык лучше знать. единственное что на ютюбе появиться больше возможностей для заработка, яндекс тут не нужен

[@amir_ibragimov]

Рил?Я его не скачивал из-за того что он жрет много

I have created my self some scripts to aid me in editing videos, I have one that detects the intensity of the voice to make smaller or bigger text for overthevideo subtitles, so I already do make srt files for all my videos, this sounds like a great addition to add more ways to make content avariable for everyone.

[@droneactiongermany]

Jeez, it frightened me when the video started. At first I thought it was an ad, but then i realized it was the actual video playing. Cool function

[@gor1kartem]

Actually yandex can translate videos. It has been working for a year. Now it can translate only into russian from english, german, french, spanish and italian. Also translation from chinese into russian will be available soon. It can't translate into other languages, but it translates much better. It detects voices of people, and gives each person his own voice (yandex has 6 male and female voices). It can translate any video with supported languages, it uses voice to text (It does not use youtube automatic subtitles). Also stream translation is on beta, but now we can use it only on limited amout of channels like NASA, english speaches.

[@lythd]

huh interesting, ill have to try it out thank you :)

[@redpepper74]

Oh wow this must be really cool for Russian speakers

[@runtelol]

Was just about to mention this, heard this is a very useful feature from some of my Russian friends (that only speak Russian)

[@danko5866]

Yandex has a youtube competitor? Edit: I searched up, it's called "Yandex Zen"

[@DnvGoodwin555]

@@danko5866 I think it's not Yandex Zen. Yandex Browser - chromium clone with yandex services. yandex zen is another service. And it's not part of yandex now.

[@rafaeltorovip]

Let me tell you, I tried your video translated to Spanish and even though is not 100% perfect it is very very good. The translation and synchronization was almost perfect. Great job. 👍

[@ThioJoe]

Yep from my understanding, Google Translate is really good

[@GianniLeonhart]

Man what's up with people low standards? do we Spanish speaking people are so desperate to get attention from the people at the north? Lmao That dub was trash, the sync was terrible, no attempt to match lips, generic monotone voice not even close to Thio's You can get dramatically better results asking an amateur south american fandubber on fiverr to do the dub for pennies lmao Impressive work, there's no doubt about that, great job indeed. But all that effort for a less than mediocre result? the least we can do is be honest with Thio about the results so things can improve in the future

[@rafaeltorovip]

@@GianniLeonhart I could report your comment as racist or discriminatory but, them your comment would be deleted and other people won’t be able to see what kind of poor mentality you have. Sorry for you. 👎

[@SurmenianSoldier]

@@rafaeltorovip lmao even i am a spanish speaking person myself yet i think the translation is crap

[@rafaeltorovip]

@@SurmenianSoldier I said that the translation was not perfect but, it is the beginning of something that eventually is going to improve. Everyone just wants to criticize and complain about the efforts done by others but, the same people complaining and criticizing do nothing. Always is like that.

[@gama8417]

I know a Brazilian channel with that feature and it is called Manual do Mundo.

[@kol2456]

I have been wanting just the multi-channel audio feature to be rolled out publicly for a while now. Language tracks are the obvious use, but imagine all the times the music in a video was too loud or just annoying. Imagine being able to mute just the music. Or with sports broadcasts, being able to turn off the crowd sounds or the commentators individually. One day, bandwidth might even be good enough for live content to have multiple channels that you can individually adjust. No more trying to tell a streamer that their music is too loud or the game is too quiet.

[@HamzaOmari1]

BRO IS MAKING KZbin BETTER ALONE FOR FREE

[@tauon_]

"alone for free" - like literally any other open-source programme?

[@s0haku]

This is very wonderful. I could hardly feel any discomfort when listening to the Japanese audio tracks. However, I think Deepl is more accurate than Google Translate when it comes to translating between Japanese and English. This text was also translated by Deepl!

[@Enforcedcraft]

Let me know if there is a difference in this. I'll write this in DeepL and next one above it is gonna be from Papago. I use that to translate Japanese to English . So do tell me. この違いがあれば、教えてください。 これと異なる点があれば教えてください。これはDeepLで書いて、その上の次のものはPapagoから書くつもりです。日本語から英語への翻訳に使っています。だから、私に教えてください。 This is Papago one. Let me know how accurate it is compare to the DeepL . I use this Papago to translate(right now it's with Honorifics) and Romaji to translate from Japanese to English. It should be more "accurate" since its a Korean company and Japanese and Korean have similarity. これはPapagooneです。 DeepLと比較してどれくらい正確なのか教えてください。私はこのPapagoを使って翻訳しています(今は敬語と一緒です)。ローマ字を使って日本語から英語に翻訳しています。 韓国の会社と日本語と韓国語は似ているので、もっと「正確」でなければなりません。 korewa poppagūn'desu dīparuto hikakushite dorekurai sēkakunanoka oshietekudasai watashiwa kono papagōo tsukatte hon'yakushiteimasu imawa kēgoto ishodesu rōmajio tsukatte nihon'gokara ēgoni hon'yakushiteimasu kan'kokuno kaishato nihon'goto kan'kokugowa niteirunode motto sēkaku de nakereba narimasen'

[@s0haku]

@@Enforcedcraft I found DeepL more accurate than Papago. DeepL has a fairly natural way of translating Japanese grammar and words, making the text very readable.

[@ParvTV]

@@s0haku I agree, DeepL is my go-to for Japanese and Papago is for Korean

[@vibaj16]

hmm, the sentence "I could hardly feel any discomfort when listening to the Japanese audio tracks" seems like a bit of a weird way to phrase that thought. Is that exactly what you meant, or was Deepl doing something weird?

[@itdepends604]

@@vibaj16 there are a lot of similar things even in human translated text. You would probably have to sacrifice some accuracy in order to avoid this kind of thing.

[@GYTCommnts]

This is great, however, I hate apps and sites that apply some settings automatically and (sometimes) the configurations you select don't stay or aren't available. So, for example, if you know more that one language, it may apply your system one, even if it's not the one you want. I like all my media in original language with subtitles. It's not a big deal IF the option will be there to change it. BUT the tendency of choosing everything for you, creates problems when there are not manual overrides. An example is Netflix. They decide what subtitles are available from where you live. So, for example, I watched a movie that the subtitles were better in english, but later they just didn't allowed my to choose them anymore. Because of my location. That is not OK and is happening more and more with different things. In some cases I had to pirate stuff that I HAVE AVAILABLE IN A PAID SERVICE because they don't allow me to choose whatever language configuration I want. Not cool.

[@Lampe2020]

When the video suddenly stopped for a brief moment to load because it hadn't buffered enough it reverted back to English, so maybe there's a bug that automatically changing the quality reverts the audio language back to the original?

[@HavingFunTimes]

Love this so much! I don't think all channels have the ability to upload videos with multiple audio tracks yet (only my big channel has an audio track thing next to where you add subtitles) but this will be really useful in the future. One feature I would love to see in this is the ability to set multiple speakers.

[@user-ir6xh2mx9d]

@Zaydan Alfariz idk I have the option on channel that has less than 10k subscribers but I did submit my passport for identifying verification for using advance features. Also you can't add additional audio tracks once your video has gained 100k views, it greys out the option after that.

[@khalilboss2855]

Now I'll wait for Linus tech tips to review this app. Great work!

[@victorblade6465]

I'VE BEEN WATCHING YOU EVER SINCE 2014/5 MAN; I JUST WANT TO TAKE A MINUTE TO SHOW YOU HOW MUCH APPRECIATION I HAVE TOWARDS YOU, CAUSE U PLAYED AN IMPORTANT PART IN MOTIVATING ME GROWING UP MAN. LOVES FR❤️‍🔥✝️💓.

[@dailycast8859]

Hello from Russia, the Russian company Yandex has done what it says in this video, but better) There is a Yandex Browser where the video translation function is available, it translates it by pressing the button at the top of the video within a couple of minutes, selects the timbre of the voice being played suitable for the timbre of the person from the video (distinguishes female/male voice as well) And watched this video through it, and works much better

[@Thunderbolt4]

今までは海外の自動翻訳された動画を見るときは字幕を追うので必死で映像を見れませんでしたが、音声があることで楽に映像を集中して見ることができるようになりました！！ これはすごい！！ KZbinの公式機能として欲しいですね！

[@sube_wa_tame]

数年前に自動翻訳の字幕で海外の動画も見れるスゲーってなってたのにもう吹き替えが来るとは！

[@triplearielinfinite2]

そうですね

[@Baselatx]

Here's the transalation: Until now, when watching videos that were automatically translated overseas, I was desperately chasing the subtitles, so I couldn't watch the video, but now that I have the audio, I can easily concentrate on watching the video! ! it's amazing! ! I want it as an official function of KZbin!

[@adonesjb]

Também acho incrível poder ler seu comentário traduzindo automaticamente pelo Google. A internet está ficando incrível. Conhecimento ultrapassando as fronteiras. 🥳👏🇧🇷

[@vannillaAJofficial204]

@@adonesjb eu tbm rsrs

[@santiagocabascango6514]

Empresas: es caro y mucho tiempo e imposible hacerlo fácil para que parezca traducción oficial Este sujeto : gratis y casi automatizado

[@coca7895]

aun le sigo buscando la explicacion de porque para una empresa es caro pero para alguien independiente es barato

[@santiagocabascango6514]

@@coca7895 eso mismo... Muchas veces no es solo caro del todo, sino sacar mas ganancia

[@francocastilloAR]

@@santiagocabascango6514 Exactamente. También es por la "mentalidad de empresa".

[@solounusuariomasdeinternet]

@@coca7895 Me imagino porque buscan perfeccionarlo... creo.

[@alwaysgrowing369]

I'm Brazilian and I'm in love with this resource, I found out about it in a video on how to make a blog from scratch, it was an American video, I was scared by the guy speaking my language! I jumped and went to the KZbin player and saw an additional feature of the KZbin player! Dubbing!! This is so perfect, you really seem to speak Brazilian Portuguese NATIVELY! WTF

[@fawkewe]

Honestly its kinda crazy youtube waited so long to add it. KZbin already automatically translates the title into your language. I remember all the way back in like early 2015 i kept clicking on Spanish Fnaf KZbinrs videos and being mad they were in Spanish, despite being no indication of it. People were also already making dub channels, such as El Smosh.

[@ilyhalight]

На самом деле это хорошо, что появился такой инструмент. Хоть русский перевод и не такой идеальный как в проприетарном решение от Яндекса в их браузере, но все равно это очень не плохо

[@user-pu5ht3fl6n]

Как раз таки решение от Яндекса, имеет абсолютно другой механизм перевода. Поэтому в данном случае, очень глупо их как либо сравнивать.

[@ilyhalight]

​@@user-pu5ht3fl6n понятное дело, что там используется полностью другой механизм перевода, но идея коммента была в том, чтобы сравнить итоговый вариант

[@Datosdelconejo]

En español hay veces en que acelera la voz y también hay veces donde la desacelera, pero la voz se escucha muy natural no parece uña Inteligencia artificial

[@nutsalhan8729]

Однако в яндексе голос просто перекрывает видео со всеми остальными звуками (ну вроде так было). А тут же можно иметь отдельный файл со всеми звуками, которые будут применены ко всем языкам.

[@Zovalista]

@@nutsalhan8729 la polla esto la verda y tormundo hablandon su idioma

[@muizzsiddique]

Watching the Japanese dub, you might want to allow some sentences to have a maximum slow speed (so short sentences don't get stretched over originally long sentences), or merge sentences that have a completely different duration than their original sentences to prevent rapid fire sentences being immediately followed by drunkenly slow sentences. For the second one, it is also needed when sentences are fragmented over multiple subtitles and the translator bugs out thinking they are competely separate sentences.

[@AccSwtch50]

Speaking Indonesian here, it has the same problem

[@jacoL8]

I'm an Arabic speaker, and honestly other than a few hitches, it works really well! I still needed to get used to the constant slowing down of the voice to match your speech, but I think that's a good compromise for how revolutionary this tech is for youtube videos!

[@YOUR_NARRATOR975]

This was automatically set to Russian and it was talking at 300mph 💀

[@nataniss_yt]

lol

[@lajawi.]

Even though translations are great, I hate the implementation of it, in the sense that there is no setting to enable or disable whether it automatically selects the language your app is set to. The same for video title translations.

[@namesurname4666]

i agree, the only way is to change the system language on android

[@qwqeqrqtqz]

Watching the german dub I have to say, I'm quite impressed with the result. I think someone hearing just the dub would be able to understand pretty much everything. However there are a lot of weird phrases. The vibe of the german dub is very different. I think most of the problems are because translating the trascript snippet by snippet, there is a lot of context lost and it shows in the translated text. Another thing is that the translator tends to use very formal speech, so it sounds more like a coporate presentation than you just explaining your program to a casual audience. Sometimes the AI voice is unusually slow/fast and it sound like a bit like those shady ads (think: "This self taught kid genius build a simple device to cut your energy consumption in half. Scientists hate him for it"). All in all still a huge leap in accessibility (even if I doubt most people know where/how to switch audio tracks).

[@TulgaD5]

Exactly, 100 % agree

[@shindousan]

It's really impressive that all of this can now be automated. I've tried several languages I'm familiar with, there's often a significant difference in speed between each sentence as expected, but the only way around this would be to change the translated text to reduce these differences (perhaps approximating a proportion of the original number of syllables , though not necessarily). Future work. But that alone meets many dubbing needs.

[@taki_val]

As an english speaker, I'm very surprised from the quality of the voice translation! It feels almost life-like!

[@JimmyC0]

As an Arab, Dubbed version in Arabic is way better than what I expected. Still needs some edits here and there but overall good enough to understand. Great job as always ❤

[@Ltwpe]

Yes

[@moelbossaty9563]

fr fr Reminds me of spacetoon dubs ☺️

[@Ali_Mener]

But also that it pronounced j in Arabic as g

[@RiasatSalminSami]

From making troll videos to actually programming things that improve KZbin experience. YT should just hire you at this point.

[@maredin-G]

처음 영상을 켰을 때 유투버가 한국어로 말하는 줄 알았다. 전문용어가 많은 영상이지만 몇몇 부분을 제외하면 거의 완벽하다고 본다. 놀라운 기능을 알려줘서 고맙습니다.

[@SalMightyOne]

I hate it that KZbin shows a title in your own language because it happens so often that the video itself isn't in your own language. KZbin should fix this.

[@cyrilio]

This is quite amazing. Good job Joe. Appreciate all you do around KZbin.

[@_g3nss]

As a Spanish speaker I could say that it is a great tool, it is very interesting and capable to do great stuffs, but it has their flaws. Listening this video in Spanish I could say that it feels unnatural. I think that almost everything is properly translated, but the problem with that is on English, the speech has their own rhythm, but on Spanish because of the words and structures, there is some phrases that are really slowed down and the next one is speed up, and you have this some sort of roller coaster that can easily confuse and annoys you. I think, on companies that dubbs content, this issue is resolved by re translating the phrases to make it ended with the same length as the original speech, e.g. Spanish 10:17 "Entonces de esta manera, todo esta cubierto" English 10:17 "So this way, everything is covered" The speed on English is almost double as the Spanish translation, so on the Spanish audio it sounds really fast, but to make it sound at the same speed as the English audio, we could say and use instead: Spanish 10:17 "Entonces todo esta cubierto" This is just an example, I am pretty sure that there are better ways to translate these as a context. Also, the language that you use on your original audio should be Neutral, because when you started to talk about prices, everything was "converted" to the target language, e.g. "$20" does not sounds like "$20 dollars", instead sounds like "$20 pesos" which is a lot cheaper of what you think, and could confuse people; maybe next time say literally "20 dollars". Also, for example, there are "North American Expressions" that cannot be translated and made it understandable by other languages and cultures. As I said at the beginning, is a great tool, but needs some tweaks (and also consider taking advices from the subtitles community, which most of them translate subtitles and knows better how to do it).

Si, exacto, claramente esto se puede solucionar con una revisión del subtitulo traducido usado para generar el audio, pero ya involucra intervención humana.

[@slevinshafel9395]

i think the same mimic of lips dont sinc with sound. and some time the voice go fast or to slow. That is because the sound have to much the second on srt subtitles. Vamos que no me gusta. Prefiero subtitulos bien hechos que abarcar de todo y que esten a medias.

@@slevinshafel9395 Yes, a way for the program to read the original audio and try to match vowels and silences would be cool to see, but again, dub studios do this manipulating the script to match it better. Maybe we will have ai that does this at some point.

[@GYTCommnts]

This will certainly be improved over time with A.I. However I think the options should be always available. I like watching media in it's original language, whatever is it, and with subtitles. So, I don't like that the app chooses automatically to play the dubbed version, even if I have my settings in general for not doing so. But as there is the option to change it, it's all Ok. I hope that option still be there in the future...

[@pabloalvaradolopez3941]

Aún así está bastante bien

[@Moonsdino]

I think this is soo cool, I think in the future aswell we could use this for audio description too to make this more accessible too, love this!!

[@XdGoldenTigerYT]

3:32 Yep, So apparently I live in Goose Island now 🤣

[@apurvaaryan7984]

Tested the Hindi one, would be very cool if youtube inplements that natively. Definitely would remove the language barrier significantly. Really appreciate your initiative.

[@freezinfire]

Yes, the translations are pretty accurate and will definitely help people connecting all over even more.

[@royalkumar795]

As hindi speaker, it's very good features it was easy for me to understand content But, it's very pure Hindi translation you have to turn on english subtitles to understand them

[@wazzaforever3704]

The Hindi translation is absolutely spot on but it is a bit too pure when compared to the Hindi that is commonly spoken. It does throw around a healthy amount of English words but a few more here and there would make it perfect "Hinglish"

[@BonJoviBeatlesLedZep]

Considering India is such a massive market for KZbin, I feel like this can be such a game changer

[@wardengamer374]

I'm Pakistani, and Hindi sounds similar to Urdu to me. Both languages have many words that are the same, but some can be different.

[@Marsman512]

I feel like this is useful for monolingual audiences, but I'm bilingual and prefer to watch videos in their original languages without being automatically translated. I have my KZbin set up in Japanese because it kept translating video titles from Japanese to English. It usually doesn't happen the other way around. I don't need the translations and it often ruins the original meaning of the title. When I clicked on this video it started playing in Japanese. No thank you. This video was originally recorded in English, I want to watch it in English. KZbin really needs to work on making options to disable certain translations for multilingual people, or at least provide an option to not translate at all.

[@lucadoesthings]

Most people aren't multilinugual and this is what it's for.

[@Marsman512]

​@@lucadoesthings Right, but KZbin's default behavior is to translate video titles and descriptions to your interface's language with no regard for whether or not you can speak the original language of the video in question. There's no way to disable this behavior, which is fine for most people, but not for me, and I'd imagine most other multilinguals. Remember, KZbin serves hundreds of millions of people, and one percent of 100 million is still a million. Even if only one percent of KZbin's users are multilingual, that's still millions of users being done a disservice. It's be a shame to see the audio go the same route with no way to disable or change this behavior.

[@Liggliluff]

@@lucadoesthings Most people _are_ multilingual. Very common that people speak two or three languages.

[@mrcryptozoic817]

@@Liggliluff Prove it. I don't think so. And knowing 20 words in another language doesn't make one multilingual. At least in the USA.

[@chlorobyte_projects]

​@@mrcryptozoic817 Murrica is not the entire world. There are 5 whole other continents filled with people who learn English (or some other language, German is common here too) as their secondary language.

[@babanavava]

日本人ですが、話し方は驚くほど自然ですが、字幕の表示されている長さに合わせて話し方がゆっくりになったり早くなったりするのが面白いと思いました。

[@elcanaldelucas6187]

As an argentinian, this is very a very good and revolutionizing tools, specially since it's just begining, but it has some flaws and i want to point out a major one for me: For some reason, the program that translated the subtitles to other languajes took every set of subtitles as a different sentence, which if the other languajes has different rules to english it will probably screw things up. For example(screw ups in spanish only): -In english, adjectives go before nouns. While in spanish, adjectives go after nouns. This can cause that words are put out of order, and worse, if the adjective in english is also a noun. For example, the sentence "i like strawberry flavoured gum", which should be translated as "i like gum with flavour of strawberry", is translated as "i like the strawberry. Gum with flavour" -In english, objects are not gendered. In spanish however, they are(they shouldn't be and their gender is arbitrary, but that doesn't matter), if the pronoun and the noun are in different subtitle sets, the program may use a different pronoun than the one it should. -In english, verbs are usually conjugated by putting another word before it. If the word before the verb and the verb and in different sets, the verb is conjugated incorrectly and the sentence doesn't make sense.

[@77dreimaldie0]

I immediately checked the German dub (my native language), even before the involvement of AI was revealed. It was instantly apparent from the abysmal grammar. But fair enough, it's better than nothing I should mention: as a computational linguist o adore this project from a technical standpoint! Never think the effort isn't appreciated

[@DigitalJedi]

I had the same feelings on the German dub. Terrible grammar in lots of places, but better than nothing. I'm not sure if I prefer it over just watching in English tbh.

[@lalez88]

@@DigitalJedi I was so damn confused when the bad german started... I switched back to English so fast.

[@paulustrucenus]

Bruh that dub feature is SO AWESOME

[@emahu]

Es un excelente desarrollo! La voz y textos en español son casi perfectos, ¡felicidades! Sería estupendo si esta aplicación pudiera aplicarse cuando somos sólo usuarios y no creadores de contenidos, ¡sería un hitazo! Me apunto para la beta! Por lo pronto, suscrita y lo mando a mis creadores favoritos!

[@Jdbye]

KZbin's time stretching has no degradation in quality? Have you tried watching anything with music in it at non-default speed? It's *really* noticeable, the audio gets "choppy" in a way. As long as the video is just talking, I often don't notice anything wrong, but a lot of videos do contain music.

[@AL_O0]

Speaking two languages i can say the dubbed version works fine, it's got the issues you'd expect from machine translation and speech synthesis but i can understand what's being said, so good job!

[@Leonhart_93]

Are the subtitles machine translated? I was under the impression that they are actual translations, otherwise there isn't much point as you can auto-translate youtube subtitles anyway.

[@FlameRat_YehLon]

@@Leonhart_93 KZbin auto translation is a one-size-fit-all solution, so it still make sense to run machine translation manually. That's said, in its current state the difference in quality probably isn't that good yet, especially that technical words are just as easy to fail.

[@liltousin]

Такая функция уже давно есть в Яндекс браузере. Там если ты смотришь видео на английском (даже прямую трансляцию) оно будет переводится на русский голосом ассистента Алисы (аналог Сири в России)

[@Maxi460]

Но иногда на английском могут слова некоторые зажеваться, из-за ветра или ещё каких-либо посторонних шумов, которые Яндекс уже не переведёт

[@MrBrigadierArchived]

+Яндекс доступен не всем и не везде. И браузер, и весь остальной сервис.

[@franknention6791]

@Mister Chuvacok Алекса от Amazon

[@Orakcool]

@@MrBrigadierArchived Ютуб не доступен в Китае(как минимум), а гуглосервисы ограниены.

[@MrBrigadierArchived]

@@Orakcool учитывая политический характер, то проще говорить что вообще доступно в Китае, чем то, что там запрещено)

[@alwaysgrowing369]

Dude, this is revolutionary, I'm Brazilian and I think you were actually speaking PT-BR! Find someone on the KZbin or Microsoft team and show them this video!!

[@peiseoagency]

Huh. This video taught me that it's very difficult to tell if a voice is real or AI when it's spoken in a language I do not understand.

[@XPStartupSound]

Would be interesting if LTT would be able to integrate this with their new ai voice Spanish workflow

[@HighSoldierTF2Official]

I cannot express how cool i think this is. I would never have imagined that dubs like this would be possible!

[@Macusercom]

For German: - A few grammar/translation mistakes - Audio speed changes are noticeable - It switches between the impolite "du" and polite "Sie" for addressing the audience - At 15:00 there is some weird overlapping of the dubs But: this is truly amazing work. The fact that these are the only things I've noticed given that this is all an artificial translation is mind-blowing. I'm so stunned right now 🤯

[@XPCuts]

the fact that there's now going to be comments in many languages now in this single video is amazing

[@Koozmich]

Cool! Btw, Yandex had already made fully dubbed multi-voice KZbin video translations and now finishing work under the Streaming dubbing )

[@tptcx]

its so weird remembering you as the guy who made troll tech tutorials like 7 years ago and seeing you do wonders for the platform now. not complaining tho, you're amazing

[@user-ri3bt8ox4d]

We have a similar function in Yandex browser in Russia. However, she only translates into Russian. It is built into the browser and translates videos from almost all hosting sites.

[@BenTechYT]

It sounds super easy, but it really took me long to realize that at the end, you have to pay a lot to be able to make this voice over with either google cloud or azure, and then if you have to pay, maybe it's better to pay a voiceover actor it will be cheaper, easyer, and more natural. Maybe it's me that i can't put this python to work, but at the end everyone is stuck at a paywall.

[@dedse3]

It was one of the best experiences I've ever had watching a video on KZbin! I played around and changed the audio track a bunch of times, turning on and off the captions, it was amazing seeing how it works, nice work!

[@pyp2205]

ThioJoe: "No app with these features? Fine I'll do it myself!"

[@Atedos]

Instead there actually is

[@Gaak]

Error with prices 11:07. In english the symbol "$" means USD, but audio track in spanish says "pesos", maybe mexican "pesos". In this case, $ is "dólares americanos" (in spanish).

[@flaviorinaldi7450]

I tried in italian and It is really great, but there may be a few things that could be improved. 1. Translating by “bits” of subtitles isn’t ideal, as sentences are often cut off, making the translator decontextualize them and see as two different phrases to translate (translators work at their best when having context, i.e. at least one whole sentence). Here is an example, coming from an italian sentence: “Ho visto il cane di Matthew ed è molto carino” (“I have seen Matthew’s dog and it’s very cute”). Let’s say that the sentence is split into “Ho visto il cane” and “di Matthew ed é molto carino”, the translator will output the following “I have seen the dog. By Matthew and it’s very cute”. Of course one could easily understand it, but given that the text-to-speech will also add a pause in between (again, it sees it at two sentences) it can get distracting at times. 2. In order to save a little bit on the second tts API pass, one could use a documentary style dubbing. Meaning that one could still hear a little the original sound as a background, removing the need to stretch the audio clips that are slow (gaps will be filled with low-volume original sound). I completely understand that, especially for the first point, it would mean more man work in order to achieve it so there’s the question of whether the hustle is worth. While for the second, more than anything else it’s a style thing. Hopefully i didn’t come out as cocky in this😭🙏

[@erich_ika]

It was really cool watching your video in Portuguese! It felt like low budget dubs from TV shows that do not care about lip sync like documentaries from Discovery Channel for example, with the added detail that the narrator speaks suddenly fast or slowly to keep up with the original video. Personally, I would prefer the awkward pause to the guy going Turtle Mode every once in a while, but what you did is really cool! You make an amazing software developer and I can't wait to share your future videos to my Brazilian friends. Cheers!

[@Liggliluff]

Yes, the software should run one pass, and look at how much each clip needs to be sped up and slowed down. Then it should try to even out the differences, so you don't have a fast clip between two slow ones.

[@JetSetDman]

finally i can watch thiojoe subbed in japanese the way god intended

[@cuanlm]

More like thio jo jo 😎😎🥶

[@selgeaus]

You mean "Dub" ; "sub" is subtitle

[@Juanguar]

HOLY SHIT THIS IS AMAZING

[@ag992009]

I am watching in Japanese with English subtitles. My dose of anime for the day 😂😂😂

[@UltraCenterHQ]

ThioJoe is the best anime

[@geraldthorburn1123]

As a modest linguist, Thio, I found this utterly fascinating - and brilliant! In my late 70's, oh to be in my teens again studying languages, when in those far-off days so much of it was a book-driven drudgery! I have reached the conclusion that today's youngsters now have no excuses for failing an exam, given the astonishingly wide range of 'aids' available.

[@MrDonee]

how do i turn this feature off? its already fing annoying

[@Kibodi]

German has a little problem with pronouns. In the second person singular we have "Du" and "Sie". "Sie" (not to be confused with "sie" without a capital s, the feminine third person singular) is the formal form, used for example in official letters. "You," on the other hand, is the more personal form, used in conversations with friends and family, for example. Google Translate uses "you" because it translates formally, but it felt a little strange to be called "you" in the video. "You" would be a bit better, but I don't think it's the best solution either, and I don't know why I feel that way. Translating other languages into German is apparently a bit difficult for Google Translate, or Google Translate is just overall bad, idk. Still, it was very interesting to see how far we've come with our technology when it comes to translating audio tracks, and that a script by just one person can do the whole thing relatively well.

[@MudakTheMultiplier]

I wish more people made use of the secondary audio tracks, there are some neat things you could do. For example, there are a lot of ASMR videos that specify a specific pronoun for the viewer, or have background music that not everyone might like. If a creator wants to provide options currently they can only upload a second version of the video, which is likely to do bad things to their engagement ratings since the viewership would be split up. Having separate audio tracks could really help channels willing to put in that extra effort.

[@trashpanda4]

"that specify a specific pronoun of the viewer, or have background music that not everyone might like" uh some feature for that already exists right here on youtube, it is called 'exit the video and watch other one which u like' and for the pronoun thing, first does it matter that a random youtuber who is barely aware of your existence doesn't get your pronoun right? and second most youtuber refer their audience as YOU or some cringy name what do you wana fix on that? different people have different preference and nowadays different pronouns too, its hard to include all people in a ~10 min video.

[@MudakTheMultiplier]

@@trashpanda4 I don't care whether a KZbinr *does* it or not. My point is that the ones who do are *punished* for it, and that sucks. I'm not sure if you watch ASMR or not, but part of the process involves being relaxed and comfortable and some people are not comfortable being referred to by pronouns they don't like (or just don't like music with their sounds). Typically they'll just skip the vid, but sometimes there are creators who are awesome and literally make different versions with different pronouns (or with and without music, ect.) and then publish both even though it's going to do bad things to their overall metrics because very few people will watch both versions. I'm not suggesting we force creators to do this, I'm saying that there's already people who do, and this would be a system that would allow their hard work to not actively harm them.

[@trashpanda4]

@@MudakTheMultiplier KZbin algorithm favours quantity over quality and if its a well established creator then the *getting punished* part is making the exact same video with slight variations. all things besides, idk what asmr people do over there but isn't there only one 2nd person pronoun which is 'you' (in English) unless its a group conversation simulator asmr how does pronouns enter the game?

[@MudakTheMultiplier]

@@trashpanda4 you greatly overestimate the viewership of most ASMRtists. I would say the bulk of the mid to large scale ASMRtists get 30-60k views on most videos, which is absolutely in range where getting one video that doesn't do as well will drop overall metrics. And several of the people I've seen do this actually only have one public facing video. And then private the other one and link to it in the description to combat this, which will result in some nontrivial fraction of your viewers clicking through to a separate video which will not contribute to algorithm weight. there are other gender defining terms (boyfriend/girlfriend, brother/sister, and so on) and it's not uncommon to refer to the listener in third person if only to cut down on saying "you" 50 times in one script. Unfortunately the alternatives are usually a bit clunky. Often a creator will pick the pronouns that match the bulk of their viewers, which then means that anyone who feels uncomfortable just loses a channel that they might love otherwise. That's why this feature is so cool because it's literally designed for accessibility, even if the original designers in 1990 would have never expected it to work like this.

[@trashpanda4]

@@MudakTheMultiplier oh that makes sense, i personally don't not watch asmr content so i have no idea what happens there

[@m1kmain]

The dedication you put towards this is outstanding. KZbin should just hire you already

[@w8irure7iwjebdi2hhri4]

Very helpful feature, this feature should be available for every video. Will expand content library for billions of people.

[@aarspar]

I'm not gonna lie, I'm very surprised and delighted that you chose to do Indonesian audio. It's very rare to come across non-Indonesian content creators who even has Indonesian subtitles for their content, let alone dubbing. Many kudos to you 👍👍 Also, the translations is very good, just that it's very rigid. Indonesian uses the passive voice much more often than English, so some sentences seem rather superfluous. Still, it's very good for AI!👍

[@arrotino9501]

You should have an option to manually edit the written translation before creating the voice, to catch any translation errors made by the program

[@Pystro]

One problem that I think I noticed: The translation tool seems to translate each subtitle line separately. And this predictably went pear-shaped in the EN-DE translation at 1:37, where a sentence is split between two lines. The translation of "[...], then you can tell your [-] favorite creators about it." did NOT make any sense in German. I suspect that the translation software simply could not handle the fragments, because they are already effectively nonsensical in the English subtitle lines. To mitigate that, you'd have to REALLY pay attention to split your subtitles into lines at natural break points (periods, commas, ...). And this can easily become infeasible if the sentences (or sentence fragments) become too long for one subtitle line. To combat that, I think you may need to define markup tags for your app that you can place at such line-splits in the subtitle file, to tell the app where the translation software should consider 2 or more lines as one unit. And even though this will produce translated strings that are too long to be a single subtitle line, I don't see a reason why for subtitle-displaying you shouldn't be able to just split them into fragments with equal number of characters (or better syllables) and display those for an equal amount of time. You'd just have to live with the small difference to the point in time when the text-to-speech voice reaches those line split points. But even WITH those small blemish marks of nonsensicallity on the AI voice: I would bet that for someone with only rudimentary English skills, it would be *tremendously* more convenient and pleasant to watch a video that is voiced in their first language and only having to look at the original (English) subtitles wherever the App messed up. The remaining issues are just small problems with the accuracy of the translation software and the text-to-speech AI.

[@lucky_lol]

This is awesome!👏👏 As an Indian, I would recommend not to use Hindi but use Hindi (Hindi+English) I know it feels absurd, but in India nobody talks in Pure Hindi as in the Translated but in Hinglish where there is use of Many English words.

[@ManInDark]

The two pass system is really apparent in the german version, some parts are quite quick and some are insanely slow. Additionally there are some grammatical faults which aren't hard to spot, and it is in general very weird.

[@selgeaus]

Listening to the Japanese version - I am surprised how accurate it is, although the way the speed changes constantly reminds me of how you change the rpm on a record player.

[@Ricardocomics14]

Esto es la HOSTIA, necesitamos audio tracks en los videos y ya llevan como 2 años sin ser implementados

[@karl3970]

Yandex has already implemented this feature and added it to its browser (and I translated this text from Russian into English using Yandex translator).

[@liab-qc5sk]

I tried Arabic. In reality, spelling the word and letters and also punctuation are so weird

[@Ous_Sama]

also the dude speaking is the same voice in the national geographic documentaries lol

[@naeemulhoque1777]

This is definitely one of my favorite channels on KZbin 🤩

[@ArrivedBog593]

11:08 In the Spanish audio the word "dollars" was translated as "pesos" (when it should be "dólares"), and in addition to the fact that there are many different pesos that have different values ​​in relation to the dollar such as the "Mexican peso", the " Argentine peso", the "Colombian peso", etc.

[@Weeping-Angel]

Bruh this is kinda trippy. It actually sounds like a real person talking and not like a bot

[@rafaelnm]

Eu ao contrário do que vcs estão dizendo, achei fascinante isso, e ajuda muito. A voz está bem natural, logicamente que não será igual uma fala natural, masssss é só o começo, e para um começo está muito bom de verdade. Isso me deixa muito empolgado pois facilita ver qlqr áudio. E ao contrário que alguns estão dizendo, essa ferramenta possui função para facilitar o entendimento de um vídeo e não para substituir dubladores de filmes ou afins. O programa pensa em transmitir a informação do áudio em qualquer idioma e não passar emoções pela voz igual um dublador faz.

[@xnicklc]

CARAMBA QUE INCRÍVEL!!!!!

[@Readyplayer11]

As an English speaker the speaking was really natural. I can't believethis was translated

[@SirSpudly]

This may be the most game changing program for KZbin