How We DRASTICALLY Improved AI Vocals

[@per3974]

This channel has become one of my favorite things to watch on KZbin over the last two months since I discovered it.

[@michaelspencer-arscott]

Ditto!

[@dirg3music]

Same, his deep dives are truly second to none.

[@karstdeviz3858]

Same

[@AscensionismOnOsu]

same here

[@mpusch88]

Extremely solid & consistent content, some of KZbin's best

[@JamesRamboPearce]

The music industry needs more Benn Jordans at the top, with a moral compass and a thorough understanding of how things work for both the artists and businesses - I look forward to seeing how the voice swap platform grows.

[@urbnctrl]

its not going to look good either way. for creatives.

[@cobuspotgieter]

Love your content, most inspiring channel I've found lately. Keep going, Benn 🤙🏼

[@NateTheMeh]

Some of the best content on the whole platform, Lucy has a beautiful singing voice!

[@DJFresh]

Beautiful

[@popsarocker]

Benn Jordan: doing the internet right. Favorite channel by far, and thank you for fighting the good fight with such an important technology

[@CaptainJack2048]

I have been fascinated with work done in AI since the early 1970's (yeah, I'm old) and I got into a technology career very much based on that interest. I've been following a lot of the interest, concerns, and comment on AI in the last couple of years. The amount of misinformation and speculation-as-fact out there is staggering, and I really appreciate the way you give a thoughtful, measured, and reasonably objective view of it all. I only play music as a hobby for myself and my family, so I'm not affected financially by changes in the industry. I am staunchly egalitarian, and I believe strongly that creates have the right to be paid for what they create. Distributors and other folks should certainly be paid for what they do, but the oppressive nature of people with money trying to make more money off the backs of people who do the real work has always bothered me. I hope that in the process of enhanced software tools to create and develop musical ideas, there also arises an economic model where fairness and proper recompense can be placed where it belongs.

[@3nertia]

That will never happen under capitalism though 😢

[@CaptainJack2048]

@@3nertia​ Possibly, but I hold out hope. I think that there are two things that have to happen. The young, hopeful creators have to be educated to understand that their path to success is in their hands, and that buying their way in (watch out for sales pitch with an exclamation point in the title!) being gifted their way in ("just sign here, don't worry about the fine print") is not a viable option. The other is that any artificial scarcity has to be diluted, to the point where people who only want more money have to look elsewhere (which does only shift the problem, but maybe the worst of the worst can be spread out a bit to lower the average impact). I see both of those things happening now in places (such as this channel) and I keep hoping to see more of it. Capitalism is likely to always be with us, and of the two I'd rather have that than violence as a tool of oppression. I'm mostly hopeful that those of us that want to create can use some of the tools available to us to level the playing field, at least to some extent.

[@3nertia]

​@@CaptainJack2048 Ah, the irony :( Capitalism encourages our worst most predatory traits - violence is its bread and butter and why the system has a monopoly on it heh What we need is socialism, true socialism. Not some dictator/capitalist claiming to be a "socialist" because that's the contemporary buzzword that convinces the idiots that they (the dictator) actually give a shit about anyone other than themselves - selfishness, another trait encouraged by capitalism, aside from exploitation which is inherently violent anyway heh

[@CaptainJack2048]

@@3nertia I don't disagree with that, but I know from history that change has to come in phases, or it won't stick. I certainly don't know how we get to a point where we all would rather take care of each other versus a few people clawing their way over everyone else, but I think it has to be a combination of education and empowerment, and possibly pitting the miscreants against each other.

[@3nertia]

@@CaptainJack2048 I actually have some ideas for that heh

[@peachtime]

BUT I WANNA WATCH IT NOW!

[@billsybainbridge3362]

Benn - A longstanding problem will still need to be solved, whether by Content ID algorithm or some other means (advanced watermarking, perhaps): proper and correct attribution of sources. Certain efforts such as The Creative Passport or Sound Credit may have made some progress, but this will remain an issue for artists as well as AI developers in this space (including VoiceSwap AI).

[@medalf]

I don't know how I missed it but you are the flashbulb ! Your music has been in my mind since I was a young adult and I'm really happy to have discovered your content twice in a little over ten years. Thank you.

[@joelcarloaymat]

AI dog vocals is what this technology was built for

[@Thunderkiss655]

I only discovered your channel a few months ago when I wanted to start making music again after a 15 year break. I absolutely love watching your videos and how passionate you are. Can't wait for more quality content :)

[@PlonkapplePrequel]

Really love the foundation set for this new platform. As a producer who doesn't delve a whole lot into vocal performances, I'd like to use Voice-Swap to lean more into either unique folk voices that don't belong to big artists already making music, or fictional voices with cool timbres and character to play with. Do you know if more of these kinds of voices will be added to Voice-Swap in the future?

[@bchspace]

For what its worth, "Crepuscular", "miles and miles", "blurry figures, far away" , "Skeletons" and "dead on the fourth of july" are some of my favorite tracks of yours. The (I assume, your) vocals really make it personal and help transfer the emotion.

[@avgriffin]

reminds me of that ''jazz doooos' sound , it's just so human its like religious almost. or those barber shop quartets, or boys II Men, all great natural vocals just cant be beaten to this day!!! 🤣

[@t3hgir]

I hate leaving messages on these things Is anyone listening? I've been by myself for days Someone please say something

[@bchspace]

@@t3hgir Something

[@jeffbrownstain]

Also most of the HoloLive vtubers have really good AI clones made of their singing voices. Plenty of covers to find from all of them as well as Neuro-sama.

[@hukasu]

The Japanese have many AI based voice synthesis applications tailored for singing voice generation, and the more recent ones are use Deep Learning. The official VoicePacks all use paid actors for the recording and some are even installed as VSTi. Since they are tailored for Japanese, maybe it will be hard for you to look at them by yourself though. Some examples are NEUTRINO, Synthesizer V, Voidol, Seiren Voice.

[@ImNotQualifiedToSayThisBut]

Synthesizer V has a vast English speaking community though

[@hukasu]

@@ImNotQualifiedToSayThisBut they all probably have some English support, I just don't know to what extent, I'm not an user of any of them

[@ImNotQualifiedToSayThisBut]

@@hukasu For some reason my comment got deleted, so I'll repeat myself: SynthV has full English support and English voicebanks. Creuzer has very good examples of it (using Eleanor Forte AI and NineZero for example).

[@yuvrajsingh-gm6zk]

The melody Man, just touch the bottom of my heart!

[@ELopez]

I recently listened to the debut album by headache - “The head hurts but the heart knows the truth” which is a spoken word album produced by Vegyn. Turns out the entire album was narrated by an AI, and I honestly wasn’t able to tell the first few times I listened

[@digicat247]

This was such a great practical video. Holly Herndon has been in front of this for quite some time. A further discussion with her on how AI will enable artists to do more and express more would be a fantastic follow up to this.

[@danlindy9670]

The idea of pairing professional production/training for AI voice models with artist rights/compensation is brilliant. I worry that these models could be copied and tweaked to skirt liability; but overall, the business model seems solid. Plus... your AI vocal demonstrations were freaking awesome.

[@Kosro]

Holy smokes, the results are pretty amazing!

[@BACKLIGHT]

really appreciate your no-bullshit approach to video presentation. i can always trust your videos to be concise and easy to follow

[@abzulooks6012]

I speculate that the same may happen with AI as with vocaloid. Voice models started appearing in the early 2000's but it took off when the right voice with the right corresponding image (Hatsune Miku) came along 16 years ago. And then (in Japan at least) the scene took off, and OK possibly peaked in 2015 or so but is still very much a scene, and still breaks artists (like Ayase from YOASOBI). After 16 years of Miku, maybe we can make some predictions/ observations: 1) a handful of voice models will become everyone's favourite and everyone will use them 2) Those producers who really get in and tweak the performances will get the best results (just like you showed) 3) In some respects it levels the playing field: everyone from beginners to seasoned musicians can work with the same vocalist, you don't need A-list connections. So posting Miku songs to social media is Japan is one way to try and break as a producer/ songwriter. (Again.... Ayase from YOASOBI but many, many others) and lastly, and most importantly 4) Producers/ songwriters will try and parlay their fame from working with vocaloid/AI into being singers themselves or working with human vocalists (Kenshi Yonezu for the former, Ayase from YOASOBI and Ryo from Supercell for the latter). So it will be interesting to see how much Vocaloid acts as a precedent for AI. Oh one more thing: Miku's voice was provided by a voice actress, Saki Fujita. Did she get royalties? No idea. I'd suspect not.

[@cybergalacticnova]

Vocaloid AI is already a thing! The newest version, Vocaloid6 is an AI based vocal synthesizer. It has vocalochanger, which allows people to insert an acapella or existing wav and an AI bank will sing the wav back but in their own voice. But also it did set a precedent. SynthesizerV AI, CeVIO AI, Voisona, Ace Studio, etc. they are AI-based vsynths that follow the same gig as Vocaloid. SynthesizerV being the very best in quality.

[@abzulooks6012]

@@cybergalacticnova Yes I was intrigued by that. I'm currently running Piapro with Miku V4X and was tempted to try out Vocaloid6. And SynthesiserV looks very interesting too.

[@groundcontrol6876]

Man, this was amazing. That blending of singing with musical tones etc. to make it come out amazing is just so cool.

[@johnoestmannmusic]

I love your content Benn, and you’ve been a huge influence on my understanding of music business, but I did feel a little taken back only finding out your involvement with a Voice AI venture only towards the end of the video, given that the first half was critiquing, essentially, your competitors. Regardless of validity of your argument, it does create a commercial bias when you are critiquing other Voice AI platforms. While I look forward to your experienced guidance for your friend’s platform, it might be worth exploring how to frame these videos now that you’re no longer in a “sideline observer” position. Regardless, love your work, keep it up!

[@freds2052]

I’m an AI-skeptical musician and this actually cleared up a lot of the questions I was having about the current state of affairs honestly. Thanks as always for your honesty!

[@kasai1575]

6:26 I would recommend reading the actual decision that was made here yourself, as a lot of news outlets are twisting the judge's words for a headline. The work in question was entirely generated by an AI, not just *with* an AI. The process was random with no human involvement, not even a prompt. The case says whether prompt-generated images or other creative works that have a degree of human involvement should be copyrighted is still on ongoing legal and ethical question. Since something like an AI generated image requires some degree of human creativity, it is still (for now) eligible for copyright.

[@IanGrams]

This is super interesting and they sure couldn't have picked a better person to help figure out how to do this ethically. I hope this goes well and ends up being a viable income stream for vocalists.

[@AironExTv]

Woho. Thanks for bringing this to the forefront in such an easy to grasp manner. The demos are wonderfull. It almost parallels the amazement I felt when I tuned a vocal over 20 years ago with Melodyne 1 for the first time. I'm a post production re-recording mixer now, but I can appreciate the value of the method and more importantly the underlying data set, i.e. the performer it's based on. I hope this shakes out well for performers and producers alike.

[@elliotmarks06]

Benn, you just seem like the coolest guy! Not only do you make awesome music-based content, but it's extremely entertaining and educational in the process. Always fun to hear your opinions on stuff, especially a controversial and complex topic like this one.

[@coversine479]

There are also pre-designed AI singer plugins you can now buy from Japanese companies like Synthesizer V Studio. Currently Japanese sounds more convincing than English, but voices like Ryo sound incredibly similar to a human now

[@demp11]

There not ai from what I know they are purely synthesized

[@Nahnono]

I was checking out how far vocaloids had come a year or two ago and the synthV stuff was crazy, pretty much indistinguishable from a real person. Especially ones like Stardust, even when it's used for english. I saw a lot of people were intentionally trying to artifically make it sound less real to get the old vocaloid vibe back.

[@valley-artifact]

@@demp11 it says on their website that there is a neural network involved alongside the more traditional DSP stuff

[@themidimachine]

@@demp11 they are trained on real voices using Neural Networks. While they are (at least mostly) synthesized, they have the character of the singer they are modelled on (including accents). I've fooled many musicians using Synth-V. Getting great results can take a bit of work and they're far from perfect, but they are miles ahead of where they were even just a few years ago, and very usable.

[@hearhear_jb]

I bought Synth V some months ago and I hate to say it but, much to my surprise, I've fooled every musician friend I've played my latest working song demos for, including several top shelf Nashville backup vocalists. I had been looking around for something like it for prototyping vocals based on melodies and lyrics I had written and played on piano. I was on a forum and mentioned I was looking at Vocaloid when a member chimed in saying I really needed to check out Synthesizer V, it blows the doors off Vocaloid. They pointed to a KZbin video someone made of a Paramore cover with Synth V over the original backing tracks. I recall thinking, well now, this is much farther along than I had imagined! If you want to check it out, search KZbin for Paramore Misery Business Synthesizer V. And then compare it with the original version. It has been somewhat of a steep learning curve to turn out really good results, but the fact is the first time I imported a MIDI file into it and plugged in my lyrics, it did a perfectly acceptable job for the most part. I was suitably impressed. That said, I intend to continue hiring pros to cut final tracks, but even they admit the Synth V roughs I've been sending really help them get to what I'm looking for in a performance a whole lot quicker. FYI, the voice models are based on actual singers (whom they name) that the company has hired to work with in training their neutral network, so no ethical issues here. There aren't that many models to choose from in English yet but the two I liked enough to purchase are Kevin and Solaria (the one used on the Paramore cover). Each was in the $80-$90 dollar range. Anyways, I apologize for the long read but I hope someone finds this helpful, and thanks to Benn for addressing this topic.

[@goochmusic668]

I think Benn is the most intelligent and decent person I pay attention to on a regular basis. He makes me want to be a harder worker and a better person. Thanks, Benn!! (Suggestions for other music-related youtubers who are particularly intelligent and thoughtful would be appreciated!)

[@dexterman6361]

This channel is amazing, so much info, so well described. Maybe in a future video, I wish to see the breakdown of copyright law and why taylor had to re-record her songs. Also a video on how you research the topics you cover on your videos. I was blown away with the amount of research you do - papers, articles, studies, references. It would definitely help people like me who struggle with focus and get overwhelmed when researching a topic. Thank you!

[@epiccipe2941]

Benn's out here just casually dropping names like DJ fresh... God damn. Great and interesting video as always! Keep up the good work :)

[@Bthelick]

Haha great to see Jamie's name again, A few of my earlier tracks were released with Jamie on Vocals. ❤❤

[@cmd_f5]

This is pretty exciting indeed. I remember Google's 2016 wavenet tech and people were going crazy with it. I sort of evolved my opinion as AI/machine learning developed. I was initially against the technology. The results in audio applications was poor, both in spoken word and definitely sung content. Also I was tired of seeing AI AI everywhere. Buzzword and crappy memes galore. However, it's just a tool like anything else. It can be used for harm or for good like anything else. This new Voice Swap AI could have some real perks for experimental projects or for multilingual applications. Keep up the great work Benn.

[@ErikPelyukhno]

Ben Jordan, you’re such an incredible guy. I never could have anticipated one person to combine massive music production experience, empathy for the embetterment of artist’s lives, bleeding edge technology and corporate world experience to provide new tools to artists. You’re one of a kind

[@juliekovich-recordingartist]

You asked for thoughts--- I did a search looking for AI back-up vocals quite awhile ago. I had seen some online software where you could do this in the past (and use older choir sample based versions of similar), but the voices all sounded the same pop style and weren't what I was looking for. A few months pass, I look again, and I was SHOCKED when I came across the options to use famous voices. To me it seems first of all cheezy to want to use some famous voice in my song where they weren't actually doing the part, but more importantly, it seems invasive to the person who's voice was copied without their consent. It's even worse than stealing a sample. It is something that needs to be dealt with, not just for protecting famous people, (I'm not always very fond of the bunch:-) but just as a moral cultural viewpoint that will develop over time. Your new venture on the other hand sounds great! I know I would definitely make use of it.

[@JordonBeal]

Really glad I stumbled upon your channel, Benn. You've reignited the audio and production nerd in me!

[@kryptyk7712]

Good luck with the project. We're only just getting started with AI, and a brilliant mind like yours is already forging a path.

[@tommorton7211]

I've been using an AI model recently for high-octave harmonies and it is much better than just using little alter boy or something like it as I did before. That said you have to give it a good vocal take that has probably been comped and tuned. While better models will produce better results, I think the example shown with a poor unedited take would have been poor in any model and the better results you achieved were based as much on better input as on a better model.

[@SineEyed]

True..

[@NightOwlAmbient]

I absolutely agree with this approach. As a native Nashvillian with friends who depended on royalties to raise their families, what Spotify is doing is abominable. And that's after decades of the record companies doing even worse. Why should an artist have to work as a bartender or waiter, as so many of my friends have done, in order to be able pay rent? And why would an artist put their work on Spotify for essentially royalty free? I only buy music on Bandcamp now because the artist gets 80-90% of what I pay for their music. What you are doing sounds like the first ethical business model for AI music I have ever seen, so I subscribed to your channel. I'm about to click on the URL when I hit send. I really hope you succeed. Keep us posted on what we can do to help that along.

[@SlimThrull]

7:32 Better than I can do, dude. 9:15 Holy ****. I've worked with computers for 30 plus years. But I've never done any sound work. I realized that this COULD be done, but damn seeing it go from pretty decent to downright epic over two minutes is a thing of beauty.

[@mikewebsdale4777]

I feel like a better person every time I watch one of your videos. You consider everything so carefully and logically with fun and interest stacked in there. I appreciate your channel, your thoughts and your music. Thank you.

[@RichardAmesMusic]

One thing that will dictate the way it goes is whether musicians turn over AI-based rights management to something like a PRO. As someone who has watched ASCAP give away a lot of my music for next-to-nothing and, in many cases, for free, I want other options. It's basically impossible to get around the PROs, at least in the US, but they're entirely unnecessary in the modern world. Engineers generate IP in the form of patents and collect royalties just like musicians but there's no PRO doing it for them (and they seem to do just fine). I want that system because it gives me the power to dictate the terms of the license agreements. Alternatively, you can pass on the management of the rights to another legal entity should you choose to do so. Regardless, there needs to be a way to do it that doesn't monopolize control in the hands of a few PROs.

[@denarisnoctem4562]

Everything you do is so far beyond me, but I love how you explain everything clearly enough I can keep up (somewhat).

[@CheerSounds]

Excellent intellectual discussion about the tech, legal aspects, and innovative production workflows. What a breath of fresh air. Thanks for the conversation.

[@fatmatrow]

I try to tell people that, for music, AI is great for things that are impossible for someone to do, like spectral noise reduction, removing instruments from a mixed track with high accuracy, cleaning up and isolating things from older music, essentially editing stuff that wouldn’t be possible otherwise.

[@kaitlyn__L]

Indeed those three things were what was done for the Now And Then demo tape to finally release it

[@iskariot3180]

just a thanks for all the work you put into this, engagement in the comments for the algorithm!

[@Melvin420x12]

Bro I love these nerding sessions with you experimenting and just figuring things out recognizing framer and then digging in devtools to find they indeed use it. You should stream some time! Glad to have found you back man, I enjoy your content a lot, I recognize so much of how my own brain works in your video. haha

[@rkaid7]

10:56 - that vocal is fire

[@eleni-84]

Lucy needs her own channel, make it happen! 😍

[@dreamstaticsounds]

Hear hear!

[@Silverfox.J]

As Voiceover artist, this is pretty wild stuff. I get told that my voice sounds a little AI a lot and the amount of AI voiceover on the platform that I work on (Fiverr) is getting frightening. Of course I used Siri voice to text to write this message. Lol.

[@crnkmnky]

Yeah, I've been wondering about that. That beard guy is still advertising his voiceover course while the industry is about to get the snow globe treatment. Are the "thought leaders" offering any glimmer of hope for newbies and veterans?

[@sinane.y]

Holy cow. You're starting a revolution.

[@WhizPill]

Another fantastic, down to earth analysis of the latest not just in music but the underlying engine that keeps the industry machine moving towards new directions.

[@jameshasseriousedoubtsabou560]

This is absolutely amazing. I hope to god that this artist first approaches becomes the norm for AI tools across the creative industry. Thankyou so much Ben

[@masoudmaani]

beautiful, once again, musicians do it right to pave the path for the rest of the artists that, good sir, makes me a proud musician and your approach is exactly what I would have done, just buy the data from artists, and then everyone can have a meal. I know it's asking too much too soon, but if you could put a kinda royalty for the artists, then your model is complete for the years to come. thank you for your beautiful work and approach anyway, very glad to see it happen.

[@bronsoncarder2491]

I know you said a few times that you're going to discuss some of these concepts in depth later, but I just want to say that I REALLY hope you go over the specifics of the formula you use for natural vibrato. a+Sin(Songtime*pi*2*5*c)*(b-0.5) and then the input is: Input*0.05+0.5 I'm assuming "Songtime" is the tempo of the song? I've been using FL Studio for years and had no idea you could do that. lol Why "pi*2*5*c"? Why not just "pi*10*c"? (I also didn't know you could use pi, but I also haven't worked with the formula controller a whole lot lol).

[@nickwallette6201]

FWIW, I often keep terms separate so I can more easily deconstruct it later. (Fewer "magic numbers" where later I'm like, how did I come up with '10'?) In my case, this is usually in code that gets compiled (e.g., C), so there's literally zero cost to doing it that way. The compiler will simplify any terms it can before emitting executable code, so you take an insignificant hit once during compilation, and then it runs, optimized, from then on. Meanwhile, the source code is easier to understand.

[@LOLCoolJ]

I'm subscribed and have all notifications selected but I never received a notification for this video. Glad I manually searched for it. Very impressed with this. I'll definitely be using this service in the future.

[@apoplexiamusic]

Amazing subject matter and video as usual! I love how Benn just name drops DJ Fresh as his good friend. Like holy shit, man is a legend!

[@minhuang8848]

Have only read the description plus subs so far, so nuance might escape this comment, but I wanted to still point out that the "better data, better model"-paradigm is very rapidly coming to a full standstill... or at least so one might guess. It really depends on the particular domain you're dealing with, but the better we get at conceiving of great ANN architectures, the less it matters how great your recordings are as long as there is some semblance of signal left in there somewhere. It's the new, wilder West we're dealing with here. Machine learning state-of-the-art keeps flipping around like it's part of a pinball machine, and I would not be at all surprised if the same is true for all sorts of vocal-adjacent endeavors. Not that there aren't plenty of great additional reasons to have a solid corpus of HQ recordings on top, but I guess that kind of illustrates the point too: having both crusty and good data can help create a model with latent mapping between those two "states," if you will, which might result in neat RX-like cleanup tools and much, much more. Can't wait for all this to be old news in a week or two, nor can I wait for the ultimate vst allowing you to just learn any sound or expression in the world. Imagine actual choirs without phrase builders, even if you have humans perform it in the end, drawing up mock lyrics with a great vst alone would do so much already. I mean, emvoice is a thing and it's actually really great for that, if the artificial voice sounds decent, the properly recorded vocals aren't going to suddenly suck either. Arrangement >> everything else, after all.

[@daw00t]

did you really just admit up front to replying before watching the video

[@minhuang8848]

​@@daw00t Wild what you can get away with when your parents decide you should live a bland, regular life and attend school. Some of us even get away with learning how to read and all that jazz, I know, it's a completely out-there notion, but I am telling you: don't knock it till you've tried it Or do whatever you want, after all it's you who thinks looking at colorful pictures alongside arbitrary information augments your understanding of the video's contents. Which might or might not track for you, someone else be the judge of that, lol

[@CarlosKTCosta]

I know that, unfortunately, I don’t speak for the majority but the way you appear to be concerned with fair compensation makes me want to use the platform just to support it

[@dqvid]

Such a good video! I find all this super interesting, but it also bums me out that years from now it'll be Taylor Swift or Frank Sinatra models doing vocals on all tracks, or whatever the average taste will devolve to. I want to hear new voices, things i haven't heard before. I remember being struck by Joanna Newsom or Brittany Howard. i just don't know that ai could come up with such voices... And that's the thing i like about vocalists, when the voice is arresting, otherworldly, but never ever in the uncanny valley.

[@tntero]

This is really cool! The thing I like the most about your channel is that you keep doing things that I had daydreamed about but left in the "what if" realm because I didn't have the time/drive/patience/knowledge to put into practice. Hit me up if the project needs another software developer

[@natekelly35]

What song is that of yours at 10:54 ? I saw you live in Kansas City some years ago and you performed it with the absolute coolest backdrop video.

[@ianbest3677]

OMG _ What a great project. When you see how large corporation working on replacing are abusing artists and creators, it's great to see a project that can benefit artists and not the big corp. I wish you good luck on this endeavour.

[@CirTap]

great content as ever. I never heard of any of these "chart topping artists" that voice-swap are marketing.

[@MatsueMusic]

I truly think that AI is one area where we have to think long term. I don’t think the tech is the problem, I think culture and society are. On the one hand social context and human action shape technological progress. However, we also create technologies without considering the long term implications. The smart phone is a great example. Who knew that the dopamine hit of getting a notification on your phone would create generations of attention seeking drones, that require external validation for their value, entire industries that leverage that and campaigns that profit of FOMO. Likewise with AI EXCEPT the asymptotic progression of AI vs human skill won’t take decades to reach diminishing returns. Also, if people and society reject AI it doesn’t mean that businesses won’t leverage LLM or stop automating tasks using them. Regulations will make it prohibitively expensive for ordinary people to leverage AI to earn money because being “up to code” will be super daunting and expensive. Like the alcohol industry or at home growers or restaurants. I think UBI is the answer because the democratization of skill, knowledge, access that AI can bring shouldn’t be dismissed. The harm it would do to the very few “skilled”, “talented” people that would actually have the ability to EXECUTE performance arts or tasks shouldn’t outweigh the huge percentage of people who have great ideas but weren’t born with social, cultural, or biological privileges that others have. I don’t buy Artisan bread from Walmart, I pay 4x the price at a local farmers market. That isn’t going away, those who want human made art will continue to buy human made art and the price should be proportional to the amount of AI used to produce it since reproduction and scale is also proportionate to the use of AI. My family was quite poor growing up, we couldn’t afford music lessons, pianos, or other forms of music education so I never had the opportunity to train my ear and so I could never compete. But, I know I have great ideas my limitations has always been not knowing what is available, where to start, especially in the music industry. I never even considered pursuing it as a career because it was unattainable. Now when I hear that AI can help me create music, help me understand how to publish it, what the best way to accomplish my vision is and people are out here trying to horde that knowledge and skill like a weird monopolistic company.

[@1981GLAD]

Last vid on Doppler was a huge amount of work and was last week!! Does this man even sleep?! Not complaining, just don't want my favorite channel to burn out in an explosion of awesomeness 💖💖💖

[@nerds-nonsense]

This project sounds really cool, it's dope to see you putting your money where your mouth is (again). Excited to see how this project turns out.

[@-arb-]

I have a mute friend, and I hope very soon she'll be able to hire an actress to provide a voice for her to use in a text to speech application. She doesn't use text to speech at all because the current technology doesn't sound very good, but it could! There needs to be a legal framework in place for this to move forward.

[@chillwalker]

A Friend of mine, a now kinda famous Voice Actor and Radio Host (and Station Voice) of germans "Deutsche Welle" somehow gave about 10-15 Years ago speech samples for a Car Navigation Tool for an One Way Payment away. And from this monment he was one of the most recognizable voices, because (due a propper copyright of this) his "sillabels and ohonetics" where used in every freaking tool. Even in the first synthesis tools. I jokes around, that I found his modell everywhere And don't have to book him ever again. He was so pissed for years...but now, with ai...his clone is fading away, slowly ;-) (I am not allowed to say his Name, but anyone with a little experience in the job know whom I am speaking of)

[@DeadDad1]

I just found your channel and subscribed! You produce extremely well done content and I sincerely appreciate it, thank you!

[@LordSmyrnian]

This is probably one of the most groundbreaking technologies of the 21st century! Just imagine....a beer that tastes like Oreo. Seriously though, I'd love to hear from Hindi speaker as to how that one part sounded. That's some amazing potential. And Lucy was just frightening. Imagine if you heard actual words...lol. There's an old short sci-fi story where they plug a device into a computer and it says something like "finally" or "at last" when it actually gets a voice. I can't remember much about it though except I think it was an Asimov story.

[@manOmanyTrades]

Awesome video. I discovered your channel last week and I am so glad I subscribed. Your content is very relevant and on point. Keep up the great work!

[@Clarity-808]

Very thoughtful video that left me with one looming question: When is Lucy’s mumble rap debut?

[@ElevateInnerGrowth]

Could you make a special video about the entire process of creating this song, especially focusing on singing in multiple harmonized voices at the same time? I would be really interested!

[@ashleyrose6438]

Once theres a large enough dataset, what's to stop companies from just not working with new artists? If you have 2,000 voices, then why work with a new person to have 2,001 voices?

[@andivax]

For successful voice transform you should train your own models in RVC2. Quality will be decent. RVC3 will be much faster and better but it's not released yet. Now I just make back vocals and doubles using RVC2 and artists are amazed. Using RVC for making lead vocals is not good and you will potentially have a copyright problem. PS and yes, it's not AI. It's Machine Learning. Very dumb algorithm to be honest )

[@Que.R]

I will try this! Thanks

[@ts4gv]

Lucy's guest verse was amazing

[@mymodularjourney]

I’ve had a good laugh hearing AI Johnny Cash singing Barbie Girl. But I’m sure the technology will evolve over time to have a decent vocal playable like an instrument.

[@MrDrury27]

The models are first trained on millions/billions of samples to begin with, and only then are they finetuned for specific use cases (e.g. to emulate a specific voice). It's easy to see how you'd go about gaining permission and offering compensation for the latter part, not so much the former. This is cool if your goal is to pay at least some people on the input side, but it's an ethical contradiction to say the least. It also flies in the face of the argument that ML works better if you prioritize quality over quantity. You really need both for good (or even passable) results. I'm skeptical inviting money into the mix would solve the quality part of the equation in the first place, but that's probably just a me problem. The popular tools all being open source backs me up a little though.

[@DJFresh]

Hey there. The base model was trained on an open source corpus that was ethically sourced 🤓

[@MrDrury27]

@@DJFresh I thought you were being ironic before checking your channel. How is open source inherently more ethical? Stable Diffusion was both trained on an open source corpus and released free and open source, and that was precisely what kickstarted the whole ethics debate.

[@DJFresh]

Hey there. Sorry if I wasn’t clear, I wasn’t saying open source is inherently more ‘ethical’ I was saying that our base model is trained on an open source data set that has been ethically sourced (where the recorded voices have been given permission). Permission is the crux here. Stable Diffusion didn’t have permission from artists to use their data.

[@djbayr9516]

Benn, are you planning to do an album any time soon? I loved the flashbulb stuff and I go back to that every so often.

[@sonofwhyachi4870]

DUUUUUDEEEE! I JUST TRIED THE WEBSITE! THIS IS AMAZINNNNNNNGGGGGGGGGGGGGG

[@mwimwits5104]

really like this video, thank you for the insight, benn! and a question, what song is at 10:56? hopefully it isn’t an unreleased track, i really like the sound of it

[@attilakun8416]

I would also like to know

[@danielhadida3915]

ME TOO !

[@keysthomas1966]

As usual, u have very solid ideas! ... and my little Lucy makes crazy sounds upon waking as well!

[@blankblank4949]

Yo, when you first kicked in a beat on amazing grace the bass was doing this breathy thing that felt like was way more than a sidechained compressor. were you doing something extra? or did the harmonics of it just happen to make that vwomvwomvwom kinda sound when the compressor hit?

[@Forr0n]

When your channel blows up, it will be a big bang.

[@amateurmakingmistakes]

Great to see someone who has some decent ethics! Goodonya!

[@stefank.3983]

If you travel at the speed of sound, following an object traveling at the speed of sound, would you be able to hear one continues sonic boom ?

[@LinoWalker]

Wait, I'm confused. How do artists get royalties if all AI-generated stuff is free use? Is it because your platform has a better-quality generator than other platforms? And if you want to use the output of that platform you have to pay royalties? But then how do you pay royalties on something that's free use?

[@nonprofet]

very happy your doing this, great video !!

[@_thomaswarburton]

What is this wizardry @8:10 ? I'd like to know more about creating that nice vibrato. Good stuff 👌

[@Stormsurf001]

You never cease to amaze. Great work Jordan!

[@stephenrodgers9698]

How are you consistently the coolest human in the music world... thanks for blessing us with these videos.

[@venjsystems]

OMFG I just lost it as Lucy's voice being made "human" hahahahahahhahhaaaaaa

[@ViRiXDreamcore]

That'd be pretty cool. Musicfy is one that seems to work alright as well although it probably doesn't have any kind of royalties.

[@arothmanmusic]

In all honesty, unless the consumers buying the product care about whether their music contains 'ethically sourced vocals', this will remain a concern primarily for producers and lawyers. As digital / AI tools get closer and closer to replicating human sounds indistinguishably (they're almost already there for photography and text), it becomes mainly a question of whether it's cheaper and/or faster to hire people or machines for any given project. If you want Mavis Staples, you'll still have to buy her time, but if you want "female vocalist that sounds like Mavis Staples" you would likely be able to get it for essentially free. It comes down to time, budget, and ethics. I suspect we're heading back to the pre-recordings era where only the personalities and live performances of musicians will be monetizable. The decreasing cost of music production and distribution will eventually drive the value down to the point that nobody can make a living at it anymore.

[@DocBolus]

Glad that somebody is starting to think about how artists can genuinely retain some control.

[@rochefsky]

Thank you Benn! You are on the right side of ground breaking, too-fast moving music tech!

[@thereneramirez]

I have recently been doing this, but one of the biggest thing that just is difficult, if you are a small artist with no money, buying another subscription to an AI model is hard. But if they were to offer a free royalty version of 1-3 models for small artists like us, to get hooked into ai it would be nice. Plus, using the free data that gets uploaded to transpose the vocal it's a great tool for training models. I get it you have to make money one way, but if you have a huge amount of free data from a royalty free model to test more voices to then sell more premium stuff I think that is the way to go on about this.

[@thereneramirez]

kzbin.info/www/bejne/m4DLg35rdqd9j6ssi=rxmDweKyUiQBEJGf

[@thereneramirez]

This is a sample of what I'm working on. That was just 2 layer vocal, but it's a great tool to learn singing, and maybe use it for reference to hire a singer to get your vocals going