New FREE/Open Source Music Generator - It Destroys Google's Model

[@shun_almya]

Strange days Who would have thought that I would love Facebook one day

[@tyler.walker]

Before they became Meta, I thought the entire company was dead. After they became Meta, I thought I was watching a reanimated corpse being fucked by Zuckerberg. After all of their ML research and VR work, I feel like they actually make sense as a company.

[@ДаниилРабинович-б9п]

yeah, they've been killing it lately with the opensource models

[@edwardecl]

Facebook do genuinely put out some good open source stuff like Z Standard compression, Linux filesystems like BTRFS can use it to compress/decompress data in real time, for example the Steam Deck uses BTRFS+ZSTD for the OS, not really sure why they don't use it for the data/home partition though. I'm not usually a fan of Meta/Facebook, but that alone impressed me.

[@randombleachfan]

Well Facebook has collected a lot of data back in the day. So it’s not a complete surprise for me for them to do be this well, but still surprised me for them to release open source tools.

[@rccsab]

Lex just dropped an interview where they say facebook will be doing this since it's part of the deal with top ai researchers to open source their work. I have to say, I hope everyone leaves google and go work for them.

[@lasagnadipalude8939]

I'm kinda confident that multimodality will solve a lot of stuff because to associate multiple types of data to the same semantic meaning it has to have a very good internal model of the world and every single generation of anything will be a lot more coherent out of the bag

[@nuvotion-live]

The singularity = 1 super model that’s as universal as the human mind

[@lasagnadipalude8939]

@@nuvotion-live except for the fact there will be billions of those and with time a big portion would even be open source

[@nuvotion-live]

@@lasagnadipalude8939 I think it will converge on one and it will be open source. It’s in the name “singularity”

[@vatekehcorlon867]

audio, image, video, text, Brain(data), groundedPysical(robotic-data), animation and much more. probably only need 5 of these I bet

[@markemad1986]

Yes definitely once a model has explicit multimodal association, which is very achievable, the coherency problems should mostly disappear instantaneously

[@Corruptinator]

Rather than using generated sounds for your soundtrack, you can take note of the rhythm, sounds, and the effect of any music generated and remix/redo them in your own way to make it unique, not in terms of originality but something of a standout that just simply fits.

[@joey4track]

I can see how for anyone who doesn't make music this seems impressive but after testing both of these tools extensively, there is a very long way to go. Still, this is really cool but we are many months away from anything actually useful. I have been sampling this stuff into actual music but it still takes a lot of skill to turn this output into anything listenable tbh

[@LustyLichKing]

Same with AI art generators when they first came out then. Progress is iterative; this is the worst it will ever be going forward.

[@IM2awsme]

Disco dubstep turned out ok?

[@kidinacage]

Give it a couple months

[@joey4track]

@@Sir_Sway looking at it now, does this actually generate stuff? it just looks like sample packs and i can't figure out how to create anything

[@edwardbradshaw6850]

I agree but it’s pretty amazing. I’m looking forward to someone actually make a useful tool from these. I know it’s going to be possible. Watch out sample library producers 😅

[@gionicol_]

I just love Matt's face when he accidentally generates demon-related stuff in these AIs

[@atomictraveller]

i didn't get that far in this video. i've used procedural methods for decades.

[@LydianMelody]

I can’t wait to go full music geek at it. “Giant steps in the style of Smash Mouth. 16 bar guitar solo by Mateo Mancuso. Nested quintuplets in the drumbeat.”

[@limeflashlight4101]

Hehehe

[@sub-jec-tiv]

Audio quality is better but it sounds like it was trained on copyright-free music and midi-music. The actual results may have superior audio quality but two-year old Jukebox still gives a better more musical result because it was trained on actual music by artists. This kind of tech won’t really start cooking until the open source community steps up with versions of these tools for people to use at home, to train using their own sources. There’s still a ways to go, as music is a time-based media. Like text-to-video, the coherence problem is yet to be solved over longer segments.

[@joelface]

Oh man, the set up for the MusicGen guitar solo.... I was expecting something even better than the Google one... and then... THAT happened.

[@fritt_wastaken]

They really should make a music model giving midi outputs. As it is it's not really useful, but imagine if you could right away tweak a decent output into a great one

[@zu__880]

Exactly. The first one who is able to modify this to be midi will actually have a significant effect on the industry.

[@jaybird922]

aiva ai does it this way and it is ok. makes some really nice piano melodies and some ok electronic stuff

[@TsukuneASMG]

as a musician i agree 100%, A Note Midi generator that works by prompting what kind of Melody or beat it should generate would be 100% more useful, i predict whoever does that first will have the "Win"

[@DJWolfy23]

@@TsukuneASMG that can't generate full music. The thing with audio AI is that it also generates things like guitar very very well. It just depends on what you want to do.

[@fritt_wastaken]

@@DJWolfy23 why not make an AI that generates full music based on midi that's generated with another AI? Like with image generation where you can make something from scratch or give it an input image and make it follow it as close as you want in any aspect you want

[@rhizomorph-music]

I'm an ambient and experimental electronic artist, and I can't wait to use AI-generated music pieces... not in the hope that it comes up with something "normal" sounding, but in hopes to get some really weird stuff out of it to layer into my compositions. And it looks promising. I will probably wait several weeks before digging in, though, becuse it seems ike it has a little bit more maturing to do.

[@DanielLehman-en7ds]

Hi, I want to thank you for all your videos, this one and the MusicGen application blew me away, in a few seconds you can do incredible things. We are going to see an upheaval in society in the coming months. Keep up the good work, greetings from Switzerland

[@MattVidPro]

You're very welcome!

[@robertdzupin8502]

Just checked it out using local install. Running LARGE model will consume about 11.4 GB for 30 second long clip. In my test I used github sample clip for 'happy rock'. It looks like for shorter clips even 12GB GPU is good enough to run highest quality model.

[@vinco1]

That's super exciting, thanks for letting us know! I was disappointed at the vram requirement since I also only have a 12GB GPU

[@yudhaxn1195]

how do you even run it on gpu? is there a setting for it?

[@theodorington]

YESSSS! YOU ARE THE GOAT AI NEWS CHANNEL 🔥🔥🔥

[@RedEyeLazer]

Yeah, his videos are really informative 🔥💯

[@Garbanzobear]

Those guitar solos from musicgen were really something else man, so beautiful and harmonic.

[@neogeo9117]

😂

[@edp_445]

I don’t think any of us could realize how big this day truly is.

[@micahwilliams1826]

It's really not that useful

[@J3R3MI6]

It’s revolutionary

[@Crasterius]

@@J3R3MI6 It's not. It's just a toy.

[@allanlomas5133]

@@micahwilliams1826 pls elaborate

[@micahwilliams1826]

@allan lomas From a producer standpoint, it needs some more tuning options, like the ability to play in a certain key, or having it play with a specific chord progression, tempo, ect...

[@GreenTea13121]

12:06 Well... it would be perfect background music for some sort of Backrooms / Liminal Space / creepy project 😆

[@salvadormarley]

That's true, creepy sounds have a place in video production

[@alexeialpaka]

To me, the melody one could be the most powerful and useful. I could make simple clear melodies with the piano, and boom, new soundtrack for my incoming videos

[@joshgiu]

Would love to see a vid for the install on local system :)

[@games528]

Do you have 16gb of VRAM though?

[@joshgiu]

@@games528 I can neither confirm nor deny this

[@Dron008]

Meta is doing much better gifts for us sharing open source models unlike Google which develops some cool things but hides them. Even Bard is still not available in my country.

[@ajitsen6927]

You are not missing much . I just gave it a text and just asked it to pick out prepositions ... it FAILED !! Bard couldnt even pick out simple prepositions properly ! ChatGPT did it perfectly in 1st try.

[@jj8703]

the melody condition thing is SUPER cool, cant wait for a tutorial for gpu

[@eyevenear]

please do the gpu version of this, i bet it's gonna take SECONDS to generate compared to Hugging face.

[@WafflesGiraffles]

The guitar solo from MusicLM actually sounded great, could've been a real performance in my book.

[@PDZofficial]

PLEASE do a tutorial for the local installation! I'd love to get this up and running on my PC but it doesn't seem like anyone has given a detailed tutorial online.

[@Mavrik9000]

I wouldn't be surprised if Nerdy Rodent had one already.

[@Madcore.]

I vote for this also

[@edp_445]

Many music creators was scared of the ai voices, they are truly not ready for the storm coming in the next few hours. 😅

[@tyler.walker]

It’s way more coherent than GoogleLM, but even with longer tracks, I still wouldn’t call it “useable” quite yet. But goddamn, it’s close. I’d say within the next 6-12 months we could expect to see decent music start to be made using tools like this. Excited to see where this goes!

[@INVALIDZEROTheTitSucker69]

​@@tyler.walker way too long. Expect that in about the next 4 months. Let's see how well this reply ages.

[@hypelitics]

I love both your optimism but personally think we’re still a few years out before tools like these become properly meaningful in the music industry

[@joey4track]

next few hours? lol nah, this tech is quite a long ways away from being useful. i have been making music for 30+ years and have been extensively testing this and googleLM, both need a lot more work to be useful to anyone actually trying to make music in any serious fashion. it will get there eventually but we are at least months away from having anything other than an interesting toy to make nightmare fuel with tbh

[@tyler.walker]

@@joey4track The hype is so over exaggerated on all of this tech, it drives me crazy. I love it, I think it’s amazing, and it’s an actual wonder to me that we can make music just by giving music to a computer and telling to to make more. However, it’s 40% dogshit still. Before, it was 60% dogshit. We’re taking steps forward, but it’s not going to replace the music industry “within hours, or even months, lmao

[@dontmindmejustwatching]

ammount of "INSANE, crazy!!111, IT DESTROYS" is too damn high.

[@pongtrometer]

Music LM, sometimes just plays for the sake of playing, like it has a repertoire of phrases for genres and emotions, almost like fixed parameters but isn’t creatively listening to the prompt…. Music Gen’s got the juice for now, especially with the audio prompt and parameter tweak options, it’s much more on par , and could become the MJ of Txt to music creation.

[@TonyBologna9]

See you all in like a month when they somehow manage to jump 3 generations into the future of this AI.

[@Sanjubaba00007]

Ai evolving so fast can't even imagine

[@postmodern.cyberpunk]

oh man, that sad piano piece is sublime

[@FusionDeveloper]

Yeah, as I said in the other video, The ONLY practical reason I would upgrade my GPU, is for better/faster AI generating. I would rather run games in low resolution with low quality as a trade-off for better/faster AI generation.

[@smithjohnsonwilliams]

Can you prompt it for specific chord progressions and specific melodic guidelines like "lydian melody on a guitar over Cm7 Fm7 G7 progression on a piano"?

[@tiagotiagot]

Shit's gonna hit the fan; copystrikes are gonna rain as the big labels panics and try to keep the public from getting in on the fun...

[@caseyarmstrong7113]

Cool, but Google's did accompaniment (figured out what music would go along well with something acapella). That's the major use case that will make waves because lots of people can sing but can't play instruments.

[@SSLCLIPS-TV]

You shouldve named the video MusicGen VS MusicLM

[@yorth8154]

It's insane that the large mode is "only" 3.3b parameters. I can't begin to think how much better a model ten or a hundred times the size would be able to do, if given even more data.

[@internetwarrior666]

It's not always about more data but fine tuning on what data exists.

[@yorth8154]

@@internetwarrior666 when it comes to pretraining, it's about more data. I think chinchella showed that pretty well

[@thebicycleman8062]

As a musician, the google LM guitar solo was not bad at alllll, however where google lm really shines is edm, anythin with 4 4 house style it rele shines with bass, it has rele nice rich basslines and very bouncy

[@Otis151]

The large model understood the assignment. Good bongos.

[@hurktang]

Is possible that "guitar solo" means nothing to the AI, did you meant classic, classic guitar, pop, rock, heavy metal, punk, flamingo, experimental music? Without context, probably half of the neuron of the AI light up when you ask for "guitar solo". And that would explain the result.

[@PriestessOfDada]

So I've been messing with this a lot. Your settings down at the bottom are really important. If it sounds off, what you need to do is adjust them. No idea what they are? No problem. Start at 250 in the first column, and work your way around. Don't mess with temperature unless you like it weird.

[@ItsXylnt]

As a music producer I love it and hate it. Definitely a great tool for creating ideas and making the music making process faster, but I can definitely see how this could be abused and affect the music industry as ai voices have. I support the technology and think this is our future, but playing devils advocate I can definitely see where others would disagree.

[@henrythegreatamerican8136]

Sad thing is a few of these samples were better than half the stuff on the American Top 40 chart the last few years.

[@ItsXylnt]

@@henrythegreatamerican8136 god the hurtful truth of this stings 😭

[@SynthoidSounds]

Commercial music has already descended into a realm of streamable tracks, almost always sounding the same within the parameters of the predetermined genre' category. AI will basically enforce even more such genre' confinements, with known formulaic elements, designed to catch attention in the first few seconds of play, be no longer than a given length, inflate the likes and follows on social media platforms, and "blow up" the song of the moment, with the ultimate goal to translate this into profitable live performance tours. Meanwhile, there will still be "organic" music artists, with the challenge to invent original content that is distinct from formulaic AI constructs.

[@ElParacletoPodcast]

This tool is awesome, can wait for it to create entire song.

[@FilliD1]

No matter how good it sounds it is not human music. I prefer my energy connected to and empowering human energy.

[@xorqwerty8276]

I wonder if one of the fundamental issues with music generation and artificial intelligence at the moment there is no look back option so music normally follows a pattern of something like a verse chorus verse chorus Bridge chorus chorus. AA seems to only feedforward where I can generate something that sounds good for a short period of time but I can’t look back to what it’s already generated to create that structure and consistency that we like

[@JorgetePanete]

You could kind of make it iterate, with chain of thoughts or similar techniques

[@dockdrumming]

I think you are correct. For most part, the music just seems to meander with no structure.

[@SNCKPCK]

TIME TO PARTY!

[@whoman0385]

i was almost first commenter..

[@hipjoeroflmto4764]

​@@whoman0385oh no now you're life is ruined because you didn't get the first comment oh noooo 😢

[@EugeneBartholomewMcJigglebutt]

​@@whoman0385ur pfp is unhinged sick messed up drawing from Ted Bundy's last interview

[@EugeneBartholomewMcJigglebutt]

Psychopath

[@EugeneBartholomewMcJigglebutt]

Psychopath

[@FusionDeveloper]

I really would prefer to get a video card with at least 24gb RAM, since my current one is 11GB, if I went to 16GB, it wouldn't be much of an upgrade (RAM wise), it would just bring me to 1 "level" higher for AI stuff (aside from massive speed increase). I would love for them to come up with a way that we can add more RAM in video cards, so that you could, for instance, buy a 16GB video card, and have like 4 empty ram slots to add at least 8GB RAM sticks to bump it up to 32GB more RAM for a total of 48GB RAM. I hate how they have been "stalled out" on increasing the RAM size for their GPUs since at least the GTX 10 series. I hope Jensen realizes that AI needs a lot of VRAM and starts making budget GPUs with high VRAM at the expense of GPU cores/speed. I would be perfectly happy with my 1080 TI 11gb if it had 32GB or 64GB RAM. I don't need Ray Tracing or Shadows. I imagine like a new series, from GTX to RTX to AIX. I don't want to go through all the specs and try to design the specs accurately and realistically. It would be useful if I did, because I would likely be able to estimate a real price. Since I don't know what specs would fit, I'll just pick a random one. Anyway, I imagine a AIX 5070 for $600 with slightly better than 1080 TI ftw3 11gb specs, but with VRAM in the range of 32GB to 64GB. It may or may not have any raytracing cores, depending on the cost. Of course it also needs to have at least 3 video outputs. I don't need blazing fast, top of the line, cutting edge VRAM, I just need it to have enough VRAM to be capable of running larger AI models. Since AI model size REQUIRES a specific minimum VRAM amount, that should be the #1 focus and let everything else slack to keep the AIX GPU price affordable. They can have higher-end AIX cards, but if they really want to help get AI to everyone, have a high VRAM GPU for cheap. Remember, most people have a 4GB VRAM video card. Let's change that. Let's make it so that most people have a low end GPU with 32GB VRAM.

[@andrewaverbah4809]

They want you buying A100, so nGreedia will never go above 24G VRAM on consumer cards

[@hannibalbarkas1350]

@@andrewaverbah4809 yep novideo is making fucking bank with fucking customers over.

[@coloryvr]

So WOW! Open Source and Local! A step by step instal Video will be very nice and will save a lot of time for people like me ;) ...BIG FAT FANX!

[@hojowarf6488]

You typed heavy drums and synth pad and that's what you got. That's all you got. Maybe it needs each element specified and won't make the assumption that you want a lead synth, guitar riffs, aux percussion, strings melody etc on top.

[@CyberSleuthCitizen]

#1 subscriber here! I'm 100% positive your followers and community would love an install tutorial! Thank you Matt! ❤️

[@alternatuber6698]

19:56 loved this one very deeper sounds

[@Distty]

Best reactions on AI space, makes me not to think about AI for few sec. priceless.

[@Travestyalpha]

Because of all the more aggressive copyright enforcement for music than for art. I expect music will be way behind in AI gen than images. Yet - so many musicians are selling the rights to their music catalogs, that some of those corps that own that music will probably start creating AI continuations of an artist music, but the rest of us won't be able to.

[@sub-jec-tiv]

As long as there is open source there will be people generating their own models. The official facebook one will create meh music because of copyright restrictions, but there will be individuals out there secretly training their own models.

[@SynthoidSounds]

Commercial music has already descended into a realm of streamable tracks, almost always sounding the same within the parameters of the predetermined genre' category. AI will basically enforce even more such genre' confinements, with known formulaic elements, designed to catch attention in the first few seconds of play, be no longer than a given length, inflate the likes and follows on social media platforms, and "blow up" the song of the moment, with the ultimate goal to translate this into profitable live performance tours. Meanwhile, there will still be "organic" music artists, with the challenge to invent content that is distinct from formulaic AI constructs.

[@HikingWithCooper]

Woah! This one is actually good. All of the prior ones have been quite lame. Cool, but lame. I would love to see a local install tut. Maybe after the model is released.

[@FrancisBourre]

We're still very far from anything usable.

[@karlosmartos4646]

every ai video talking about music generation : - it creates total trash - "im liking this, this is so good"

[@Sport-buz]

You are correct about there being a lot of new AI’s. I have a new one that I am developing called Divora.

[@Cosniewiym]

Great video...just a question, does anybody know how to see and select different models and length of music output? thanks

[@Nutronic]

I wonder what the bongo beat would sound like uploaded as the melody and used with that 80s prompt!

[@supermalavox]

Another disadvantage of MusicLM is that it's not available outside of the US, apparently. Anyway, I tried to generate a song played with a sitar and MusicLM via a Discord server didn't let me. MusicGen, on the other hand, let me even do a combination of sitar and church organ!

[@gawni1612]

12:04 SO MANY MEME FACES IN ON SEQUENCE. I completely lost my shit.

[@wally6265]

Good review, the only thing you left out was if the Meta one is open source. /s

[@giri.goyo_yt]

Matt's face 12:22. Priceless.

[@nangld]

MusicGen also hallucinated some synth to go with piano.

[@SMmania123]

17:42 sounded good!

[@addapp]

Dying to know how to put this on my own machine. Thanks!!

[@parkbyrd]

Thanks so much. Super interesting stuff... Focusing on a type of guitar and style may yield better results...

[@VaibhavShewale]

damn meta doing some awesome work!

[@spydar05]

2:13 Is this conceptually more like Music-GAN or Music-Firefly? Or does that comparison even make sense/matter?

[@Astrologon]

So that's how the aliens generated all 90s music.

[@Mr.ThomasAnderson]

I’d watch a 2nd channel full of Matt’s face reacting to cursed generations😂

[@Baekstrom]

It’s not quite there yet but it is starting to get to where you can maybe use it for cheap commercials or video games.

[@ytubeanon]

15:38 nice piano melody, how did you end up making it? 'sad piano, emotion of doom evoked', but how did you get rid of the background beat

[@andresshamis4348]

Pretty sure these models are made to generate samples rather than full songs/full instrumental layers

[@nerdglider]

Meta is on a comeback tour

[@alexkaa]

I'm not sure - but shouldn't the prompts be much much more specific and elaborated, to generate good results? Have to try this one out myself first, though... - just a guess when you anticipate the eleorated multi-sentence prompts for Midjourney, DALL-E and the like.

[@JorgetePanete]

That depends a lot on the AI, each one may be different

[@DefinitelyNotNormalLol]

12:32 I definitely heard a dial up tone slipped in there.

[@backroomss]

Can musicgen create more longer music with colab or on local PC?

[@KeithPhillips]

Is the wait list for MusicLM really that bad? I signed up for it weeks ago and got access in less than 24 hours..? 🤷‍♀️

[@awgrym6426]

A certain quote comes to mind, "Writing about music is like dancing about architecture."

[@rickt774]

Where can you ajust duration? And where quality? All those settings have been thrown away since a week or so i think

[@andrewhryhorov8357]

It would be great to give it a try👍

[@Marc42]

Power user with local interests here ✋Thanks in advance! 😎

[@Dante02d12]

I didn't manage to get proper music clips with that. It couldn't replicate melodies nor did it give good representation of the prompts I tried. But I guess it will be like Stable Diffusion : base model isn't terrific, but its power will reside in its customizaton and the abiliy to create new models _based_ on that default one.

[@goldenfurniture123]

I feel like the google AI is better at knowing what the prompt is and being smart, but Facebook Ai Has better audio definition

[@C_Corpze]

All this stuff is cool. But the requirements are pretty painful. My RTX 4070 ti only has 12 gb so it would be 4 gb short to run the model. Only very few people will have 16 gb likely. And most people likely want to be able to train it as well since while sounding good, most music still sounds rather generic and soulless. I’m excited for what this tech will bring us though! It might be very new and recent still so optimizing and improving it likely still needs to be done. I wonder why Stable Diffusion can run with 4 gb while a music AI needs 16 gb? It’s such a big difference and a lot of the music also does not sound properly mixed/mastered and a bit muddy.

[@eafortson]

I would love a tutorial for how to install this on an M1 Max MacBook Pro. Assuming it would run since the M1 Max has 64GB of unified memory. Right?

[@johnmitchelltaus]

Too many people trying this out. It's taking hours to extend a song. Think it might be stuck. After 3 hrs it gave up.

[@alexsindane14]

Can I use it for my video

[@Future80srecords]

Vangelis has left the chat.

[@eyesyt7571]

Why? Why would a company as horrible as Meta do anything good? What are they getting out of it?

[@MattVidPro]

Separate divisions of the company

[@UuU1001.]

12:04 - 12:35 this is the music that mark zuckerberg listens to in order to concentrate and recharge his battery

[@HostileRespite]

I liked the rattlesnake basedrop! 😂

[@Maisonier]

Yes please, create that tutorial... There are tons of gpu being not used for mining nowadays

[@HazySkies]

So MusicGen is like a clearer reskinned version of OpenAI's Jukebox? Interesting...

[@Magicguy13]

Thanks Matt

[@hoopstalkcentral]

I wonder if you can use this and upload to music distributors?

[@miikkapoikela1237]

So did you want to making a craiyon v3 for a dnd monsters text to image ?

[@johnwinslow8841]

it worked after creating an account, but the results was terrible. used a clean vocal melody audio file.

[@godned74]

Its gonna struggle for along time because a data set required for audio would be drastically larger then what was used for gpt 4. Probably 40 petabytes at least

[@iqmarketers]

Goog has jumped the shark on Ai. They are getting lapped. 😂 and do not realize it...Go Matt Go!😊