AI Like OpenAI’s Sora...But Free To Try!

❤️ Check out Lambda here and sign up for their GPU Cloud: lambdalabs.com...
Try it out: lumalabs.ai/dr...
Full "This is fine" comic strip (you have been warned!): www.theverge.c...
Ben Nash extending the video: x.com/bennash/...
📝 My paper on simulations that look almost like reality is available for free here:
rdcu.be/cWPfD
Or this is the orig. Nature Physics link with clickable citations:
www.nature.com...
🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Kyle Davis, Loyal Alchemist, Lukas Biewald, Martin, Michael Albrecht, Michael Tedder, Owen Skarpness, Richard Sundvall, Taras Bobrovytsky, Ted Johnson, Thomas Krcmar, Tybie Fitzhugh, Ueli Gallizzi.
If you wish to appear here or pick up other perks, click here: / twominutepapers
Thumbnail background design: Felícia Zsolnai-Fehér - felicia.hu
Károly Zsolnai-Fehér's research works: cg.tuwien.ac.a...
Twitter: / twominutepapers

[@pranjal9830]

What a time to alive.

[@spyfox260]

What a be to try alive!

[@pandoraeeris7860]

What a time to be AI!

[@twylxght]

Alive time be to!

[@OhioNPC911]

What do you even understand? Or just saying “what a time to alive” just for like? Stupid doesnt even know what true art is

[@SockOrSomething]

Time a what Alive be to

[@Nulley0]

I remember once upon a time (4 years ago), when I was generating rudimentary chaotic AI videos from music in low res. We've come really far.

[@test-uy4vc]

IMO I feel that Sora is like the Dalle of video generation. And these other ones are like a Dalle-Mini. And in a year or two, we will get a "Dalle-2" of video generation. Also found that adding a dimension of time, makes the "images" or 2 dimensions really good. A snapshot of a sora video is way better than Dalle-3 could produce,and I wonder what adding the dimension of 3D space would do.

[@Nulley0]

TLDR: Higher dimensional models are objectively better despite an order more huge, yet we don't have much real world volumetric images to train them on. These models, even though they were only trained on 2D images, they have a 3D model (depth info) that understands the world in 3D. Just like us who can assume depth with one eye closed. Training an entire 3D volumetric space generating AI with Time dimension, frees up unnecessary overhead for the AI to understand 3D to 2D projection, but we need a lot of "volumetric images" for them to train. This may not be feasible, Super intelligence is more likely to happen than this.

[@research417]

​@@Nulley0 It's totally possible, we just need better 2d images -> volumetric AI/software that take in multiple 2d images and recreate the 3d geometry. Even if that is far off, I wouldn't say it's impossible just for the fact that 3d data is easy to generate en masse. Imagine VR becomes more popular, or even just something as simple as an app comes out and allows users to scan in their rooms and capture the 3d data. Maybe this would be so they could decorate it with virtual furniture and preview what that would look like. Regardless, if it becomes as popular as TikTok, we could have over a billion different unique rooms in a matter of weeks. An unfathomable amount of data, because it was collected and submitted by users. Clearly we can see it's a possibility, although I think that route would be unlikely to create a public dataset because of the ethicality problems. I think it's more likely that a 'scan' the earth app/initiative would lead to that scenario. Either way, it's not as unlikely as you were suggesting. The bigger issue IMO would be the size of the data and the training cost. We'd need to vastly improve the general efficiency ML algorithms soon if we want future research to be successful.

[@hundredfireify]

I remember when I used to record music visualizations with winamp on my old computer and finding it really cool. We've come really fast

[@sanketvaria9734]

me: giving trolly problem to AI The AI: run over everyone with train instead. LMAO

[@galmud1508]

We in the future: AI, solve climate-change AI: Runs over everyone with a train

[@nangld]

Perfect demonstration of the dangers of AI

[@rickymort135]

AI in one year: split off the last carriage down the other path to get the remaining human

[@laviwastaken9845]

Multi-track drifting.

[@rubywhistler869]

All lives are equal, I guess...

[@DoctorMandible]

Appreciate you clarifying "free TO TRY". Too many paid and/or licensed products masquerading as "free"

[@Maxie962]

multiple accounts goes brrrrrrrrrrrrrrrrrrrrrrr!

[@Traf063]

What a time to become alive. - AI probably

[@MrRandomPlays_1987]

3:35 - WOW the elephant walking on the finger is one of the coolest, it's unbelievable, it's shockingly cool

[@Napert]

how nice of the ai to give her a high-vis vest at 0:16

[@Nulley0]

That legendary C4 meme, is now a video

[@Patralgan]

I tried Luma and I loaded a picture of an artwork for an album and it created a fly-through animation of it and I was shocked how consistent it was. The art has a cherry blossom tree with a swing and a river (and a wooden pier) by it and mountains on the background. Luma understood the relative positions where all the things should be in the 3D space.. Well, almost. Just very minor inconsistencies, which are not obvious. Very impressive

[@Qubot]

Runway just droped GEN 3, and it is even better

[@reifuTD]

I think I remember someone saying June was going to be lite with video generation models because of Sora and the time needed for everyone else to catch up, I have to say June is pretty lit.

[@test-uy4vc]

​@reifuTD IMO I feel that Sora is like the Dalle of video generation. And these other ones are like a Dalle-Mini. And in a year or two, we will get a "Dalle-2" of video generation. Also found that adding a dimension of time, makes the "images" or 2 dimensions really good. A snapshot of a sora video is way better than Dalle-3 could produce,and I wonder what adding the dimension of 3D space would do.

[@jimmykrochmalska3501]

@@test-uy4vc In a few years we'll be able to make 2k 5 minute long videos locally with a trashy cpu only, imagine being bored with no wifi then

[@jenkem4464]

@@test-uy4vc The dimensions of the subject in 2D image gen are already inside the model, we just need a way to coax it out better. You can already force it to output multiple angles of the same subject in the same image and you can tell it really nails the cohesion and form of the 3D...with some odd things here and there. It's a little difficult to get to that stage with just prompting but once you get a good seed it's pretty interesting to see. That's probably something they've been leveraging in order to create the cohesion we're seeing now in the videos. (just a guess)

[@blasttrash]

@@jenkem4464 how are the dimensions of the subject in 2d image already inside the model? just curious coz the 3rd dimension data is not encoded in rgb data anywhere right? its all up to imagination which we humans could do to a certain extent but even we humans are wrong at times like whats behind the cat? does it have a black mark on its tail? we cant say that with just image. With 3d model, we can rotate and check the texture etc.

[@xotekissu9938]

what a ( terrifying) time to be alive! thank you so much for uour ongoing coverage of these topics and your endless enthusiasm!!

[@patrik5188]

Wtf why terryfying? You mean best time ever to be alive

[@dangernoodle235]

I can just think of the endless stream of disinformation and fake news that can now be generated with such tools lmao

[@-dtuyhnjhggvjjjn]

@@patrik5188because corporations are trying to use it to crush the entire concept of art and ai might unemploy everyone but the ultraweathy and also potentially kill us all etc. There’ll be upsides too of course but it’s going to get messy and predicting the future is impossible

[@blasttrash]

@@patrik5188 coz we will loose jobs. already happening in many industries. Once these tools become standardized and easy to use more jobs will be cut and some roles might be deprecated altogether while cost of living rises. We wanted AI to solve the "DDD"(difficult, dangerous and dirty) problems while we could focus on interesting jobs like Art, Music etc. But now AI is doing the interesting stuff and we are left to do physical work like construction, plumbing, cleaning toilets etc

[@OrlOnEarth]

Aren't you aware of what all the ai forefather are saying? Or do you really just listen to the AI hype?

[@jenkem4464]

Well if the servers weren't already chock full...now they will be. lol. We really need to get some local options and hardware specs for these things. AI is quickly not becoming "open".

[@Dave-rd6sp]

It's hilarious that this drops just as Runway Gen 3 is announced. AI moves faster than your ability to produce videos.

[@H0b0sexual]

Not if you use AI to make the videos ;)

[@JayYu-lr4ro]

2 papers down the road: no more human drops: AI just posts videos, podcasts, and social media posts simultaneously!

[@oofusmcdoofus]

The "It is release day my dudes" image was funny lmao

[@MrCharkteeth]

Microphant

[@AKU666]

Now i expect "In queue" being more often than it's was...

[@laviwastaken9845]

"Generation is not complete" for ever.

[@rafaelsanchez5853]

R. I. P. Hollywood

[@rockumk]

I have made a lot of videos with Luma Labs Dream Machine. Basically most looked like nightmare machine. That said I appreciate their kindness in letting up create videos for free.

[@phase...8849]

It's like there's no time to rest! Amazing video as always ❤

[@stafomosakez]

I just tried Luma for the first time and used excerpts from my novel to see what scenes it came up with. This is super neat, man! Thank you. Looking forward to where the tech will be by the time I finish this final draft.

[@thehanke]

AND.. WELL.. OR.. ALL.. AND THAT.. Very entertaining delivery!

[@Juan-qv5nc]

Thanks for this video, fellow scholar Károly. Although very impressive, the scholars in the video were holding papers but not holding "to" their papers. Important difference that the system needs to learn.

[@azhuransmx126]

Finally we are entering in the Times of 2045 Technological Singularity.

[@blabby102]

This video totally broke the site. High demand and I've been queuing for ages.

[@RegahP]

So this is where all those videos of vines with extended endings were coming from!

[@Matthew_Fog]

3:05 ok that's funny

[@MrRandomPlays_1987]

There's now Gen 3 of Runway ML and it's super amazing and cool like on par with Sora almost, yet another competitor to Sora, we're in for a treat !

[@onlyyoucanstopevil9024]

WHAT A TIME TO BE ALIVE 😊😊😊

[@theftking]

I'm ready for Rufus' incredible return.

[@klaushermann6760]

Closest free thing we have to Sora Ai. Much better than runway models.

[@WifeWantsAWizard]

Apparently they added an "extend" button for clips to add 5s at a time this morning. Time to edit that paper.

[@willhart2188]

I have not also made a "Very Funny Animated Luma AI / Dream Machine Videos Part 2"

[@vivekpadia9901]

Imagine integrating such AI into games

[@The_Last_AI_of_Humanity]

Check out the Last AI of Humanity, "Level 2 AI Adoption is coming before AGI"

[@ChairmanHehe]

how can we create a truly open source version?

[@test-uy4vc]

What a video time to be generated alive! 🎉

[@Andrey-kh9ef]

2:23 the meme becomes funnier than it was 😂

[@wellwizard]

tested Luma with the Sora tokyo street prompt. Sora is definitely better, at least the demo video. But Luma is still amazing for the level of funding and resources v Open AI. Interesting times ahead.

[@markmuller7962]

Runaway just announced their new AI video Sora competitor!

[@mnstrz0]

And when everyone's super...

[@arandomguy46]

here we go again

[@JJFliesAndMakes]

"Hey AI, can you watch those 29 millions AI generated movies created yesterday and tell me which one is worth to watching?" New level for "too much information" era is here and I love it. What a time to be alive!

[@metakron]

Same for me brah i Always knew It came "the era of too much and infinite informations" and i love It so much 😊

[@GeekyGami]

"For free." But your only option to sign up is Google, which has your phone number, linking directly to your real identity.

[@GuiadoCasual-tl7mh]

superb! ive literately made a few sec ANIME, no slow, no deformation, its rlly incredible *-

[@Grap_Jelly4343]

You didn’t make it, the artists that the ai was “trained” off of did.

[@GuiadoCasual-tl7mh]

y but who cares no one can copyright me for that xD

[@Elidessinnn]

You didn't. You just typed a prompt.

[@SaumonDuLundi]

What a time to be alive!

[@Tanvir1337]

Not great but awesome. Very excited.

[@KosiiqsMusic]

I experimented with AI music videos for my UDIO AI music. Luma looks really promising!

[@gu9838]

its not quite sora level though theres still some lack of sharpness in some scenes and all but its getting there. its def a step up to other stuff like runway gen2 and pika for sure.

[@ingusmant]

You know the Escobar memes are video stills right? We don't need to animate them it was already animated...

[@Patralgan]

The extend feature is nice but it would be better if it would retain in memory what is shown in the first picture and the first clip so it wouldn't invent new things which shouldn't be there.

[@nutzeeer]

NVIDIAs physicalized AI will improve image and video generation fundamentally.

[@nientaiho7997]

Brain Dance is coming true

[@smetljesm2276]

AI: what a time to be alive!

[@jadusiv]

I’ve tried it. It’s pretty bad honestly. Almost everything I try looks terrible. Tried about 25 times and maybe one was ok.

[@asterlofts1565]

And Runway Gen-3 is in alpha too.

[@Saykator]

Hoppá, miért hallottam magyar nevet?

[@seraphin01]

To be honest, Luma is absolutely garbage as a text to video. But if you doing image to video then you can have some quite decent result if you're lucky

[@defaultHandle1110]

and another one was just released. Mid journey 3 ?

[@ChrisGuerra31]

Music videos are about to be cuh-razy

[@LifeAsANoun]

no spiders!

[@Pretty-blood-offical.856]

A LUMA IS CURSED

[@lordvadher4]

I tried the will smith eating spaguetti, but it did not turn out okay

[@Vurt72]

it's really crappy though, out of maybe 10 videos you are lucky to get one decent. in 30 generations maybe you have one which is good. I imagine Sora is very, very likely similarly cherry picked where maybe 1 video out of 100 comes out as amazing as in their showcases. Not that this is a new issue, or worse than expected, it's just how it's always been with new image/video AI.

[@boriskaragiannis]

YOU SOUND LIKE THE PUPPET IN FUR TV

[@Airbag888]

I tried it and.. it generated a nightmare lol

[@Axel_Andersen]

Syndrom: "when everyone is super, no one will be" ... eventually this will mean that creating this kind of videos as described by TMP her will be worth ... not much.

[@douglasteixeiradeabreu]

Gen-3 Runway ❤

[@gmlgml780]

Yeah. I'm pretty sure that this guy is generated, too.

[@itziknadel]

Are we alive?

[@killyGHILLIE]

imagine my dissociation just two papers done the line

[@Ponk_80]

Signing up is not free, you pay with your personal data.

[@tomoki-v6o]

You have your own .studios now

[@myelinsheathxd]

2: 24 new chick

[@BrianSantosF]

Is their website login down for anyone else? can't use it

[@petemoss3160]

2:27 lol

[@palnetalan2754]

Imádom hogy magyar vagy

[@LilBurntCrust99]

HOLYSHITHOLYSHITHOLSYSHUSHUIS

[@LictordeThrax]

I've tried two videos. Awful results. It WILL improve, though.

[@nadiaplaysgames2550]

Is it local ? No then it useless

[@Zsamoff]

I can't create an account! :O

[@raidtheairwaves42]

Llllmmmaaaooo!!!! 3:09

[@JohnDoe-ln8jp]

It really fails to deliver (LUMA). Yeah it has it's moments, but in general it is more like the first google deep dream, than say stable diffusion. The videos in their showcase are not representative of usual outputs. It does a little bit better with image-to-video editing, but still it is more reminiscent of the famous will smith spaghetti.

[@marnig9185]

In 2 papers down the line,is the new just one more lane❤

[@mstreich]

Why do AI generators get spelling wrong? By design or ? Even when I use quotes around the word, it messes it up. 1:12

[@lamborghinicentenario2497]

I think it's a limitation

[@cube2fox]

Because it uses tokens (groups of characters) as primitives, not characters.

[@mstreich]

@@cube2fox as a compiler developer, our tokens would include String that would know the entire string. But your point is taken.

[@ajs1998]

​@@cube2foxit has nothing to do with tokens. Tokens are perfectly capable of spelling things correctly. Image generation just sucks at enforcing structure. It diffuses noise and approximates the prompt.

[@cube2fox]

@@ajs1998 Nah bro, someone tested it with single character tokens. Somewhere on Arxiv.

[@jamaljamalwaziat1002]

Disgusting is like nightmare not creativity

[@Tiniuc]

5 seconds, and image to video? Yeah, that'll work! Can't wait to see what the furries cook up, haha!

[@axeldaval3410]

I tried...let's say it's like dalle 2, not do great

[@NoGu_167]

I feel pretty nauseous pretty fast watching AI-generated videos.

[@ajs1998]

Gotta say, I wish you wouldn't make so many videos about image and video generators. It won't be impressive unless it's using a workflow a human would. Humans don't diffuse noise, they use Photoshop or Blender to build a scene or animation and render it. Design and rendering need to be separated and until they are, this remains an unimpressive waste of time and resources.

[@3D_TUTS]

AI hype is declining..

[@AZTECMAN]

I've been watching this channel for years, but frankly I'm sick of the advertisements for AI startups and so forth. I came here for academia. Unsubscribing.

[@rericksen]

it's not boner pills/slim wallets/RAID SHADOW LEGENDS, it's literally useful tools for people doing the kind of work he always talks about, and always at the end of the video.

[@ThisIsntmyrealnameGoogle]

Adios

[@DanikaOliver]

That's academia, all of this came from research papers from people trying to advance computer vision. Academia doesn't stop only in research.

[@AZTECMAN]

@@DanikaOliver 2 minute papers, but 6 minutes - and no paper. Obviously it was optimistic to hope this channel would continue to give well digested summaries of actual papers.

[@AZTECMAN]

I really should have watched the video first [proceeds to drop papers clumsily]

[@PacômeHégésippeAdélardLadislas]

Just tried it. It's super woke

[@cestvraimenthorrible3465]

Can Ai fix the way he speaks?

[@MikkiPike]

unfortunately this one doesn't like imagination very much because any kind of fantastical character or idea is replaced with humans or humanoids talking so far. still lacking in imagination I think...

[@Trollguy76]

can you please stop talking like this

[@ShahabEslamian]

Haiper is better

[@0sliter0]

I love how useless video about sora were. As much so as there's another AI that was trained JUST LIKE sora drops to public before sora itself.... I hate AI at this point and we're all doomed....

[@boredtotears_]

first

[@SrIgort]

Congrats! 🎉

[@xxThat90sGuyxx]

Just say all the words in the same sentence!!!! Is this an AI voice? 2x playback speed seems to make you sound like you're speaking at a normal cadence, but makes it sound even more AI generated. Did. You. Learn. To. Speak. From. Reddit.?

[@lamborghinicentenario2497]

Quit saying scalars bro

[@claudiaxander]

It's a time to be alive! What?

[@telotawa]

I feel like it being "not as good as sora" is because openai probably cherry picked the best results to show publicly