A Deeper Look at OpenAI's Sora

OpenAI is on a roll. From ChatGPT to Dall-E to...Sora? Interesting choice of name. Regardless, the quality of the video generation I'm seeing here is insane enough to genuinely scare me.
Check out my Twitch, Twitter, Discord more at t3.gg
SOURCES
openai.com/sora
/ 1758249750909096142
• Smoother animation ≠ B...
/ 1755864359283093609
S/O Ph4se0n3 for the awesome edit 🙏

[@wlockuz4467]

When its publically available, the first thing we are obligated to do is create Will Smith eating spaghetti.

[@oxxal7357]

Thery are trying to ban the use of celebs in their AI which is arguably good.

[@bluegoatfanclickyes1825]

@@oxxal7357but not even eating spaghetti is sad

[@nolem5417]

@@oxxal7357Meme aside, I was hoping that was the case

[@hannibalrex2191]

Arguably lame ​@@oxxal7357

[@Airbender131090]

@@oxxal7357is dalle banned from celebs? Mj aloe them

[@gavrshchuk]

Waiting for netflix AI generated series by text

[@movieidiots5542]

Black mirror next season

[@entropywilldestroyusall1323]

Joan Is Awful incoming

[@Caldaron]

well, won't be that much worse...

[@karikeone]

Netflix’s proprietary AI is actually quite advanced and is used for a lot of things.

[@everyhandletaken]

Bring on the day where they generate specific content for us, so we don’t have to scroll for days.

[@radulotus]

Some people think AI is a passing gimmick just like NFTs were. Nope.

[@Malix_off]

NFT didn't make my productivity bump 3x like any other tool I used before

[@antman7673]

No, no crypto is web 3. Haha Lmao. Get rugged. Who would have guessed digital abundance would win over digital scarcity.

[@dv_interval42]

WHO is making these dumb predictions?

[@neociber24]

​@@dv_interval42 people that just look the hype around these tech trends and nothing more

[@simonfarre4907]

This is not AI.

[@peteredmonds1712]

13:11 Maybe that is just what homemade mobile phone videos will look like in 2056

[@frogery]

and giant people will exist

[@cfsouzajr]

beat me to it! ;)

[@amirhosseinahmadi3706]

Damn Sora was playing 4D chess...

[@pkolodz]

More like 2026

[@GoatedGuild]

Just wanted to say thanks for putting out such great content. It was one of the inspirations for me to start posting. You rock!

[@jason_v12345]

You're one of few KZbinrs whose actually added thoughtful commentary and not just described what we can all see with our own eyes with a few oohs and aahs mixed in

[@Tekay37]

16:00 that's one of the main issues I have when using AI creatively. It's easy to get AI write a scene of a chapter, but it's really hard to iterate on that specific scene and ask for changes, because the AI will always fall back into it's patterns half way through. It's like AI somehow does actually lose it's context and just moves along from where it started. It doesn't remind itself of what it was doing and doesn't re-check if it is doing what it is supposed to be doing. A bit like people with ADHS (like me) where a new impression or a new thought can send me miles away from what I was doing within a few seconds. But just reminding myself of how "Will Smith eating spaghetti" looked a year ago, I'm pretty confident we will see a similar jump of quality over the next year.

[@simonfarre4907]

You know why that happens? Because it's not AI.

[@celestemtz587]

The AI models 2nd Gen, like GPT, Gemini, Dalle, Midjourney and so on, doesn't have short-term memory, they forget every thing in the next instruction just like 80s pcs.

[@celestemtz587]

3rd Gen Will be able to remember who are you and how you treat them

[@Tekay37]

@@celestemtz587 I didn't specify, but I was talking about GPT-4. When sending a prompt, it also gets the most recent conversation as a context, and even with that context it has a hard time changing course because it falls back into its patterns.

[@clemenswirth]

For me as a filmmaker and visual artist, this is devastating - I'm really scared of the future

[@luisfernando5998]

Why it will make ur life much easier now u don’t need all that equipment to make ur content

[@ZotyLisu]

yeah, it will just enable creative people like you to produce more far more quickly, and the traditional camera stuff will still be needed for capturing real events and places

[@clemenswirth]

@@luisfernando5998 There is no need to hire visual artists like me when you can get it for free and in seconds - that's the scary thing..

[@Felhek]

Don't feel like that. These models have censorship. Some clients will need results that these AIs cannot make because of their policies.

[@fantasticube]

I am also scared as I am currently 15 years old and I don't know if I will be able to get a good job.

[@Kay8B]

What would be insane, is you could take a novel and feed it to AI and it would generate a movie for it. I would love that!

[@dazhoiden]

I'm actually working on this very problem. I have the beginnings of a platform that leverages various GenAI APIs for artwork, sound effects, music and narration. Right now with the state of the art it's still very labour intensive with countless retries to get content that is meaningful. With Sora, things will only get easier. I can't wait for this to be available!

[@thescatterpiratesquarepant7935]

In the future, you will go watch a movie in which AI replaces the lead with you, or your dad, or your GF.. or whoever you want, and you'll be the main character! That would be awesome

[@complexity5545]

CliffNotes on steroids.

[@southpaw9393]

here comes Lord of the rings remake

[@MountainViews90]

Imagine the ai version of harry potter being 1000x better than the ones produced by Hollywood. Could be for many reasons. Like having the actors do stunts not physically possible. VFX that would take 10+ years to make, longer runtimes with no need to keep tabs on film, optimized audio, etc.

[@ColinTimmins]

I really liked this video going in-depth. Really fascinating stuff. =]

[@JarrodSears]

Great video. Really enjoyed the detailed commentary on artifacts and difficulties in animation that I would not have noticed or realized how difficult they were to do well. Thanks. @OpenAI Get this guy a demo!

[@MegaloBlazemon]

Your analysis helped like, calm my mind. Something about all this has been screwing with my heard, like I'll watch Nature Footage that used to bring me joy & peace, but now my 1st thought is "Well it's probably not real, probably AI Generated" even tho I know it isn't. Like I can feel the wiring in my brain shifting to a place where I feel like I can't trust anything on a computer or TV is really real, and that's sort of scary. Like I fear the day where like, if there's a horrific event and someone shows me the news coverage on the phone, that I'll just look at them and go "Well, like, we don't know if that's real, it could be fake" I really think this is going to mess with people's mental health & sense of reality.

[@Airbender131090]

Look man, its hard but you never supposed to trust what is on tv or internet anyways. 90% of info out were is a lie. This ai thing will teach people to check, think and use critical thinking. You need to calm down and get prepared for future ahead. Trust me, your first reaction always should be “its fake till i prove its not” and not the other way around.

[@MegaloBlazemon]

@@Airbender131090 The average person is not going to think that way, they're not going to meditate on it and suddenly become a critical thinker, humans are reactionary emotional creatures. What you are saying it will teach people is a utopian fantasy that has a 0% chance of happening. And yeah, when we can no longer communicate and tell if what we're hearing or seeing is real or not, that's a problem, a huge problem. And you wanting to assume everything is fake instead of real is incredibly nihilistic, it's not a world humans can live in for very long.

[@AnalJhala]

this scares me a lot. Because now I can't trust any image, video or audio file. And you can't use them as proof in courtrooms as well.

[@CarpenterBrother]

Soon cameras will digitally sign every image or video they take to prove their authenticity.

[@m_hall]

That's a good point, that does seem like a necessary step to help ensure the authenticity of images and videos

[@h.hristov]

@@CarpenterBrotherunless the camera’s digital signature is compromised or the camera itself is compromised

[@sharkinahat]

Bane of all AI - misalignment. If only we'd have tools that let us mess with the latent space of the generators to tweak the output in a meaningful way, instead of rolling dice and hoping to roll a nat 20 on cohesion. Heh, maybe one more paper down the line.... ?

[@markhaus]

Not totally the same thing but we’re using more numerous, smaller and specialized models in a composition to try and improve among other things observability. First it was a way to save on training complexity while achieving similar results but we started realising it might also make it easier to observe and tweak what the larger composition does because we can control how the specialized smaller models communicate with each other. The difficulty is in making smaller models flexible enough despite their size and it's a pretty horrible balancing act, but it shows a lot of promise.

[@simonfarre4907]

Yeah, and after that paper they're going to say "It really is AI now", even though it won't be 😄

[@patricknelson]

What a time to be alive!

[@animanaut]

once you see it you cannot unsee. the woman in the first one switches legs midstride. similar to the threadmill guy later

[@timseguine2]

It is a common failure mode for these models. The much worse google model from last year had it too.

[@timothy6966]

Yeah, her left legs passes over her right leg and then becomes her right leg.

[@TC-ch8xu]

I dont see it could you timestamp pls

[@animanaut]

3:56 for example, looks like a shuffle step but if you go to the linked page you can see it happening midstride and also when one right foot is in the front, switches to her left and moves backwards again(4:18). can be noticed in pretty much all walking videos. human or animals. as noted in a comment above its inherent to the current model. it does not (yet) understand anatomy, its just very good at mimmicking moving images

[@jomito8784]

But the image generating models had these problems too and now they are gone

[@Fanaro]

Reality is getting close to game over.

[@uagrotema]

Thank you so much for the right time and right place👌

[@claudiocapitao3833]

I wonder the price per second/frame/clip of this? Maybe the 7 trilion dollars are to be able to properly scale this thing.

[@dazhoiden]

Yeah, I was thinking about pricing last night. I've seen various podcasts now with critiques on the Sora videos, and it does have many limitations (as wonderful as it looks). Based on this, I'm thinking pricing will be reasonable.

[@antheus_s]

It's amazing how far this tech has gotten in a relatively short period of time.

[@Zeragamba]

and scary

[@JEsterCW]

Short? This has been around for many years, lol if 20+ years is short then idk

[@Zeragamba]

​@@JEsterCW ya but these latest advancements have only been in the last 3 years

[@hombacom]

@@Zeragamba Maybe there is research progress that not reach clickbaity social media with a few demos and wild guesses about future

[@Airbender131090]

Its not even reletivly short. Its literaly short. From nothing to chatgpt and SORA in 20 months

[@jks234]

People already ignore verified truths and choose to believe discounted ideas, I don’t imbue these kinds of things with the “terrifying” power of influence that Theo speaks of. People have done this since forever, by the way. So yes, some people may be influenced, but they always have been.

[@HedgehogGolf]

Cool video, I agree with almost all your points. However, I'd like to point out that it's pretty easy to do screen replacements for the TV scene, and text replacement on the car. Especially the car. The text is always in shadow, so you don't have to worry about animating shadows. So the job is almost entirely just tracking the new text onto the car. Which I would be surprised if it wasn't trivial to do in DaVinci Resolve or After Effects, and still quite possible to do in Blender, with its new-ish corner-pin planar tracks

[@cherubin7th]

So next year we can do this with an open source model.

[@ShaneCodes]

It'll be interesting to see the first completely AI generate persona/actor that gets "famous". They could completely make up a person put them in movies, TV shows and commercials. Create a whole story for them to cater to specific groups of people. The next 10 years is going to be insane.

[@alastairtheduke]

24:57 Totally missed it lol

[@il804]

😂

[@il804]

Also with the grandma one. I think the main problem was that the candles didn't go off.

[@nfrankiksa4596]

and the direction of the flames, it wasn't consistent. @@il804

[@Kane0123]

That’s the next production milestone - chunking for re-prompting segments

[@BoldCG]

on the 1950s TV Screens example - I believe it could be possible to generate a kind of UV map-like grid on the screens and then update each part of the grid. This would be more like inpainting, a technique we will definitely see in AI-generated videos. The biggest issue would be how to handle depth in scenes. However, this could improve over time, similar to the advancements from GPT-3 to GPT-4. Video is somewhat different, but I could imagine it gaining all the capabilities that stable diffusion has, such as inpainting, outpainting, various models and styles, motion capture/posing, mixing, etc. it's impressive how fast and how good these things are getting.

[@andybrice2711]

Actually the Lagos clip is accurate because in 2056 phones will be able to fly.

[@BlueSkyCountry]

I am sure flying phones already existed since 2018. All you need is a quadcopter drone big enough to hold a smartphone. Not exactly useful though, as most hobby drones already have high resolution cameras for that purpose.

[@mathesonstep]

What would be really cool is if it could have a concept of layers so for example in the case of a bunch of TV's you could bring it into your editing software OBS, etc

[@Malix_off]

> AI blockbuster before 2027 > Cost 1/10 of an organic one > Never seen scenes and animations > "10/10 wtf" > Everyone freaks out about job securities > "yeah but we had to do many edits" > Everyone is calm again

[@leroysmith733]

Does anyone knows where we can put the prompts? they talk about whats on the page, and how to use mid journey but no now says anything about where on SORA should be, is it available ?

[@dwip7_7]

not available for general public right now

[@danieljohnson6481]

I wonder if asking for green screens displayed on the tvs would work -- making it easier to map your own footage

[@HoldingThisHandle]

I'm curious how it would handle cartoons or anime. Creating anime battles on demand would be cool.

[@bruhism173]

Attack on titan, I don't even want to make a different ending, I just want it animated with wits style, so AI can do it

[@KforKanishk]

Insane amount of data is required here! Wild times people! its Wild times!

[@fedorvlasov3336]

Whats the name of the plugin, he is using, the one which brings up a google searchbar anywhere?

[@patricknelson]

5:51 - There _are_ tools out there that will actually take a prompt and use it to edit existing video as you see fit. I heard of it recently but I can’t remember the name of it for the life of me, unfortunately.

[@ych3455]

I am curious how are they doing it??

[@EDIIIZ]

Soon some device will be able to read our thoughts and visualize them with this kind of tech.

[@jalenh371]

Great video dude.

[@rubemkleinjunior237]

Wondering if anyone's eyes felt weird when looking at these videos? I felt confused visually, through what's on the screen looks very realistic.

[@behailuseyoum9367]

How did you generet Sora's animation

[@thescatterpiratesquarepant7935]

1) short prompt usually give better and more HQ results in my experience. 2) the AI usually loves to add random watermarks since most reference images that are used to create the model are videos or images that are watermarked/signed etc

[@haithem8906]

shorts and 20s generated videos.... i hope that doesn't become a deadly combination...

[@wlockuz4467]

One thing thats most sketchy about it is the training data. To have this level of quality in the output you would need massive amounts of diverse data, when I say massive I mean petabytes. There is no way they had licensed every bit of it. Not only would that be expensive, but extremely impractical. So I am assuming a large part of the training data was just KZbin videos. I mean, where else would you get the same video, in multiple resolutions, multiple languages, captions, multiple genres and basically anything you could ask for. When you think about it, KZbin is just a massive unorganised training data if you have the resources. If you had a popular video on social media, then chances are you contributed to training Sora. Whether you like it or not. I would hope theres some regulation in place which prevents these companies from just straight up stealing the data to train their models or someone would strong arm OpenAI to _open_ about their data but with Microsoft behind them, its just wishful thinking.

[@ilovescience7655]

when it becomes open source , IT IS GOING TO BE CHAOS. I am not prepared for that yet

[@j946atFIVEFOUR88AA]

I've been messing around with Runaway AI to create videos, I cannot wait for this to get better and better

[@artzyally5020]

I been bawling since I got up....I feel like my time here is over, my jobs at risk now and Im crushed!

[@gabereell]

I agree, I'm about to go into post-secondary for 3D and now my parents want me to reevaluate my career choice.

[@sajithnuski3813]

When it available to use?

[@tazimrahbar7882]

I can sense video AGI. One of the worst time to be alive.

[@mohitkumar-jv2bx]

lol, I am not exactly what's the correct word in english for it is, but was that a low key knock at Prime at the start?

[@vaap]

if u mean it in like a negative way it could be a "diss" "slight" or "jab". if u mean it in a positive way it could be a "nod" or another word im forgetting

[@Zenteee]

Kingdom heaaaaarts lets goooooo

[@brennan123]

Am I missing something? I'm using the SORA ChatGPT plugin and it just generates images, not video.

[@jonkeren]

Its not open to the public yet

[@FUBAR_87]

In 10-20 years kids are going to look back at us and laugh thinking why would you watch a movie or play a game someone else made up. You prompt your own movie or game and you're in your own imagination

[@Felhek]

The grandma with the cake, she was not looking at it and she can't blow off the candle.

[@breakaway2x]

Still waiting for Star Trek's Holodeck to be real. We're getting close to it with videos and photos, but I want to be immersed into a virtual reality holodeck and interact physically with my created prompts LMAO.

[@kellymoses8566]

The papercraft video does not look bad. It looks amazing.

[@Nightstorm-2516]

Eventually instead of inputing text to get a image, you will be inputing directions for a movie, and getting whole fledge films, actors, directors, and scenes from a single sentence, and you could watch a movie, any movie, with any style and prompt. Wild Times!

[@Irrelavant__]

great, even more crappy movies on the horizon! truly wild times

[@yikan1107]

I’m not sure if this is new cuz stable diffusion has been been doing video . I didn’t realize this was such a revolution.

[@ParthaSarathylink]

For the AI voiced video's and still-image videos getting millions of views, for them it's a gold mine. As for the audience it's a galactic pile of trash. Soon we better standardize labelling these or something else

[@_evillevi]

I didn't know Theo likes Flume, that's so cool!

[@Fanaro]

I can see these uncanny videos becoming some sort of niche, a sort of vaporware-like thing.

[@Ramesh_Sharma_Jee]

This would be revolutionary, if this creates a real like video based on a script 😘

[@BlackMita]

Image sora using literal dream footage as an input.

[@drummermike5150]

My new rule....if I didn't see it happen right it front of me, it is fake until I prove otherwise.

[@benjaminpike]

finding out theo is a flume fan is such a nice surprise

[@exoZelia]

Just a heads a up, a long sentence doesn't mean its a run on sentence. Run on sentences are actually barely a thing, it was more of a short hand for teaching kids to not write long sentences

[@caerphoto]

Marcel Proust approves of this comment.

[@Celph]

so not available to the public> ?

[@edgardsimon983]

im sure they will make a movie where a guy is in an IA dimension and he need to find the coherent part to survive like in Matrix but this time with generative IA and they will use actual Generative IA mistake to scare u out, cause thats some serious cursed thing

[@nfrankiksa4596]

Bro really missed what was wrong in the treadmill video 🤣🤣

[@llmtime2178]

It's so weird to watch you immediately pick it apart for every little flaw as if that was insightful in any way. You missed the forest for the trees. You're supposed to look at where we were a year ago and see this and shit your pants because every little (and big) flaw you found will be moot by next year.

[@cfsouzajr]

It might not be that hard to do post-edit on that. Someone managed to create a gaussian splatting. With that, you can do pre much anything using Unity or the likes. kzbin.info/www/bejne/norWio1-ncqZnc0

[@oowaz]

i didn't even think of that, thanks for sharing

[@kasper369]

Ai genrated Anime ? Personalized just for me. Future is looking good

[@neociber24]

In the same way a single person can now make an entire music album, may be posible for a single person to generate entire 24min 12 episode series in a future. This will be crazy

[@darwindelrosario3785]

i wanna impromt :shopping on black friday with a voice. liked the AI commercials

[@oowaz]

i find funny that you guys are looking at this pointing out what looks odd, which of course it does, as if it's not glaringly obvious, despite the fact that the overall quality of these videos is beyond comprehension already, i was stunned when i learned about sora and i'm still trying to process this. if it didn't output anything incoherent at all i'd be concerned OpenAI already has AGI. a model would actually need to be able to fully understand the human perspective to fool us with every single generated video

[@d3stinYwOw]

About this 'danover' -> Simply they don't want to be sued, jailed to hell model powers that nonetheless. Bias check? Will they do that from both sides of the fence? ;)

[@marcindomanski5654]

These films resemble our thoughts. They are not perfect and full of details. This can be perfectly seen when you want to, for example, draw what you are thinking and you catch yourself missing details

[@deatho0ne587]

Can not do celebrities, but what if you are the celebirty and just think this would be easier than doing it yourself?

[@margaretisolan9189]

I already don’t like the dark side of this, although this is amazing. The propaganda that can be created with this is…

[@forderdrek8757]

6:05 this isn't the first apparent error. In the Tokio one at one point the woman suddenly switches legs (left becomes right and vice versa). Still really impressive though

[@AbhiShake-pl3cf]

Man, he talked about fps for forever but saying the word

[@Controvi]

its really amazing when you think about how our minds work here. When really seeing a video shot of real people we say "look at her, what is she doing" as in "what the actual F is she doing". We see that person as real. But now we have generative AI giving us a video and we still talk about the representations of people as real people like saying "Look at granny" or "What is the girl in the background doing". The mental aspects and psychological part of it is fasanating

[@renejotas]

25:02 did he not notice the backwards treadmill?

[@thescatterpiratesquarepant7935]

The clip at 17:00 would be a thing kids would listen to all day. Edit some Audio to it and BOOM

[@luisfernando5998]

AI blockbuster 2025 beats titanic and avatar adjusted for inflation

[@ashneilroy]

I like your hoodie

[@voidmind]

Maybe in 2056, cell phones will levitate without the need for drones, and that's how you will shoot home videos

[@kon-jakub]

26:23 people in the background are sooooo creepy. New movie genre: "AI generated liminality".

[@alastairtheduke]

18:19 what's the point of comparing this to copilot? Of course copilot is easier to edit, it's working with text. This is video!

[@henninghidalgo782]

At this point in time at least just regulate AI so it doesn't cause too many problems .

[@nyarlathotep2621]

Well, Sora is actually capable of editing videos, look at the last article on openAI's research index

[@Airbender131090]

Its capable of way more than people realize. And its just the begining

[@AngeloXification]

"This is the worst its going to be"

[@hehe-bm5gc]

stock footage sales 📉

[@andybrice2711]

One the other hand: Stock footage value as training data 📈

[@naughtiousmaximus7853]

Its over for realitycels

[@powerblades]

Just wait a year or 2 😅

[@flareragnarok]

if you want to play basketball you need to *become* the basketball the gods are being godly as they should

[@mbaxter22]

The porn implications alone are staggering. This is going to ruin our sex lives.

[@arismaruto8291]

I think we should be here to admire an attempt of new mind-blowing tech generation, not to examine and criticize a old-published product of some factory

[@ari613]

OpenAi, let @Theo try your tool.

[@alastairtheduke]

29:17 what the hell is Theo saying? Theo is generated by AI

[@user-op6uh3vx1j]

)))

[@JP-MOS]

All ai generation is based on massive amounts of data, preferably video with a fixed format and flow, and it is foreseeable that adult film actors may lose their jobs🙄

[@nikolaikalhof8364]

Just like everything else on the internet, this will definitely be used for an immense amount of porn.

[@complexity5545]

Its only getting worse for "hollywood" guys. Actors and Actresses are about to be apart of the "Days of Yesterday." We're getting closer and closer to the perfect Robot (or demi-god).