How to Create A Short Film with AI | Midjourney & RunwayML Tutorial

[@MDMZ]

How to generate consistent characters: kzbin.info/www/bejne/mKayepSqlLeipM0

[@thomasscott2426]

As someone who recently learned AI just 3 days ago, i can confirm that this is doable. Wrote my first script for a movie and AI helped me bring it somewhat to life. Its just a trailer but i plan i going all the way with it.

[@seouliteusexpat1424]

I would love to see the trailer.

[@chrismccathern4653]

which program did you use

[@thomasscott2426]

@@chrismccathern4653 I used about four different programs for what I was doing but mind you it does cost a subscription. It all depends on how serious you want to get into AI free stuff normally comes with limitations.

[@wmurray003]

Let us see it.

[@thomasscott2426]

@@wmurray003 my second version is being uploaded now.

[@shadrackkiban4716]

Use the prompt below: "I'm working on this cool short film project. My plan is to use MidJourney to generate a bunch of images. Then, I'll run those images through another AI called Gen2 to bring them to life. After that, I'll piece all those animated scenes together to tell a really captivating story. Here's the story idea: A 30 year old man who's stuck out in the middle of the sea during a sea storm, they finally spot an island in the distance. Here’s where I need your help: 1. Provide 10 simple comma-separated keywords describing the main character; what they look like and what they're wearing 2. Develop the story further and break it down into different shots, for each shot, provide 10 simple comma-separated keywords in this this structure: - Camera angle and position - Action, environment, time of day, lighting. - Don't include: Camera movement, sound."

[@Davindersingh-bo6ek]

Why haven't this comment pinned?

[@theycallmelucky]

here's where i need your help: 1. provide 10 simple comma - separated keywords describing of the main character; what they look like and what they are wearing 2. Break the story further and break it down into different shots for each shots provide 10 simple comma - separated keywords in this structure - camera angle and position -Action, environment , time of day, lighting -Dont include: camera movement sound PROMPT might help.

[@AMidnightsTale]

Your a champion for this dude!

[@Inspired_JG]

Solid feedback.

[@molugusatyapriya2]

Excellent how-to video for creating short films with RunwayML and Midjourney! The manual provides insightful analysis and useful suggestions for utilizing AI in filmmaking.

[@anawakoff]

Wow, I'm really impressed. I tried out some of the AI processes and tools you mentioned to make a short fantasy film. But I have to admit that your work is on a whole other level. Congratulations

[@MDMZ]

🙏

[@PHANTAZMA]

Now with Sora AI, it's about to get real crazy! Great Video! 🔥🔥🔥

[@MDMZ]

Can't wait to use it

[@zeeesea]

Is there more information about the release? Btw this was an awesome video@@MDMZ

[@MDMZ]

@@zeeesea it's probably gonna take a while before it's available to everyone, I can't wait

[@akinyiyvonne3512]

DONT YOU THINK ITS GOING TO TAKE OUR JOBS AS FILM MAKERS @@MDMZ

[@jrod4God777]

@@MDMZ how do you add voiceover and text to speech?

[@chaos_visuals]

it's absolutely game changing for creatives who want to try out putting their ideas to life💯

[@LabyrinthofLogic]

This is actually insane. I threw and image in Gen 2 and I about hit the floor. The future is now.

[@MDMZ]

you can use rotoscoping and composite in post

[@oleole4340]

Man, as soon as consistency in AI generation reached the whole cinema industry changes drastically

[@MDMZ]

I agree

[@S9universe]

maybe implode

[@AICLASSROhM]

underated comment. none of the AI developers are focussing on character consistency. even the same seed value doesnt do justice.

[@patrickt6642]

Has long has its not woke and a good story that's what matters.

[@veilofreality]

more than consistency I think the biggest problem is getting the AI systems to understand the borders between character and elements in the surrounding

[@theprophetminstrel1597]

I was just thinking about this technology last night.

[@imagineartsstudio]

Dude is a Super-Genius! Thanks!😂😂😂

[@Toolgdskli]

The ultimate AI power: you’re able to automatically convert a novel into a movie.

[@fortunato1957]

I'm born 1965. Considering the evolution that only I have witnessed, I am sure, we have a good chance to do this locally at home in some years.

[@MDMZ]

You can now create entire films using just a computer !!🎥💻 The first 500 people to use my link will get a 1 month free trial of Skillshare skl.sh/mdmz01241

[@Hammy5641]

And what's the maximum duration for these 'films'?

[@jacksan-ck7xo]

Thank you❤

[@Inspired_JG]

One additional recommendation with midjourney is to use the style reference (-sref url). I have a short on my channel showing the results for reference.

[@MDMZ]

great tip, I think it's better to use cref, since sref is more for style referencing

[@unfltrdpassion]

is there another video on consistent characters? seems to be the missing piece for really bring this all together

[@MDMZ]

try to be as specific as you can in the prompt, describe your character in details, it helps

[@conhuir]

Runway ML is still king. Pika was a major let down! Great tutorial dude

[@PlayOfLifeOfficial]

They recently put a pay wall on pika, it blew my mind ahah

[@gu9838]

not bad at all runway gen2 CLEARLY has limations but if you can work around it you can make some cool stuff! BIG TIP get unlimited if you can afford it at least for the first month or so. that way you can produce unlimited videos without credits (can wait till near the end to use the credits before they reset) and have all the time you want to learn how to use the tools and understand its limatiions and work around it helped for me!

[@MDMZ]

thanks for the tip, good things is that all these tools are getting better and better, sometimes I share my opinion and one month later, it's no longer relevant 😅

[@TahaSumra]

How to get consistent character looks. That part you missed and that's the only thing I was looking for

[@MDMZ]

I have another video on that

[@TahaSumra]

@@MDMZ Link please. I searched but not able to find. Thank you.

[@lym-ro8ws]

Great video as always! I’ve been using Stylar for a while now, and I’d love to see how you think it stacks up against Midjourney.

[@MDMZ]

I haven't tried it myself to be honest

[@solipedia]

Thank you so much. valuable information

[@MDMZ]

Glad it was helpful!

[@smeech5177]

Also, I think utilizing Adobe After Affects could work tremendously if done right for movement but idk

[@mauvaissinges5254]

I'm currently wrking on a short animated story for children. The difficult part is consistancy between shot, poses, the character are never the same. More easy with realistic style

[@MDMZ]

it's true, you might be able to get more consistency if you're prompts are more detailed and descriptive

[@TomHimanen]

Could you make a tutorial about doing all this with ComfyUI? I think that MJ and RW do nothing that ComfyUI can't do. The most difficult part is imho video to video, if needed. For complex movement I think it is definitely required with consistent characters and consistent background.

[@MDMZ]

I do have a tutorial on how to animate images with ComfyUI

[@TomHimanen]

@@MDMZ At the moment, animating images with Comfy seem to morph and flicker more than for example Midjourney-made img2vid outputs. There are some opem source models that seem to avoid morphing when you are patient and lucky, but I haven't seen them being used in ComfyUI. I'm not sure if they could be used in Comfy with custom nodes. By the way, I love your tutorials. I have followed many of them and joined your Disco server yesterday,

[@TomHimanen]

@@MDMZ I guess model such as Genmo AI Mochi 1 would be required to prevent morphing and flickering from happening. But haven't seen any way to use it in ComfyUI.

[@chamindagunasekara97]

Amazing🥰😍

[@ShaqShalom]

Thanks. What is the best combination of tools as of today, to generate AI top videos?

[@MDMZ]

I'm working on a video about that

[@vishnu6729]

nice video❤

[@MDMZ]

Thank u 🙏

[@amang39]

Hey Buddy! Thanks for making this video. It’s helpful. Can you please share the prompt which you used in ChatGPT to describe the characters etc ?

[@MDMZ]

hi there, it's linked in the description, available for free on my patreon

[@AbdulLathief2090]

BRILLIANT 🎥

[@prernasharma5710]

Awesome!

[@hussainrabeea6017]

this is fantastic tutorial. thanx all best please keep going

[@MDMZ]

glad you liked it

[@TheBlessingReport]

Great job

[@RacherW]

How it is possible you have such good quality of this videos? My videos after editing photos in Runway ML looks like 480p.

[@MDMZ]

not sure, I haven't done much beyond what's covered in the video, maybe you can try upscaling your video using Topaz video AI

[@RacherW]

@@MDMZ Thanks, i'll try 😁

[@Mrc.182]

Adorei o vídeo. Vou começar a usar no meu canal. Sou do Brasil obrigado pelas dicas

[@moraissblessed]

marcha

[@rafatenangel]

Where can I watch the final result?

[@MDMZ]

just added to description

[@cafe_dastan]

Hello, thank you for answering my question When we create a character with AI , how can we make the photos with the look we created in the first photo in the next photos. That is, don't make the character of the story with a new look in every photo

[@MDMZ]

Check the pinned comment 😉

[@digital_magic]

Thanx for the great tutorial 🙂

[@MDMZ]

My pleasure 😊

[@sameeramin822]

Love your videos ❤

[@vlada5358]

how to make sure that the same man appears in all pictures? It constantly generates different ones.

[@MDMZ]

I have a follow up tutorial on that coming soon

[@vlada5358]

@@MDMZ great 🧡🧡🧡🧡

[@08tothe19]

Is it possible that I put the real person's image into the CHATGPT or Midjourney, so I make some shots based on that person? I am currently working on a new short film, and I wanna make some footages in the film by AI imaging technology.

[@PopCultureCollage]

I think you can use Midjourny. Just upload the reference image to midjourney. Copy the image URl (right click image to do this after uploading it). Write your prompt, maybe describe your person, then write --cref and-paste-the-link-here Good luck. :)

[@MDMZ]

which AI voice ?

[@ondrej73]

still getting an error --------------------------------- !!! Exception during processing!!! Error(s) in loading state_dict for ImageProjModel: size mismatch for proj.weight: copying a param with shape torch.Size([8192, 1024]) from checkpoint, the shape in current model is torch.Size([8192, 1280]). ---------------------------------- any idea? thanx

[@AIHahb]

helpful video

[@MDMZ]

Glad you think so!

[@StoryInspirations]

One question: images and videos generated by free AI tools such as midjourney ....monintizable in KZbin?????? Pls reply

[@degreatech]

I love you bro

[@rashadcolvin3518]

This is dope!!

[@aDAMoscar]

You said you will link the comfy ui work flow! I cant find it in the description 😢

[@MDMZ]

pretty sure I haven't, because I didn't use comfyui in this video 😅

[@nonstopvibes3493]

legend boss

[@explorewithaera8110]

how can I always keep the same character? It always change my main character to a new character?

[@MDMZ]

check the pinned comment 😉

[@explorewithaera8110]

@@MDMZ Thank you for the reply!

[@Mmmmk9797]

How much do I have to pay to use this effect?

[@Shhvisualcontent]

I did a video a week back.. it was a guge disposable cup flying.. when the cup fly then textture of the cup changes. I asked many people all they said was something to do with motion ..but did not explain clearly.any solution?

[@MDMZ]

it's really hard to tell, there are many factors involved, try to experiment with different settings

[@Shhvisualcontent]

@@MDMZ thanks

[@micheleastolfi]

HELP: How is it possible to be able to recreate the same protagonist but in different shots and scenarios? Thanksss!!!!!!

[@MDMZ]

YES! I have a tutorial on that coming tomorrow

[@spinfoamcanvas2823]

can u add dialog scenes in midjourney

[@alihamza-nh2ft]

CAN we generate motion vidoe by Using only Mid journey?

[@MDMZ]

Midjourney doesnt generate motion

[@healthwealthgurus]

Hi how do you ensure a character generated in mid journey remains consistent in each generated image. Ie they look the same ?

[@MDMZ]

you can use a face swap tool after selecting your images, and having detailed prompts with consistent character description helps a lot too

[@KDG2425]

How to make charcters consistent when generating images

[@MDMZ]

you can describe the character in details in the prompt, it helps

[@rrikisolmain]

Short question: in paid subscription of any of ai tools! Do you have The Rights to sell them (image/photos) in stock Photography?

[@Gamervidsman2000]

No

[@spinfoamcanvas2823]

how do access the prompt you use in the ur short film...

[@MDMZ]

everything is in the description

[@tallescarvalhofilmes5040]

I can´t acess the link of the prompt. Can you help me?

[@MDMZ]

what happens when you click on it ?

[@philipdratcliffe9648]

I've tried runway and watched the pros do it too and honestly, it fucking sucks with certain things. With human movement especially facial and eyes it fails every time without fail. Sora and veo will leave this thing buried

[@Rohittarkar175]

Can i make it 15 min longer?

[@MDMZ]

you can animate as many scenes as you want therefore making your video longer

[@Seth_hahaha]

thanks bro love u 3000

[@MDMZ]

Enjoy

[@imgnryvansh]

Sora will change the world forever.

[@MDMZ]

I agree

[@smeech5177]

Am I able to use a seed for midjourney from NightCafe?

[@MDMZ]

you can try :)

[@sajinscartoonvine]

I tagged you in my video in my channel that i create after watching your short film... You may not see....

[@MDMZ]

Great job!

[@sajinscartoonvine]

@@MDMZ Thank you

[@doyouknow1022]

Can we generate same face for all scenes The main issue with AI is, it will generate different different images for different scenes

[@MDMZ]

I'm planning to make a video on this

[@doyouknow1022]

@@MDMZ please make

[@SustainabilityJobslist]

i like the way you talk.......

[@enricociacchi94]

Is it free video ai gneerator?

[@joelgarner667]

Do you take on commission projects?

[@MDMZ]

Check the related video for full workflow

[@jmbeats_prod]

Is there a video of yours that teaches you how to use your own video and use artificial intelligence and make another video with a moving background?! For example, I want to take a video of myself with my face and then play it on the A.I. and put a moving scene, where in my eyes it shows a city with buildings. When I open my mouth, smoke comes out and in the background there is nature full of animals flying and walking. I want to make a video like this for my album.

[@MDMZ]

that's to specific, might as well use traditional editing tools, lots of masking, rotoscoping, and you can use AI for certain parts, it's even more fun that way

[@listentobaladu]

Looks decent but it is not even close to a film or even a film strip, and could have been produced in 1980. I hope there is some decent AI movie programs coming soon - this wait is way too long.

[@dujohngilatar]

How to get consistent pictures

[@MDMZ]

make sure your prompts are as specific as possible

[@yashgood4481]

where can we see the video u created?

[@MDMZ]

I uploaded it as a short

[@opianist4]

Can I use sounds the movies ?

[@MDMZ]

not sure if I understand your question

[@Niveditaaaaaaaa]

Please make a video on same topicbit for free ai generators like leinardo, genmo please

[@lunazamoraart]

This is wonderful but other than being super creative, how is it useful?

[@MDMZ]

creative IS useful, we'll see this and other similar methods implemented in commercials/movies really soon

[@DemocracyDecoded]

I just saw an AI generated video, that's 10 mins long, with the same characters, is it fake or real? It looked so good. I thought it was animated, i was like i need to see a 10 mins AI generated video so i came to look here, i couldn't find any AI generated video like it, i think its animated. So confused.😢

[@MDMZ]

it's possible that it's AI, what makes you doubt it ? is it the realism?

[@mezoadel3211]

Like it

[@MDMZ]

I'm glad you like it

[@StoryMingles]

Is this possible to do on IPhone?

[@MDMZ]

hmmm, time to test some stuff 😁

[@zytnpltr]

Yeah, I've watched all these types of videos and It's still nearly impossible to get the consistency to really flow. Makes it nearly impossible to tell a story with a fluid character. Even when using the same descriptive prompts for each shot, adding the new scene, it's practically a new character each time lol. This will be killer in a few updates but for now.....sigh

[@MDMZ]

I guess you can try to use some face swap AI after generating

[@tmx680]

where´s the movie?

[@sanjayt4862]

System requirements??

[@MDMZ]

no specific system requirements for midjourney, Gen2 works on the cloud so it's irrelevant, your best bet is to give it a shot

[@777dot77]

Can you use your own photos and videos for A.I video editing? If so what sites and programs can you use?

[@andrewrozhen513]

Too bad the tutorial doesn’t cover the most important thing - how to make characters consistent.

[@MDMZ]

that's where prompting comes in, try to be as specific as you can with the description and use it in all your prompts

[@andrewrozhen513]

@@MDMZ thanks for the reply

[@cheddartheater]

Great info and pace, but that AI voice is a nightmare.

[@mayurpandya2237]

how we can create same face image in every shot?

[@varunsiriah1user557]

Biggest question in AI film making ever Consistency

[@MDMZ]

you can guide it as close as possible with prompts, I've shared some tips in the video

[@thewestindianboy]

wow

[@lukewilliams7020]

This sadly is not a film just a series of basically still images

[@MDMZ]

These tools are only gonna get better and this is the core principle of it, just like how the first flight ever only lasted 12 seconds

[@sixgunguerrilla]

@@MDMZ The first flight was still a flight. These are drawings with a zoom effect.

[@sedisdead]

@@sixgunguerrillamay not be a full film but definitely can see how it would be useful when making a film. Maybe for a specific short scent or something like that

[@RichardsShortHorrorFilms]

@@sixgunguerrillaExactly! It’s maddening talking to AI enthusiasts.

[@sniperkota]

IA video

[@Hammy5641]

How do you get away from that cheesy ‘Top Gun: Maverick’ look/aesthetic & get something, overall, more classy - like Hugh Hudson’s shots of Istanbul - at the start of ‘Midnight Express’; or, to use a contemporary example: ‘Baltimore/Rose’s War’ (Imogen Poots)

[@MDMZ]

you can try to specify that in the prompt

[@germanmairen]

No matter how people try to sell this. The end result is rubbish

[@RasoulSnappQ]

It was not a short film! it was a photo video clip.

[@MDMZ]

You can use the same techniques i used here to create any sort of visual story

[@dishaa8691]

the midjourney discord version is also paid....????

[@MDMZ]

Midjourney is a paid service, totally worth it tho

[@dishaa8691]

@@MDMZ fine sir

[@xXTeamaXx]

Can I put my face in the video and how?

[@MDMZ]

use a face swapper

[@xXTeamaXx]

​@@MDMZ What software do I use?

[@강진매엔맨]

No free

[@prajmipare2493]

Disappointed did not see the end result.

[@MDMZ]

It's in the intro

[@prajmipare2493]

@@MDMZ oops, sorry, Thank you.

[@kanohane]

That man looks 45+, not 30....😭

[@MDMZ]

Too much time in the sea do that to you 🤣

[@OvenGlobalDomination]

Thats basicly an animated comic overview or a really simple basic anime making tool lol

[@NGW_Studio]

Personally i use pika to animate my short stories but runway so expensive

[@MDMZ]

Glad it works for you

[@JoshForet]

less attempts with runway makes it about the same

[@bluetreacle]

i'm not sure it'll ever work. it's like eating sugar free food. it's not the same.

[@MDMZ]

did you try?