Bring Images to LIFE with Stable Video Diffusion | A.I Video Tutorial

[@MDMZ]

The future of video seems WILD 🚀 ThinkDiffusion: bit.ly/3PoCkoN - if you can't see the manager after installing, make sure you have the latest version of GIT and re-install, if it persists, retry the installation methods here: civitai.com/models/71980/comfyui-manager

[@asialsky]

If you want longer videos, just save the final frame and run it again as the new source image. It may take a few attempts to get camera motion that doesn't make you sick, but it'll get the job done.

[@MDMZ]

great idea, but there might be a visible cut, did you try it ?

[@VinylSolutions]

@@MDMZdepending on the output it may or may not work from outputs we've seen on various image to video/gif creations.

[@drimscape]

i tired of this tuts where no one tells how to control camera movement. u can use online version it has camera control button (zoom in, wiggle etc) but no one tell how to do it locally

[@ZakariaNada]

Your tutorials are much easier to follow than most KZbinrs. Keep it up! But if you have time, you still need to do the controlnet animatediff or any video generation method using controlnet.

[@MDMZ]

Noted! and thanks a lot

[@apdirectedit]

BIG BIG BIG FACTS!!! FULL SUPPORT INSTANT SUB'D!

[@DJTide]

ty for the workflow! it really saves me alot of hassle having to use other websites just to animate my images!

[@MedAmineTN]

The best person to explain the montage method in A.I

[@MDMZ]

Thank you bro

[@timoi7777]

Bro you are the best tutorial person I've found. Thanks

[@MDMZ]

glad I could help

[@romeoottaviani5325]

hello, great guide. Congratulations. But I have a question, if for now the maximum limit is 5 seconds, how do some people animate the models that run on instagram? Just faceswap?

[@MDMZ]

I think you are referring to video2video, I just posted a video on that, this is img2video

[@romeoottaviani5325]

@@MDMZ tks

[@Thomas_Leo]

Very concise tutorials. All your tutorials always help me. 👍👍

[@MDMZ]

glad to hear

[@holly11111111]

THE BEST TUTORIAL!!! THANKS THANKS GRACIIIIIAAAAAAAAS 😍😍😍

[@MDMZ]

Glad it was helpful!

[@Morri91]

What ever the others are. You re allways next level and complete describer. Thank you brother❤

[@MDMZ]

this means a LOT 🙏

[@shireensingh2834]

a question- do we really have to use this complex method (comfy UI) like you have done for other things as well. or is there any other less complex way to do it? THANKS

[@MDMZ]

other things such as...?

[@deepinworld1]

Hey I don't have pc Can you do this on mobile especially in moon valley ai

[@MDMZ]

I've actually covered this topic in the video

[@matt_fpv]

Great tutorial, thank you! Do you know a way to upscale the video directly in comfyUI? Not everyone has Topaz Video AI, its quite expensive...

[@MDMZ]

there are ways to do it with custom nodes, I covered one example here: kzbin.info/www/bejne/oZ69ZoZojZponLcsi=ehay3olnLfJckLUN&t=547

[@A1Ch3353]

Hey good tutorial. Your custom workflow causing error and other one runs fine. How to fix this?

[@MDMZ]

what kind of errors? can you share on discord ?

[@Saidgarciac]

Could you help me with the installation process of img2vid for Comfyui to run it from stability matrix?

[@JefHarrisnation]

Great and easy tutorial, and thanks for the workflow template. Though I'm not able to find the Video Combine node.

[@MDMZ]

did yo utry installing it from the ComfyUI manager ?

[@jvtosiartist]

Thanks again!!! Great tutorial

[@MDMZ]

Glad you liked it!

[@jvtosiartist]

@@MDMZ Thanks to your help, I just tried Think Diffusion, it is very good, it is very fast and easy to use. This is what I did: kzbin.infoGy-HeW3Stj8?si=sc1i8Z6qiuACuC9V

[@hosseinahmadi1855]

is it possible to control the result of the video by giving some prompts? And another question. Is there a model to produce transition between two images/frames (I have frame images, but I want to turn them into a video and frames are not very close to each other so it cannot be used in conventional video editing software)?

[@MDMZ]

I guess you can add prompt nodes to have more control, but I'm yet to try it myself, you can try using RunwayMl instead. I don't have an answer for the second question, sorry :/

[@ruggio.o]

I'm currently using stable diffusion video on Mac m2 pro and I have the same problem with the ksampler... in particular this one: "Conv3D is not supported on MPS". Do somebody know how to deal with it and fix the problem? It would be very nice if someone can help me 🙏🏻

[@MDMZ]

running on MAC can be tricky sometimes, looks like a PyTorch issue, you might be able to get some help here: github.com/cocktailpeanut/mac-svd-install Please let me know if that worked If it persists, I would definitely try an online solution instead

[@spacemultimedia2979]

Can you do this on the normal Stable diffusion? With the normal UI, I hate this type of UI with tons of modules everywhere. If I installed comfy UI locally, would it be a separate ui from the normal Stable diffusion? It wouldn't launch the same for sure. Would that mess up my current SD local install?

[@MDMZ]

this won't interfere with your local A1111 installation, I highly recommend you give it a shot, it looks complicated but it's much more flexible and gets easier with time

[@reviewflicks]

I am looking for the video combine node, cant seem to see it anywhere. Need it to change output file type, any help? cheers!

[@MDMZ]

I believe it's been changed to another node, you can use my custom workflows from the description, those still have it

[@AbdulKhamis]

very useful! Thank you!

[@illuminat_empire]

Really good content 👍

[@MDMZ]

Thank you 🙌

[@ertezsssz]

is there a way to add a Clip Text Encode module between the SVD_img2vid_Conditioning and the KSampler to add positive and negative prompt to have more control with the camera movement ?

[@MDMZ]

interesting, not sure about that, but will look into it

[@ertezsssz]

@@MDMZ thank you very much, i think it could be an even greater tool to be able to influence the animation with prompts, not sure if its possible or not or even if the curent models are able to do that

[@MDMZ]

@@ertezsssz I remember seeing motion models that allow that, but you can also try including movement description in the prompts, it usually helps

[@ertezsssz]

I searched on civitai without finding one

[@SefaKaratekin]

I get this error. Does anyone have any idea about how to fix it? Prompt outputs failed validation: Exception when validating node: VideoCombine.VALIDATE_INPUTS() got an unexpected keyword argument 'frame_rate' VHS_VideoCombine: - Exception when validating node: VideoCombine.VALIDATE_INPUTS() got an unexpected keyword argument 'frame_rate'

[@luqaszoq]

I had the same problem and fixed it by Update All in Manager.

[@SefaKaratekin]

@@luqaszoq It worked for me, thanks a lot for your help, mate!

[@tendest1]

Hi dude, is there any method to increase the length of the generated video?

[@MDMZ]

not that I know of :/

[@simonhick9124]

The video Combine does not upload the video, it has an output that says filenames, i dont know what node I should use to see the video

[@MDMZ]

you mean the video is not being saved to your output folder ?

[@nachoquiroga307]

con una laptop se puede hacer? tengo un i5 de 10ma generacion y una rtx 3060

[@parscrypto]

Error occurred when executing SVD_img2vid_Conditioning: 'NoneType' object has no attribute 'encode_image hello how to fix?

[@MDMZ]

hi, you can head over to discord for help

[@lightndreamsmachine]

love this video

[@prabuddhagupta519]

I installed the comfy ui manager, but the option isn't visible there. i updated the comfy ui but still the manager option is not showing. can you tell me what to do?

[@MDMZ]

make sure you have the latest version of GIT and re-install, if it persists, retry the installation methods here: civitai.com/models/71980/comfyui-manager

[@sameeramin822]

Awesome bro 😎 big fan ❤

[@michelingesoft]

What requirements are neede to run this locally?

[@MDMZ]

you might be able to run this on low settings with 8GB VRAM

[@TheBeastiaryPetshop]

followed all the steps but i get Prompt outputs failed validation ImageOnlyCheckpointLoader: - Value not in list: ckpt_name: 'svd.safetensors' not in (list of length 21) any help?

[@MDMZ]

have you downloaded the svd model? make sure it's in the right folder

[@Thomas_Leo]

You have to actually select the downloaded checkpoint from the list. The default name doesn't match the one you downloaded.

[@AymanSaeed-c5m]

can I make loop animated pictures, or what called cinemagraph with AI?

[@MDMZ]

I don't think there are ways to make it look so seamless using these tools, unless there's a method I don't know of.

[@AymanSaeed-c5m]

@@MDMZ Thank you.

[@Samuirobotics]

I have a problem like this% ERROR:root:!!! Exception during processing !!! ERROR:root:Traceback (most recent call last): and here goes long list with the cherry on top - RuntimeError: MPS backend out of memory (MPS allocated: 15.61 GB, other allocations: 2.90 GB, max allowed: 18.13 GB). Tried to allocate 562.50 MB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure). Prompt executed in 3701.64 seconds Appreciate for any help. MacBook Pro m1 16GB

[@MDMZ]

plz check the pinned comment

[@andresfc435]

incredible, really helfull ❤❤

[@MDMZ]

Glad it helped!

[@sinc356]

When I queue prompt, my problem is Cuda out of memory how to fix that?

[@MDMZ]

it means your VRAM is a little low for it, did you try setting a lower resolution ? and maybe a smaller steps number

[@sinc356]

@@MDMZ how to set lower resolution?

[@kakashi99908]

Can you do this with stable diffusion too?

[@MDMZ]

this IS stable diffusion

[@muhammadsyafiq1004]

is it possible to run this with laptop? Seems load heavy.

[@MDMZ]

It will depend on the specs, you can definitely give it a shot

[@socialsculptmedia-vs8jm]

Doesn't work 'NoneType' object has no attribute 'encode_image' "How to solve this?

[@MDMZ]

which node is showing in red when yo uexecute?

[@Draqee4]

am facing this issue in local device Error occurred when executing ImageOnlyCheckpointLoader: Error while deserializing header: HeaderTooLarge File "K:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "K:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "K:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "K:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy_extras odes_video_model.py", line 21, in load_checkpoint out = comfy.sd.load_checkpoint_guess_config(ckpt_path, output_vae=True, output_clip=False, output_clipvision=True, embedding_directory=folder_paths.get_folder_paths("embeddings")) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "K:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\sd.py", line 498, in load_checkpoint_guess_config sd = comfy.utils.load_torch_file(ckpt_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "K:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\comfy\utils.py", line 15, in load_torch_file sd = safetensors.torch.load_file(ckpt, device=device.type) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "K:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\python_embeded\Lib\site-packages\safetensors\torch.py", line 311, in load_file with safe_open(filename, framework="pt", device=device) as f: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[@MDMZ]

can't say for sure, but it's possible that you are downloading the wrong files, make sure you get the correct models and place them in the right folders

[@Draqee4]

@@MDMZ yeah i was doing it wrong now it s working

[@kaleabspica8437]

cant find the manager option

[@MDMZ]

check the pinned comment

[@lilillllii246]

is it possible text prompt?

[@MDMZ]

yes, I've covered it in the video

[@leylauk]

Gen 2 is easier?

[@MDMZ]

it definitely is, but to me SVD gives better results

[@ThaMoonwalkerFSDBetaChannel]

is this only for PC?

[@MDMZ]

you might be able to make it run on MAC, dome people did

[@decanecaudeoscar6961]

I am the only one deleting all his souvenirs to get enough space for all these models ?

[@MDMZ]

the struggle is real!

[@jamiemunsey]

Is anyone else getting "500 - Internal Server Error" when trying to upload images? Or know how to fix that?

[@MDMZ]

hi Jamie, might need more context, does this happen when u try to generate ? or literally when u browse for an image and try to upload it ? are u using ThinkDiffusion or running locally

[@jamiemunsey]

@@MDMZ I get that error when trying to upload an image! Running it off Google Cloud (think that's the only way, since I have an Intel Mac?)

[@MDMZ]

@@jamiemunsey I see, not sure how hosting and image loading works over there, can u try loading different images from different locations and see if any of it works? othwerwise you can check other services such as thinkdiffusion, if it's cheaper than google cloud it might make more sense to use that instead

[@cucciolo182]

wow any updates?

[@MDMZ]

Luma dream machine is now better than this, at least in my experience

[@ruthskiba-otway-peelvirtua9678]

Can this be done in Automatic1111?

[@MDMZ]

I believe so

[@rohithkumarsp]

how to do this locally

[@MDMZ]

it's covered in the video

[@alberto64674yt]

i hate comfy ui any method to use that on automatic1111 ui?

[@MDMZ]

why!?

[@alberto64674yt]

@@MDMZ is harder to me understand that module mess better the automatic1111 its just tabs and options

[@alberto64674yt]

node*

[@DIAMIN305]

RuntimeError: input must be 4-dimensional

[@MDMZ]

do you see any nodes turning red when you execute? I'm suspecting that you're loading a non-video file, or a corrupt video file

[@druqsdude]

because you have an AMD card and running it with --directml thats what i found online

[@raymondvaldes_]

not a TIFF file (header b'n' not valid)

[@raymondvaldes_]

please help man, i dont get this

[@raymondvaldes_]

happens with jpegs, png work fine

[@RodieOsc]

Time to cook my 4090

[@MDMZ]

welcome to the club

[@suzanazzz]

Awesome videos thank YOu! Question is it possible to upload my custom models to thinkDiffusion (comfyUi)?

[@MDMZ]

Yes you can!

[@시니어티브]

비디오 생성기

[@AllSeasonsVideo]

Think diffusion is a rip off of a much larger and much better service. Everyone uses that one. Way more apps. Way better service. Think diff is nothing special.

[@maresionut-laurentiu7128]

Except it's totally free

[@Justafakestory]

Lol what service

[@nelsonduffle496]

What service are you referring to?

[@tooluuke7638]

​@@maresionut-laurentiu7128 Think diffusion is free just for 30 minutes