ComfyUI AnimateDiff Prompt Travel: Getting Started.

For Unlimited Animation lengths, Watch Here:
• ComfyUI AnimateDiff Pr...
This video is a complete start to finish guide on getting ComfyUI setup with the addition of the ComfyUI-Manager and AnimateDiff with Prompt Travel. This will give you an almost one to one workflow with the AnimateDiff-CLI-Prompt-Travel workflow. I have other video's on the way to get us closer to a one to one workflow with the CLI.
The repos are located here:
github.com/comfyanonymous/Com...
github.com/ltdrdata/ComfyUI-M...
github.com/Kosinkadink/ComfyU...
The workflow file can be downloaded from here (Updated to allow for unlimited frames):
drive.google.com/file/d/1bG9-...
The models (checkpoints) downloaded for this tutorial series are here:
civitai.com/models/25694/epic...
civitai.com/models/134442/hel...
The VAE used can be downloaded from:
huggingface.co/AIARTCHAN/aich...
The motion_modules and motion_loras can be found on the original AnimateDiff repo where you will be offered different sources to download them from:
github.com/guoyww/AnimateDiff
Or here's a quick link to civitai:
civitai.com/models/108836/ani...
civitai.com/models/153022
The newer motion module I mentioned in the video can be downloaded from here(huge shout out to manshoety):
civitai.com/models/158389/imp...
Socials:
x.com/c0nsumption_
/ consumeem

[@SpacePirates_Studios]

Thank you for all these awesome walkthroughs - very appreciated!

[@aipamagica1]

Thanks for the straightforward tut! My favorite moment was you telling us not to worry about how it's working, just focus on doing your art! :)

[@johngardnergraphics]

Awesome, thanks for the step by step! Very helpful!

[@GuyXotic]

I have to say you are giving away the best tutorial with a precise explanation and I really appreciate you taking your time to make them simplified

[@c0nsumption]

Thanks for the positivity 🙏🏽

[@ai.manipulator]

Whassup dawg 😂

[@GuyXotic]

@@ai.manipulator wassahh holmes 😂🙌🏻

[@bardaiart]

Awesome tutorial! Keep up the great work :)

[@simonzapata1636]

Fantastic work! Thank you so much

[@sakari.niittymaa]

Great tutorial and well explained 👍

[@c0nsumption]

Glad it was helpful! IPAdapter, ControlNets, LoRAs, vid to vid, and much much more is on the way 🧍🏽‍♂️

[@MrPlasmo]

dude- you just made me convert from A1111 to Comfy. I gave it a 2nd shot after your tutorial, and everything "just clicks" now and makes much more sense. Maybe i didn't get a proper tutorial when I first tried Comfy. Anyways, it looks like a Comfy weekend for me, and can't wait until SDXL works iwth ComfyUI AnimateDiff - thx again

[@c0nsumption]

No worries :) it does btw. Kosinkadink (creator of animatediff-evolved) made support for HotShotXL. I’ll make a video eventually but I got a couple of em backlogged

[@MrPlasmo]

@@c0nsumption that's great! so SDXL models and SDXL Loras work with ComfyUI AnimateDiff?

[@c0nsumption]

@@MrPlasmo no, with HotShot-XL which is the closest thing we have to it yet

[@FrauPolleHey]

you are a genious, thank you for this tutorial, you have amazing teaching skills!

[@FrauPolleHey]

@c0sumption sorry for the stupid question... but i dont know how to reopen comfyUI after closing everything...the bat file doesnt work if i double click it, can you please help :)?

[@francaleu7777]

thank you! very good tutorial!!

[@cgonestudio2752]

Gracias master estuvo genial espero sigas subiendo mas tutoriales ..thanks

[@c0nsumption]

More on the way. Very busy with work sorry 🙏🏽🥺

[@mikberg1824]

Very good explanation!

[@yumincao5878]

this is so dm good. this is the best tutorial.

[@ryanontheinside]

Bro. Thank you for your time

[@karim_yourself]

Thanks a lot for this tutorial ✌

[@c0nsumption]

Feel free to tag me in what you build. Love seeing what everyone’s making with it. Y’all are so creative!!

[@arcangeel4828]

Great video dude!!🔥🔥🔥

[@c0nsumption]

Thanks 🙏🏽

[@c0nsumption]

Hey everyone 👋🏽 It was brought to my attention that there was an issue generating past 30 frames. I have updated the workflow file to allow for this functionality. My apologies, I left out a crucial node and am making a video now to discuss what caused this. Will link when done. Here's the updated workflow file, note that it's also been updated in the description: drive.google.com/file/d/1bG9-leZVnQm8HwvGJMu101HOaQmutmzc/view?usp=sharing

[@highly_illiquid]

That link is giving me this: No preview available. File is in owner's trash

[@c0nsumption]

@@highly_illiquid fixed. updated the one in description, forgot to update this one.

[@aestendrela]

Unbelievable, muchas gracias. It would be amazing if you could make a tutorial where you use a reference video and apply AnimateDiff on it

[@c0nsumption]

Thanks man. It’s on the way. Was trying to get this here so that as I go through workflows (quickly and concise), it’ll act as a reference for anyone who is confused and doesn’t know how to install ComfyUI or the manager

[@afigueroa0331]

​@@c0nsumptionamazing how complex but yet so simple you make all this thank you

[@EpromLee]

thank you~

[@sync.e]

legend

[@pascal_dop]

this is way easier than the last three tutorials ... Is it possible to do 120 frames or more ? and will you add controlnet to the workflow in the upcoming videos?

[@c0nsumption]

Increase your batch size value in the “Empty Latent Image” node. Yes, IPAdapters and ControlNets coming up 🧍🏽‍♂️

[@c0nsumption]

Here: 19:10

[@c0nsumption]

Just tested, making an updated video showing how to do longer generations. My apologies 🙏🏽

[@bovjer]

Thanks for the tutorial! You are very good at this❤ But i found the video quality on the output of videos to be so much better on the first tutorial you made. Even if its so much more tedious to do, and i have to edit the prompt.json everytime, but the quality was so much better. It was less vram consuming too. Do you know why? I would love to have the same output in comfyui, but still have not made it.

[@c0nsumption]

With the CLI? I agree. The CLI is incredible. You can get similar results with ComfyUI you just gotta know how to mess around with the nodes some. Did you use the IPAdapter last time? If so, that most definitely has huge influence over it all. I’ll try to release IPAdapter video tonight, so tired 😴

[@bovjer]

Yes I mean the CLI. I am so new to this 😅 Yeah, I haven't been able to get the same results with comfy yet. I also didn't.manage to get any longer videos than 30 fps in this comfy set up. I get an error that says it's restricted to 30frames.

[@c0nsumption]

@@bovjer I’m posting a new video to day 100% that allows for longer vids and the other things mentioned 🙏🏽 Going to eat and film. It’s funny, I have some saying the CLI produces better results and others saying that ComfyUI does. Think it’s all about which one makes you feel more natural to interact with

[@MrPlasmo]

Quick question if you have time: Is there a setting or way for ComfyUI to save all of the individual image frames (in addition to saving the .mp4/png file) in the outputs folder? thanks

[@c0nsumption]

Yeah, I’ll throw it in my next video. Still At work 😣

[@58gpr]

can this also be used with Adetailer? Great video again btw. Thanks, mate!

[@c0nsumption]

Yes it can but it requires a bit more nodes. A little confusing for new comers but I’ll try to set up a node network that incorporates it. Pretty much made this video so that now I can drop short videos that showcase workflows. This way it’s just a 3 to 10 min that’s to the point on some advanced node networks. One with Adetailer is prime example

[@58gpr]

@@c0nsumption nice! I'm really looking forward to those future videos from you! BTW: any idea why comfyui produces much better results than the CLI prompt travel version? I like it. a lot! :)

[@eraniopetruska5701]

Hello. I have a question. What's the difference between having comfyUI installed with the python embedded from the portable mode and opening a venv every time you start comfy? Thanks

[@metaivisuals]

Thank you! Btw you can use your stable diffusion models without copying them, by adjusting the modelpath #### How do I share models between another UI and ComfyUI? See the [Config file](extra_model_paths.yaml.example) to set the search paths for models. In the standalone windows build you can find this file in the ComfyUI directory. Rename this file to extra_model_paths.yaml and edit it with your favorite text editor.

[@mehradbayat9665]

Would the text that was produced in the video be caused by the LoRa or the checkpoint?

[@aminshallwani9369]

Hello thanks for such a helpful video. I have followed all the steps you mention errorless, In the end when hit Queue Prompt got this error ERROR:root:!!! Exception during processing !!! ERROR:root:Traceback (most recent call last): File "E:\DATA\Resources_ComfyUI\ComfyUI\execution.py", line 153, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "E:\DATA\Resources_ComfyUI\ComfyUI\execution.py", line 83, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "E:\DATA\Resources_ComfyUI\ComfyUI\execution.py", line 76, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) File "E:\DATA\Resources_ComfyUI\ComfyUI\custom_nodes\ComfyUI_FizzNodes\ScheduledNodes.py", line 101, in animate animation_prompts = json.loads(inputText.strip()) File "C:\Python3109\lib\json\__init__.py", line 346, in loads return _default_decoder.decode(s) File "C:\Python3109\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Python3109\lib\json\decoder.py", line 353, in raw_decode obj, end = self.scan_once(s, idx) json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 5 column 1 (char 382) Would appreciate your help. Thanks

[@c0nsumption]

There’s an error in your prompt list for key frames. Make sure there is no comma at the end of your last prompt

[@aminshallwani9369]

@@c0nsumption Thanks its working now🙂

[@trippyvortex]

Is this possible to operate in a colab notebook for users like myself who are gpu negative?

[@c0nsumption]

I’ve been studying the heck out of Runpod and can say that I’ve finally figured the blunt of all of it out. I will be dropping a tutorial very soon. I have to setup some .sh files to make things easier as well as film 🧍🏽‍♂️

[@curcicreations]

Is it possible to use keyframe prompts without any extra extension in in auto1111 or is it just for comfyui animatediff evolved.

[@c0nsumption]

As far as I know, only CLI and ComfyUI. I’ve seen some stuff in Japanese post allowing for keyframing in auto1111 but it’s extremely dodgy as it requires moving your ControlNet install in to different parts of the webUI directory and manipulating Python files. In that, it will break a lot of things so you’d have to do a separate install to keep things safe and there’s no guarantee you’ll get it working. Plus minimal support as there’s almost no community behind it

[@58gpr]

BTW: you can Ctrl + click a link in CMD window and it'll open the link in your default browser.

[@c0nsumption]

This doesn't work on windows 10. Only linux and mac. Been doing it for years. One of my biggest frustrations with windows command line.

[@Myth1n]

is it possible to add control net to get a bit more consistancy? where would i connect it, and which control net module

[@c0nsumption]

This vid: kzbin.info/www/bejne/eqPLaZ2mn6pkr9Esi=3twkgVUNlWLjS_Yh

[@LIMANDRZAVA]

How to "slow down" animation, not to shift into something new every half second, besides frame rate? Nice tuts bro

[@anoubhav]

What resources do you recommend to really understand how each node works?

[@c0nsumption]

I got vids coming this week. Just knocked out some work that was eating up all my time 🙏🏽 Most useful resource will always be the repos and documentation for ComfyUI and any extensions though

[@anoubhav]

@c0nsumption got it! Thanks for these videos. Looking forward to the coming ones!

[@heymyfriend4445]

I am getting error: Cannot import C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-VideoHelperSuite module for custom nodes: module 'cv2.gapi.wip.draw' has no attribute 'Text'. Some nodes are not working.

[@Spajra-music]

how do you install nodes into comfy UI via runpod jupyter notebook? I git cloned into the workspace/ComfyUI/custom_nodes/ path and it is not working. not sure how to proceed? trying to get the manager working :/

[@c0nsumption]

You would have to ask in the r/Animatediff subreddit as that’s beyond the scope of the tutorial. Haven’t tried it myself

[@robertcrystals]

when I do a batch size much over 16 (using sd_15_v2), it just produces noise. Batch size is the amount of frames, right?

[@c0nsumption]

Just tested. Filming video now for “Generating longer videos”

[@haydnmann]

Where is the discord server you mentioned?

[@rishabhsingh8334]

Can someone help me. I am getting this error after running comfyUI. RuntimeError: mixed dtype (CPU): all inputs must share same datatype.

[@user-ro2ov5ee8y]

Can i install this on my mac book ?thank you !

[@c0nsumption]

Yes, you’ll have to reference the actual repo though: github.com/comfyanonymous/ComfyUI

[@kenrock2]

Is there a way to specify the length of the frame like the previous tutorial?

[@c0nsumption]

Can you reframe the question? A bit confused 🤔

[@c0nsumption]

The frame rate? 19:10

[@c0nsumption]

Making video now for generating longer generations

[@c0nsumption]

kzbin.info/www/bejne/gmWYiaSqoJ19Zqs

[@kenrock2]

@@c0nsumption thanks... this is exactly what i mean

[@ai.io10101]

can i make rx7 loras with this workflow?

[@SheRoMan]

I have a Question ......... I downloaded the ComfyUI Windows portable version and I didn't install requirements.txt or virtual environment but it is working just fine .......... So should I do all of that now or what?

[@c0nsumption]

I recommend not using the portable version so that if you run into trouble with extensions it’s easier for people to help you. If you don’t plan on downloading extensions you should be fine

[@SheRoMan]

Thank you @@c0nsumption

[@izac9382]

Might be a silly question but after I've got this working, how do I launch it again next time?

[@c0nsumption]

Head to directory where you have venv for it: - Type ‘cmd’ into the address bar. - in command line activate virtual environment: venv\Scripts\activate Then in command line head into comfyUI and run application: cd ComfyUI python main.py

[@izac9382]

@@c0nsumption Perfect thank you! I've been playing with what you shared for the last 7 hours, so much fun.

[@TheMaxvin]

Length of animation is 30 frames, what the purpose in new prompt from "30" ?

[@c0nsumption]

Why not just test it out? Great experiment for yourself 🧍🏽‍♂️

[@mannychop]

I'm missing the BatchPromptSchedule node, and searching in the manager I cant locate it, any tips for me? thank you!!

[@c0nsumption]

The video show you what to do: Drag workflow file into the ComfyUI interface. Use the ComfyUI manager to download the missing nodes

[@c0nsumption]

Here: 7:30

[@c0nsumption]

Is it working now? Make sure you grab the updated prompt file

[@mannychop]

@@c0nsumption I seem to have borked my install of comfy somehow. My system is running outta space and I was uninstalling stuff and I think I did too much 😂 so I’m going to refresh everything and follow again from your first tut for the fresh install. Thanks

[@hatakeventus]

Can i use this with AMD RX 6700

[@c0nsumption]

You would have to test that out on your own. I solely use Nvidia GPUs. Reference the ComfyUI repo: github.com/comfyanonymous/ComfyUI

[@Oinkmooo]

dumb question. How do i access comfyui? the gpu.bat file ain't working

[@c0nsumption]

6:28 .... Please watch the video before asking questions like this. This is literally what the video is about.

[@Oinkmooo]

@@c0nsumption my bad, it's my 2nd day learning about AI.

[@c0nsumption]

@@Oinkmooo lol 😆 okay makes sense. My b my b. Don’t quit. Legit. You’ll get the hang of all of it. Everything is so much easier than you think. As you work on more and more stuff you’ll look back and laugh at how hard the little things were 🙏🏽

[@TrippyVideosGirl]

You know it was a good tutorial because i was able to get it up and running haha. But now after closing it, how the heck do I relaunch it? There's not bat file like for 1111.

[@c0nsumption]

Go to directory with venv folder in it and cmd in the address bar. Activate the venv just the same: venv\Scripts\activate Go into ComfyUI directory: cd ComfyUI Start it up: python main.py If confused at all feel free to ask questions

[@TrippyVideosGirl]

@@c0nsumption Oh ok, I didn't realize I had to do that every time. Have a friend who has it set up easier. There's a run Nvidia batch file in there somehow on his. Simple click of the mouse...

[@c0nsumption]

@@TrippyVideosGirl you should get in the habit of learning the venv thing though. Pretty much every new project that drops is probably gonna require it. Otherwise you can just make a .bat file real easy that just has all those commands in it. Then you’d just click that file and it’ll do the rest

[@TrippyVideosGirl]

@@c0nsumption OK thank you freind🙏, Don't really understand it all haha, but I will do that.

[@chadddemoss]

I'm getting all kinds of errors when i get to installing the -r requirements.txt " WARNING: The script isympy.exe is installed in 'C:\Users\chadd\AppData\Roaming\Python\Python310\Scripts' which is not on PATH."

[@c0nsumption]

Da hell. Your venv for sure activated? 🤔 Please share on r/animatediff subreddit

[@wagmi614]

next up: init images 3 image interpolation

[@c0nsumption]

On its way 🙏🏽 For sure this week

[@speaktruthtopower3222]

This is not working for me. Says I'm missing. "'ADE_AnimateDiffLoaderWithContext". Wont allow me to uninstall Kosinkadink Aninmate diff version., Not sure what to do.

[@c0nsumption]

Couple options: Delete Animatediff evolved from custom nodes folder. Restart ComfyUI and reinstall. 7:38 is time stamp for installing missing nodes. Start there, lmk if it works. Remember to restart comfyUI after installing

[@c0nsumption]

kzbin.info/www/bejne/gmWYiaSqoJ19Zqs

[@speaktruthtopower3222]

@@c0nsumption I ended up making a completely new install (Your instructions worked fine). I was doing this from an existing install, I think the issue might have been a previous ControlNet installed. I'll play with my previous installation to see if I can fix it to help others

[@KDawg5000]

@@speaktruthtopower3222 @c0nsumption I also cannot install the Kosinkadik custom node. I keep deleting it, and then trying to install it w/the manager, but it doesn't work. I even tried git cloning it, but no luck. Not sure what is preventing it from installing. EDIT: I read below and saw your suggestion of updating all. I did that and it worked. Thanks

[@metaivisuals]

@@speaktruthtopower3222 in the comfyui folder you can also type cmd and "git pull" then it will update. Worked for me afterwards. I had the same issue

[@TheDreamingMachineFilms]

Can this run on Google colab? (Pro)

[@c0nsumption]

Yeah, I believe they may have posted about it on r/animatediff subreddit. Id recommend using Runpod instead though.

[@TheDreamingMachineFilms]

@c0nsumption Thank you!

[@AtheistExpert]

45 minute generation time for your example. must be broken for me :/

[@c0nsumption]

45 min?! What kind of GPU you got? Maybe pip uninstall torch and then do that same command from the tutorial again

[@AtheistExpert]

@@c0nsumption gtx1080, 8gb vram

[@elowine]

Same for me on my old 1080. I think when older cards hit the GPU memory limit, instead of crashing like they used to, they will go very very slowly. When you lower the resolution you are generate it will probably be a lot quicker. Guess we have to try and get this to work in Colab too :P

[@c0nsumption]

@@elowine just use Runpod. You’ll pay by the hour for any gpu you want and they have a ComfyUI template 👍🏽

[@elowine]

​@@c0nsumption interesting something to check out tomorrow thanks for the tip.

[@fernando749845]

No chapters...? I appreciate the video but come on man, I don't have time for half an hour and rewatch stuff that I've already seen.

[@Lewdstorygirl]

I keep trying to reinstall Animatediff custom nodes like you did but still have these errors : When loading the graph, the following node types were not found: ADE_AnimateDiffLoaderWithContext ADE_AnimateDiffUniformContextOptions ImportError: cannot import name 'attention_basic' from 'comfy.ldm.modules.attention' (C:\Users\Admin\Documents\1_ComfyUI\ComfyUI\comfy\ldm\modules\attention.py)

[@c0nsumption]

So you: - downloaded the comfyUI manager via git cloning in custom nodes. - you then started up comfyUI. - you then dragged the file in. - it warns you that you missed the custom nodes. - you go into the manager and click install missing nodes - you then restart ComfyUI via ctrl+c in the command line. And running it again (python main.py) Did you do all these?

[@Lewdstorygirl]

Yes 5 times.@@c0nsumption

[@c0nsumption]

I’ve come across it too. Try: - refreshing page. If that doesn’t work: - go into manger - update all (I think that’s what it’s called) - refresh page If that doesn’t work: - delete animatediff-evolved from custom nodes folder (make sure to backup models so that you don’t have to download again - install again - shut down comfyUI - start back up again - refresh page These three have worked for me

[@D3coify]

I have a laptop with nvidia Geforce 840m, will i be able to run the run_nvidia_gpu.bat? or should i just use the run_cpu.bat

[@c0nsumption]

Did you watch the video? This video does not use that. Please watch video before asking questions as they may not be in the scope of my training

[@D3coify]

@@c0nsumption i did, nevermind

[@c0nsumption]

@@D3coify did it resolve the issue or you still need help?

[@D3coify]

@@c0nsumption the tutorial is amazing, unfortunately i think my pc is too old, haha.

[@c0nsumption]

@@D3coify try Runpod.io They have a template already setup dor ComfyUI and you can rent GPUs by the hour. This way you don’t have to shell out a whole 10 bucks and you get something better than google colab 👍🏽