How to convert movie to anime video using ComfyUI, AnimateDiff, ControlNet and LCM

[@FutuTek]

Because many ones asked, I inputed all models and loras links to the description, let's have fun

[@dmitrykonovalov9366]

Nicely explained! Many thanks.

[@larryleng8531]

偉大的創舉，我一直在嘗試畫面的協調的問題，畫面和動作以及風格，你做到了，我一直想簡化流程單事實證明這沒有捷徑必須一點一點的處理細節，你的控制部分太棒了給了我很多的啓發，應該還能更高，非常感謝你。

[@ian2593]

Great workflow! This is the first one that has applied the style without artifacts for me. I'm wondering how many frames it can convert on a mid range machine with an RTX 4070 series.

[@FutuTek]

You should convert the max 15s video with 12fps, then use Film FVI to improve the interpolation to upgrate to 24fps

[@nholmes86]

Superb conversion

[@MisterWealth]

My footage just comes out discolored/purple for some reason, do you know why?

[@FutuTek]

Do you mean you use my footage in description? or you own footage? if you use your own footage please try to re-create it with higher frame rate

[@MisterWealth]

@@FutuTek My own footage, for some reason if its not slow motion video, itll have a weird purple cast, or it won't detect the faces, or generally have some terrible artifacts

[@STAR_SOLOS]

this workflow doesnt work anymore ! can you do a updated version ! idk why but i get error !

[@ComfyUIstudio]

Hi, love your video style. can you add to the workflows model links in a note node?

[@FutuTek]

Hello, it is very easy, you load my workflow to your ComfyUI, then you will see the "modelname.safetensor" or "modelname.ckpt" in every workflow, just type this full "modelname.safetensor" to Google and then you will seed a lot of sources to download it

[@quangdinhdota2388]

Thank for your video. Can you note detail file for each box in step 2. Same file I don’t know how I should add exactly file fill in box. Name file different in your video ❤

[@KanaNoClown]

So good

[@MrCalmwaters112]

Can't hear anything. Why would you have the music so loud?

[@adukomik720]

Super awesome! and thanks for the workflow. However I am missing some nodes and being a newbie on comfyui I don't know the workaround. When you find time please help. Thanks

[@FutuTek]

Hello please follow below steps: 1. Update your ComfyUI to the latest version: Manager / Update All, then restart ComfyUI 2. Load my workflow in the video discription 3. Installing missing notes: Manager / Install Missing Custome Nodes, then have fun

[@adukomik720]

@@FutuTek I appreciate you for responding. Yes, I have updated to the recent comfyui, still figuring out how to fix import failed ComfyUI-Advanced-ControlNet and AnimateDiff Evolved. Thanks

[@adukomik720]

@@FutuTek what models did you use. SDXL or v1.5?

[@FutuTek]

@@adukomik720 Try to install them separately in Manager / Custom Nodes Manger

[@FutuTek]

I use SD 1.5, the SDXL is not much good for AnimateDiff

[@Isi-uT]

Nice!

[@dmitrykonovalov9366]

I searched and found most but not all loras/vae/models. It would be nice if you could add the links to the safetensors you are using.

[@FutuTek]

I inputed all models and loras links to the description, let's have fun

[@mohammedzaidkhan5687]

How much time did it take to convert the video you showed us, Please respond, i would appreciate it brother

[@FutuTek]

I use a RTX 4090 GPU then it took me 30 min

[@mohammedzaidkhan5687]

@FutuTek thanks a lot brother

[@miratdrati]

Can I change the video style bro? where to change, I'm new to comfyui

[@FutuTek]

What you mean "video style"?

[@miratdrati]

@ yes. i want change style video, eg : chibi ?

[@FutuTek]

@@miratdrati you just go to civitai.com, type chibi to the search, enter, then select model SD 1.5 on the left, there are many chibi model for download, just download to comfyui\models\checkpoints, then refresh the workfow. After that you can choose new model in the Group Node 2 (load checkpoint)

[@ivanbelyakov]

Thanks for the video! I need help. It issues errors, and then the calculation stops at 9%... lora key not loaded: lora_unet_up_blocks_0_attentions_1_transformer_blocks_0_attn1_to_k.lora_down.weight ERROR lora diffusion_model.output_blocks.3.0.in_layers.2.weight shape '[1280, 2560, 3, 3]' is invalid for input of size 11059200

[@ivanbelyakov]

changed LCM SDXL to LCM 1.5 and the process started. Then it hung at 9%, with 95% GPU load. I waited 5 hours, but nothing happened. The scheme and example files are yours.

[@FutuTek]

@@ivanbelyakov There is not enough RAM (not VRAM), you need a computer at least 32G RAM, then please reduce the upscale resolution to 1024x576 then run test for 1 second (24 frame only), then increase it gradualy

[@ivanbelyakov]

@@FutuTek I have 64gb RAM

[@FutuTek]

@@ivanbelyakov I have 64G too, and sometiem the same problem still occured, just reduce the resolution or shorten the length then the problem will be solved

[@quangdinhdota2388]

Same problem. Have you solved problem?

[@sprocketamerica1]

Getting: Prompt output failed validation: Exception when validating node: name 'full type name' is not defined VHS_VideoCombine

[@FutuTek]

You can try by right click to the note and select Fix node(recreate)

[@sprocketamerica1]

@@FutuTek no sorry does not fix it.....please review your work flow you posted because i think it is differnt then you video

[@FutuTek]

@@sprocketamerica1 I have just added a new workflow to the folder, name "...241213", I already tested it, if the problem still occur pls. check your software and computer setting

[@sprocketamerica1]

@@FutuTek what version of comfyUI are you running this may be my issue because i still get he (Prompt outs failed validation) Exception when validating node : name 'full type name' is not defined VHS_VideoCombine

[@jorgediaz9093]

Lower the background music is to loud to annoying

[@larryleng8531]

我們一直嘗試在讓AI既能發揮他的能力又需要規範他的動作，所以我們不得不把更多的控制也就是約束加給他，這讓我很迷惑 到底是我們在創造AI還是AI在探查我們的底綫。這太難了 另外這個工作流的消耗確實是驚人的，我的頂級設備，在運行的時候遇到了兩次溫度報警。但這依然是給我非常多的啓發，非常感謝你的貢獻我愛你，最誠摯的尊重，來自東方的祝福。

[@FutuTek]

You should reduce the video resolution and shorten the video to avoid the high temperature