Thx, Mochi1 seems to be great at nature scenes and listens to cam instructions very well (I didn't do much cherry picking for this one). I hope it keeps these advantages once there is a good way to use it with start or end frames.

Right, this was CogVideoX-Fun-2B (Start to Endframe Animation/Interpolation). A couple of new motion models came out in the last days - and you can use some of them with 12GB Vram in Comfy. So expect more experiments in the next weeks.

Thx, settings wouldn't explain how this one was done: I created this Flux image init clip first: schielo.at/768story.mp4 (x2 - 2nd version slightly time-shifted). Then I simply alternated between the 2 ip-adapter clips (+ added some movements according to the transitions) with an XL model. I used pure Flux (NF4) with my Schwarzwald clip though, but didn't have problems with overbaking (just some troubles with Loras and blurry backgrounds).

@@schielo aahh that's the reason so, the deforum part wasn't flux. Thx the same 4 answering btw I am kinda getting it to work with flux after many tries

Thx, this was a little bit more complicated. The background clip had something like schielo.at/20241008221237_settings.txt (CN clip was just some geometric structures, not really necessary) I didn't get the birds with the prompts and these movements, so I added them with another another CN layer: 2 bird animation init clips from pixabay, no cam movements, 2-3 CNs, strength values audio reactive with framesync.xyz (not very noticeable due to the low number of steps I used) then I removed the backgrounds with REMBG in comfy and blendet the layers.

Some (hyper-) realistic images look sharper than they really are, but you are almost right. Most of my clips are interpolated/upscaled to 4k@60fps.

@@schielo what programmed used to upscale?

@@tornadowatcher-my5ej Deforum, RealESRGAN is good enough and one of the faster options.

Wormhole malfunction. Sucks, but happens. Could have been worse than Yutani Space though - just imagine Data getting sucked into the Warhammer Universe ...

Thx, I used some tricks (playing parts in reverse, CNs, Forge Couple, ...) - it just took a really long time to find the right prompts/init images and match my cam movements.

@@schielo it was well worth it

20 verschiedene ... ;) Jede Szene besteht aus einem Teil der verkehrt abgespielt ist und ist angepasst aufs init img (+ ein paar CN/Layer Tricks). Der Clip war hauptsächlich editing.

Comfy init clip (Flux) + IPadapter in forge, yes.

Ein wenig anders. 1. Szene in diesem Clip: Deforum Zoom-Out verkehrt abgespielt 2. Szene: AnimateDiff Background Animation (mit gleicher init wie Szene 1) -> Deforum (AnimateDiff war CN1, CN2 war ein Clip von der Schwertkämpferin). 3. Szene: Deforum-Zoom von Szene 1 war CN 1, running girl war CN2 Letzte Szene waren verschiedene Tanzanimationen (CN1) + AnimateDiff Background Animation (CN2)

Der Trick war ein Flux init image zu verwenden, dass in der Mitte jeder einzelnen Szene steckt (möglichst kaschiert - d.h. mit Loras für die es sowohl Flux als auch für XL Versionen gibt, zB civitai.com/models/633639/azure-sketch-illustration-sdxl-pony-flux). Mehr als die Hälfte des Clips ist verkehrt abgespielt, eine meiner inits wäre zB das Titelbild (Hexe + Mulder in der ersten Szene war auch eine init, allerdings Forge Couple).

Thx, bis jetzt - nein. Ich verwende lediglich init images + init clips Clips (zB SVD mit Controlnet) um die Flux Kompositionen in Forge Deforum (mit XL models) reinzubekommen. Es gibt bereits eine Flux Deforum Version - github.com/XLabs-AI/deforum-x-flux - aber die hat noch sehr wenige Features und lässt sich nur über CLI bedienen.

Weight values sind mit www.framesync.xyz/ erstellt worden (ca 0.75, -0.5, Bend 1 @ 30fps), für die cam movements empfehle ich graphtoy.com/

Actually, this was not that bad, my audio reactive 20 steps clips take much, much longer. I used Cad 2 with Flux inits + 6-9 steps with lightning/hyper XL models. Calculating the camera motion and "cuts"/editing/ Flux inits took probably longer than the Deforum part.

Haven't dared to use the X-Labs version with 12 GB VRAM yet, my GGUF8 inits with 2 Loras took 2-3 minutes already (comfy). The animation itself was old Forge + Forge Deforum again.

Nothing new to see ... For the last two clips used the same workflows as in my Pax Miranda clip: kzbin.info/www/bejne/nWO4h5JjfsyGj9U (see description, augmentation 0.04-0.06, bucket id, ~60, cfg 1.8-2.2, - don't forget to mask the parts of the image you want to animate for the faster 24 frames clips, the stitched 48frames clips need no masking). Flux workflow was very basic, I didn't even use (or know about) the flux guidance node back then.

Thought the last scene gave it away ... First I created an init clip with Flux images and 48 frames SVD animations ( + 4x interpolation, since using only 2x for the cannons in the beginning caused some stuttering). Then I simply used Deforum CNs (Canny, Scribble, Depth) for style and cam movements.

Thx, releasing settings would not make sense this time (again), since I created an init clip first that you'll need for this to work. Cam movements were pretty simply (and less than perfect), CN values were a little bit more tricky - but I did not write them down correctly.

Thx, for the cam movements I can recommend graphtoy.com again. I used (adapted) cosinus functions that I changed every 50 frames if necessary. Model was zavychroma + hyper8CFG, since I don't see any quality loss with most prompts. I tried hyper4 but prefer 3 img2img steps + higher CFG for the negative prompt. The most difficult transition was the start: Drone view -> street view ... and I kept the drone over the castle in the end that I got with this token because I found it funny. ;)

@@schielo I saw the helicopter, was funny

@@schielo yeh I chuckled at the drone, it happens in Hollywood too! I looked at graphtoy and I realised how little I know lol Again, this is just awesome. I have released 3 video clips on my artist page done with deforum but with basic camera movements. I really need to step it up for the next one, they are too similar.

Some tips for my (current) graphtoy methods: if you use hyper4 or hyper8 you can usually leave strength at fixed 0.65 to get quick 2 or 3 steps animations. I wish I had this speed improvements when I learned this stuff (since it has a pretty steep learning curve you might have to do a couple of clips to understand everything). I almost never use Rotation3D Z. Most of my functions have this example format: 0: ((0+(5*(cos((3.141)*(t-50)/100)**2)))) - where 5 is the maximum strength at frame 50 repeating every 100 frames and if you increase the **2 in the end you get faster acceleration, but be careful, if you use odd numbers (**21 for example) you invert your function for the next set of 100 frames. Replace t with x for graphtoy visualization.

@@schielo thanks for that🙏although it took me a few reads to wrap my head around it, but I think I most of it. Graphtoy still looks alien to me but I'll give it a shot and see how I go. Gotta get my prompts right also

Thx. You are right, this was canny, depth leres and ttplanet.

Keep going my friend! Good work!

@@schielo thx bro!

Thx! Since I used Forge+Cadence, the little errors/bugs give a hint for my method: I used stop&resume together with graphtoy.com . The problem is, I used Cadence 2, and this version "doubles" the cadence every time you resume (results in half-broken frames that disturb the camera/motion flow for a tiny moment). The good news is, I just tested the new reforum branch from @reallybigname github.com/reallybigname/sd-webui-deforum/tree/reforum - I could not reproduce this bug with his automatic1111 version!