Multimodal AI Video Relight with IC-Light (ComfyUI, non-AnimateDiff)

[@risunobushi_ai]

Try RunComfy and run this workflow on the Cloud without any installation needed, with lightning fast GPUs! Visit www.runcomfy.com/?ref=AndreaBaioni , and get 10% off for GPU time or subscriptions with the Coupon below. REDEMPTION INSTRUCTIONS: Sign in to RunComfy → Click your profile at the top right → Select Redeem a coupon. COUPON CODE: RCABA10 (Expires August 31) Workflow (RunComfy): www.runcomfy.com/comfyui-workflows/comfyui-product-relighting-video-workflow?ref=AndreaBaioni Workflow (Local): openart.ai/workflows/YxuxT1PkjeWCmnpSpwPs

[@ВикторияВасилькова-д7и]

Oh my god, this morning's discussion on reddit was so heated that you repeated and improved that controversial workflow and provided it for free. Bravo!!!

[@risunobushi_ai]

ahah I wish I could create a workflow in a matter of hours, but this was created last week in the span of a good couple of days!

[@yotraxx]

Andrea, you are a genius. Thank you so much for sharing your acknowledgement.

[@risunobushi_ai]

I don’t know about being a genius, but I have fun trying to solve things!

[@xandervera7026]

Love it, can't wait to fiddle with this!! ALSO what do you have against set/get nodes 😆I hate spaghetti!! Just set and get so everything looks clean and color code them xD

[@kennedysworks]

Dear Andrea Baioni, The workflow you created is absolutely brilliant and has had a significant impact on my video work. I believe its versatility is truly limitless. I have one question: Is there a way to adjust the flickering of the lighting in the final video output? Currently, the flickering is consistent regardless of the lighting I use. I wonder if it’s possible to either reduce the flickering or increase it. If it’s not too much trouble, I would appreciate your advice.

[@Zanroff]

I have green screen footage of a person I'd like to key out and subtly relight using this. Perhaps it would be better to use the still photo relighting workflow and use the incremental batch image loaders for the footage, luma matte and lighting footage? I'll split everything up into still frame images. Any suggestions beyond that?

[@DanielPartzsch]

I think you should always use lanczos algorithm in the resizing nodes. Nearest exact gives very pixilated results.

[@tonon_AI]

Hey man no problem. Thanks for sharing such nice content.

[@risunobushi_ai]

Hi! didn't know you had a YT channel, I linked to your linkedin profile but I'll add the YT link as well. Thanks for the RIFE suggestion once again!

[@tonon_AI]

@@risunobushi_ai no worries I dont post anything in here. And happy to help

[@Scrapemist]

The previews of the result are a bit small. To hide the inconsistency/artefacts?

[@risunobushi_ai]

yes! you are the first one to realize this is all a conspiracy from which I have basically nothing to gain! :) in all seriousness though, the examples are pretty clear in full screen on a 27", I literally point out the artefacts and inconsistencies when they appear, and I provide the workflow to test it all out yourself, I'm not sure if I can be accused of "hiding" stuff.

[@Scrapemist]

17:24 “minimal artefacts”.. The wood texture is dancing as if it is water. It is simply unusable professionally.

[@risunobushi_ai]

@@Scrapemist the chapter is literally called "Frame Interpolation Issues", again, not sure I'm hiding anything here. It's also the last stage, which is the one I'm warning about since it has the most issues deriving from a RIFE overuse. I agree that the last RIFE pass is too much, but the "minimal artefacts" is relative to the AnimateDiff pipelines we'd been used to up till now.

[@Scrapemist]

@@risunobushi_ai fair point

[@RoN43wwq]

thanks

[@jonathanperreault4503]

@risunobushi_ai can you make a video on how to relight flat images like a picture of a wall or floor ? im making textures for video games and id like to remove all lighting from a picture i took or change the lighting bakes in a texture also would it be possible to isolate different aspects of the lighting a bit like segment anything? im trying to find a way to make diferent maps for my materials

[@risunobushi_ai]

hi! can you send me some sample textures so I can understand better? [e-mail] would the normal map creation method with IC-Light help you, or would the "baked in" lighting not be solvable with that?

[@jonathanperreault4503]

@@risunobushi_aiill send you a few images but they are basically images of walls and floors shot with a phone up close i just saw your workflow for normal maps and its pretty much perfect but i also need a completely flately lit version of the image for the albedo color of the shader ,im not sure if you mean solvable trough a.i or standards means here but im sure ic-light could do that just not sure how since i havent got to try it yet to make tests if the image completely fills the screen will it still work correctly ? the perfect workflow would let you also let you separate the parts of the image that are metallic, rough , wet .etc and by how much also , i dont think ic-ight does this but make the image seemlessly tileable is really important for textures could you remove lens distortion as well using ic-ligt?

[@castleliao7357]

Very good video, thanks for sharing

[@PaganDeus]

Hi, your user interface is unique, but it causes me an error. I tried to find a solution, but it didn't work out. Maybe you, as the father of this interface, will decide) Error occurred when executing LoadAndApplyICLightUnet: IC-Light: Could not patch calculate_weight - IC-Light: The 'calculate_weight' function does not exist in 'lora'

[@risunobushi_ai]

did you download and place the ic-light fc model in the unet folder?

[@ismgroov4094]

keep going, sir!