DaVinci Resolve and GPT Integration Part 3: Creating Fusion Nodes

Do you find Resolve's Fusion difficult to master? What if you could build effects simply by describing then in your own words?
With a bit of help from OpenAI GPT and Blackmagic Resolve APIs, we may just get there! In this video you will see the baby steps of what compositing will look like in the near future.
Please note that this is a proof-of-concept technology demonstration, not a publicly available product. You will not find this in your Resolve's menus!
I'm eager to hear your thoughts on this integration. Are there any other potential use cases for GPT AI that you would like to see in Resolve? Please leave your comments below.
Make sure not to miss the next video in this series where I present AI integration in DaVinci Resolve. Don't forget to subscribe! For more information about my other coding projects involving DaVinci Resolve and Baselight, visit www.metafide.com.

[@TransientMobile]

Fantastic proof of concept Igor. This could go very far. Loving this series!

[@hisroyalillness]

great series. Please Keep it going.

[@lfedit5202]

This is amazing stuff Igor! A workflow tool for repetitive fusion comps would go along to eliminate manual repetitive tasks. For example. Log footage on ingest, create a fusion ".setting" file, Ask the AI resolve assistant to apply the .setting file to all clips on the timeline with a specific logged metadata. Look forward to more information as you continue. Cheers.

[@StefanRingelschwandtner]

This is amazing!

[@XSession]

This is mind-blowing❤❤❤

[@KamarleyBlake]

This is amazing man 🤯🤯

[@galacticgreg8261]

Not there yet. When it can truly take natural language and be able to convert to use the proper node(s) will be key.

[@cosplayfx]

I look forward to: animate the lightsabers in this battle scene. Sith red color and Jedi blue.

[@abhishekak9619]

this might have worked with gemini and its 1 million tokens. and i think the gemini tokenizer is different because other llms run out of tokens in fewer words. its bad in most cases until it has enlugh context and then it starts working as intended.

[@raccoonair]

Interesting; not really for building on the fly, buy I can see possibilities for alternate scripts for the (near) future. Thanks for sharing.

[@WarrenPeaceOG]

I've been using OpenAssistant and ChatGPT for help learning Fusion. I asked it how to get scopes on the Fusion page and it told me how to get scopes on the Color page (that don't exist in the Fusion page). When I corrected it, it acted like it hadn't made a mistake🙄If it's gonna respond to thanks, it should admit to being wrong appropriately. (I did find Fusion's display scopes). Anyway, it's a pretty good learning tool, used with caution

[@Behnam_Moghaddam]

wow! very promising. to me it would be interesting how it copes with stuff that needs an analysed picture like " blur the church in the picture throughout.

[@raphaelquinones4002]

Hi there Igor, I love your videos and I basically learned how to use the API from you. I just have a question that I think you might have the answer to. Why does the SetClipProperty() only work with “Start TC” and not the other properties? Are there any workarounds on setting the other properties? Like maybe changing the XML save file?

[@Hdhead]

Yes you can change them but only the mutable properties. For example, you can change 'Clip Name' but not 'Sample Rate" because the sample rate is intrinsic to the file represented by the clip.

[@raphaelquinones4002]

@@Hdhead What about 'End TC' and 'Duration'? I think those two are mutable, but I can't seem to change those stuff. Been trying to read the forums, but still can't figure it out. Btw, thank you so much for replying

[@Hdhead]

@@raphaelquinones4002 I'd have to double check, but I think 'End TC' and 'Duration' in Resolve parlance are not 'Mark Out' and "Duration" derived from markOut-markIn.

[@raphaelquinones4002]

@@Hdhead Ohhhh, thanks for correcting me. I got it wrong. So what I'm currently trying to change is the start and end of a video clip For example, I have a 10s video clip in my desktop. I want to put it in a video track where the starting point is at 2s and ending point is at 5s, so that the duration in that track is only 3s What properties should I take into account? I really appreciate you helping me out and this means a lot to me. Thank you so much again

[@WarrenPeaceOG]

4:20 I'm guessing Chat GPT responded to the node-free directions based on the node-based directions in the line above, ie. it remembers

[@Hdhead]

It does not "remember." This is actually GPT 3.5-Turbo LLM. Chat GPT is the openAI's webapp which is not what we're using here. None of them are able to remember the past conversation. The way it's done is by resubmitting all the past conversation before submitting each new prompt. The problem here is that the API is now limited in the number of tokens that can be sent and retrieved. 16K is the limit of this model. Fusion comps are very verbose and if I tried to fake the memory persistence like Chat GPT does I'd would very quickly exceed the token limit. BTW, I do fake the memory persistence in the Resolve Helpbot in part 1 of this series. I can do it there because we're dealing with a smaller amount of text.

[@WarrenPeaceOG]

@@Hdhead Cool, thanks for the thoughtful response. I missed part 1. I'll check that out🙏🏻

[@majorediting8164]

I would be happy if I could just ask “if I want to do XYZ, what node(s) do I need to use?” That would solve fusion for me!!!

[@Hdhead]

Check out my DaVinci Resolve Assistant custom GPT.

[@majorediting8164]

@@Hdhead will do. Is it a plugin or script? What about the VO script, are you developing that for release too?

[@Hdhead]

@@majorediting8164 It's neither. It's a custom GPT: chat.openai.com/g/g-suSgYbcpa-davinci-resolvetm-assistant I'm kind of working on the VO thing, but I move slowly...

[@aqueminteressarpossa3445]

this AI thing sound too much like a digital "ouija board" to me. i don't like this concept at all. I dont want this kind of help, if you know what i mean, but love your work as always.

[@Hdhead]

Thanks!