Audioreactive Graffiti - TouchDesigner x StreamDiffusion Tutorial 1

In this tutorial, we're looking at how to create graffiti based on abstract generative shapes and colors in TouchDesigner, using @dotsimulate's latest .tox for StreamDiffusion. It's all still very new and unexplored, so this is just one of my first approaches and I'm happy to hear any feedback :)
Really looking forward to many more examples!
This also works without StreamDiffusion, you can simply watch the part before!
StreamDiffusion .tox by dotsimulate: / streamdiffusion-97939064
Setup Tutorial: • Real-time diffusion in...
Snare Detection: / 98175337
Video File: / 98175466
Previous video: • Sonic Flowers - TouchD...
Music video made with StreamDiffusion: • elekktronaut - altered...
Resolutions: / sdxl_resolution_cheat_...
The prompt used for this example:
graffiti, text, graffiti art, wall, grunge
You can use any prompt you like. Not entirely sure but order seems to matter, first "weigh" more. I usually don't use sentences, but simply add words that fit with commas.
Pro Tips: TD now includes Nvidia Upscaler, I can highly recommend using that at the ned of the network!
Specs: TouchDesigner 2023.11340, Windows 11, RTX 3070 Laptop, Intel i7 11 Gen, 32GB Ram
-----
00:00 Intro & Overview
02:11 Timeline Setup
05:22 StreamDiffusion Setup
07:17 Audio Setup
09:54 Instancing & Render Setup
13:39 Audioreactivity & Post Processing
15:55 Prompt & Step
17:34 Outlines & Breaks
21:27 Adding Noise
24:26 Frame by Frame
26:21 Seed
27:14 Exporting*
28:28 Misc
30:24 Pep Talk
31:26 Outro
-----
*To see the file, check the project folder or define one on your moviefileout.
Files, exclusive content and more:
/ elekktronaut
The special thanks is integrated at the end of the video ❤
If you have any questions, feel free to ask.
IG: @elekktronaut
/ elekktronaut

[@Azraillevisuals]

Your tutorials helped me with Touchdesigner so much. I can now create textures and lots of cool stuff using your techniques. Bless you!

[@elekktronaut]

❤

[@FernandoMedrano]

Incredibly clear tutorial. Amazing. Thanks!

[@gnome3d750]

Probably one of the best TD tutorials out there. I learned so many different techniques just from this one video alone. Thank you for sharing your work with us! This is also the closest thing to concert visuals I've seen a TD tutorial. Would love to see more projects and videos of similar complexity to help some of us learn more about making groundbreaking concert visuals!

[@freivonstil]

On point as usual! Honestly, i was trying to find a way to do smooth recording with SD and there you are with a tutorial for exactly that. Thank you! :)

[@elekktronaut]

you're very welcome :)

[@thechrisdo]

Really appreciate these videos! It's great to learn a lot about Touchdesigner, eventhough I've only used the program once before. In other words your great at doing tutorials, and I'd love to see more of these AI explorations!

[@elekktronaut]

❤️🙏

[@aulerius]

When you say you started incorporating SD into feedback loops, were there any particularly important problems to solve or "gotchas" about it? Like working with Realtime flag off, checking when things finish cooking etc? I've been using TD in feedback loops since before StreamDiffusion, using A1111 API for example (either by DotSimulate or Oleg Chomp), and I've repeatedly had issues synchronizing everything and having it ticking reliably if Diffusion happens somewhere in the middle of a complex network. The issue usually was that these operators execute in an async manner, meaning it's not following the usual TD dependency chain of operations, leading to unexpected behavior. It could be that none of that is even an issue with SD by DotSimulate. If so I'd need to investigate.

[@nedschi]

Thanks a lot for all your videos. I watched you complete beginner course. And I have to admit, I'm still a little confused. Even though I'm not unfamiliar with nodes in Foundry Modo and Fusion (Davinci Resolve). Do you also have experience with Cavalry? I'm currently torn about which software to learn. TouchDesigner seems to be even more powerful. But also harder to learn. Or am I wrong? I could't find a comparison between Cavalry and TouchDesigner.

[@thechrisdo]

most of the times it's a good signal when something is harder to learn

[@nedschi]

@@thechrisdo You could be right. 🙄

[@markus_knoedel]

Great share. Thank you. To bad it only works with TD 23.

[@amkkart]

Danke für dieses tutorial aber ich bräuchte echt deine hilfe

[@gnome3d750]

Quick question, in the beginning when I change the time to Run Independently - when I hit pause in the main project folder, it pauses. And then when I go into the "independent" node and look at the local node, it also seems to pause. I'm unable to get them to run separately but even after following your steps, I dont see the timeline running independently when my project is paused. Can you help?

[@gnome3d750]

nevermind! I restarted TD and it worked lol

[@ysy69]

Wonderful. All the applause for your contribution. Really cool! When replacing with another music, in order to gain more influence, I need to play and make changes into the parameters right? Like your example uses snare, but could be another element from the music right? Say, the piano notes?

[@elekktronaut]

thanks so much ❤️ yea exactly. there are other beat detection components on my patreon or use any channel from the audio analysis in the pallete :) piano notes could come from something like my midi keyboard visualisation tutorial

[@ysy69]

@@elekktronaut Just checked the midi keyboard tutorial. I was thinking how a simpler method would be to use a 3rd party tool that isolates each instrument, then bring the piano track into TD and use it to drive the visuals?

[@DegtyarConsulting]

This is fucking crazy man

[@anastasia.macmanous]

❤

[@pookienumnums]

dotsimulate machineburners lets fucking goooooooooooooooo

[@arsen_wasted]

Hey pal, love your channel, but sorry for off topic - did you put away Notch, and what do you think about future of TD and Notch? In theses, maybe? Just interested about your thoughts. And of course, thanks for another amazing video! Feel free to make a long comment or sort of podcast, I mean everything is changing so fast, especially real-time things and I think you're one of the guy who can tell something. Thanks again!

[@elekktronaut]

I've turned away from Notch, I want to completely focus on TD and now AI as well :) Idk about the future of Notch, it's definitely an amazing tool but I don't see a reason for me to get into it now TD is this powerful. You're very welcome!

[@AlphaMoonbase]

Did you record this in a church? the reverb is nice though :)

[@elekktronaut]

new apartment that isn't finished setting up 🙃

[@nico3144]

🤍🤍🤍🤍