Sonic Flowers – TouchDesigner x StableDiffusion Tutorial 1

In this tutorial we’re having a first look at how to integrate the image-generation AI tool #stablediffusion into TouchDesigner. We’re creating an audio-reactive texture in an independant component to be able to do frame by frame animation without worrying about losing frames. TD is running in real-time, but the process itself is not (yet) real-time, as one frame takes 5-10 seconds to be created.

Please mind that this is very experimental and just one way (my current approach). I am very open to hear about ways of improving this or ideas on how to expand it. See this more of a starting point and inspiration than a perfectly refined technique.

Make sure that you download the automatic1111 WebUI (below) and some models to work with. The models should be placed inside “stable-diffusion-webuimodelsStable-diffusion”.
Only tested on Windows, running on an Nvidia RTX 3070 Notebook version.

dotsimulate’s SD_API: https://www.patreon.com/posts/sd-api-1-22-85238082
automatic1111: https://github.com/AUTOMATIC1111/stable-diffusion-webui
Models: https://civitai.com/
Parameters explained: https://blog.openart.ai/2023/02/13/the-most-complete-guide-to-stable-diffusion-parameters/
Create your own API: https://www.youtube.com/watch?v=4khcLvGjoX8
IG Post: https://www.instagram.com/reel/Ctue_S_o9eF/?igshid=NTc4MTIwNjQ2YQ==

The prompt used for this example:
ultrarealistic surreal flowers, ultra detailed, texture, generative art, focus, wes anderson, kodak, light and shadow

—–
00:00 Intro & Overview
02:40 Independant Comp & Audio Setup
07:50 Audioreactive Circles
16:00 Circles Processing
18:30 SD API Component
27:54 Feedback Loop
32:58 Frame by Frame
36:52 Exporting
40:34 Additional Info
43:16 Outro
—–

Files, exclusive content and more:
https://www.patreon.com/elekktronaut

The special thanks is integrated at the end of the video ❤
If you have any questions, feel free to ask.

IG: @elekktronaut
https://www.instagram.com/elekktronaut/