Advanced Kinect Rendering in TouchDesigner | Instancing, PBR, Depth Refinement, Shadows, and More

Project file and depth refinement component available on Patreon: https://www.patreon.com/posts/toe-advanced-in-121763070?utm_medium=clipboard_copy&utm_source=copyLink&utm_campaign=postshare_creator&utm_content=join_link

This tutorial is a deep dive into rendering Kinect data in TouchDesigner. It covers:
– isolating the subject
– basic instancing
– improved orthographic instancing
– direct rendering (new technique using a GLSL material fragment shader)
– real-time depth map refinement
– adding shadows with hybrid rendering

It contains all the main tips and tricks I’ve learned from working with Kinect throughout the past 6+ years. Hope you find it helpful! You can check out @prismatic.visuals on Instagram to see more of my work.

Links I mentioned:
– Interactive Immersive Kinect tutorial: https://www.youtube.com/watch?v=p-lhmCMxn2g&ab_channel=TheInteractive%26ImmersiveHQ
– Paketa12 normal map tox: https://derivative.ca/community-post/asset/normal-map-v2/65562
– Shadertoy: https://www.shadertoy.com/
– GLSL material documentation: https://docs.derivative.ca/Write_a_GLSL_Material
– Polyhaven HDRIs: https://polyhaven.com/hdris
– Lucas Morgan linear vs sRGB article: https://forum.derivative.ca/t/linear-vs-gamma-pbr-workflows-in-touch/135812/2
– Learn OpenGL shadow mapping: https://learnopengl.com/Advanced-Lighting/Shadows/Shadow-Mapping
– Exsstas render buffers tutorial: https://www.youtube.com/watch?v=vUxXJ5IcaUQ&t=21s&ab_channel=exsstas

Contents:
0:00 – intro
0:40 – outline
1:56 – isolating the subject
8:49 – basic instancing
10:52 – improved instancing
19:10 – direct rendering setup
26:21 – direct rendering implementation
32:03 – adding reflections
34:59 – depth refinement
41:34 – shadows
49:01 – outro