Texturing 3D scenes with AI

Exporting depth passes and clay renders from a kitbashed 3D scene to guide AI generation I was able art direct the texturing of a scene.

final textured result

To ensure I could art direct the scene as I wanted. I created several generations with different seed values and cherry picked my favorite pieces of each. I brought those into Photoshop and photobashed my favorite pieces together. I then passed that composite through Stable Diffusion again at a low denoise value to tie them all together and make the lighting consistent. The final result had exactly the texturing I wanted for each part of the render.

Stable Diffusion takes an inverted Z depth pass, where white is close to the camera with black being the far point

Clay render of the scene to help guide the generation more

I also created other variations of the scene after adding in some posed characters I downloaded from Mixamo and by using higher denoise values.

alt version of the render with added characters

Watch the tutorial: