Art-directing Appearance using an Environment Map Latent Space
Problem: Artists need to make stylistic 3D lighting edits uniquely for different environments, e.g. "boost rim-light when sunny", and "add highlights when cloudy". Current artistic editing techniques can't adapt to different changing lighting environments, which is needed for real-time games and AR apps.
We developed a new workflow for artists to art-direct 3D appearance across different lighting conditions. To do this we learned a latent space of varied lighting environments, that enables the following:
- In look-development, artists can intuitively browse the latent space and preview many environments organised in semantic clusters.
- Artists then paint directly on the space to specify how 3D lighting edits should vary across environments.
- Games and AR apps with dynamic environments, encode them into the latent space, and look-up the desired artistic appearance.
(a) Our environment map (EM) latent space expresses semantic clustering in 2D, for navigating a varied EM database during look development.
(b) Art directing material and lighting edits by 2D painting across the latent space.
(c) Art-directed appearance is preserved in similar EMs, and dynamically adapted to changing environment lighting.
Character model by Emiliano Colantoni