Decomposing shading into a tree-structured representation. Our method enables the decomposition of given shading into a shade tree. This representation can be reused to generate new shade trees and edit the shading of objects.

Abstract

We study inferring a tree-structured representation from a single image for object shading. Prior work typically uses the parametric or measured representation to model shading, which is neither interpretable nor easily editable. We propose using the shade tree representation, which combines basic shading nodes and compositing methods to factorize object surface shading. The shade tree representation enables novice users who are unfamiliar with the physical shading process to edit object shading in an efficient and intuitive manner. A main challenge in inferring the shade tree is that the inference problem involves both the discrete tree structure and the continuous parameters of the tree nodes. We propose a hybrid approach to address this issue. We introduce an auto-regressive inference model to generate a rough estimation of the tree structure and node parameters, and then we fine-tune the inferred shade tree through an optimization algorithm. We show experiments on synthetic images, captured reflectance, real images, and non-realistic vector drawings, allowing downstream applications such as material editing, vectorized shading, and relighting.

Introductory Video

Qualitative Comparison

Decomposing In-the-world Shadings

Our method can not only work on synthetic data but can also be widely used in the decomposition of in-the-wild shading. The shadings in (a, b, c) are collected from the Internet. In (d), we show that our method can do decomposition of the shadings from Lopez-Moreno et al. In (e), we compare our method with Richardt et al. We first extract shading from the vector drawing, and then we use our method to do decomposition to the shading sphere.

Citation

The website template was borrowed from Michaël Gharbi and Jon Barron.