Tree-Structured Shading Decomposition
We study inferring a tree-structured representation from a single image for
object shading. Prior work typically uses the parametric or measured
representation to model shading, which is neither interpretable nor easily
editable. We propose using the shade tree representation, which combines basic
shading nodes and compositing methods to factorize object surface shading. The
shade tree representation enables novice users who are unfamiliar with the
physical shading process to edit object shading in an efficient and intuitive
manner. A main challenge in inferring the shade tree is that the inference
problem involves both the discrete tree structure and the continuous parameters
of the tree nodes. We propose a hybrid approach to address this issue. We
introduce an auto-regressive inference model to generate a rough estimation of
the tree structure and node parameters, and then we fine-tune the inferred
shade tree through an optimization algorithm. We show experiments on synthetic
images, captured reflectance, real images, and non-realistic vector drawings,
allowing downstream applications such as material editing, vectorized shading,
and relighting. Project website: https://chen-geng.com/inv-shade-trees
6502a153e9230a325e02309c
Chen Geng
claimed_verified
2023-09-15T07:40:22.344Z
6305b64699870e13d3e1cb50
/avatars/e38fc1ad7b147036b98cd5ac6cd37516.svg
Chen Geng
user
MrMorning
6502a153e9230a325e02309d
Hong-Xing Yu
claimed_verified
2023-09-15T07:40:19.281Z
64ca8e0a8a16b1748fe44f21
/avatars/0b56a1634dcd562c6516f26b5e9cde14.svg
Hong-Xing "Koven" Yu
user
Koooven
6502a153e9230a325e02309e
Sharon Zhang
6502a153e9230a325e02309f
Maneesh Agrawala
admin_assigned
2023-09-14T07:49:03.843Z
642bb3e44f379cdc3deb26cd
/avatars/556e792f013bcbe3748d7c9c719070d8.svg
Maneesh Agrawala
user
magrawala
6502a153e9230a325e0230a0
Jiajun Wu
admin_assigned
2023-09-14T07:49:25.150Z
647b7b8345616c3de3627863
/avatars/8196364bffed71086ba9a0a4a62027d3.svg
Jiajun Wu
user
jiajunwu
Tree-Structured Shading Decomposition
We study inferring a tree-structured representation from a single image for
object shading. Prior work typically uses the parametric or measured
representation to model shading, which is neither interpretable nor easily
editable. We propose using the shade tree representation, which combines basic
shading nodes and compositing methods to factorize object surface shading. The
shade tree representation enables novice users who are unfamiliar with the
physical shading process to edit object shading in an efficient and intuitive
manner. A main challenge in inferring the shade tree is that the inference
problem involves both the discrete tree structure and the continuous parameters
of the tree nodes. We propose a hybrid approach to address this issue. We
introduce an auto-regressive inference model to generate a rough estimation of
the tree structure and node parameters, and then we fine-tune the inferred
shade tree through an optimization algorithm. We show experiments on synthetic
images, captured reflectance, real images, and non-realistic vector drawings,
allowing downstream applications such as material editing, vectorized shading,
and relighting. Project website: https://chen-geng.com/inv-shade-trees
A hybrid approach combining an auto-regressive model and optimization algorithm is proposed for inferring a shade tree representation from images, enabling intuitive shading editing.
shade tree representation auto-regressive inference model optimization algorithm material editing vectorized shading relighting