Real-Time Neural Appearance Models
We present a complete system for real-time rendering of scenes with complex
appearance previously reserved for offline use. This is achieved with a
combination of algorithmic and system level innovations.
  Our appearance model utilizes learned hierarchical textures that are
interpreted using neural decoders, which produce reflectance values and
importance-sampled directions. To best utilize the modeling capacity of the
decoders, we equip the decoders with two graphics priors. The first prior --
transformation of directions into learned shading frames -- facilitates
accurate reconstruction of mesoscale effects. The second prior -- a microfacet
sampling distribution -- allows the neural decoder to perform importance
sampling efficiently. The resulting appearance model supports anisotropic
sampling and level-of-detail rendering, and allows baking deeply layered
material graphs into a compact unified neural representation.
  By exposing hardware accelerated tensor operations to ray tracing shaders, we
show that it is possible to inline and execute the neural decoders efficiently
inside a real-time path tracer. We analyze scalability with increasing number
of neural materials and propose to improve performance using code optimized for
coherent and divergent execution. Our neural material shaders can be over an
order of magnitude faster than non-neural layered materials. This opens up the
door for using film-quality visuals in real-time applications such as games and
live previews.
645480ec7f7b2bed1a0391a1
Tizian Zeltner
645480ec7f7b2bed1a0391a2
Fabrice Rousselle
645480ec7f7b2bed1a0391a3
Andrea Weidlich
645480ec7f7b2bed1a0391a4
Petrik Clarberg
645480ec7f7b2bed1a0391a5
Jan Novák
645480ec7f7b2bed1a0391a6
Benedikt Bitterli
645480ec7f7b2bed1a0391a7
Alex Evans
645480ec7f7b2bed1a0391a8
Tomáš Davidovič
645480ec7f7b2bed1a0391a9
Simon Kallweit
645480ec7f7b2bed1a0391aa
Aaron Lefohn
Real-Time Neural Appearance Models
We present a complete system for real-time rendering of scenes with complex
appearance previously reserved for offline use. This is achieved with a
combination of algorithmic and system level innovations.
  Our appearance model utilizes learned hierarchical textures that are
interpreted using neural decoders, which produce reflectance values and
importance-sampled directions. To best utilize the modeling capacity of the
decoders, we equip the decoders with two graphics priors. The first prior --
transformation of directions into learned shading frames -- facilitates
accurate reconstruction of mesoscale effects. The second prior -- a microfacet
sampling distribution -- allows the neural decoder to perform importance
sampling efficiently. The resulting appearance model supports anisotropic
sampling and level-of-detail rendering, and allows baking deeply layered
material graphs into a compact unified neural representation.
  By exposing hardware accelerated tensor operations to ray tracing shaders, we
show that it is possible to inline and execute the neural decoders efficiently
inside a real-time path tracer. We analyze scalability with increasing number
of neural materials and propose to improve performance using code optimized for
coherent and divergent execution. Our neural material shaders can be over an
order of magnitude faster than non-neural layered materials. This opens up the
door for using film-quality visuals in real-time applications such as games and
live previews.
Real-time rendering of complex scenes is achieved using learned hierarchical textures interpreted by neural decoders, supported by hardware-accelerated tensor operations in ray tracing shaders.
hierarchical textures neural decoders reflectance values importance-sampled directions graphics priors learned shading frames microfacet sampling distribution anisotropic sampling level-of-detail rendering compact unified neural representation hardware accelerated tensor operations real-time path tracer neural material shaders coherent execution divergent execution