RenderFormer: Transformer-based Neural Rendering of Triangle Meshes with
  Global Illumination
We present RenderFormer, a neural rendering pipeline that directly renders an
image from a triangle-based representation of a scene with full global
illumination effects and that does not require per-scene training or
fine-tuning. Instead of taking a physics-centric approach to rendering, we
formulate rendering as a sequence-to-sequence transformation where a sequence
of tokens representing triangles with reflectance properties is converted to a
sequence of output tokens representing small patches of pixels. RenderFormer
follows a two stage pipeline: a view-independent stage that models
triangle-to-triangle light transport, and a view-dependent stage that
transforms a token representing a bundle of rays to the corresponding pixel
values guided by the triangle-sequence from the view-independent stage. Both
stages are based on the transformer architecture and are learned with minimal
prior constraints. We demonstrate and evaluate RenderFormer on scenes with
varying complexity in shape and light transport.
6837c23acce400abe6f18790
Chong Zeng
claimed_verified
2025-05-29T07:42:38.730Z
60747cbf3ea03830676542b5
https://cdn-avatars.huggingface.co/v1/production/uploads/60747cbf3ea03830676542b5/wGr1Jzz520JM9nZ-UcLyb.png
Chong Zeng
user
NCJ
6837c23acce400abe6f18791
Yue Dong
extracted_pending
2025-05-29T02:11:08.981Z
63299011bdb6242b42b77f57
/avatars/056aec97eb3c10d3b63eb13238e1d2a4.svg
doyleconan
user
doyleconan
6837c23acce400abe6f18792
Pieter Peers
extracted_pending
2025-05-29T13:00:36.601Z
68385a2de4660611f713a529
/avatars/b994efd6170d985d7da46aad9cc90dcf.svg
Pieter Peers
user
PieterPeers
6837c23acce400abe6f18793
Hongzhi Wu
6837c23acce400abe6f18794
Xin Tong
RenderFormer: Transformer-based Neural Rendering of Triangle Meshes with
  Global Illumination
We present RenderFormer, a neural rendering pipeline that directly renders an
image from a triangle-based representation of a scene with full global
illumination effects and that does not require per-scene training or
fine-tuning. Instead of taking a physics-centric approach to rendering, we
formulate rendering as a sequence-to-sequence transformation where a sequence
of tokens representing triangles with reflectance properties is converted to a
sequence of output tokens representing small patches of pixels. RenderFormer
follows a two stage pipeline: a view-independent stage that models
triangle-to-triangle light transport, and a view-dependent stage that
transforms a token representing a bundle of rays to the corresponding pixel
values guided by the triangle-sequence from the view-independent stage. Both
stages are based on the transformer architecture and are learned with minimal
prior constraints. We demonstrate and evaluate RenderFormer on scenes with
varying complexity in shape and light transport.
RenderFormer is a transformer-based neural rendering pipeline that renders images from triangle representations without per-scene training and with full global illumination effects.
neural rendering pipeline global illumination effects sequence-to-sequence transformation tokens reflectance properties pixel patches transformer architecture view-independent stage view-dependent stage triangle-to-triangle light transport ray bundles