title: RenderFormer: Transformer-based Neural Rendering of Triangle Meshes with
    Global Illumination
summary: We present RenderFormer, a neural rendering pipeline that directly renders an
  image from a triangle-based representation of a scene with full global
  illumination effects and that does not require per-scene training or
  fine-tuning. Instead of taking a physics-centric approach to rendering, we
  formulate rendering as a sequence-to-sequence transformation where a sequence
  of tokens representing triangles with reflectance properties is converted to a
  sequence of output tokens representing small patches of pixels. RenderFormer
  follows a two stage pipeline: a view-independent stage that models
  triangle-to-triangle light transport, and a view-dependent stage that
  transforms a token representing a bundle of rays to the corresponding pixel
  values guided by the triangle-sequence from the view-independent stage. Both
  stages are based on the transformer architecture and are learned with minimal
  prior constraints. We demonstrate and evaluate RenderFormer on scenes with
  varying complexity in shape and light transport.
paper_authors._id: 6837c23acce400abe6f18790
paper_authors.name: Chong Zeng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-29T07:42:38.730Z
paper_authors._id: 60747cbf3ea03830676542b5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60747cbf3ea03830676542b5/wGr1Jzz520JM9nZ-UcLyb.png
paper_authors.fullname: Chong Zeng
paper_authors.type: user
paper_authors.user: NCJ
paper_authors._id: 6837c23acce400abe6f18791
paper_authors.name: Yue Dong
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-05-29T02:11:08.981Z
paper_authors._id: 63299011bdb6242b42b77f57
paper_authors.avatarUrl: /avatars/056aec97eb3c10d3b63eb13238e1d2a4.svg
paper_authors.fullname: doyleconan
paper_authors.type: user
paper_authors.user: doyleconan
paper_authors._id: 6837c23acce400abe6f18792
paper_authors.name: Pieter Peers
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-05-29T13:00:36.601Z
paper_authors._id: 68385a2de4660611f713a529
paper_authors.avatarUrl: /avatars/b994efd6170d985d7da46aad9cc90dcf.svg
paper_authors.fullname: Pieter Peers
paper_authors.type: user
paper_authors.user: PieterPeers
paper_authors._id: 6837c23acce400abe6f18793
paper_authors.name: Hongzhi Wu
paper_authors._id: 6837c23acce400abe6f18794
paper_authors.name: Xin Tong
paper_title: RenderFormer: Transformer-based Neural Rendering of Triangle Meshes with
    Global Illumination
paper_summary: We present RenderFormer, a neural rendering pipeline that directly renders an
  image from a triangle-based representation of a scene with full global
  illumination effects and that does not require per-scene training or
  fine-tuning. Instead of taking a physics-centric approach to rendering, we
  formulate rendering as a sequence-to-sequence transformation where a sequence
  of tokens representing triangles with reflectance properties is converted to a
  sequence of output tokens representing small patches of pixels. RenderFormer
  follows a two stage pipeline: a view-independent stage that models
  triangle-to-triangle light transport, and a view-dependent stage that
  transforms a token representing a bundle of rays to the corresponding pixel
  values guided by the triangle-sequence from the view-independent stage. Both
  stages are based on the transformer architecture and are learned with minimal
  prior constraints. We demonstrate and evaluate RenderFormer on scenes with
  varying complexity in shape and light transport.
paper_ai_summary: RenderFormer is a transformer-based neural rendering pipeline that renders images from triangle representations without per-scene training and with full global illumination effects.
paper_ai_keywords: neural rendering pipeline global illumination effects sequence-to-sequence transformation tokens reflectance properties pixel patches transformer architecture view-independent stage view-dependent stage triangle-to-triangle light transport ray bundles