title: Seeing the World through Your Eyes
summary: The reflective nature of the human eye is an underappreciated source of
  information about what the world around us looks like. By imaging the eyes of a
  moving person, we can collect multiple views of a scene outside the camera's
  direct line of sight through the reflections in the eyes. In this paper, we
  reconstruct a 3D scene beyond the camera's line of sight using portrait images
  containing eye reflections. This task is challenging due to 1) the difficulty
  of accurately estimating eye poses and 2) the entangled appearance of the eye
  iris and the scene reflections. Our method jointly refines the cornea poses,
  the radiance field depicting the scene, and the observer's eye iris texture. We
  further propose a simple regularization prior on the iris texture pattern to
  improve reconstruction quality. Through various experiments on synthetic and
  real-world captures featuring people with varied eye colors, we demonstrate the
  feasibility of our approach to recover 3D scenes using eye reflections.
paper_authors._id: 648bcc25a566447524183c7d
paper_authors.name: Hadi Alzayer
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-19T07:16:03.563Z
paper_authors._id: 648d554fba166fc822582b14
paper_authors.avatarUrl: /avatars/93072cde421c9d7d25614f967a9aa45a.svg
paper_authors.fullname: Hadi Alzayer
paper_authors.type: user
paper_authors.user: HadiZayer
paper_authors._id: 648bcc25a566447524183c7e
paper_authors.name: Kevin Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-16T17:38:39.331Z
paper_authors._id: 648c6e5111ee31147d61aa0d
paper_authors.avatarUrl: /avatars/5c90681d63cb3297176b1a68d024cc39.svg
paper_authors.fullname: Kevin Zhang
paper_authors.type: user
paper_authors.user: kzhang0
paper_authors._id: 648bcc25a566447524183c7f
paper_authors.name: Brandon Feng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-16T18:50:47.635Z
paper_authors._id: 634ae1fbe97a3d183f2cf063
paper_authors.avatarUrl: /avatars/3e05289d1876d7b7ef2c6a595a7f39d7.svg
paper_authors.fullname: Brandon Feng
paper_authors.type: user
paper_authors.user: brandonyfeng
paper_authors._id: 648bcc25a566447524183c80
paper_authors.name: Christopher Metzler
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-16T13:57:23.898Z
paper_authors._id: 631621faf5e32157c51cc38d
paper_authors.avatarUrl: /avatars/0659d41aab160b3997f52c9c8fd5f2c1.svg
paper_authors.fullname: chris metzler
paper_authors.type: user
paper_authors.user: strikemetz
paper_authors._id: 648bcc25a566447524183c81
paper_authors.name: Jia-Bin Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-16T07:20:18.098Z
paper_authors._id: 641c139b73296f7ee256970c
paper_authors.avatarUrl: /avatars/5a2550d95e686640242840ad3bd0e680.svg
paper_authors.fullname: Jiabin Huang
paper_authors.type: user
paper_authors.user: YellowAddice
paper_title: Seeing the World through Your Eyes
paper_summary: The reflective nature of the human eye is an underappreciated source of
  information about what the world around us looks like. By imaging the eyes of a
  moving person, we can collect multiple views of a scene outside the camera's
  direct line of sight through the reflections in the eyes. In this paper, we
  reconstruct a 3D scene beyond the camera's line of sight using portrait images
  containing eye reflections. This task is challenging due to 1) the difficulty
  of accurately estimating eye poses and 2) the entangled appearance of the eye
  iris and the scene reflections. Our method jointly refines the cornea poses,
  the radiance field depicting the scene, and the observer's eye iris texture. We
  further propose a simple regularization prior on the iris texture pattern to
  improve reconstruction quality. Through various experiments on synthetic and
  real-world captures featuring people with varied eye colors, we demonstrate the
  feasibility of our approach to recover 3D scenes using eye reflections.