Seeing the World through Your Eyes
The reflective nature of the human eye is an underappreciated source of
information about what the world around us looks like. By imaging the eyes of a
moving person, we can collect multiple views of a scene outside the camera's
direct line of sight through the reflections in the eyes. In this paper, we
reconstruct a 3D scene beyond the camera's line of sight using portrait images
containing eye reflections. This task is challenging due to 1) the difficulty
of accurately estimating eye poses and 2) the entangled appearance of the eye
iris and the scene reflections. Our method jointly refines the cornea poses,
the radiance field depicting the scene, and the observer's eye iris texture. We
further propose a simple regularization prior on the iris texture pattern to
improve reconstruction quality. Through various experiments on synthetic and
real-world captures featuring people with varied eye colors, we demonstrate the
feasibility of our approach to recover 3D scenes using eye reflections.
648bcc25a566447524183c7d
Hadi Alzayer
claimed_verified
2023-06-19T07:16:03.563Z
648d554fba166fc822582b14
/avatars/93072cde421c9d7d25614f967a9aa45a.svg
Hadi Alzayer
user
HadiZayer
648bcc25a566447524183c7e
Kevin Zhang
claimed_verified
2023-06-16T17:38:39.331Z
648c6e5111ee31147d61aa0d
/avatars/5c90681d63cb3297176b1a68d024cc39.svg
Kevin Zhang
user
kzhang0
648bcc25a566447524183c7f
Brandon Feng
claimed_verified
2023-06-16T18:50:47.635Z
634ae1fbe97a3d183f2cf063
/avatars/3e05289d1876d7b7ef2c6a595a7f39d7.svg
Brandon Feng
user
brandonyfeng
648bcc25a566447524183c80
Christopher Metzler
claimed_verified
2023-06-16T13:57:23.898Z
631621faf5e32157c51cc38d
/avatars/0659d41aab160b3997f52c9c8fd5f2c1.svg
chris metzler
user
strikemetz
648bcc25a566447524183c81
Jia-Bin Huang
admin_assigned
2023-06-16T07:20:18.098Z
641c139b73296f7ee256970c
/avatars/5a2550d95e686640242840ad3bd0e680.svg
Jiabin Huang
user
YellowAddice
Seeing the World through Your Eyes
The reflective nature of the human eye is an underappreciated source of
information about what the world around us looks like. By imaging the eyes of a
moving person, we can collect multiple views of a scene outside the camera's
direct line of sight through the reflections in the eyes. In this paper, we
reconstruct a 3D scene beyond the camera's line of sight using portrait images
containing eye reflections. This task is challenging due to 1) the difficulty
of accurately estimating eye poses and 2) the entangled appearance of the eye
iris and the scene reflections. Our method jointly refines the cornea poses,
the radiance field depicting the scene, and the observer's eye iris texture. We
further propose a simple regularization prior on the iris texture pattern to
improve reconstruction quality. Through various experiments on synthetic and
real-world captures featuring people with varied eye colors, we demonstrate the
feasibility of our approach to recover 3D scenes using eye reflections.
A method for reconstructing 3D scenes beyond a camera's line of sight using eye reflections is proposed, refining cornea poses, radiance fields, and iris textures.
cornea poses radiance field iris textures iris texture pattern regularization