UrbanIR: Large-Scale Urban Scene Inverse Rendering from a Single Video
We show how to build a model that allows realistic, free-viewpoint renderings
of a scene under novel lighting conditions from video. Our method -- UrbanIR:
Urban Scene Inverse Rendering -- computes an inverse graphics representation
from the video. UrbanIR jointly infers shape, albedo, visibility, and sun and
sky illumination from a single video of unbounded outdoor scenes with unknown
lighting. UrbanIR uses videos from cameras mounted on cars (in contrast to many
views of the same points in typical NeRF-style estimation). As a result,
standard methods produce poor geometry estimates (for example, roofs), and
there are numerous ''floaters''. Errors in inverse graphics inference can
result in strong rendering artifacts. UrbanIR uses novel losses to control
these and other sources of error. UrbanIR uses a novel loss to make very good
estimates of shadow volumes in the original scene. The resulting
representations facilitate controllable editing, delivering photorealistic
free-viewpoint renderings of relit scenes and inserted objects. Qualitative
evaluation demonstrates strong improvements over the state-of-the-art.
648bc91aded16248631ede5c
Zhi-Hao Lin
admin_assigned
2023-06-16T17:02:03.385Z
632a340c5f2ff1958c014191
/avatars/d51ccb2e2b3e9756dea3d06b6e62f438.svg
Chih-Hao Lin
user
chih-hao-lin
648bc91aded16248631ede5d
Bohan Liu
648bc91aded16248631ede5e
Yi-Ting Chen
648bc91aded16248631ede5f
David Forsyth
648bc91aded16248631ede60
Jia-Bin Huang
admin_assigned
2023-06-16T16:49:05.020Z
641c139b73296f7ee256970c
/avatars/5a2550d95e686640242840ad3bd0e680.svg
Jiabin Huang
user
YellowAddice
648bc91aded16248631ede61
Anand Bhattad
admin_assigned
2023-06-16T17:04:59.622Z
6303d5924ec2dfa82a5627de
/avatars/1e433b7fb9d30b9f42dec487d658b1e3.svg
Anand Bhattad
user
anandbhattad
648bc91aded16248631ede62
Shenlong Wang
UrbanIR: Large-Scale Urban Scene Inverse Rendering from a Single Video
We show how to build a model that allows realistic, free-viewpoint renderings
of a scene under novel lighting conditions from video. Our method -- UrbanIR:
Urban Scene Inverse Rendering -- computes an inverse graphics representation
from the video. UrbanIR jointly infers shape, albedo, visibility, and sun and
sky illumination from a single video of unbounded outdoor scenes with unknown
lighting. UrbanIR uses videos from cameras mounted on cars (in contrast to many
views of the same points in typical NeRF-style estimation). As a result,
standard methods produce poor geometry estimates (for example, roofs), and
there are numerous ''floaters''. Errors in inverse graphics inference can
result in strong rendering artifacts. UrbanIR uses novel losses to control
these and other sources of error. UrbanIR uses a novel loss to make very good
estimates of shadow volumes in the original scene. The resulting
representations facilitate controllable editing, delivering photorealistic
free-viewpoint renderings of relit scenes and inserted objects. Qualitative
evaluation demonstrates strong improvements over the state-of-the-art.
UrbanIR computes an inverse graphics representation from video for realistic relighting and object insertion in urban scenes with novel lighting conditions.
inverse rendering NeRF geometry estimates shadow volumes free-viewpoint rendering photorealistic