Dynamic NeRFs for Soccer Scenes
The long-standing problem of novel view synthesis has many applications,
notably in sports broadcasting. Photorealistic novel view synthesis of soccer
actions, in particular, is of enormous interest to the broadcast industry. Yet
only a few industrial solutions have been proposed, and even fewer that achieve
near-broadcast quality of the synthetic replays. Except for their setup of
multiple static cameras around the playfield, the best proprietary systems
disclose close to no information about their inner workings. Leveraging
multiple static cameras for such a task indeed presents a challenge rarely
tackled in the literature, for a lack of public datasets: the reconstruction of
a large-scale, mostly static environment, with small, fast-moving elements.
Recently, the emergence of neural radiance fields has induced stunning progress
in many novel view synthesis applications, leveraging deep learning principles
to produce photorealistic results in the most challenging settings. In this
work, we investigate the feasibility of basing a solution to the task on
dynamic NeRFs, i.e., neural models purposed to reconstruct general dynamic
content. We compose synthetic soccer environments and conduct multiple
experiments using them, identifying key components that help reconstruct soccer
scenes with dynamic NeRFs. We show that, although this approach cannot fully
meet the quality requirements for the target application, it suggests promising
avenues toward a cost-efficient, automatic solution. We also make our work
dataset and code publicly available, with the goal to encourage further efforts
from the research community on the task of novel view synthesis for dynamic
soccer scenes. For code, data, and video results, please see
https://soccernerfs.isach.be.
65029e427b875426c918a24f
Sacha Lewin
admin_assigned
2023-09-14T07:42:50.811Z
633df08b55a63a501a4e264b
https://cdn-avatars.huggingface.co/v1/production/uploads/633df08b55a63a501a4e264b/vfwCnuPH47VPCVotffGZ7.png
Sacha lewin
user
iSach
65029e427b875426c918a250
Maxime Vandegar
65029e427b875426c918a251
Thomas Hoyoux
65029e427b875426c918a252
Olivier Barnich
admin_assigned
2023-09-14T07:43:10.997Z
6311e8b7ae8896941da3bdd1
/avatars/86c72945efa3bf6f291ad87b3649e778.svg
Olivier Barnich
user
olivierbarnich
65029e427b875426c918a253
Gilles Louppe
extracted_confirmed
2023-09-14T11:03:40.183Z
64103068928400b41642ad8a
/avatars/8acbe90178973081ecd1bb386e1646b5.svg
Gilles Louppe
user
glouppe
Dynamic NeRFs for Soccer Scenes
The long-standing problem of novel view synthesis has many applications,
notably in sports broadcasting. Photorealistic novel view synthesis of soccer
actions, in particular, is of enormous interest to the broadcast industry. Yet
only a few industrial solutions have been proposed, and even fewer that achieve
near-broadcast quality of the synthetic replays. Except for their setup of
multiple static cameras around the playfield, the best proprietary systems
disclose close to no information about their inner workings. Leveraging
multiple static cameras for such a task indeed presents a challenge rarely
tackled in the literature, for a lack of public datasets: the reconstruction of
a large-scale, mostly static environment, with small, fast-moving elements.
Recently, the emergence of neural radiance fields has induced stunning progress
in many novel view synthesis applications, leveraging deep learning principles
to produce photorealistic results in the most challenging settings. In this
work, we investigate the feasibility of basing a solution to the task on
dynamic NeRFs, i.e., neural models purposed to reconstruct general dynamic
content. We compose synthetic soccer environments and conduct multiple
experiments using them, identifying key components that help reconstruct soccer
scenes with dynamic NeRFs. We show that, although this approach cannot fully
meet the quality requirements for the target application, it suggests promising
avenues toward a cost-efficient, automatic solution. We also make our work
dataset and code publicly available, with the goal to encourage further efforts
from the research community on the task of novel view synthesis for dynamic
soccer scenes. For code, data, and video results, please see
https://soccernerfs.isach.be.
Dynamic NeRFs show promise for photorealistic novel view synthesis of soccer scenes, though they do not yet meet the quality standards required by the broadcast industry.
neural radiance fields NeRFs dynamic NeRFs novel view synthesis soccer scenes