From Sim-to-Real: Toward General Event-based Low-light Frame Interpolation with Per-scene Optimization
<|reference_start|>From Sim-to-Real: Toward General Event-based Low-light Frame Interpolation with Per-scene Optimization: Video Frame Interpolation (VFI) is important for video enhancement, frame rate up-conversion, and slow-motion generation. The introduction of event cameras, which capture per-pixel brightness changes asynchronously, has significantly enhanced VFI capabilities, particularly for high-speed, nonlinear motions. However, these event-based methods encounter challenges in low-light conditions, notably trailing artifacts and signal latency, which hinder their direct applicability and generalization. Addressing these issues, we propose a novel per-scene optimization strategy tailored for low-light conditions. This approach utilizes the internal statistics of a sequence to handle degraded event data under low-light conditions, improving the generalizability to different lighting and camera settings. To evaluate its robustness in low-light condition, we further introduce EVFI-LL, a unique RGB+Event dataset captured under low-light conditions. Our results demonstrate state-of-the-art performance in low-light environments. Project page: https://naturezhanghn.github.io/sim2real.<|reference_end|>
@article{zhang2024from,
    title={From Sim-to-Real: Toward General Event-based Low-light Frame
  Interpolation with Per-scene Optimization},
    author={Ziran Zhang, Yongrui Ma, Yueting Chen, Feng Zhang, Jinwei Gu, Tianfan
  Xue, Shi Guo},
    journal={arXiv preprint arXiv:2406.08090},
    year={2024},
    doi={10.1145/3680528.3687649},
    archivePrefix={arXiv},
    eprint={2406.08090},
    primaryClass={cs.CV}
}