title: SportsSloMo: A New Benchmark and Baselines for Human-centric Video Frame
    Interpolation
summary: Human-centric video frame interpolation has great potential for improving
  people's entertainment experiences and finding commercial applications in the
  sports analysis industry, e.g., synthesizing slow-motion videos. Although there
  are multiple benchmark datasets available in the community, none of them is
  dedicated for human-centric scenarios. To bridge this gap, we introduce
  SportsSloMo, a benchmark consisting of more than 130K video clips and 1M video
  frames of high-resolution (geq720p) slow-motion sports videos crawled from
  YouTube. We re-train several state-of-the-art methods on our benchmark, and the
  results show a decrease in their accuracy compared to other datasets. It
  highlights the difficulty of our benchmark and suggests that it poses
  significant challenges even for the best-performing methods, as human bodies
  are highly deformable and occlusions are frequent in sports videos. To improve
  the accuracy, we introduce two loss terms considering the human-aware priors,
  where we add auxiliary supervision to panoptic segmentation and human keypoints
  detection, respectively. The loss terms are model agnostic and can be easily
  plugged into any video frame interpolation approaches. Experimental results
  validate the effectiveness of our proposed loss terms, leading to consistent
  performance improvement over 5 existing models, which establish strong baseline
  models on our benchmark. The dataset and code can be found at:
  https://neu-vi.github.io/SportsSlomo/.
paper_authors._id: 64f1a4c3d52925c13309bc75
paper_authors.name: Jiaben Chen
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2023-09-05T20:01:14.056Z
paper_authors._id: 64f788ea9980b96c33e2b8fe
paper_authors.avatarUrl: /avatars/b9fc9a93550df79033f86758fc8cbc5d.svg
paper_authors.fullname: Jiaben Chen
paper_authors.type: user
paper_authors.user: Jiaben
paper_authors._id: 64f1a4c3d52925c13309bc76
paper_authors.name: Huaizu Jiang
paper_title: SportsSloMo: A New Benchmark and Baselines for Human-centric Video Frame
    Interpolation
paper_summary: Human-centric video frame interpolation has great potential for improving
  people's entertainment experiences and finding commercial applications in the
  sports analysis industry, e.g., synthesizing slow-motion videos. Although there
  are multiple benchmark datasets available in the community, none of them is
  dedicated for human-centric scenarios. To bridge this gap, we introduce
  SportsSloMo, a benchmark consisting of more than 130K video clips and 1M video
  frames of high-resolution (geq720p) slow-motion sports videos crawled from
  YouTube. We re-train several state-of-the-art methods on our benchmark, and the
  results show a decrease in their accuracy compared to other datasets. It
  highlights the difficulty of our benchmark and suggests that it poses
  significant challenges even for the best-performing methods, as human bodies
  are highly deformable and occlusions are frequent in sports videos. To improve
  the accuracy, we introduce two loss terms considering the human-aware priors,
  where we add auxiliary supervision to panoptic segmentation and human keypoints
  detection, respectively. The loss terms are model agnostic and can be easily
  plugged into any video frame interpolation approaches. Experimental results
  validate the effectiveness of our proposed loss terms, leading to consistent
  performance improvement over 5 existing models, which establish strong baseline
  models on our benchmark. The dataset and code can be found at:
  https://neu-vi.github.io/SportsSlomo/.
paper_ai_keywords: SportsSloMo panoptic segmentation human keypoints reconstruction