Motion-Corrected Moving Average: Including Post-Hoc Temporal Information for Improved Video Segmentation
<|reference_start|>Motion-Corrected Moving Average: Including Post-Hoc Temporal Information for Improved Video Segmentation: Real-time computational speed and a high degree of precision are requirements for computer-assisted interventions. Applying a segmentation network to a medical video processing task can introduce significant inter-frame prediction noise. Existing approaches can reduce inconsistencies by including temporal information but often impose requirements on the architecture or dataset. This paper proposes a method to include temporal information in any segmentation model and, thus, a technique to improve video segmentation performance without alterations during training or additional labeling. With Motion-Corrected Moving Average, we refine the exponential moving average between the current and previous predictions. Using optical flow to estimate the movement between consecutive frames, we can shift the prior term in the moving-average calculation to align with the geometry of the current frame. The optical flow calculation does not require the output of the model and can therefore be performed in parallel, leading to no significant runtime penalty for our approach. We evaluate our approach on two publicly available segmentation datasets and two proprietary endoscopic datasets and show improvements over a baseline approach.<|reference_end|>
@article{mendel2024motion-corrected,
    title={Motion-Corrected Moving Average: Including Post-Hoc Temporal Information
  for Improved Video Segmentation},
    author={Robert Mendel (1), Tobias Rueckert (1), Dirk Wilhelm (2), Daniel
  Rueckert (3, 4), Christoph Palm (1, 5) ((1) Regensburg Medical Image
  Computing (ReMIC), Ostbayerische Technische Hochschule Regensburg (OTH
  Regensburg), Regensburg, Germany, (2) Department of Surgery, Faculty of
  Medicine, Klinikum rechts der Isar, Technical University of Munich, Munich,
  Germany, (3) Artificial Intelligence in Healthcare and Medicine, Klinikum
  rechts der Isar, Technical University of Munich, Munich, Germany, (4)
  Department of Computing, Imperial College London, London, UK, (5) Regensburg
  Center of Health Sciences and Technology (RCHST), OTH Regensburg, Regensburg,
  Germany)},
    journal={arXiv preprint arXiv:2403.03120},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.03120},
    primaryClass={cs.CV}
}