Gemini Network for Temporal Action Localization
Temporal Action Localization in untrimmed videos is an important yet difficult task. It's important for accurate temporal action localization to model temporal structure. In this work, we propose a novel model, Gemini CNN, which can effectively model temporal structure via several separated CNNs. Our first contribution is the idea of modular design of temporal structure modeling. Temporal structure is divided into long range temporal structure and short range temporal structure. Two types of temporal structures are respectively modeled in two subnets. The other contribution is the idea of auxiliary supervision. We add classifiers at not only the end of our framework, but also intermediate layers. The loss of auxiliary classifiers increases the gradient signal that gets propagated back, and boosts our framework's performance. We achieve state of the art performance for temporal action localization on two challenging temporal action localization datasets, THUMOS14 and ActivityNet.
PRCV