  Previous work has demonstrated that virtual accelerometry data, extracted
from videos using cross-modality transfer approaches like IMUTube, is
beneficial for training complex and effective human activity recognition (HAR)
models. Systems like IMUTube were originally designed to cover activities that
are based on substantial body (part) movements. Yet, life is complex, and a
range of activities of daily living is based on only rather subtle movements,
which bears the question to what extent systems like IMUTube are of value also
for fine-grained HAR, i.e., When does IMUTube break? In this work we first
introduce a measure to quantitatively assess the subtlety of human movements
that are underlying activities of interest--the motion subtlety index
(MSI)--which captures local pixel movements and pose changes in the vicinity of
target virtual sensor locations, and correlate it to the eventual activity
recognition accuracy. We then perform a "stress-test" on IMUTube and explore
for which activities with underlying subtle movements a cross-modality transfer
approach works, and for which not. As such, the work presented in this paper
allows us to map out the landscape for IMUTube applications in practical
scenarios.

Computer Vision and Pattern Recognition