  The development of approaches for trajectory prediction requires metrics to
validate and compare their performance. Currently established metrics are based
on Euclidean distance, which means that errors are weighted equally in all
directions. Euclidean metrics are insufficient for structured environments like
roads, since they do not properly capture the agent's intent relative to the
underlying lane. In order to provide a reasonable assessment of trajectory
prediction approaches with regard to the downstream planning task, we propose a
new metric that is lane distance-based: Lane Miss Rate (LMR). For the
calculation of LMR, the ground-truth and predicted endpoints are assigned to
lane segments, more precisely their centerlines. Measured by the distance along
the lane segments, predictions that are within a certain threshold distance to
the ground-truth count as hits, otherwise they count as misses. LMR is then
defined as the ratio of sequences that yield a miss. Our results on three
state-of-the-art trajectory prediction models show that LMR preserves the order
of Euclidean distance-based metrics. In contrast to the Euclidean Miss Rate,
qualitative results show that LMR yields misses for sequences where predictions
are located on wrong lanes. Hits on the other hand result for sequences where
predictions are located on the correct lane. This means that LMR implicitly
weights Euclidean error relative to the lane and goes into the direction of
capturing intents of traffic agents. The source code of LMR for Argoverse 2 is
publicly available.

Computer Vision and Pattern Recognition;Artificial Intelligence;Machine Learning;Robotics