  Not all positive pairs are beneficial to time series contrastive learning. In
this paper, we study two types of bad positive pairs that impair the quality of
time series representation learned through contrastive learning ($i.e.$, noisy
positive pair and faulty positive pair). We show that, with the presence of
noisy positive pairs, the model tends to simply learn the pattern of noise
(Noisy Alignment). Meanwhile, when faulty positive pairs arise, the model
spends considerable efforts aligning non-representative patterns (Faulty
Alignment). To address this problem, we propose a Dynamic Bad Pair Mining
(DBPM) algorithm, which reliably identifies and suppresses bad positive pairs
in time series contrastive learning. DBPM utilizes a memory module to track the
training behavior of each positive pair along training process. This allows us
to identify potential bad positive pairs at each epoch based on their
historical training behaviors. The identified bad pairs are then down-weighted
using a transformation module. Our experimental results show that DBPM
effectively mitigates the negative impacts of bad pairs, and can be easily used
as a plug-in to boost performance of state-of-the-art methods. Codes will be
made publicly available.

Machine Learning