UniCL: A Universal Contrastive Learning Framework for Large Time Series Models
<|reference_start|>UniCL: A Universal Contrastive Learning Framework for Large Time Series Models: Time-series analysis plays a pivotal role across a range of critical applications, from finance to healthcare, which involves various tasks, such as forecasting and classification. To handle the inherent complexities of time-series data, such as high dimensionality and noise, traditional supervised learning methods first annotate extensive labels for time-series data in each task, which is very costly and impractical in real-world applications. In contrast, pre-trained foundation models offer a promising alternative by leveraging unlabeled data to capture general time series patterns, which can then be fine-tuned for specific tasks. However, existing approaches to pre-training such models typically suffer from high-bias and low-generality issues due to the use of predefined and rigid augmentation operations and domain-specific data training. To overcome these limitations, this paper introduces UniCL, a universal and scalable contrastive learning framework designed for pretraining time-series foundation models across cross-domain datasets. Specifically, we propose a unified and trainable time-series augmentation operation to generate pattern-preserved, diverse, and low-bias time-series data by leveraging spectral information. Besides, we introduce a scalable augmentation algorithm capable of handling datasets with varying lengths, facilitating cross-domain pretraining. Extensive experiments on two benchmark datasets across eleven domains validate the effectiveness of UniCL, demonstrating its high generalization on time-series analysis across various fields.<|reference_end|>
@article{li2024unicl:,
    title={UniCL: A Universal Contrastive Learning Framework for Large Time Series
  Models},
    author={Jiawei Li, Jingshu Peng, Haoyang Li, Lei Chen},
    journal={arXiv preprint arXiv:2405.10597},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.10597},
    primaryClass={cs.LG cs.AI cs.CL}
}