TS-TCD: Triplet-Level Cross-Modal Distillation for Time-Series Forecasting Using Large Language Models
<|reference_start|>TS-TCD: Triplet-Level Cross-Modal Distillation for Time-Series Forecasting Using Large Language Models: In recent years, large language models (LLMs) have shown great potential in time-series analysis by capturing complex dependencies and improving predictive performance. However, existing approaches often struggle with modality alignment, leading to suboptimal results. To address these challenges, we present a novel framework, TS-TCD, which introduces a comprehensive three-tiered cross-modal knowledge distillation mechanism. Unlike prior work that focuses on isolated alignment techniques, our framework systematically integrates: 1) Dynamic Adaptive Gating for Input Encoding and Alignment}, ensuring coherent alignment between time-series tokens and QR-decomposed textual embeddings; 2) Layer-Wise Contrastive Learning}, aligning intermediate representations across modalities to reduce feature-level discrepancies; and 3) Optimal Transport-Driven Output Alignment}, which ensures consistent output predictions through fine-grained cross-modal alignment. Extensive experiments on benchmark time-series datasets demonstrate that TS-TCD achieves state-of-the-art results, outperforming traditional methods in both accuracy and robustness.<|reference_end|>
@article{wang2024ts-tcd:,
    title={TS-TCD: Triplet-Level Cross-Modal Distillation for Time-Series
  Forecasting Using Large Language Models},
    author={Pengfei Wang, Huanran Zheng, Silong Dai, Wenjing Yue, Wei Zhu, and
  Xiaoling Wang},
    journal={arXiv preprint arXiv:2409.14978},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.14978},
    primaryClass={cs.AI}
}