title: LETS-C: Leveraging Language Embedding for Time Series Classification
summary: Recent advancements in language modeling have shown promising results when
  applied to time series data. In particular, fine-tuning pre-trained large
  language models (LLMs) for time series classification tasks has achieved
  state-of-the-art (SOTA) performance on standard benchmarks. However, these
  LLM-based models have a significant drawback due to the large model size, with
  the number of trainable parameters in the millions. In this paper, we propose
  an alternative approach to leveraging the success of language modeling in the
  time series domain. Instead of fine-tuning LLMs, we utilize a language
  embedding model to embed time series and then pair the embeddings with a simple
  classification head composed of convolutional neural networks (CNN) and
  multilayer perceptron (MLP). We conducted extensive experiments on
  well-established time series classification benchmark datasets. We demonstrated
  LETS-C not only outperforms the current SOTA in classification accuracy but
  also offers a lightweight solution, using only 14.5% of the trainable
  parameters on average compared to the SOTA model. Our findings suggest that
  leveraging language encoders to embed time series data, combined with a simple
  yet effective classification head, offers a promising direction for achieving
  high-performance time series classification while maintaining a lightweight
  model architecture.
paper_authors._id: 668f3348b3991ac0c3f2568b
paper_authors.name: Rachneet Kaur
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-07-11T01:23:47.850Z
paper_authors._id: 663c57534edaac49a8afe286
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/663c57534edaac49a8afe286/fUB5uphwK-lqZE6EN2Z8F.jpeg
paper_authors.fullname: Rachneet Kaur
paper_authors.type: user
paper_authors.user: rachneetkaur
paper_authors._id: 668f3348b3991ac0c3f2568c
paper_authors.name: Zhen Zeng
paper_authors._id: 668f3348b3991ac0c3f2568d
paper_authors.name: Tucker Balch
paper_authors._id: 668f3348b3991ac0c3f2568e
paper_authors.name: Manuela Veloso
paper_title: LETS-C: Leveraging Language Embedding for Time Series Classification
paper_summary: Recent advancements in language modeling have shown promising results when
  applied to time series data. In particular, fine-tuning pre-trained large
  language models (LLMs) for time series classification tasks has achieved
  state-of-the-art (SOTA) performance on standard benchmarks. However, these
  LLM-based models have a significant drawback due to the large model size, with
  the number of trainable parameters in the millions. In this paper, we propose
  an alternative approach to leveraging the success of language modeling in the
  time series domain. Instead of fine-tuning LLMs, we utilize a language
  embedding model to embed time series and then pair the embeddings with a simple
  classification head composed of convolutional neural networks (CNN) and
  multilayer perceptron (MLP). We conducted extensive experiments on
  well-established time series classification benchmark datasets. We demonstrated
  LETS-C not only outperforms the current SOTA in classification accuracy but
  also offers a lightweight solution, using only 14.5% of the trainable
  parameters on average compared to the SOTA model. Our findings suggest that
  leveraging language encoders to embed time series data, combined with a simple
  yet effective classification head, offers a promising direction for achieving
  high-performance time series classification while maintaining a lightweight
  model architecture.
paper_ai_summary: Using a language embedding model with a convolutional and MLP classification head achieves high accuracy and significantly reduced parameter count in time series classification.
paper_ai_keywords: language models pre-trained large language models time series classification language embedding model convolutional neural networks multilayer perceptron time series classification benchmarks