Scaling-laws for Large Time-series Models
<|reference_start|>Scaling-laws for Large Time-series Models: Scaling laws for large language models (LLMs) have provided useful guidance on how to train ever larger models for predictable performance gains. Time series forecasting shares a similar sequential structure to language, and is amenable to large-scale transformer architectures. Here we show that foundational decoder-only time series transformer models exhibit analogous scaling-behavior to LLMs, while architectural details (aspect ratio and number of heads) have a minimal effect over broad ranges. We assemble a large corpus of heterogenous time series data on which to train, and establish, for the first time, power-law scaling relations with respect to parameter count, dataset size, and training compute, spanning five orders of magnitude.<|reference_end|>
@article{edwards2024scaling-laws,
    title={Scaling-laws for Large Time-series Models},
    author={Thomas D. P. Edwards, James Alvey, Justin Alsing, Nam H. Nguyen,
  Benjamin D. Wandelt},
    journal={arXiv preprint arXiv:2405.13867},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.13867},
    primaryClass={cs.LG cs.AI}
}