README.md

---
license: cc-by-4.0
library_name: YingLong
tags:
- time-series
- forecasting
- foundation-models
- pretrained-models
- time-series-foundation-models
- large-time-series-models
---


# YingLong

YingLong model is introduced in this [paper](https://github.com/wxie9/YingLong/blob/main/YingLong_manuscript.pdf). This version is pre-trained on **78B** time points. More details can be found at our [github](https://github.com/wxie9/YingLong/).


 
## Quickstart

```bash
pip install xformers transformers
pip install flash-attn --no-build-isolation
git clone https://github.com/Dao-AILab/flash-attention && cd flash-attention
cd csrc/rotary && pip install .
cd ../layer_norm && pip install .
```
The flash attention is not required. If you use V100 or other GPU doesn't support flash attention, just change the FlashAttention2Available = RequirementCache("flash-attn>=2.0.0.post1") to
FlashAttention2Available = False in the model.py file. It should be able to run. 

```python
import torch
from transformers import AutoModelForCausalLM

# load pretrain model
model = AutoModelForCausalLM.from_pretrained('qcw2333/YingLong_50m', trust_remote_code=True,torch_dtype=torch.bfloat16).cuda()

# prepare input
batch_size, lookback_length = 1, 2880
seqs = torch.randn(batch_size, lookback_length).bfloat16().cuda()

# generate forecast
prediction_length = 96
output = model.generate(seqs, future_token=prediction_length)

print(output.shape)
```

A notebook example is also provided [here](https://github.com/wxie9/YingLong/blob/main/quickstart_zero_shot.ipynb). The sample codes for long-term forecasting tasks and gift-eval tasks are provided at [link](https://github.com/wxie9/YingLong/tree/main).


<!-- ## Specification -->


## Citation
Coming soon...
<!-- ```
@inproceedings{liutimer,
  title={Timer: Generative Pre-trained Transformers Are Large Time Series Models},
  author={Liu, Yong and Zhang, Haoran and Li, Chenyu and Huang, Xiangdong and Wang, Jianmin and Long, Mingsheng},
  booktitle={Forty-first International Conference on Machine Learning}
}

@article{liu2024timer,
  title={Timer-XL: Long-Context Transformers for Unified Time Series Forecasting},
  author={Liu, Yong and Qin, Guo and Huang, Xiangdong and Wang, Jianmin and Long, Mingsheng},
  journal={arXiv preprint arXiv:2410.04803},
  year={2024}
}
``` -->

## Contact

If you have any questions or want to use the code, feel free to contact:

Xue Wang ([email protected])

Tian Zhou ([email protected])

 
## License

This model is licensed under the cc-by-4.0 License.