Also" Think" Step-by-Step. arXiv preprint arXiv:2306.14050 (2023).

[157] Mu Li, David G Andersen, Jun Woo Park, Alexander J Smola, Amr Ahmed, Vanja Josifovski, James Long, Eugene J Shekita, and Bor-Yiing Su. 2014. Scaling
distributed machine learning with the parameter server. In 11th USENIX Symposium on operating systems design and implementation (OSDI 14). 583–598.
[158] Shenggui Li, Fuzhao Xue, Chaitanya Baranwal, Yongbin Li, and Yang You. 2021. Sequence parallelism: Long sequence training from system perspective.

arXiv preprint arXiv:2105.13120 (2021).

[159] Shanda Li, Chong You, Guru Guruganesh, Joshua Ainslie, Santiago Ontanon, Manzil Zaheer, Sumit Sanghai, Yiming Yang, Sanjiv Kumar, and Srinadh

Bhojanapalli. 2023. Functional Interpolation for Relative Positions Improves Long Context Transformers. arXiv preprint arXiv:2310.04418 (2023).
TheEfficiencySpectrumofLargeLanguageModels-AnAlgorithmicSurvey