6 Conclusion

Acknowledgements

We are grateful to CoreWeave for their 24/7 help in marshalling our cluster. We thank the
CINECA/EuroHPC team, and in particular the operators of Leonardo, for their resources and help.
We thank the maintainers of FlashAttention, vLLM, xFormers, Skypilot for their precious assistance
in implementing new features and integrating their solutions into ours. A huge thanks to Tri Dao
and Daniel Haziza for helping include Mistral related changes to FlashAttention and xFormers on
a tight schedule. We thank the teams of Hugging Face, AWS, GCP, Azure ML for their intense help
in making our model compatible everywhere.

6
Mistral7B