The idea behind this project is to pretrain a GPT-J-6B on an Arabic corpus using JAX and TPUs of course.
Some resources:

Automate Text-based Workflows at Scale 16
GPT-J-6B: 6B JAX-Based Transformer – Aran Komatsuzaki 23