Learn how to use JAX/Flax with Transformers  + 

We partnered-up with Google’s Flax, JAX, and Cloud teams to organize a new community week from July 7th to July 14th. We want to teach you how to effectively use JAX/Flax for Natural Language Processing (NLP) and Computer Vision (CV).
Free access to a TPUv3-8 VM will kindly be provided by the Google Cloud team !
We can guarantee TPU access for the first 400 participants, so it might be worth to sign-up quickly .
TLDR; All important announcements will be made in an internal Slack channel. To get access to the Slack, you can sign-up here 751. Important information is summarized here 426  .

What it is about 

The goal of the JAX/Flax community week is to make compute-intensive NLP and CV projects (like pre-training BERT, GPT2, CLIP, ViT) practicable for a wider audience of engineers and researchers.
To do so, we will teach participants how to effectively use JAX/Flax on TPU and help them define a fun project to complete during the community week.

How does it work 

Participants can propose ideas for an interesting NLP and/or CV project. Teams of 2 to 5 will then be formed around the most promising and interesting projects.
Make sure to read this 276 document on how to propose projects, comment on other participants’ project ideas, and create a team.
To help each team to successfully finish their project, we have organized talks by leading scientists and engineers from Google, Hugging Face, and the open-source NLP & CV community. The talks will take place before the community week from June 30th to July 2nd. Make sure to attend the talks to get the most out of your participation!
Each team is then given free access to a TPUv3-8 VM from July 7th to July 14th. In addition, we will provide training examples in JAX/Flax for a variety of NLP and Vision models to kick-start your project. During the week, we’ll make sure to answer any questions you might have about JAX/Flax and Transformers. We will try to help each team as much as possible to successfully complete their project!
At the end of the community week, each team can submit a demo of their project. All demonstrations will be evaluated by a jury and the top-3 demos will be awarded a prize! In a few days, we will share a document that gives a couple of propositions on how to submit your project with a nice demo.

What do I need to do to participate 

An interest in JAX/Flax and ideally an idea for a fun NLP/CV project! Compute in the form of a TPUv3-8 will kindly be provided by Google’s Cloud team. To sign up for the event, please use this 751 Google form to fill in your name, email address as well as your Hugging Face account name. You will then be added to an internal Slack channel in which all the relevant information will be given.
An overview of the event is shown here 426.

What do I get 



enjoy a bit of Hugging Face vibe by joining the JAX/Flax community week.


learn about JAX/Flax and how to effectively use it for compute-intensive training.


interesting presentations by leading researchers and engineers in NLP & CV.


a prize if you are under the top-3 projects!



Summary - Timeline 

23.06. - Official announcement of the community week. Make sure to sign-up in this 751 Google form.
23.06. - 30.06. Participants will be added to an internal Slack channel. Project ideas can be proposed here and groups of 2-10 are formed. Read this 276 document for more information.
30.06. - Release of all relevant training scripts in JAX/Flax as well as other documents on how to set up a TPU, how to use the training scripts, how to submit a demo, tips & tricks for JAX/Flax, tips & tricks for efficient use of the  hub.
30.06. - 2.07. Talks about JAX/Flax, TPU, Transformers, Computer Vision & NLP will be held.
7.07. Start of the community week! Access to TPUv3-8 will be given to each team.
7.07. - 14.07. - The Hugging Face & JAX/Flax & Cloud team will be available for any questions, problems the teams might run into.
15.07. Access to TPU is deactivated and community week officially ends.
16.07. Deadline for each team to submit a demo.
Open-sourcely yours,
The , , Flax and Cloud team