logo3584×1792 183 KB

In HuggingArtists 11, we can generate lyrics by a specific artist. This was made by fine-tuning a pre-trained HuggingFace Transformer 1 on parsed datasets from Genius 3.
As well, we use Weights & Biases integration to automatically log model performance and predictions.
 2
Project repository:
 11

About
Create training script for TPU with Flax/JAX. So users can choose GPU or TPU in Colab.

Model
A pre-trained GPT2 model in English and fine-tune it to learn English lyrics.
The GPT2 model, pre-trained in English: link 3.

Datasets
Parsed with this script 4 from Genius 3.

Training scripts
GPU with PyTorch is available in project Colab:
 3
So we need to create for TPU with Flax/JAX.

Project outcome
Create training script for TPU with Flax/JAX. So users can choose GPU or TPU in Colab.

Challenges

I am new with Flax/JAX, so I need some help.
There is no other challenges 



Conclusion
We can integrate this project with Video and Music generation so we can create our AI-Artist 
I hope to find like-minded people and create a cool project