metadata
library_name: transformers
language:
- pl
pipeline_tag: text-generation
model-index:
- name: gpt2-small-III
results: []
datasets:
- allenai/c4
- clarin-knext/arguana-pl
- JonaszPotoniec/wikipedia-with-statistics-pl
- JuDDGES/pl-court-instruct
- speakleash/PES-2018-2022
Model Card for Model ID
A small GTP-2 model trained on 6.94 GB (3 permutations * 2.31 GB) of Polish text
Model Details
Model Description
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
- Developed by: romanoza
Uses
A base model for other models.
Training Details
Training Data
Training data size: 1_584_191 * 1_024 = 1_622_211_584 tokens
Training Procedure
Training Hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-04
- train_batch_size: 16
- lr_scheduler_type: linear
- num_epochs: 2
- warmup_steps: 500
Environmental Impact
Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
- Hardware Type: 1 * A100
- Hours used: ~50h
- Cloud Provider: Google Colab