Hugging Face's logo

LSX-UniWue
/
ModernGBERT_134M

Feature Extraction
Transformers
PyTorch
Safetensors
German
modernbert
fill-mask
masked-lm
long-context
Model card Files
xet
Community
2

ModernGBERT 134M

This is a German ModernBERT 134M language model trained from scratch using the ModernBERT codebase and the same German portion of RedPajama V2 as our LLäMmlein family. Find more details in our preprint!

Usage 

from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("LSX-UniWue/ModernGBERT_134M")

tokenizer = AutoTokenizer.from_pretrained("LSX-UniWue/ModernGBERT_134M")

Performance

We evaluated our model on the SuperGLEBer benchmark.

Downloads last month
282
Safetensors
Model size
159M params
Tensor type
F32
·
Inference Providers NEW
Feature Extraction
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Dataset used to train LSX-UniWue/ModernGBERT_134M

Collection including LSX-UniWue/ModernGBERT_134M