modernBERT for Norwegian.

#6
by hoxmark - opened

Have you thought about training a Norwegian modernBERT(https://huggingface.co/answerdotai/ModernBERT-base) model?

That would be very useful.

Language Technology Group (University of Oslo) org

Yes, we are planning to release s collection of new NorBERTs that will be more optimized for inference speed :)

Great news! Thanks you.

Will that also include the possible token length?

Language Technology Group (University of Oslo) org

What exactly do you mean by that? :)

Thank you for answer and my apologies, I some how stopped mid sentence.

My question was suppose to say:
In the blog post introducing modernBERT they also say that they will increase the possibility for a sequence length of up to 8192 tokens. Is this something you will look into doing? :)

Language Technology Group (University of Oslo) org

I see :) Yes, we will increase the sequence length. But note that even the current NorBERT3 is able to accept longer sequences than the 512 tokens it has been trained on, thanks to its bucketed relative positional encoding.

Language Technology Group (University of Oslo) org
edited Jun 10

Hi again, it took longer than we thought, but the NorBERT4 models are finally out! They were trained on sequence length of 16834 tokens and offer the same efficiency as ModernBERT. Thanks for pushing us to do this :)

All sizes of the NorBERT4 family:

Wonderful! Thank you very much, I'll test them out in our product next week!

Sign up or log in to comment