Quite excited by the ModernBERT release! 0.15/0.4B small, 2T modern pre-training data and tokenizer with code, 8k context window, great efficient model for embeddings & classification!
text
Quite excited by the ModernBERT release! 0.15/0.4B small, 2T modern pre-training data and tokenizer with code, 8k context window, great efficient model for embeddings & classification!
new_line
new_line
This will probably be the basis for many future SOTA encoders! And I can finally stop using DeBERTav3 from 2021  :D 
text
This will probably be the basis for many future SOTA encoders! And I can finally stop using DeBERTav3 from 2021  :D 
new_line
new_line
Congrats 
text
Congrats 
@answerdotai
mention
answerdotai
, 
text
, 
@LightOnIO
mention
LightOnIO
 and collaborators like 
text
 and collaborators like 
@tomaarsen
mention
tomaarsen
 !
text
 !
new_line
new_line
Paper and models here 👇https://huggingface.co/collections/answerdotai/modernbert-67627ad707a4acbf33c41deb
text
Paper and models here 👇https://huggingface.co/collections/answerdotai/modernbert-67627ad707a4acbf33c41deb
Quite excited by the ModernBERT release! 0.15/0.4B small, 2T modern pre-training data and tokenizer with code, 8k context window, great efficient model for embeddings & classification!

This will probably be the basis for many future SOTA encoders! And I can finally stop using DeBERTav3 from 2021  :D 

Congrats @answerdotai, @LightOnIO and collaborators like @tomaarsen !

Paper and models here 👇https://huggingface.co/collections/answerdotai/modernbert-67627ad707a4acbf33c41deb
👍
jobreu csabakecskemeti pszemraj reonyy calmodovar morsczx IlyaTakov Felladrin prsyahmi KS112 mlburnham GordonM
👀
John6666 reonyy
64d0341901931c60161f2a06
/avatars/0e097958c5835a62d5d318414257e1d8.svg
Mike Burnham
mlburnham
user
5fb15d1e84389b139cf3b508
https://cdn-avatars.huggingface.co/v1/production/uploads/1613511937628-5fb15d1e84389b139cf3b508.jpeg
Moritz Laurer
MoritzLaurer
user
5ff60d4352c26e9bc240badd
https://cdn-avatars.huggingface.co/v1/production/uploads/5ff60d4352c26e9bc240badd/HzoknJibrSasc1ZzU71XA.png
Benjamin Clavié
bclavie
user