Hugging Face's logo
Yasas's picture
5 2 4

Yasas

ysenarath
gauravbahl's profile picture
Ā·
http://www.ysenarath.com

AI & ML interests

NLU, KB, Crisis Informatics, Social Media Mining

Recent Activity

reacted to tomaarsen's post with ā¤ļø 2 days ago
An assembly of 18 European companies, labs, and universities have banded together to launch šŸ‡ŖšŸ‡ŗ EuroBERT! It's a state-of-the-art multilingual encoder for 15 European languages, designed to be finetuned for retrieval, classification, etc. šŸ‡ŖšŸ‡ŗ 15 Languages: English, French, German, Spanish, Chinese, Italian, Russian, Polish, Portuguese, Japanese, Vietnamese, Dutch, Arabic, Turkish, Hindi 3ļøāƒ£ 3 model sizes: 210M, 610M, and 2.1B parameters - very very useful sizes in my opinion āž”ļø Sequence length of 8192 tokens! Nice to see these higher sequence lengths for encoders becoming more common. āš™ļø Architecture based on Llama, but with bi-directional (non-causal) attention to turn it into an encoder. Flash Attention 2 is supported. šŸ”„ A new Pareto frontier (stronger *and* smaller) for multilingual encoder models šŸ“Š Evaluated against mDeBERTa, mGTE, XLM-RoBERTa for Retrieval, Classification, and Regression (after finetuning for each task separately): EuroBERT punches way above its weight. šŸ“ Detailed paper with all details, incl. data: FineWeb for English and CulturaX for multilingual data, The Stack v2 and Proof-Pile-2 for code. Check out the release blogpost here: https://huggingface.co/blog/EuroBERT/release * https://huggingface.co/EuroBERT/EuroBERT-210m * https://huggingface.co/EuroBERT/EuroBERT-610m * https://huggingface.co/EuroBERT/EuroBERT-2.1B The next step is for researchers to build upon the 3 EuroBERT base models and publish strong retrieval, zero-shot classification, etc. models for all to use. I'm very much looking forward to it!
updated a model 3 days ago
ysenarath/roberta-base-hoeken2024hateful-augmented-v2
published a model 3 days ago
ysenarath/roberta-base-hoeken2024hateful-augmented-v2
View all activity

Organizations

Spaces-explorers's profile picture