Gordon's picture

9 3 34

Gordon

GordonM

·

https://gordonmcd.com

AI & ML interests

Data Science for good

Recent Activity

reacted to MoritzLaurer's post with 👍 10 days ago

Quite excited by the ModernBERT release! 0.15/0.4B small, 2T modern pre-training data and tokenizer with code, 8k context window, great efficient model for embeddings & classification! This will probably be the basis for many future SOTA encoders! And I can finally stop using DeBERTav3 from 2021 :D Congrats @answerdotai, @LightOnIO and collaborators like @tomaarsen ! Paper and models here 👇https://huggingface.co/collections/answerdotai/modernbert-67627ad707a4acbf33c41deb

reacted to MoritzLaurer's post with 🔥 10 days ago

🚀 Releasing a new zeroshot-classifier based on ModernBERT! Some key takeaways: - ⚡ Speed & efficiency: It's multiple times faster and uses significantly less memory than DeBERTav3. You can use larger batch sizes and enabling bf16 (instead of fp16) gave me a ~2x speed boost as well - 📉 Performance tradeoff: It performs slightly worse than DeBERTav3 on average across my zeroshot classification task collection - 🧠 Use cases: I recommend using it for scenarios requiring speed and a larger context window (8k). - 💡 What’s next? I’m preparing a newer version trained on better + longer synthetic data to fully leverage the 8k context window and improve upon the training mix of my older zeroshot-v2.0 models. I also hope that there will be a multilingual variant in the future. Great work by https://huggingface.co/answerdotai ! If you’re looking for a high-speed zeroshot classifier, give it a try! 📄 Resources below: 👇 Base model: https://huggingface.co/MoritzLaurer/ModernBERT-base-zeroshot-v2.0 Large model: https://huggingface.co/MoritzLaurer/ModernBERT-large-zeroshot-v2.0 Updated zeroshot collection: https://huggingface.co/collections/MoritzLaurer/zeroshot-classifiers-6548b4ff407bb19ff5c3ad6f ModernBERT collection with paper: https://huggingface.co/collections/answerdotai/modernbert-67627ad707a4acbf33c41deb

liked a model about 1 month ago

Varosa/SeamlessExpressive

View all activity

Organizations

GordonM's activity

New activity in multimodalart/dreambooth-training almost 2 years ago

I keep getting errors

#79 opened about 2 years ago by

CUDA out of memory

#66 opened over 2 years ago by

New activity in open-llm-leaderboard/open_llm_leaderboard almost 2 years ago

Scores of GPT3.5 and GPT4 for comparison

#30 opened almost 2 years ago by