content.raw: ‼️Sentence Transformers v3.0 is out! You can now train and finetune embedding models with multi-GPU training, bf16 support, loss logging, callbacks & much more. I also release 50+ datasets to train on. 
content.type: text
content.value: ‼️Sentence Transformers v3.0 is out! You can now train and finetune embedding models with multi-GPU training, bf16 support, loss logging, callbacks & much more. I also release 50+ datasets to train on. 
content.type: new_line
content.type: new_line
content.raw: 1️⃣ Training Refactor
content.type: text
content.value: 1️⃣ Training Refactor
content.type: new_line
content.raw: Embedding models can now be trained using an extensive trainer with a lot of powerful features:
content.type: text
content.value: Embedding models can now be trained using an extensive trainer with a lot of powerful features:
content.type: new_line
content.raw: - MultiGPU Training (Data Parallelism (DP) and Distributed Data Parallelism (DDP))
content.type: text
content.value: - MultiGPU Training (Data Parallelism (DP) and Distributed Data Parallelism (DDP))
content.type: new_line
content.raw: - bf16 training support; loss logging
content.type: text
content.value: - bf16 training support; loss logging
content.type: new_line
content.raw: - Evaluation datasets + evaluation loss
content.type: text
content.value: - Evaluation datasets + evaluation loss
content.type: new_line
content.raw: - Improved callback support + an excellent Weights & Biases integration
content.type: text
content.value: - Improved callback support + an excellent Weights & Biases integration
content.type: new_line
content.raw: - Gradient checkpointing, gradient accumulation
content.type: text
content.value: - Gradient checkpointing, gradient accumulation
content.type: new_line
content.raw: - Improved model card generation
content.type: text
content.value: - Improved model card generation
content.type: new_line
content.raw: - Resuming from a training checkpoint without performance loss
content.type: text
content.value: - Resuming from a training checkpoint without performance loss
content.type: new_line
content.raw: - Hyperparameter Optimization
content.type: text
content.value: - Hyperparameter Optimization
content.type: new_line
content.raw: and much more!
content.type: text
content.value: and much more!
content.type: new_line
content.raw: Read my detailed blogpost to learn about the components that make up this new training approach: 
content.type: text
content.value: Read my detailed blogpost to learn about the components that make up this new training approach: 
content.href: https://huggingface.co/blog/train-sentence-transformers
content.raw: https://huggingface.co/blog/train-sentence-transformers
content.type: link
content.type: new_line
content.type: new_line
content.raw: 2️⃣ Similarity Score
content.type: text
content.value: 2️⃣ Similarity Score
content.type: new_line
content.raw: Not sure how to compare embeddings? Don't worry, you can now use 
content.type: text
content.value: Not sure how to compare embeddings? Don't worry, you can now use 
content.code: model.similarity(embeddings1, embeddings2)
content.raw: `model.similarity(embeddings1, embeddings2)`
content.type: inline_code
content.raw:  and you'll get your similarity scores immediately. Model authors can specify their desired similarity score, so you don't have to worry about it anymore!
content.type: text
content.value:  and you'll get your similarity scores immediately. Model authors can specify their desired similarity score, so you don't have to worry about it anymore!
content.type: new_line
content.type: new_line
content.raw: 3️⃣ Additional Kwargs
content.type: text
content.value: 3️⃣ Additional Kwargs
content.type: new_line
content.raw: Sentence Transformers relies on various Transformers instances (AutoModel, AutoTokenizer, AutoConfig), but it was hard to provide valuable keyword arguments to these (like 'torch_dtype=torch.bfloat16' to load a model a lower precision for 2x inference speedup). This is now easy!
content.type: text
content.value: Sentence Transformers relies on various Transformers instances (AutoModel, AutoTokenizer, AutoConfig), but it was hard to provide valuable keyword arguments to these (like 'torch_dtype=torch.bfloat16' to load a model a lower precision for 2x inference speedup). This is now easy!
content.type: new_line
content.type: new_line
content.raw: 4️⃣ Hyperparameter Optimization
content.type: text
content.value: 4️⃣ Hyperparameter Optimization
content.type: new_line
content.raw: Sentence Transformers now ships with HPO, allowing you to effectively choose your hyperparameters for your data and task.
content.type: text
content.value: Sentence Transformers now ships with HPO, allowing you to effectively choose your hyperparameters for your data and task.
content.type: new_line
content.type: new_line
content.raw: 5️⃣ Dataset Release
content.type: text
content.value: 5️⃣ Dataset Release
content.type: new_line
content.raw: To help you out with finetuning models, I've released 50+ ready-to-go datasets that can be used with training or finetuning embedding models: 
content.type: text
content.value: To help you out with finetuning models, I've released 50+ ready-to-go datasets that can be used with training or finetuning embedding models: 
content.raw: https://huggingface.co/collections/sentence-transformers/embedding-model-datasets-6644d7a3673a511914aa7552
content.id: sentence-transformers/embedding-model-datasets-6644d7a3673a511914aa7552
content.type: collection
content.type: resource
content.url: https://huggingface.co/collections/sentence-transformers/embedding-model-datasets-6644d7a3673a511914aa7552
content.type: new_line
content.type: new_line
content.raw: Full release notes: 
content.type: text
content.value: Full release notes: 
content.href: https://github.com/UKPLab/sentence-transformers/releases/tag/v3.0.0
content.raw: https://github.com/UKPLab/sentence-transformers/releases/tag/v3.0.0
content.type: link
rawContent: ‼️Sentence Transformers v3.0 is out! You can now train and finetune embedding models with multi-GPU training, bf16 support, loss logging, callbacks & much more. I also release 50+ datasets to train on. 
  
  1️⃣ Training Refactor
  Embedding models can now be trained using an extensive trainer with a lot of powerful features:
  - MultiGPU Training (Data Parallelism (DP) and Distributed Data Parallelism (DDP))
  - bf16 training support; loss logging
  - Evaluation datasets + evaluation loss
  - Improved callback support + an excellent Weights & Biases integration
  - Gradient checkpointing, gradient accumulation
  - Improved model card generation
  - Resuming from a training checkpoint without performance loss
  - Hyperparameter Optimization
  and much more!
  Read my detailed blogpost to learn about the components that make up this new training approach: https://huggingface.co/blog/train-sentence-transformers
  
  2️⃣ Similarity Score
  Not sure how to compare embeddings? Don't worry, you can now use `model.similarity(embeddings1, embeddings2)` and you'll get your similarity scores immediately. Model authors can specify their desired similarity score, so you don't have to worry about it anymore!
  
  3️⃣ Additional Kwargs
  Sentence Transformers relies on various Transformers instances (AutoModel, AutoTokenizer, AutoConfig), but it was hard to provide valuable keyword arguments to these (like 'torch_dtype=torch.bfloat16' to load a model a lower precision for 2x inference speedup). This is now easy!
  
  4️⃣ Hyperparameter Optimization
  Sentence Transformers now ships with HPO, allowing you to effectively choose your hyperparameters for your data and task.
  
  5️⃣ Dataset Release
  To help you out with finetuning models, I've released 50+ ready-to-go datasets that can be used with training or finetuning embedding models: https://huggingface.co/collections/sentence-transformers/embedding-model-datasets-6644d7a3673a511914aa7552
  
  Full release notes: https://github.com/UKPLab/sentence-transformers/releases/tag/v3.0.0
reactions.reaction: 🔥
reactions.users: osanseviero alvarobartt lunarflu bourdoiscatie philschmid mattmdjaga DmitryRyumin not-lain gabrielmbmb cschroeder Ali-Khaled wonderinos urchade nickprock SamuelNthoroane Rybens vaclavkosar Abhinay45
reactions.reaction: ❤️
reactions.users: louisbrulenaudet alvarobartt lunarflu bourdoiscatie philschmid gabrielmbmb urchade MexIvanov
reactions.reaction: 🤗
reactions.users: lunarflu bourdoiscatie philschmid gabrielmbmb urchade
reactions.reaction: 👍
reactions.users: trenkert