Hugging Face's logo

Decius
/
sft_model_project

Text Classification
setfit
Safetensors
sentence-transformers
bert
generated_from_setfit_trainer
Eval Results
Model card Files Community

SetFit with sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Sources

Model Labels

Label Examples
delete_category
  • 'Supprimer une catégorie'
  • 'I want to delete a product category'
  • 'Remove a category from the list'
delete_product
  • 'I want to delete the red t-shirt'
  • 'Remove this item from inventory'
  • 'Supprimer un produit'
greet-who_are_you
  • 'how can you help me'
  • "pourquoi j'ai besoin de toi"
  • 'je ne te comprends pas'
create_website
  • 'أريد إنشاء موقع إلكتروني لمتجر الملابس الخاص بي'
  • 'ساعدني في تصميم موقع أعمالي الخاصة بالتدريب الرياضي'
  • 'ساعدني في إنشاء موقع لمطعمي'
read_category
  • 'Can I see all the categories?'
  • 'What categories are available?'
  • 'Affiche-moi les catégories'
update_store
  • 'Update store information'
  • 'Modify the store contact details'
  • 'Je veux changer les coordonnées du magasin'
update_category
  • 'Je veux changer le nom d’une catégorie'
  • 'Can I rename a category?'
  • 'Update the category name to something else'
greet-good_bye
  • 'See you later'
  • 'A plus tard'
  • 'stop'
update_product
  • 'I want to change product details'
  • 'Je veux modifier un produit'
  • 'Edit the product information'
read_product
  • 'Can I see the available items?'
  • 'List the products'
  • 'Affiche tous les produits'
delete_store
  • 'Remove store number 3'
  • 'Supprimer un magasin'
  • 'Can I delete an existing store?'
read_store
  • 'Quels sont les magasins disponibles ?'
  • 'List all registered stores'
  • 'Show me the list of stores'
greet-hi
  • 'Hello buddy'
  • 'Salut'
  • 'Hey'

Evaluation

Metrics

Label Accuracy
all 0.9744

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("Decius/sft_model_project")
# Run inference
preds = model("أحذف المنتج من المخزن")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 1 5.7846 13
Label Training Sample Count
greet-hi 5
greet-who_are_you 7
greet-good_bye 5
create_website 21
read_category 3
update_category 3
delete_category 3
read_product 3
update_product 3
delete_product 3
read_store 3
update_store 3
delete_store 3

Training Hyperparameters

  • batch_size: (4, 4)
  • num_epochs: (4, 4)
  • max_steps: -1
  • sampling_strategy: oversampling
  • body_learning_rate: (2e-05, 1e-05)
  • head_learning_rate: 0.01
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: False
  • warmup_proportion: 0.1
  • l2_weight: 0.01
  • seed: 42
  • evaluation_strategy: epoch
  • eval_max_steps: -1
  • load_best_model_at_end: True

Training Results

Epoch Step Training Loss Validation Loss
0.0011 1 0.1475 -
0.0111 10 0.1345 -
0.0222 20 0.0807 -
0.0333 30 0.0943 -
0.0444 40 0.0785 -
0.0555 50 0.1016 -
0.0666 60 0.0756 -
0.0777 70 0.0775 -
0.0888 80 0.0368 -
0.0999 90 0.0635 -
0.1110 100 0.0395 -
0.1221 110 0.0279 -
0.1332 120 0.0217 -
0.1443 130 0.0254 -
0.1554 140 0.0406 -
0.1665 150 0.0143 -
0.1776 160 0.0482 -
0.1887 170 0.042 -
0.1998 180 0.0286 -
0.2109 190 0.012 -
0.2220 200 0.0258 -
0.2331 210 0.0193 -
0.2442 220 0.0126 -
0.2553 230 0.0342 -
0.2664 240 0.0238 -
0.2775 250 0.0111 -
0.2886 260 0.0101 -
0.2997 270 0.0099 -
0.3108 280 0.0208 -
0.3219 290 0.0089 -
0.3330 300 0.0276 -
0.3441 310 0.0099 -
0.3552 320 0.0191 -
0.3663 330 0.0199 -
0.3774 340 0.0095 -
0.3885 350 0.0142 -
0.3996 360 0.0083 -
0.4107 370 0.0079 -
0.4218 380 0.0072 -
0.4329 390 0.0098 -
0.4440 400 0.01 -
0.4550 410 0.0084 -
0.4661 420 0.0024 -
0.4772 430 0.0176 -
0.4883 440 0.0068 -
0.4994 450 0.0209 -
0.5105 460 0.0038 -
0.5216 470 0.0063 -
0.5327 480 0.034 -
0.5438 490 0.0191 -
0.5549 500 0.0159 -
0.5660 510 0.0088 -
0.5771 520 0.0032 -
0.5882 530 0.0045 -
0.5993 540 0.0192 -
0.6104 550 0.0123 -
0.6215 560 0.0048 -
0.6326 570 0.0068 -
0.6437 580 0.0036 -
0.6548 590 0.0123 -
0.6659 600 0.0104 -
0.6770 610 0.0023 -
0.6881 620 0.0062 -
0.6992 630 0.0048 -
0.7103 640 0.0063 -
0.7214 650 0.0012 -
0.7325 660 0.0026 -
0.7436 670 0.0136 -
0.7547 680 0.0144 -
0.7658 690 0.0045 -
0.7769 700 0.0013 -
0.7880 710 0.0058 -
0.7991 720 0.0056 -
0.8102 730 0.004 -
0.8213 740 0.0023 -
0.8324 750 0.0047 -
0.8435 760 0.001 -
0.8546 770 0.0028 -
0.8657 780 0.0042 -
0.8768 790 0.0016 -
0.8879 800 0.002 -
0.8990 810 0.0004 -
0.9101 820 0.0034 -
0.9212 830 0.0016 -
0.9323 840 0.0076 -
0.9434 850 0.0021 -
0.9545 860 0.0027 -
0.9656 870 0.0017 -
0.9767 880 0.0024 -
0.9878 890 0.0014 -
0.9989 900 0.0015 -
1.0 901 - 0.0316
1.0100 910 0.0014 -
1.0211 920 0.0009 -
1.0322 930 0.0015 -
1.0433 940 0.0023 -
1.0544 950 0.0004 -
1.0655 960 0.0006 -
1.0766 970 0.001 -
1.0877 980 0.0005 -
1.0988 990 0.0044 -
1.1099 1000 0.0011 -
1.1210 1010 0.0008 -
1.1321 1020 0.0008 -
1.1432 1030 0.0007 -
1.1543 1040 0.0004 -
1.1654 1050 0.0009 -
1.1765 1060 0.0017 -
1.1876 1070 0.002 -
1.1987 1080 0.0008 -
1.2098 1090 0.002 -
1.2209 1100 0.0005 -
1.2320 1110 0.0012 -
1.2431 1120 0.002 -
1.2542 1130 0.0012 -
1.2653 1140 0.0025 -
1.2764 1150 0.0008 -
1.2875 1160 0.0009 -
1.2986 1170 0.0011 -
1.3097 1180 0.0004 -
1.3208 1190 0.001 -
1.3319 1200 0.0008 -
1.3430 1210 0.0005 -
1.3541 1220 0.0006 -
1.3651 1230 0.0007 -
1.3762 1240 0.0009 -
1.3873 1250 0.0008 -
1.3984 1260 0.0009 -
1.4095 1270 0.0009 -
1.4206 1280 0.0008 -
1.4317 1290 0.0007 -
1.4428 1300 0.001 -
1.4539 1310 0.0004 -
1.4650 1320 0.0004 -
1.4761 1330 0.0008 -
1.4872 1340 0.0003 -
1.4983 1350 0.0004 -
1.5094 1360 0.0096 -
1.5205 1370 0.001 -
1.5316 1380 0.0006 -
1.5427 1390 0.0015 -
1.5538 1400 0.0008 -
1.5649 1410 0.0006 -
1.5760 1420 0.0007 -
1.5871 1430 0.0009 -
1.5982 1440 0.0004 -
1.6093 1450 0.0013 -
1.6204 1460 0.0007 -
1.6315 1470 0.0004 -
1.6426 1480 0.0005 -
1.6537 1490 0.0006 -
1.6648 1500 0.0008 -
1.6759 1510 0.0007 -
1.6870 1520 0.0005 -
1.6981 1530 0.0004 -
1.7092 1540 0.0005 -
1.7203 1550 0.0007 -
1.7314 1560 0.0006 -
1.7425 1570 0.0004 -
1.7536 1580 0.0006 -
1.7647 1590 0.0005 -
1.7758 1600 0.0006 -
1.7869 1610 0.0011 -
1.7980 1620 0.0007 -
1.8091 1630 0.0005 -
1.8202 1640 0.0005 -
1.8313 1650 0.0003 -
1.8424 1660 0.0004 -
1.8535 1670 0.0006 -
1.8646 1680 0.0005 -
1.8757 1690 0.0006 -
1.8868 1700 0.0004 -
1.8979 1710 0.0004 -
1.9090 1720 0.0002 -
1.9201 1730 0.0005 -
1.9312 1740 0.0005 -
1.9423 1750 0.001 -
1.9534 1760 0.0006 -
1.9645 1770 0.001 -
1.9756 1780 0.0004 -
1.9867 1790 0.0005 -
1.9978 1800 0.0002 -
2.0 1802 - 0.0260
2.0089 1810 0.0005 -
2.0200 1820 0.0005 -
2.0311 1830 0.0004 -
2.0422 1840 0.0005 -
2.0533 1850 0.0002 -
2.0644 1860 0.0005 -
2.0755 1870 0.0007 -
2.0866 1880 0.0005 -
2.0977 1890 0.0003 -
2.1088 1900 0.0004 -
2.1199 1910 0.0003 -
2.1310 1920 0.0014 -
2.1421 1930 0.0005 -
2.1532 1940 0.0002 -
2.1643 1950 0.0003 -
2.1754 1960 0.0007 -
2.1865 1970 0.0005 -
2.1976 1980 0.0004 -
2.2087 1990 0.0006 -
2.2198 2000 0.0005 -
2.2309 2010 0.0003 -
2.2420 2020 0.0006 -
2.2531 2030 0.0006 -
2.2642 2040 0.0006 -
2.2752 2050 0.0003 -
2.2863 2060 0.0014 -
2.2974 2070 0.0004 -
2.3085 2080 0.0005 -
2.3196 2090 0.0004 -
2.3307 2100 0.0004 -
2.3418 2110 0.0004 -
2.3529 2120 0.0004 -
2.3640 2130 0.0011 -
2.3751 2140 0.0003 -
2.3862 2150 0.0003 -
2.3973 2160 0.0005 -
2.4084 2170 0.0006 -
2.4195 2180 0.0004 -
2.4306 2190 0.0002 -
2.4417 2200 0.0002 -
2.4528 2210 0.0006 -
2.4639 2220 0.0003 -
2.4750 2230 0.0002 -
2.4861 2240 0.0006 -
2.4972 2250 0.0006 -
2.5083 2260 0.0004 -
2.5194 2270 0.0005 -
2.5305 2280 0.0004 -
2.5416 2290 0.0005 -
2.5527 2300 0.0005 -
2.5638 2310 0.0006 -
2.5749 2320 0.0005 -
2.5860 2330 0.0003 -
2.5971 2340 0.0007 -
2.6082 2350 0.0002 -
2.6193 2360 0.0003 -
2.6304 2370 0.0003 -
2.6415 2380 0.0004 -
2.6526 2390 0.0004 -
2.6637 2400 0.0005 -
2.6748 2410 0.0003 -
2.6859 2420 0.0003 -
2.6970 2430 0.0003 -
2.7081 2440 0.0005 -
2.7192 2450 0.0006 -
2.7303 2460 0.0005 -
2.7414 2470 0.0005 -
2.7525 2480 0.0006 -
2.7636 2490 0.0002 -
2.7747 2500 0.0002 -
2.7858 2510 0.0002 -
2.7969 2520 0.0007 -
2.8080 2530 0.0003 -
2.8191 2540 0.0004 -
2.8302 2550 0.0003 -
2.8413 2560 0.0002 -
2.8524 2570 0.0006 -
2.8635 2580 0.0003 -
2.8746 2590 0.0002 -
2.8857 2600 0.0002 -
2.8968 2610 0.0002 -
2.9079 2620 0.0003 -
2.9190 2630 0.0003 -
2.9301 2640 0.0002 -
2.9412 2650 0.0002 -
2.9523 2660 0.0002 -
2.9634 2670 0.0003 -
2.9745 2680 0.0003 -
2.9856 2690 0.0003 -
2.9967 2700 0.0003 -
3.0 2703 - 0.0244
3.0078 2710 0.0002 -
3.0189 2720 0.0004 -
3.0300 2730 0.0002 -
3.0411 2740 0.0003 -
3.0522 2750 0.0003 -
3.0633 2760 0.0002 -
3.0744 2770 0.0001 -
3.0855 2780 0.0002 -
3.0966 2790 0.0003 -
3.1077 2800 0.0003 -
3.1188 2810 0.0004 -
3.1299 2820 0.0005 -
3.1410 2830 0.0002 -
3.1521 2840 0.0003 -
3.1632 2850 0.0002 -
3.1743 2860 0.0003 -
3.1853 2870 0.0002 -
3.1964 2880 0.0007 -
3.2075 2890 0.0002 -
3.2186 2900 0.0002 -
3.2297 2910 0.0002 -
3.2408 2920 0.0003 -
3.2519 2930 0.0002 -
3.2630 2940 0.0002 -
3.2741 2950 0.0003 -
3.2852 2960 0.0005 -
3.2963 2970 0.0003 -
3.3074 2980 0.0002 -
3.3185 2990 0.0003 -
3.3296 3000 0.0003 -
3.3407 3010 0.0002 -
3.3518 3020 0.0002 -
3.3629 3030 0.0003 -
3.3740 3040 0.0001 -
3.3851 3050 0.0003 -
3.3962 3060 0.0003 -
3.4073 3070 0.0004 -
3.4184 3080 0.0002 -
3.4295 3090 0.0003 -
3.4406 3100 0.0003 -
3.4517 3110 0.0002 -
3.4628 3120 0.0002 -
3.4739 3130 0.0002 -
3.4850 3140 0.0004 -
3.4961 3150 0.0005 -
3.5072 3160 0.0006 -
3.5183 3170 0.0002 -
3.5294 3180 0.0002 -
3.5405 3190 0.0004 -
3.5516 3200 0.0003 -
3.5627 3210 0.0002 -
3.5738 3220 0.0001 -
3.5849 3230 0.0002 -
3.5960 3240 0.0002 -
3.6071 3250 0.0001 -
3.6182 3260 0.0002 -
3.6293 3270 0.0002 -
3.6404 3280 0.0002 -
3.6515 3290 0.0002 -
3.6626 3300 0.0003 -
3.6737 3310 0.0003 -
3.6848 3320 0.0003 -
3.6959 3330 0.0002 -
3.7070 3340 0.0001 -
3.7181 3350 0.0002 -
3.7292 3360 0.0002 -
3.7403 3370 0.0002 -
3.7514 3380 0.0002 -
3.7625 3390 0.0002 -
3.7736 3400 0.0001 -
3.7847 3410 0.0003 -
3.7958 3420 0.0002 -
3.8069 3430 0.0003 -
3.8180 3440 0.0003 -
3.8291 3450 0.0002 -
3.8402 3460 0.0002 -
3.8513 3470 0.0002 -
3.8624 3480 0.0002 -
3.8735 3490 0.0002 -
3.8846 3500 0.0002 -
3.8957 3510 0.0002 -
3.9068 3520 0.0003 -
3.9179 3530 0.0001 -
3.9290 3540 0.0002 -
3.9401 3550 0.0002 -
3.9512 3560 0.0002 -
3.9623 3570 0.0003 -
3.9734 3580 0.0003 -
3.9845 3590 0.0002 -
3.9956 3600 0.0002 -
4.0 3604 - 0.0237

Framework Versions

  • Python: 3.11.11
  • SetFit: 1.1.0
  • Sentence Transformers: 3.4.1
  • Transformers: 4.44.0
  • PyTorch: 2.6.0+cu124
  • Datasets: 3.5.0
  • Tokenizers: 0.19.1

Citation

BibTeX 

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}
Downloads last month
7
Safetensors
Model size
118M params
Tensor type
F32
·
Inference Providers NEW
Text Classification
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Decius/sft_model_project

Base model

sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
Finetuned
(246)
this model

Evaluation results

View on Papers With Code