SetFit with sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
Classification head: a LogisticRegression instance
Maximum Sequence Length: 128 tokens
Number of Classes: 57 classes

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
OFFERS_AND_REFERRALS	'I want to know about offers and referral programs' 'What are offers and referral programs available' 'Show me some offers and referrals program'
CHECK_DEPOSIT_STATUS	'Check Deposit Status' 'My deposit statement' 'What are my deposit status'
JOIN_CONTEST	'How to join a Contest?' 'How can I join a Contest on PowerPlay11?' 'Contest join'
WITHDRAWAL_TIME	'Withdrawal Time' 'How much time it takes to withdraw my amount' "I didn't get my withdrawal amount yet"
WITHDRAWAL_INTRO	'Where is my withdraw' 'Withdrawals' 'Minimum withdrawal limit'
BANK_VERIFICATION_DETAILS	'Details for Bank Account Verification' 'What details I need to provide for bank account'
REFUND_OF_WRONG_AMOUNT	'Wrong amount refund' 'Amount added by mistake' 'I added amount by mistake'
REFUND_OF_ADDED_CASH	'Refund of Added Cash' 'Please refund money' 'Not refunded'
CHAT_WITH_AN_AGENT	'Chat With an Agent' 'Nope' "No, I couldn't find"
LESS_WINNINGS_AMOUNT	'My winnings are wrong' 'My winnings are incorrect' 'Why is my winning amount less than mentioned in Prize list for my rank?'
WINNINGS	'Did not receive my winnings' 'Did not receive my amount' 'Winnings'
INSTANT_WITHDRAWAL	'Instant Withdrawal' 'Instant withdrawal is available' 'When Instant withdrawal available'
WITHDRAWAL_STATUS	'Check withdrawal status' 'I want to view my withdrawal status' 'Show my withdrawal status'
PAN_VERIFICATION_FAILED	'PAN Card verification failed' 'PAN verification failed' 'Getting error while verifying PAN Card'
HOW_POINTS_CALCULATED	'How Are Points Calculated on PowerPlay11'
CHANGE_BANK_ACCOUNT	'Change My Bank Account' 'How to change Bank account'
CHANGE_PROFILE_TEAM_DETAILS	'Change My Profile or Team Details' 'Update My Profile/Team Details' 'Update My Profile'
CHANGE_MOBILE_NUMBER	'Modify mobile number' 'Change phone number' 'Change mobile number'
TAXES_ON_WINNINGS	'Taxes on Winnings' 'Are there any taxes associated with withdrawals' 'Tax deduction'
FAKE_TEAMS	'Fake teams' 'You have your own team in the leagues' 'Fake players'
NO_EMAIL_CONFIRMATION	"Why I haven't received any email confirmation link" 'No Email Confirmation' "Why I haven't received any email confirmation link"
WITHDRAW_CASH_BONUS	'Withdraw Cash Bonus'
CASH_BONUS_EXPIRY	'Cash Bonus Expiry'
CASH_BONUS	'Cash Bonus'
TYPES_BONUS	'Types of Bonus' 'What is the difference between sign-up bonus and cash bonus' 'What is difference between Cash Bonus, signup bonus, surprise bonus, winnings'
TEAM_DEADLINE	'What is the deadline for a match on PowerPlay11?' 'Team Deadline for Match' 'What is Safe Play & Regular Play'
ACCOUNT_NOT_VERIFIED	'Account Not Verified' 'How to Verify my Account?' 'Account Verification'
WHY_VERIFY	'Why do I Need to Verify my Account' 'Why do I need to Verify' 'Is it important to verify my account'
TYPES_CONTESTS	'Contests' 'Types of Contests'
VERIFY_EMAIL	'How to Verify my email address' 'Verify Email' 'How to verify email'
VERIFY_PAN	'Verify PAN Card' 'How to verify my PAN Card?' 'Pan card verification'
VERIFY_MOBILE	'How to verify my mobile number' 'Verify Mobile Number' 'How should I verify my mobile number'
ACCOUNT_RESET	'Account reset' 'PowerPlay11 account reset' 'How to reset account'
WHAT_IF_THERES_A_TIE	'What happens in case there is a tie' 'And if there is a tie' 'How to solve tie issue'
CANNOT_SEE_JOINED_CONTESTS	'Cannot See Joined Contests' 'Why can I not see my joined leagues' 'What is the meaning of C infront a league'
FAIRPLAY_VIOLATIONS	'Fairplay Violation' 'What is considered as FairPlay Violation on PowerPlay11?' 'Fairplay rules'
POINTS_NOT_UPDATED	'Points not updated' 'Points Delay' 'Player points not updated'
WRONG_SCORES	'What if a game is completed with wrong scores?'
ACCOUNT_BALANCE_DEDUCTED	'Want to know how my Account Balance can get Deducted' 'Account Balance Deducted' 'How is account balance deducted'
SIGNUP_BONUS	'Signup Bonus'
HOW_TO_PLAY	'How to Play' 'How Can I Play on PowerPlay11' 'Teach me to play'
DELETE_PAN_CARD	'How to delete PAN card' 'I want to change my PAN' 'Pan card remove'
CHECK_WALLET_BALANCE	'Wallet Balance' 'Show my wallet balance' 'How much amount are left in my wallet balance'
UNUTILIZED_MONEY	'Unutilized' 'Unutilized Amount'
NEW_TEAM_PATTERN	'What is the new team format' 'How many bolwers I can select' 'What is the new team pattern'
THANKS	'Thanks a lot' 'Tysm' 'Thank'
CAPABILITIES	'How can you help me' 'Help me' 'What can you do for me'
APPRECIATION	'You are amazing' 'Appreciate it' 'Good job'
DEDUCTED_AMOUNT_NOT_RECEIVED	'What if amount is debited from account but not added on PowerPlay11?' 'Amount is not received on PowerPlay11 even though debited from account' 'Deducted Amount But Not Received'
MATCH_ABANDONED	'What if a match is abandoned?' 'If match get abandoned will I get refund' 'Match Abandoned'
PRESENCE	'Are You Online' 'Are you there'
DOWNLOAD_POWERPLAY11	'How to download app' 'Download app' 'How to download PowerPlay11'
CRITICISM	'You are horrible' 'You are dumb' 'You are stupid'
GREETINGS_DAY	'Hey' 'Hello' 'Hi'
CONTACT_NUMBER	'Can you share your number' 'Contact Details' 'Contact me'
UPDATE_APP	'How to update the app'
FEEDBACK	'Feedback' 'I want to give feedback'

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("huiyeong/setfit-powerplay11")
# Run inference
preds = model("Give me bonus")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	1	4.8960	31

Label	Training Sample Count
ACCOUNT_BALANCE_DEDUCTED	11
ACCOUNT_NOT_VERIFIED	8
ACCOUNT_RESET	4
APPRECIATION	9
BANK_VERIFICATION_DETAILS	2
CANNOT_SEE_JOINED_CONTESTS	11
CAPABILITIES	9
CASH_BONUS	1
CASH_BONUS_EXPIRY	1
CHANGE_BANK_ACCOUNT	2
CHANGE_MOBILE_NUMBER	10
CHANGE_PROFILE_TEAM_DETAILS	12
CHAT_WITH_AN_AGENT	42
CHECK_DEPOSIT_STATUS	17
CHECK_WALLET_BALANCE	14
CONTACT_NUMBER	12
CRITICISM	5
DEDUCTED_AMOUNT_NOT_RECEIVED	6
DELETE_PAN_CARD	6
DOWNLOAD_POWERPLAY11	3
FAIRPLAY_VIOLATIONS	6
FAKE_TEAMS	12
FEEDBACK	2
GREETINGS_DAY	13
HOW_POINTS_CALCULATED	1
HOW_TO_PLAY	7
INSTANT_WITHDRAWAL	10
JOIN_CONTEST	8
LESS_WINNINGS_AMOUNT	4
MATCH_ABANDONED	3
NEW_TEAM_PATTERN	9
NO_EMAIL_CONFIRMATION	6
OFFERS_AND_REFERRALS	25
PAN_VERIFICATION_FAILED	3
POINTS_NOT_UPDATED	15
PRESENCE	2
REFUND_OF_ADDED_CASH	8
REFUND_OF_WRONG_AMOUNT	6
SIGNUP_BONUS	1
TAXES_ON_WINNINGS	9
TEAM_DEADLINE	17
THANKS	6
TYPES_BONUS	6
TYPES_CONTESTS	2
UNUTILIZED_MONEY	2
UPDATE_APP	1
VERIFY_EMAIL	13
VERIFY_MOBILE	9
VERIFY_PAN	7
WHAT_IF_THERES_A_TIE	9
WHY_VERIFY	7
WINNINGS	14
WITHDRAWAL_INTRO	14
WITHDRAWAL_STATUS	16
WITHDRAWAL_TIME	11
WITHDRAW_CASH_BONUS	1
WRONG_SCORES	1

Training Hyperparameters

batch_size: (16, 16)
num_epochs: (5, 5)
max_steps: -1
sampling_strategy: oversampling
num_iterations: 5
body_learning_rate: (2e-05, 1e-05)
head_learning_rate: 0.01
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
l2_weight: 0.01
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0034	1	0.0805	-
0.1695	50	0.1632	-
0.3390	100	0.1277	-
0.5085	150	0.0788	-
0.6780	200	0.056	-
0.8475	250	0.0406	-
1.0169	300	0.0315	-
1.1864	350	0.026	-
1.3559	400	0.0238	-
1.5254	450	0.024	-
1.6949	500	0.0203	-
1.8644	550	0.0194	-
2.0339	600	0.0188	-
2.2034	650	0.0152	-
2.3729	700	0.0144	-
2.5424	750	0.0138	-
2.7119	800	0.0161	-
2.8814	850	0.0106	-
3.0508	900	0.0111	-
3.2203	950	0.0087	-
3.3898	1000	0.0105	-
3.5593	1050	0.0085	-
3.7288	1100	0.0105	-
3.8983	1150	0.013	-
4.0678	1200	0.0097	-
4.2373	1250	0.0074	-
4.4068	1300	0.0081	-
4.5763	1350	0.0093	-
4.7458	1400	0.0086	-
4.9153	1450	0.0113	-

Framework Versions

Python: 3.11.13
SetFit: 1.1.2
Sentence Transformers: 4.1.0
Transformers: 4.52.4
PyTorch: 2.6.0+cu124
Datasets: 3.6.0
Tokenizers: 0.21.1

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

Huiyeong
/

setfit-powerplay11