SetFit with sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
Classification head: a LogisticRegression instance
Maximum Sequence Length: 128 tokens
Number of Classes: 57 classes

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
OFFERS_AND_REFERRALS	'I want to know about offers and referral programs' 'What are offers and referral programs available' 'Show me some offers and referrals program'
CHECK_DEPOSIT_STATUS	'Check Deposit Status' 'My deposit statement' 'What are my deposit status'
JOIN_CONTEST	'How to join a Contest?' 'How can I join a Contest on PowerPlay11?' 'Contest join'
WITHDRAWAL_TIME	'Withdrawal Time' 'How much time it takes to withdraw my amount' "I didn't get my withdrawal amount yet"
WITHDRAWAL_INTRO	'Where is my withdraw' 'Withdrawals' 'Minimum withdrawal limit'
BANK_VERIFICATION_DETAILS	'Details for Bank Account Verification' 'What details I need to provide for bank account'
REFUND_OF_WRONG_AMOUNT	'Wrong amount refund' 'Amount added by mistake' 'I added amount by mistake'
REFUND_OF_ADDED_CASH	'Refund of Added Cash' 'Please refund money' 'Not refunded'
CHAT_WITH_AN_AGENT	'Chat With an Agent' 'Nope' "No, I couldn't find"
LESS_WINNINGS_AMOUNT	'My winnings are wrong' 'My winnings are incorrect' 'Why is my winning amount less than mentioned in Prize list for my rank?'
WINNINGS	'Did not receive my winnings' 'Did not receive my amount' 'Winnings'
INSTANT_WITHDRAWAL	'Instant Withdrawal' 'Instant withdrawal is available' 'When Instant withdrawal available'
WITHDRAWAL_STATUS	'Check withdrawal status' 'I want to view my withdrawal status' 'Show my withdrawal status'
PAN_VERIFICATION_FAILED	'PAN Card verification failed' 'PAN verification failed' 'Getting error while verifying PAN Card'
HOW_POINTS_CALCULATED	'How Are Points Calculated on PowerPlay11'
CHANGE_BANK_ACCOUNT	'Change My Bank Account' 'How to change Bank account'
CHANGE_PROFILE_TEAM_DETAILS	'Change My Profile or Team Details' 'Update My Profile/Team Details' 'Update My Profile'
CHANGE_MOBILE_NUMBER	'Modify mobile number' 'Change phone number' 'Change mobile number'
TAXES_ON_WINNINGS	'Taxes on Winnings' 'Are there any taxes associated with withdrawals' 'Tax deduction'
FAKE_TEAMS	'Fake teams' 'You have your own team in the leagues' 'Fake players'
NO_EMAIL_CONFIRMATION	"Why I haven't received any email confirmation link" 'No Email Confirmation' "Why I haven't received any email confirmation link"
WITHDRAW_CASH_BONUS	'Withdraw Cash Bonus'
CASH_BONUS_EXPIRY	'Cash Bonus Expiry'
CASH_BONUS	'Cash Bonus'
TYPES_BONUS	'Types of Bonus' 'What is the difference between sign-up bonus and cash bonus' 'What is difference between Cash Bonus, signup bonus, surprise bonus, winnings'
TEAM_DEADLINE	'What is the deadline for a match on PowerPlay11?' 'Team Deadline for Match' 'What is Safe Play & Regular Play'
ACCOUNT_NOT_VERIFIED	'Account Not Verified' 'How to Verify my Account?' 'Account Verification'
WHY_VERIFY	'Why do I Need to Verify my Account' 'Why do I need to Verify' 'Is it important to verify my account'
TYPES_CONTESTS	'Contests' 'Types of Contests'
VERIFY_EMAIL	'How to Verify my email address' 'Verify Email' 'How to verify email'
VERIFY_PAN	'Verify PAN Card' 'How to verify my PAN Card?' 'Pan card verification'
VERIFY_MOBILE	'How to verify my mobile number' 'Verify Mobile Number' 'How should I verify my mobile number'
ACCOUNT_RESET	'Account reset' 'PowerPlay11 account reset' 'How to reset account'
WHAT_IF_THERES_A_TIE	'What happens in case there is a tie' 'And if there is a tie' 'How to solve tie issue'
CANNOT_SEE_JOINED_CONTESTS	'Cannot See Joined Contests' 'Why can I not see my joined leagues' 'What is the meaning of C infront a league'
FAIRPLAY_VIOLATIONS	'Fairplay Violation' 'What is considered as FairPlay Violation on PowerPlay11?' 'Fairplay rules'
POINTS_NOT_UPDATED	'Points not updated' 'Points Delay' 'Player points not updated'
WRONG_SCORES	'What if a game is completed with wrong scores?'
ACCOUNT_BALANCE_DEDUCTED	'Want to know how my Account Balance can get Deducted' 'Account Balance Deducted' 'How is account balance deducted'
SIGNUP_BONUS	'Signup Bonus'
HOW_TO_PLAY	'How to Play' 'How Can I Play on PowerPlay11' 'Teach me to play'
DELETE_PAN_CARD	'How to delete PAN card' 'I want to change my PAN' 'Pan card remove'
CHECK_WALLET_BALANCE	'Wallet Balance' 'Show my wallet balance' 'How much amount are left in my wallet balance'
UNUTILIZED_MONEY	'Unutilized' 'Unutilized Amount'
NEW_TEAM_PATTERN	'What is the new team format' 'How many bolwers I can select' 'What is the new team pattern'
THANKS	'Thanks a lot' 'Tysm' 'Thank'
CAPABILITIES	'How can you help me' 'Help me' 'What can you do for me'
APPRECIATION	'You are amazing' 'Appreciate it' 'Good job'
DEDUCTED_AMOUNT_NOT_RECEIVED	'What if amount is debited from account but not added on PowerPlay11?' 'Amount is not received on PowerPlay11 even though debited from account' 'Deducted Amount But Not Received'
MATCH_ABANDONED	'What if a match is abandoned?' 'If match get abandoned will I get refund' 'Match Abandoned'
PRESENCE	'Are You Online' 'Are you there'
DOWNLOAD_POWERPLAY11	'How to download app' 'Download app' 'How to download PowerPlay11'
CRITICISM	'You are horrible' 'You are dumb' 'You are stupid'
GREETINGS_DAY	'Hey' 'Hello' 'Hi'
CONTACT_NUMBER	'Can you share your number' 'Contact Details' 'Contact me'
UPDATE_APP	'How to update the app'
FEEDBACK	'Feedback' 'I want to give feedback'

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("huiyeong/setfit-powerplay11-neg")
# Run inference
preds = model("Give me bonus")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	1	5.0189	31

Label	Training Sample Count
ACCOUNT_BALANCE_DEDUCTED	11
ACCOUNT_NOT_VERIFIED	8
ACCOUNT_RESET	5
APPRECIATION	11
BANK_VERIFICATION_DETAILS	2
CANNOT_SEE_JOINED_CONTESTS	13
CAPABILITIES	10
CASH_BONUS	1
CASH_BONUS_EXPIRY	1
CHANGE_BANK_ACCOUNT	2
CHANGE_MOBILE_NUMBER	12
CHANGE_PROFILE_TEAM_DETAILS	14
CHAT_WITH_AN_AGENT	52
CHECK_DEPOSIT_STATUS	18
CHECK_WALLET_BALANCE	15
CONTACT_NUMBER	14
CRITICISM	5
DEDUCTED_AMOUNT_NOT_RECEIVED	7
DELETE_PAN_CARD	6
DOWNLOAD_POWERPLAY11	4
FAIRPLAY_VIOLATIONS	6
FAKE_TEAMS	14
FEEDBACK	2
GREETINGS_DAY	14
HOW_POINTS_CALCULATED	1
HOW_TO_PLAY	8
INSTANT_WITHDRAWAL	11
JOIN_CONTEST	9
LESS_WINNINGS_AMOUNT	4
MATCH_ABANDONED	3
NEW_TEAM_PATTERN	10
NO_EMAIL_CONFIRMATION	7
OFFERS_AND_REFERRALS	30
PAN_VERIFICATION_FAILED	3
POINTS_NOT_UPDATED	16
PRESENCE	2
REFUND_OF_ADDED_CASH	9
REFUND_OF_WRONG_AMOUNT	7
SIGNUP_BONUS	1
TAXES_ON_WINNINGS	9
TEAM_DEADLINE	17
THANKS	6
TYPES_BONUS	6
TYPES_CONTESTS	2
UNUTILIZED_MONEY	2
UPDATE_APP	1
VERIFY_EMAIL	15
VERIFY_MOBILE	11
VERIFY_PAN	7
WHAT_IF_THERES_A_TIE	11
WHY_VERIFY	7
WINNINGS	16
WITHDRAWAL_INTRO	15
WITHDRAWAL_STATUS	18
WITHDRAWAL_TIME	15
WITHDRAW_CASH_BONUS	1
WRONG_SCORES	1

Training Hyperparameters

batch_size: (16, 16)
num_epochs: (5, 5)
max_steps: -1
sampling_strategy: oversampling
num_iterations: 5
body_learning_rate: (2e-05, 1e-05)
head_learning_rate: 0.01
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
l2_weight: 0.01
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0030	1	0.2351	-
0.1515	50	0.1993	-
0.3030	100	0.145	-
0.4545	150	0.0912	-
0.6061	200	0.0751	-
0.7576	250	0.0578	-
0.9091	300	0.0415	-
1.0606	350	0.0377	-
1.2121	400	0.032	-
1.3636	450	0.0286	-
1.5152	500	0.0249	-
1.6667	550	0.0228	-
1.8182	600	0.0211	-
1.9697	650	0.0207	-
2.1212	700	0.0136	-
2.2727	750	0.0141	-
2.4242	800	0.0127	-
2.5758	850	0.0143	-
2.7273	900	0.0157	-
2.8788	950	0.0162	-
3.0303	1000	0.0115	-
3.1818	1050	0.0103	-
3.3333	1100	0.0104	-
3.4848	1150	0.0093	-
3.6364	1200	0.0091	-
3.7879	1250	0.0112	-
3.9394	1300	0.0108	-
4.0909	1350	0.0096	-
4.2424	1400	0.01	-
4.3939	1450	0.009	-
4.5455	1500	0.0077	-
4.6970	1550	0.0079	-
4.8485	1600	0.0083	-
5.0	1650	0.0081	-

Framework Versions

Python: 3.11.13
SetFit: 1.1.2
Sentence Transformers: 4.1.0
Transformers: 4.52.4
PyTorch: 2.6.0+cu124
Datasets: 3.6.0
Tokenizers: 0.21.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

Huiyeong
/

setfit-powerplay11-neg