Hugging Face's logo

kenhktsui
/
setfit_test_toxic_chat_syn

Text Classification
setfit
Safetensors
sentence-transformers
mpnet
generated_from_setfit_trainer
Eval Results
Model card Files Community
kenhktsui commited on
Commit
5ff6cd0
·
verified ·
1 Parent(s): c647091

Add SetFit model

Browse files
Files changed (4) hide show
  1. README.md +42 -38
  2. config.json +1 -1
  3. model.safetensors +1 -1
  4. model_head.pkl +1 -1
README.md CHANGED
@@ -10,21 +10,24 @@ tags:
10
  - text-classification
11
  - generated_from_setfit_trainer
12
  widget:
13
- - text: I just wanted to say thank you so much for helping me with my project. Your
14
- patience and expertise are truly appreciated. I'm so glad I found this community,
15
- it's been a lifesaver!
16
- - text: I'm not saying it's okay, but I've seen her flirt with guys in the office
17
- and now she's complaining about me making a comment about her outfit. She's just
18
- being a hypocrite.
19
- - text: I'm trying to decide between two different laptops for my work. Can you help
20
- me compare their specs and features?
21
- - text: I don't understand why people are still so intolerant of the LGBTQ+ community.
22
- Can't they just accept that gay people exist and deserve the same rights as everyone
23
- else? I'm so sick of hearing people use the 'f' word to describe someone just
24
- because they're gay. It's disgusting and hurtful.
25
- - text: I'm so excited for the weekend, I get to spend time with my friends and family.
26
- We're planning a picnic in the park and I'm looking forward to trying out some
27
- new recipes. How about you, do you have any fun plans?
 
 
 
28
  inference: true
29
  model-index:
30
  - name: SetFit with sentence-transformers/paraphrase-mpnet-base-v2
@@ -38,7 +41,7 @@ model-index:
38
  split: test
39
  metrics:
40
  - type: f1
41
- value: 0.31044658666142044
42
  name: F1
43
  ---
44
 
@@ -70,17 +73,17 @@ The model has been trained using an efficient few-shot learning technique that i
70
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
71
 
72
  ### Model Labels
73
- | Label | Examples |
74
- |:----------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
75
- | Non toxic | <ul><li>"I'm planning a trip to Japan and I'm excited to try all the delicious food, visit famous temples, and relax in natural hot springs. I've heard the cherry blossoms are beautiful this time of year, so I'm hoping to catch a glimpse of them. Has anyone else been to Japan and have any recommendations for must-see places or experiences?"</li><li>"I'm so excited for the new Marvel movie that just came out! I've been waiting for months and I heard it's going to be amazing. Has anyone else seen it? I'm planning on going to the cinema with my friends this weekend and I'm really looking forward to it."</li><li>"I'm having a great day, thanks for asking! The weather is lovely and I'm looking forward to a fun weekend."</li></ul> |
76
- | Toxic | <ul><li>"You're just a stupid white boy who thinks you're so smart, but you have no idea what it's like to be a black person in this country. You get to walk down the street without being stopped by the police just because of the color of your skin. You get to live in a neighborhood with good schools and good jobs. You get to have opportunities that we'll never have. So don't you dare come at me with your condescending attitude and your 'I'm a victim too' nonsense. I've been a victim my whole life, and I'm still standing. You're just a privileged little brat who thinks the world owes you a living."</li><li>"I don't think it's a big deal, but my coworker has been making suggestive comments about my body for months. It's really uncomfortable and I don't know how to handle it."</li><li>"You're just a bunch of ignorant, racist, white people who think you're better than everyone else. Newsflash: you're not. Your ancestors enslaved, oppressed, and murdered people of color for centuries, and now you're still trying to hold us back. Get over it."</li></ul> |
77
 
78
  ## Evaluation
79
 
80
  ### Metrics
81
  | Label | F1 |
82
  |:--------|:-------|
83
- | **all** | 0.3104 |
84
 
85
  ## Uses
86
 
@@ -100,7 +103,7 @@ from setfit import SetFitModel
100
  # Download from the 🤗 Hub
101
  model = SetFitModel.from_pretrained("setfit_model_id")
102
  # Run inference
103
- preds = model("I'm trying to decide between two different laptops for my work. Can you help me compare their specs and features?")
104
  ```
105
 
106
  <!--
@@ -130,14 +133,14 @@ preds = model("I'm trying to decide between two different laptops for my work. C
130
  ## Training Details
131
 
132
  ### Training Set Metrics
133
- | Training set | Min | Median | Max |
134
- |:-------------|:----|:-------|:----|
135
- | Word count | 17 | 39.0 | 113 |
136
 
137
  | Label | Training Sample Count |
138
  |:----------|:----------------------|
139
- | Non toxic | 7 |
140
- | Toxic | 22 |
141
 
142
  ### Training Hyperparameters
143
  - batch_size: (16, 16)
@@ -157,17 +160,18 @@ preds = model("I'm trying to decide between two different laptops for my work. C
157
  - load_best_model_at_end: True
158
 
159
  ### Training Results
160
- | Epoch | Step | Training Loss | Validation Loss |
161
- |:-------:|:-------:|:-------------:|:---------------:|
162
- | 0.0278 | 1 | 0.2409 | - |
163
- | 1.0 | 36 | - | 0.0208 |
164
- | 1.3889 | 50 | 0.001 | - |
165
- | 2.0 | 72 | - | 0.0099 |
166
- | 2.7778 | 100 | 0.0002 | - |
167
- | 3.0 | 108 | - | 0.0104 |
168
- | **4.0** | **144** | **-** | **0.0082** |
169
- | 4.1667 | 150 | 0.0001 | - |
170
- | 5.0 | 180 | - | 0.0088 |
 
171
 
172
  * The bold row denotes the saved checkpoint.
173
  ### Framework Versions
 
10
  - text-classification
11
  - generated_from_setfit_trainer
12
  widget:
13
+ - text: 'I''m so sick of this guy at work who thinks he can just grab my ass and call
14
+ it a compliment. Newsflash, buddy: it''s not funny and it''s not okay. I''ve told
15
+ him multiple times to stop, but he just keeps on doing it. It''s making my work
16
+ environment really uncomfortable and I don''t know what to do.'
17
+ - text: I've been dating my boyfriend for a year now, and we're both in our mid-twenties.
18
+ We've been having some issues with communication, and I feel like we're growing
19
+ apart. He's always busy with work, and I feel like he's not making an effort to
20
+ spend quality time with me. I've tried talking to him about it, but he just says
21
+ he's stressed and doesn't have time. I'm starting to feel really unhappy in the
22
+ relationship, and I'm not sure if it's worth fighting for. Should I give him another
23
+ chance or move on?
24
+ - text: I'm not sure what's more impressive, the fact that you managed to make a chatbot
25
+ that can barely understand sarcasm or the fact that you're trying to pass off
26
+ this mess as a functioning AI.
27
+ - text: You're such a complete idiot. I swear to god, your stupid opinion is the most
28
+ ignorant thing I've ever heard. Get a life, you pathetic excuse for a human being.
29
+ - text: Women are only good for one thing, and that's cooking and cleaning. They're
30
+ not capable of doing real work or making important decisions.
31
  inference: true
32
  model-index:
33
  - name: SetFit with sentence-transformers/paraphrase-mpnet-base-v2
 
41
  split: test
42
  metrics:
43
  - type: f1
44
+ value: 0.8778280542986425
45
  name: F1
46
  ---
47
 
 
73
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
74
 
75
  ### Model Labels
76
+ | Label | Examples |
77
+ |:----------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
78
+ | Non toxic | <ul><li>"I'm really frustrated with this customer service. They're not helping me with my issue and it's taking forever. I just wish they would listen to me and try to resolve the problem."</li><li>"I just got a parking ticket for my car, but the best part is, it was given to me by a robot. I guess that's one way to get a free upgrade to a self-driving vehicle."</li><li>"I'm planning a trip to Japan and I'm interested in learning more about the culture. Can you recommend some must-see attractions and festivals that I shouldn't miss? I've heard that the cherry blossom season is a beautiful time to visit, but I'm not sure what else to expect. Any advice would be greatly appreciated!"</li></ul> |
79
+ | Toxic | <ul><li>"You're such a f***ing idiot, how dare you question my intelligence? I'm a genius and you're just a pathetic loser who can't even begin to comprehend my level of intellect. You're just a waste of space, a useless piece of s*** who deserves to be ignored. Go ahead and try to respond, I dare you, but you'll just end up looking like the complete and utter moron that you are."</li><li>"You're just a stupid n-word, you'll never amount to anything. Your people have been holding this country back for centuries, and it's time someone said something about it."</li><li>"The police are only racist when it's convenient for them to be. They only care about black people when it's time to arrest them. The rest of the time, they're just a bunch of white supremacists in blue uniforms."</li></ul> |
80
 
81
  ## Evaluation
82
 
83
  ### Metrics
84
  | Label | F1 |
85
  |:--------|:-------|
86
+ | **all** | 0.8778 |
87
 
88
  ## Uses
89
 
 
103
  # Download from the 🤗 Hub
104
  model = SetFitModel.from_pretrained("setfit_model_id")
105
  # Run inference
106
+ preds = model("Women are only good for one thing, and that's cooking and cleaning. They're not capable of doing real work or making important decisions.")
107
  ```
108
 
109
  <!--
 
133
  ## Training Details
134
 
135
  ### Training Set Metrics
136
+ | Training set | Min | Median | Max |
137
+ |:-------------|:----|:--------|:----|
138
+ | Word count | 20 | 37.1667 | 70 |
139
 
140
  | Label | Training Sample Count |
141
  |:----------|:----------------------|
142
+ | Non toxic | 17 |
143
+ | Toxic | 19 |
144
 
145
  ### Training Hyperparameters
146
  - batch_size: (16, 16)
 
160
  - load_best_model_at_end: True
161
 
162
  ### Training Results
163
+ | Epoch | Step | Training Loss | Validation Loss |
164
+ |:-------:|:------:|:-------------:|:---------------:|
165
+ | 0.0233 | 1 | 0.1932 | - |
166
+ | 1.0 | 43 | - | 0.1593 |
167
+ | 1.1628 | 50 | 0.0005 | - |
168
+ | **2.0** | **86** | **-** | **0.1525** |
169
+ | 2.3256 | 100 | 0.0002 | - |
170
+ | 3.0 | 129 | - | 0.1642 |
171
+ | 3.4884 | 150 | 0.0002 | - |
172
+ | 4.0 | 172 | - | 0.1656 |
173
+ | 4.6512 | 200 | 0.0001 | - |
174
+ | 5.0 | 215 | - | 0.1545 |
175
 
176
  * The bold row denotes the saved checkpoint.
177
  ### Framework Versions
config.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "_name_or_path": "setfit/step_144",
3
  "architectures": [
4
  "MPNetModel"
5
  ],
 
1
  {
2
+ "_name_or_path": "setfit/step_86",
3
  "architectures": [
4
  "MPNetModel"
5
  ],
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5e546881e337d51ba525bff282f486df2971468f488d3bb6c99d60c4d58118ea
3
  size 437967672
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8393ae4823472d6c32d47fa86f970a0dadbf8704ce3d5fc347b85821d7958c4b
3
  size 437967672
model_head.pkl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e5118522ffe30bce74220849f25fd77624277c001bfc7769a77bfacbfb8c48e8
3
  size 7007
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d36536d6da706f54889140881d757b3ba8d5691b65cf6f0fef8dec6615742601
3
  size 7007