sentence-transformers safetensors roberta sentence-similarity feature-extraction generated_from_trainer dataset_size:30288 loss:MultipleNegativesRankingLoss arxiv:1908.10084 arxiv:1705.00652 base_model:manuel-couto-pintos/roberta_erisk base_model:finetune:manuel-couto-pintos/roberta_erisk autotrain_compatible text-embeddings-inference endpoints_compatible region:us
---
base_model: manuel-couto-pintos/roberta_erisk
datasets: []
language: []
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:30288
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: 'Looks like a small cockroach, but much more colorful, 0.75" long.
    [Atlanta, Georgia]  '
  sentences:
  - 'Help me win a bet: What size gi does Marcelo Garcia wear? I suspect he uses different
    size pants relative to the gi-top because of his epic thighs relative to stature.
    My buddy just says A2 all around (on average, recognizing that it varies by brand).
    What do you say? '
  - 'What little things about the Star Wars Universe do you love?  '
  - 'Looks like a small cockroach, but much more colorful, 0.75" long. [Atlanta, Georgia]  '
- source_sentence: "Clogged Construction on my brand new condo finished this summer.\
    \ Not wasting a second, I broke lease on my musky apartment, and moved in as soon\
    \ as possible. I rather enjoyed knowing I was the first resident living here:\
    \ there was no wear and tear, no smoke stains on the walls, and no damage to the\
    \ structure. The only issue was a light clattering sound whenever I used the commercial\
    \ sink in my laundry room. I rarely used it, so I didn't bring up the problem\
    \ to the contractors. Everything else worked perfectly, and my home was as sterile\
    \ as an operating table.\n\n\n\n nbsp;\n\n\n\nAfter a few months, I began noticing\
    \ water pooling at the foot of my shower. The drain must have been clogged. I\
    \ took to my tools, unscrewed the shower drain, and peered inside. I could see\
    \ a collection of fibers bunched up in the pipes. Reaching in with an unfolded\
    \ coat hanger, I pulled out mountains of dirty blond hair clogging the pipes.\
    \ I live alone, I don't have any pets, I haven't entertained a lady in over a\
    \ year, and I've been bald since I was 27.\n\n\n\n nbsp;\n\n\n\nThe odd phenomena\
    \ got me thinking about the sink in the laundry room. I detached the aerator,\
    \ placed my hand under the faucet, and turned on the water. Dozens of molars came\
    \ flying out, slipping through my fingers and into the sink, bouncing up and down\
    \ until ultimately falling down the drain.\n\n\n\n nbsp;\n\n\n\nOn a completely\
    \ unrelated note: I have a beautiful, fully furnished, barely-used condo for sale.\
    \ Located in downtown Detroit. Anyone interested? "
  sentences:
  - '3-2 defense cannot stop corner 3s? Does anyone else have this problem? My down
    low guys won''t kick out to even try to defend an open 3 shot, and the computer
    just spams this on me all day when I play offline. '
  - tw.being suicidal but knowing someone whos commit is the worst thing in the world.
    bc you see both sides. you see how it affects the people that love that person.
    including yourself. you see how it doesnt end the pain but it just passes it on
    to all the people who are left to deal with it. but then it also makes it so much
    more understandable as to why someone did it. you know what its like to want the
    pain to end. the feeling of your brain sabotaging you and your happiness constantly.
    to stop feeling like youre drowning in yourself. you get each and every point
    to it. and in a sense it makes me feel even more guilty for ever having the thought
    in the first place. for it becoming my safe space. knowing that if things dont
    fall into place that im okay with not being here anymore but not being okay leaving
    the people you love to clean up the mess / carry it with them for the rest of
    their lives. sorry. end rant.
  - "Clogged Construction on my brand new condo finished this summer. Not wasting\
    \ a second, I broke lease on my musky apartment, and moved in as soon as possible.\
    \ I rather enjoyed knowing I was the first resident living here: there was no\
    \ wear and tear, no smoke stains on the walls, and no damage to the structure.\
    \ The only issue was a light clattering sound whenever I used the commercial sink\
    \ in my laundry room. I rarely used it, so I didn't bring up the problem to the\
    \ contractors. Everything else worked perfectly, and my home was as sterile as\
    \ an operating table.\n\n\n\n nbsp;\n\n\n\nAfter a few months, I began noticing\
    \ water pooling at the foot of my shower. The drain must have been clogged. I\
    \ took to my tools, unscrewed the shower drain, and peered inside. I could see\
    \ a collection of fibers bunched up in the pipes. Reaching in with an unfolded\
    \ coat hanger, I pulled out mountains of dirty blond hair clogging the pipes.\
    \ I live alone, I don't have any pets, I haven't entertained a lady in over a\
    \ year, and I've been bald since I was 27.\n\n\n\n nbsp;\n\n\n\nThe odd phenomena\
    \ got me thinking about the sink in the laundry room. I detached the aerator,\
    \ placed my hand under the faucet, and turned on the water. Dozens of molars came\
    \ flying out, slipping through my fingers and into the sink, bouncing up and down\
    \ until ultimately falling down the drain.\n\n\n\n nbsp;\n\n\n\nOn a completely\
    \ unrelated note: I have a beautiful, fully furnished, barely-used condo for sale.\
    \ Located in downtown Detroit. Anyone interested? "
- source_sentence: 'Top 10 Movies Trailers of 2017 Must watch It  '
  sentences:
  - Im on coke n 2 mg kpin and im anxious as fuckIdk what i can do to get rid of this
    i know coke doesnt last long but the anxietys lingering n the kpins are keeping
    me borderline okay, but I've never been this anxious on coke i feel like im on
    a psychedelic having a bad trip but im not tripping its just the anxiety. Can
    anyone help me thru this
  - '[Giveaway] 10 BTS for new users  '
  - 'Top 10 Movies Trailers of 2017 Must watch It  '
- source_sentence: 'Vet says he nearly operated on himself when VA wouldn''t pay medical
    bill.  '
  sentences:
  - 'What kind of soap is best to get glitter off your skin?  '
  - 'Alvvays is nearly done tracking their next album  '
  - 'Vet says he nearly operated on himself when VA wouldn''t pay medical bill.  '
- source_sentence: Age old questions[View Poll](https://www.reddit.com/poll/m89hf3)
  sentences:
  - "GUYS I MIGHT HAVE TO DELETE THIS ACCOUNT BECAUSE MY BF KNOWS MY ACC BUT I DON'T\
    \ WANT TO IT'S A MASSIVE URGENCE I'VE HAD THIS 3 YEARS So basically me and my\
    \ boyfriend was messing around but he decided to go onto my reddit app and he\
    \ \"accidently\" saw my reddit account name and he said that he's not going to\
    \ look cause he knows he won't like what he sees but GUYS my post history is fucked\
    \ i'm fucked it makes me look more fucked then I am what the fuck do i dooooo\
    \ D:\n\nI don't wanna start over and there's a couple of subreddits that are suscriber\
    \ only so how the fuck am i gonna get back   \n\nhe's said he's been curious about\
    \ this before but he knows the sorta stuff i post and he said it would really\
    \ upset him but when he's curios he usally won't stop wondering but I like to\
    \ think that i can trust him but I''m complety FUCKED.  \n\napparently he forgot\
    \ it too but he has good memory "
  - Age old questions[View Poll](https://www.reddit.com/poll/m89hf3)
  - 'Who else is in a opposite gender dominated industry? What have been your experiences?
    I am a female in IT. I chose this field because I enjoy it, and it turns out I
    am good at it. I am not concerned about the gender bias because I feel my qualifications
    and experience speak for themselves, and so far that has been the case (the only
    time I have been discriminated against it has not affected my career progress).
    However, I''m relatively inexperienced and I would love to know other people''s
    experiences in similar environments. '
---

# SentenceTransformer based on manuel-couto-pintos/roberta_erisk

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [manuel-couto-pintos/roberta_erisk](https://huggingface.co/manuel-couto-pintos/roberta_erisk). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [manuel-couto-pintos/roberta_erisk](https://huggingface.co/manuel-couto-pintos/roberta_erisk) <!-- at revision 9aa8180ee595fe69a8d23c06dc5ee405f4f5d5ac -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 768 tokens
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: RobertaModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("manuel-couto-pintos/roberta_erisk_simcse")
# Run inference
sentences = [
    'Age old questions[View Poll](https://www.reddit.com/poll/m89hf3)',
    'Age old questions[View Poll](https://www.reddit.com/poll/m89hf3)',
    "Who else is in a opposite gender dominated industry? What have been your experiences? I am a female in IT. I chose this field because I enjoy it, and it turns out I am good at it. I am not concerned about the gender bias because I feel my qualifications and experience speak for themselves, and so far that has been the case (the only time I have been discriminated against it has not affected my career progress). However, I'm relatively inexperienced and I would love to know other people's experiences in similar environments. ",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset


* Size: 30,288 training samples
* Columns: <code>sentence_0</code> and <code>sentence_1</code>
* Approximate statistics based on the first 1000 samples:
  |         | sentence_0                                                                         | sentence_1                                                                         |
  |:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
  | type    | string                                                                             | string                                                                             |
  | details | <ul><li>min: 9 tokens</li><li>mean: 84.36 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>min: 9 tokens</li><li>mean: 84.36 tokens</li><li>max: 512 tokens</li></ul> |
* Samples:
  | sentence_0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   | sentence_1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
  |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>Actor Cory Monteith, Who Played Finn Hudson On 'Glee,' Found Dead  </code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             | <code>Actor Cory Monteith, Who Played Finn Hudson On 'Glee,' Found Dead  </code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
  | <code>Is the AW3420DW worth double the cost of a $500 monitor?I've been researching ultrawides and wanted to know people's opinion if the extra cost for the [Alienware AW3420DW ($999)](https://www.microcenter.com/product/620684/dell-alienware-aw3420dw-34-wqhd-120hz-hdmi-dp-g-sync--curved-ips-led-gaming-monitor) was worth the extra over say a [AOC CU34G2X ($449)](https://www.microcenter.com/product/618536/aoc-cu34g2x-34-qhd-144hz-hdmi-dp-freesync-ultrawide-curved-led-gaming-monitor) or [BenQ EX3501R ($649)](https://www.bhphotovideo.com/c/product/1383775-REG/benq_ex3501r_premium_grey_35_va_3440x1440.html) or another monitor in that range? If I'm willing to spend the cash for the Alienware, should I just make the leap?</code> | <code>Is the AW3420DW worth double the cost of a $500 monitor?I've been researching ultrawides and wanted to know people's opinion if the extra cost for the [Alienware AW3420DW ($999)](https://www.microcenter.com/product/620684/dell-alienware-aw3420dw-34-wqhd-120hz-hdmi-dp-g-sync--curved-ips-led-gaming-monitor) was worth the extra over say a [AOC CU34G2X ($449)](https://www.microcenter.com/product/618536/aoc-cu34g2x-34-qhd-144hz-hdmi-dp-freesync-ultrawide-curved-led-gaming-monitor) or [BenQ EX3501R ($649)](https://www.bhphotovideo.com/c/product/1383775-REG/benq_ex3501r_premium_grey_35_va_3440x1440.html) or another monitor in that range? If I'm willing to spend the cash for the Alienware, should I just make the leap?</code> |
  | <code>My first time making it to a week! Awesome! Nothing to say, just felt like sharing(: Have a good day!<br><br><br><br>**EDIT:** Oh my gosh, I meant to say month... Woops. </code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      | <code>My first time making it to a week! Awesome! Nothing to say, just felt like sharing(: Have a good day!<br><br><br><br>**EDIT:** Oh my gosh, I meant to say month... Woops. </code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
  ```json
  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `per_device_train_batch_size`: 10
- `per_device_eval_batch_size`: 10
- `num_train_epochs`: 1
- `multi_dataset_batch_sampler`: round_robin

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: no
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 10
- `per_device_eval_batch_size`: 10
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1
- `num_train_epochs`: 1
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.0
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `eval_use_gather_object`: False
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: round_robin

</details>

### Training Logs
| Epoch  | Step | Training Loss |
|:------:|:----:|:-------------:|
| 0.1651 | 500  | 0.8614        |
| 0.3301 | 1000 | 0.0012        |
| 0.4952 | 1500 | 0.0007        |
| 0.6603 | 2000 | 0.0002        |
| 0.8254 | 2500 | 0.0002        |
| 0.9904 | 3000 | 0.0           |


### Framework Versions
- Python: 3.10.14
- Sentence Transformers: 3.0.1
- Transformers: 4.44.2
- PyTorch: 2.0.1+cu117
- Accelerate: 0.32.0
- Datasets: 2.20.0
- Tokenizers: 0.19.1

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->