transformers pytorch bart text2text-generation summarization en dataset:samsum model-index autotrain_compatible endpoints_compatible region:us
---
datasets:
- samsum
language:
- en
library_name: transformers
metrics:
- rouge
pipeline_tag: summarization
widget:
- text: "Rita: I'm so bloody tired. Falling asleep at work. :-(\nTina: I know what\
    \ you mean.\nTina: I keep on nodding off at my keyboard hoping that the boss doesn't\
    \ notice..\nRita: The time just keeps on dragging on and on and on.... \nRita:\
    \ I keep on looking at the clock and there's still 4 hours of this drudgery to\
    \ go.\nTina: Times like these I really hate my work.\nRita: I'm really not cut\
    \ out for this level of boredom.\nTina: Neither am I.\n"
- text: 'Beatrice: I am in town, shopping. They have nice scarfs in the shop next
    to the church. Do you want one?

    Leo: No, thanks

    Beatrice: But you don''t have a scarf.

    Leo: Because I don''t need it.

    Beatrice: Last winter you had a cold all the time. A scarf could help.

    Leo: I don''t like them.

    Beatrice: Actually, I don''t care. You will get a scarf.

    Leo: How understanding of you!

    Beatrice: You were complaining the whole winter that you''re going to die. I''ve
    had enough.

    Leo: Eh.

    '
- text: 'Jack: Cocktails later?

    May: YES!!!

    May: You read my mind...

    Jack: Possibly a little tightly strung today?

    May: Sigh... without question.

    Jack: Thought so.

    May: A little drink will help!

    Jack: Maybe two!

    '
model-index:
- name: bart-finetuned-samsum
  results:
  - task:
      type: summarization
      name: Text Summarization
    dataset:
      name: SamSum
      type: samsum
    metrics:
    - type: rouge-1
      value: 53.6163
      name: Validation ROUGE-1
    - type: rouge-2
      value: 28.914
      name: Validation ROUGE-2
    - type: rougeL
      value: 44.1443
      name: Validation ROUGE-L
    - type: rougeLsum
      value: 49.2995
      name: Validation ROUGE-L Sum
---

# Description

This model was trained by fine-tuning the [facebook/bart-large-xsum](https://huggingface.co/facebook/bart-large-xsum) model using [these parameters](#training-parameters) and the [samsum dataset](https://huggingface.co/datasets/samsum).

## Development

- Jupyter Notebook: [Text Summarization With BART](https://github.com/adedamola26/text-summarization/blob/main/Text_Summarization_with_BART.ipynb)

## Usage

```python
from transformers import pipeline

model = pipeline("summarization", model="adedamolade26/bart-finetuned-samsum")

conversation = '''Jack: Cocktails later?
May: YES!!!
May: You read my mind...
Jack: Possibly a little tightly strung today?
May: Sigh... without question.
Jack: Thought so.
May: A little drink will help!
Jack: Maybe two!
'''
model(conversation)
```

## Training Parameters

```python
evaluation_strategy = "epoch",
save_strategy = 'epoch',
load_best_model_at_end = True,
metric_for_best_model = 'eval_loss',
seed = 42,
learning_rate=2e-5,
per_device_train_batch_size=4,
per_device_eval_batch_size=4,
gradient_accumulation_steps=2,
weight_decay=0.01,
save_total_limit=2,
num_train_epochs=4,
predict_with_generate=True,
fp16=True,
report_to="none"
```

## References

Model Training process was adapted from Luis Fernando Torres's [Kaggle Notebook](https://www.kaggle.com/code/lusfernandotorres/text-summarization-with-large-language-models): 📝 Text Summarization with Large Language Models
