safetensors t5 flan fine-tuned instruction en dataset:knkarthick/dialogsum base_model:google/flan-t5-small base_model:finetune:google/flan-t5-small license:apache-2.0 region:us
---
base_model:
- google/flan-t5-small
datasets:
- knkarthick/dialogsum
language:
- en
license: apache-2.0
metrics:
- rouge
tags:
- t5
- flan
- fine-tuned
- instruction
---

# FLAN-T5-small Dialogue Summarization

## Model Description
Fine-tuned **FLAN-T5-small** model for dialogue summarization tasks using the DialogSum dataset. Achieves improved performance in generating concise summaries from conversational dialogues.

## Training Data
- **Dataset**: DialogSum (1,837 annotated dialogues)
- **Preprocessing**: prompt_template = """
Here is a dialogue:
{dialogue}
Write a short summary.
{summary}
"""

Converted original dataset into instruction format with dialogue-summary pairs

## Training Setup
| Parameter | Value |
|-----------|-------|
| Base Model | google/flan-t5-small |
| Epochs | 5 |
| Batch Size | 16 (per device) |
| Learning Rate | 3e-4 |
| Optimizer | Adafactor |
| Mixed Precision | fp16 |
| Gradient Accumulation | 4 steps |
| Max Length | 512 tokens |

## Evaluation Results
| Metric | Value |
|--------|-------|
| ROUGE-1 | 0.3722 |
| ROUGE-2 | 0.1066 |
| ROUGE-L | 0.2794 |

## Basic Inference

```python
from transformers import pipeline
from datasets import load_dataset
from evaluate import load

summarizer = pipeline('summarization', model='ingu627/finetuned-flan-t5-dialogsum')
dataset = load_dataset('knkarthick/dialogsum', split='test')
basic_summarizer = pipeline('summarization', model='google/flan-t5-small')

rouge = load('rouge')

references = []
predictions = []
basic_predictions = []

for example in dataset.select(range(50)):
    generated = summarizer(
        f"Summarize this dialogue:\n{example['dialogue']}\nSummary:",
        max_length=135,
        num_beams=3
    )[0]['summary_text']
    basic_generated = basic_summarizer(
        f"Summarize this dialogue:\n{example['dialogue']}\nSummary:",
        max_length=135,
        num_beams=3
    )[0]['summary_text']

    references.append(example['summary'])
    predictions.append(generated)
    basic_predictions.append(basic_generated)

fine_tuned_results = rouge.compute(
    predictions=predictions,
    references=references,
    rouge_types=['rouge1', 'rouge2', 'rougeL'],
    use_aggregator=True,
    use_stemmer=True,
)

print(fine_tuned_results)
```

## Training Procedure
- **Hardware**: T4 GPU on Kaggle
- **Framework**: PyTorch with Hugging Face Transformers
- **Training Time**: ~50 minutes (Kaggle free tier)

## Recommendations
- Use beam search (num_beams=3-5) for better results
- Combine with post-processing for formatting
- Fine-tune longer for complex dialogues

## Limitations
- Struggles with multi-topic dialogues
- May miss subtle contextual cues
- Best performance on short conversations (<500 tokens)

## License
Apache 2.0 (Same as base FLAN-T5 model)

## Citation
@misc{dialogsum2021,
title={DialogSum: A Real-Life Scenario Dialogue Summarization Dataset},
author={Karthick Krishnamurthy},
year={2021},
howpublished={HuggingFace Datasets},
}