transformers safetensors t5 text2text-generation autotrain_compatible text-generation-inference endpoints_compatible region:us
---
library_name: transformers
tags: []
---
## Model Details

### Model Description

T5 Small - News Summarization
Model Description
This model is a fine-tuned version of google/t5-small for news summarization. It has been trained on the CNN/DailyMail dataset to generate concise summaries of news articles.

Base Model: google/t5-small
Fine-tuned On: cnn_dailymail (version 3.0.0)
Task: Text Summarization
Framework: 🤗 Transformers (Trainer API)

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

- **Developed by:** [Niranjan R]
- **Funded by [optional]:** [More Information Needed]
- **Shared by [optional]:** [More Information Needed]
- **Model type:** [Summarization_Model]
- **Language(s) (NLP):** [English]
- **License:** [More Information Needed]
- **Finetuned from model [optional]:** [More Information Needed]

## How to Get Started with the Model

# Load model directly
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("Niranjann0902/Summarization_model")
model = AutoModelForSeq2SeqLM.from_pretrained("Niranjann0902/Summarization_model")

[More Information Needed]

## Training Details

GPU: Google Colab (T4 GPU)
Epochs: 1 (for fast training)
Batch Size: 16
Optimizer: AdamW with weight_decay=0.01
Mixed Precision: Enabled (fp16=True)

### Training Data

"cnn_dailymail" Dataset is used, Version : 3.0.0

### Limitations:

May struggle with long-form summarization.
Fine-tuning on more diverse datasets could improve performance.
Exploring even smaller models (like DistilT5) for faster inference.

### Training Procedure

The model was fine-tuned using the Hugging Face Trainer API with the following setup:

Dataset: cnn_dailymail (Version 3.0.0)
Training Split: 10,000 samples
Validation Split: 2,000 samples
Batch Size: 16 (for both training and evaluation)
Epochs: 1 (for quick training)
Optimizer: AdamW (weight_decay=0.01)
Mixed Precision: Enabled (fp16=True)
Hardware Used: Google Colab (T4 GPU)

The model was trained on a subset of the dataset to speed up training while maintaining reasonable performance. The learning process used the "summarize: " prefix before inputs, following the standard T5 fine-tuning approach for summarization tasks.

### Testing Data, Training Data, Factors & Metrics

Training Set: 10,000 samples from cnn_dailymail
Validation Set: 2,000 samples from cnn_dailymail
Preprocessing: Tokenization using T5Tokenizer with max-length padding and truncation
Text Format:
Input: "summarize: <news article>"
Output: <summary>


## More Information

Summarization Model

## Model Card Authors 

Niranjan R

## Model Card Contact

Contact Mail: niranjanraja.99@gmail.com