|
|
--- |
|
|
language: en |
|
|
tags: |
|
|
- summarization |
|
|
- bart |
|
|
- lora |
|
|
- fine-tuning |
|
|
- agriculture |
|
|
- huggingface |
|
|
license: apache-2.0 |
|
|
datasets: |
|
|
- gutenberg-books |
|
|
library_name: transformers |
|
|
--- |
|
|
|
|
|
# LoRA Fine-Tuned BART for Agricultural Text Summarization |
|
|
|
|
|
## Model Overview |
|
|
This is a **LoRA fine-tuned version** of `facebook/bart-large-cnn`, specialized for **summarizing agricultural texts**. |
|
|
The model has been trained on processed **agricultural e-books** sourced from Project Gutenberg, using **Low-Rank Adaptation (LoRA)** for efficient fine-tuning. |
|
|
|
|
|
Books used: |
|
|
https://www.gutenberg.org/ebooks/56640 |
|
|
https://www.gutenberg.org/ebooks/67813 |
|
|
https://www.gutenberg.org/ebooks/20772 |
|
|
https://www.gutenberg.org/ebooks/40190 |
|
|
https://www.gutenberg.org/ebooks/4924 |
|
|
https://www.gutenberg.org/ebooks/4525 |
|
|
|
|
|
|
|
|
- **Base Model:** [facebook/bart-large-cnn](https://huggingface.co/facebook/bart-large-cnn) |
|
|
- **Fine-Tuning Method:** LoRA (Low-Rank Adaptation) |
|
|
- **Dataset:** Processed agricultural e-books from Gutenberg |
|
|
- **Primary Task:** Summarization |
|
|
|
|
|
## Training Details |
|
|
- **LoRA Configuration:** |
|
|
- Rank (`r`): 8 |
|
|
- Alpha (`lora_alpha`): 16 |
|
|
- Dropout (`lora_dropout`): 0.1 |
|
|
- **Training Setup:** |
|
|
- **Batch Size:** 8 |
|
|
- **Gradient Accumulation Steps:** 2 |
|
|
- **Learning Rate:** 2e-5 |
|
|
- **Epochs:** 3 |
|
|
- **Optimizer:** AdamW (bitsandbytes, if available) |
|
|
- **Precision:** Mixed-precision (`fp16`) |
|
|
- **Dataset Processing:** |
|
|
- Texts were tokenized using the **BART tokenizer**. |
|
|
- Chunking was performed using **LangChain Recursive Text Splitter** (max 300 words per chunk). |
|
|
- Training pairs were created using **LLM-based summarization** (`meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo-p`). |
|
|
|
|
|
## How to Use |
|
|
### Load the Model in Transformers |
|
|
```python |
|
|
from transformers import pipeline |
|
|
|
|
|
# Load fine-tuned model from Hugging Face |
|
|
summarizer = pipeline("summarization", model="your_username/bart-large-lora-finetuned-agriculture") |
|
|
|
|
|
# Sample text for summarization |
|
|
text = "Crop rotation helps maintain soil health by alternating different crops each season." |
|
|
|
|
|
# Generate summary |
|
|
summary = summarizer(text, max_length=100, min_length=30, do_sample=False)[0]["summary_text"] |
|
|
|
|
|
print("Summary:", summary) |
|
|
|
