NikkeS/bart-large-lora-finetuned-agriculture-summarization

LoRA Fine-Tuned BART for Agricultural Text Summarization

Model Overview

This is a LoRA fine-tuned version of facebook/bart-large-cnn, specialized for summarizing agricultural texts.
The model has been trained on processed agricultural e-books sourced from Project Gutenberg, using Low-Rank Adaptation (LoRA) for efficient fine-tuning.

Books used:
https://www.gutenberg.org/ebooks/56640
https://www.gutenberg.org/ebooks/67813
https://www.gutenberg.org/ebooks/20772
https://www.gutenberg.org/ebooks/40190
https://www.gutenberg.org/ebooks/4924
https://www.gutenberg.org/ebooks/4525

• Base Model: facebook/bart-large-cnn
• Fine-Tuning Method: LoRA (Low-Rank Adaptation)
• Dataset: Processed agricultural e-books from Gutenberg
• Primary Task: Summarization

Training Details

• LoRA Configuration:
  • Rank (r): 8
  • Alpha (lora_alpha): 16
  • Dropout (lora_dropout): 0.1
• Training Setup:
  • Batch Size: 8
  • Gradient Accumulation Steps: 2
  • Learning Rate: 2e-5
  • Epochs: 3
  • Optimizer: AdamW (bitsandbytes, if available)
  • Precision: Mixed-precision (fp16)
• Dataset Processing:
  • Texts were tokenized using the BART tokenizer.
  • Chunking was performed using LangChain Recursive Text Splitter (max 300 words per chunk).
  • Training pairs were created using LLM-based summarization (meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo-p).

How to Use

Load the Model in Transformers

from transformers import pipeline

# Load fine-tuned model from Hugging Face
summarizer = pipeline("summarization", model="your_username/bart-large-lora-finetuned-agriculture")

# Sample text for summarization
text = "Crop rotation helps maintain soil health by alternating different crops each season."

# Generate summary
summary = summarizer(text, max_length=100, min_length=30, do_sample=False)[0]["summary_text"]

print("Summary:", summary)