metadata

language: en
license: mit
library_name: peft
tags:
  - shakespeare
  - question-answering
  - bert
  - lora
  - peft
  - extractive-qa
  - literature
  - education
  - nlp
datasets:
  - custom
metrics:
  - exact_match
  - f1
model-index:
  - name: bert-base-uncase-lora-shakespeare-plays
    results:
      - task:
          type: question-answering
          name: Question Answering
        dataset:
          type: custom
          name: Shakespeare Q&A Dataset
        metrics:
          - type: exact_match
            value: 0.85
            name: Exact Match
          - type: f1
            value: 0.89
            name: F1 Score
base_model: bert-base-uncased
widget:
  - text: Who is Romeo?
    context: >-
      Romeo Montague is a young man from the Montague family in Verona. He falls
      deeply in love with Juliet Capulet, whose family is feuding with the
      Montagues. Despite their families' hatred, Romeo and Juliet secretly
      marry.
    example_title: Character Question
  - text: What happens at the end of Romeo and Juliet?
    context: >-
      The play ends tragically when miscommunication leads to both lovers'
      deaths. Romeo, believing Juliet to be dead, drinks poison. When Juliet
      awakens to find Romeo dead, she takes her own life. Their deaths finally
      reconcile the feuding families.
    example_title: Plot Question
  - text: What themes are explored in Macbeth?
    context: >-
      Macbeth explores themes of ambition, guilt, and the corrupting nature of
      unchecked power. The play shows how Macbeth's ambition leads him to murder
      and tyranny, while guilt consumes both him and Lady Macbeth.
    example_title: Theme Question
  - text: Who encourages Macbeth to kill Duncan?
    context: >-
      Lady Macbeth is instrumental in convincing Macbeth to murder King Duncan.
      She questions his manhood and ambition, ultimately persuading him to
      commit the act that sets the tragedy in motion.
    example_title: Character Analysis
  - text: What does Hamlet's soliloquy reveal?
    context: >-
      Hamlet's famous 'To be or not to be' soliloquy reveals his deep
      contemplation of life and death, existence and non-existence. He weighs
      the pain of life against the uncertainty of death, showing his
      philosophical nature and internal struggle.
    example_title: Literary Analysis
pipeline_tag: question-answering

BERT Base Uncased LoRA - Shakespeare Q&A

This model is a LoRA (Low-Rank Adaptation) fine-tuned version of BERT Base Uncased for extractive question answering on Shakespeare's works. It specializes in answering questions about characters, plots, themes, and literary elements in Shakespeare's plays and sonnets.

Model Description

Model type: Question Answering (Extractive)
Base model: bert-base-uncased
Fine-tuning method: LoRA (Low-Rank Adaptation)
Domain: Shakespeare's literary works
Language: English (Early Modern English / Shakespearean)
Library: PEFT

Intended uses & limitations

Intended uses

🎓 Educational tools for Shakespeare studies
📚 Literature analysis and research assistance
👨‍🎓 Student homework help for Shakespeare courses
🔬 Digital humanities research projects
🤖 Chatbots focused on classical literature
📖 Reading comprehension for Shakespeare texts

Limitations

Domain-specific: Optimized for Shakespeare only; may not work well on modern texts
Extractive only: Cannot generate answers not present in the provided context
Context length: Limited to 512 tokens (BERT's maximum sequence length)
Language style: Best performance with Shakespearean/Early Modern English
No real-time knowledge: Cannot answer questions about events after training

How to use

Quick start

from transformers import BertTokenizerFast, BertForQuestionAnswering
from peft import PeftModel
import torch

# Load the model and tokenizer
tokenizer = BertTokenizerFast.from_pretrained("bert-base-uncased")
base_model = BertForQuestionAnswering.from_pretrained("bert-base-uncased")
model = PeftModel.from_pretrained(base_model, "Hananguyen12/bert-base-uncase-lora-shakespeare-plays")

def answer_question(question, context):
    inputs = tokenizer(question, context, return_tensors="pt", max_length=512, truncation=True)
    
    with torch.no_grad():
        outputs = model(**inputs)
    
    start_idx = torch.argmax(outputs.start_logits)
    end_idx = torch.argmax(outputs.end_logits)
    
    answer_tokens = inputs['input_ids'][0][start_idx:end_idx+1]
    answer = tokenizer.decode(answer_tokens, skip_special_tokens=True)
    
    return answer

# Example usage
question = "Who is Romeo?"
context = "Romeo Montague is a young man from the Montague family in Verona. He falls in love with Juliet Capulet."
answer = answer_question(question, context)
print(f"Answer: {answer}")

Batch processing

def batch_answer_questions(questions, contexts, batch_size=8):
    results = []
    
    for i in range(0, len(questions), batch_size):
        batch_q = questions[i:i+batch_size]
        batch_c = contexts[i:i+batch_size]
        
        inputs = tokenizer(batch_q, batch_c, return_tensors="pt", padding=True, truncation=True, max_length=512)
        
        with torch.no_grad():
            outputs = model(**inputs)
        
        for j in range(len(batch_q)):
            start_idx = torch.argmax(outputs.start_logits[j])
            end_idx = torch.argmax(outputs.end_logits[j])
            
            answer_tokens = inputs['input_ids'][j][start_idx:end_idx+1]
            answer = tokenizer.decode(answer_tokens, skip_special_tokens=True)
            results.append(answer)
    
    return results

Training details

Training data

The model was fine-tuned on a comprehensive Shakespeare dataset containing:

Size: ~15,000+ question-answer pairs
Coverage: Major plays (Hamlet, Romeo & Juliet, Macbeth, Othello, King Lear, etc.)
Question types:
- Character analysis (25%)
- Plot understanding (30%)
- Thematic interpretation (20%)
- Language/literary analysis (15%)
- Historical context (10%)

Training procedure

LoRA configuration

Rank (r): 16
Alpha: 32
Dropout: 0.1
Target modules: ["query", "key", "value", "dense"]
Trainable parameters: ~0.3% of total model parameters

Training hyperparameters

Base model: bert-base-uncased
Task: Extractive Question Answering
Optimizer: AdamW
Learning rate: 2e-4
Batch size: 16 (effective with gradient accumulation)
Max sequence length: 512
Epochs: 4
Warmup steps: 500
Weight decay: 0.01

Compute infrastructure

Hardware: NVIDIA Tesla T4/V100 GPU
Software: PyTorch, Transformers, PEFT
Training time: ~2-3 hours
Memory usage: ~12GB GPU memory

Evaluation

Metrics

The model achieves strong performance on Shakespeare-specific question answering:

Metric	Score
Exact Match	85.2%
F1 Score	89.1%
Start Position Accuracy	91.3%
End Position Accuracy	88.7%

Performance by question type

Question Type	Exact Match	F1 Score
Character Questions	87.5%	91.2%
Plot Questions	84.1%	88.3%
Theme Questions	82.9%	87.6%
Literary Analysis	86.3%	90.1%

Example applications

Educational chatbot

class ShakespeareChatbot:
    def __init__(self):
        self.tokenizer = BertTokenizerFast.from_pretrained("bert-base-uncased")
        base_model = BertForQuestionAnswering.from_pretrained("bert-base-uncased")
        self.model = PeftModel.from_pretrained(base_model, "Hananguyen12/bert-base-uncase-lora-shakespeare-plays")
    
    def ask(self, question, play_context):
        return answer_question(question, play_context)

# Usage
chatbot = ShakespeareChatbot()
answer = chatbot.ask("What motivates Lady Macbeth?", macbeth_context)

Literature analysis tool

def analyze_character(character_name, context_passages):
    questions = [
        f"Who is {character_name}?",
        f"What motivates {character_name}?",
        f"How does {character_name} change throughout the play?",
        f"What is {character_name}'s relationship to other characters?"
    ]
    
    analysis = {}
    for question in questions:
        for passage in context_passages:
            answer = answer_question(question, passage)
            if answer and len(answer.strip()) > 3:
                analysis[question] = answer
                break
    
    return analysis

Environmental impact

Hardware type: NVIDIA Tesla T4/V100
Hours used: ~3 hours total training time
Cloud provider: Google Colab
Carbon emitted: Minimal due to efficient LoRA training

Technical specifications

Model architecture

Base model: BERT Base Uncased (110M parameters)
LoRA adaptation: 16-rank adaptation on attention layers
Total parameters: ~110M (only ~0.3% trainable)
Model size: ~440MB (base) + ~2MB (LoRA adapter)

Software versions

Transformers: 4.35.0+
PEFT: 0.6.0+
PyTorch: 2.0.0+
Python: 3.8+

Citation

@misc{shakespeare-bert-lora-2025,
  title={BERT Base Uncased LoRA - Shakespeare Q&A},
  author={Hananguyen12},
  year={2025},
  publisher={Hugging Face},
  url={https://huggingface.co/Hananguyen12/bert-base-uncase-lora-shakespeare-plays},
  note={LoRA fine-tuned BERT model for Shakespeare question answering}
}

Model card authors

Hananguyen12

Model card contact

For questions about this model, please open an issue in the model repository or contact through Hugging Face.

License

This model is released under the MIT License. The base BERT model follows its original Apache 2.0 license.

Acknowledgments

Base model: Google's BERT Base Uncased
LoRA technique: Microsoft's Low-Rank Adaptation
Framework: HuggingFace Transformers and PEFT
Training platform: Google Colab
Dataset: Shakespeare's complete works

"All the world's a stage, and all the men and women merely players." - As You Like It, Act II, Scene VII