transformers safetensors bert question-answering generated_from_trainer dataset:squad base_model:google-bert/bert-base-uncased base_model:finetune:google-bert/bert-base-uncased license:apache-2.0 endpoints_compatible region:us
---
base_model: google-bert/bert-base-uncased
datasets:
- squad
library_name: transformers
license: apache-2.0
tags:
- generated_from_trainer
model-index:
- name: debug_squad
  results: []
---

# bert-base-uncased-finetuned-squad

This model is a fine-tuned version of [google-bert/bert-base-uncased](https://huggingface.co/google-bert/bert-base-uncased) on the SQuAD dataset.

## Model description

- **Model Type:** BERT for Question Answering
- **Base Model:** bert-base-uncased
- **Language:** English
- **Task:** Question Answering
- **Dataset:** [SQuAD v1.1](https://rajpurkar.github.io/SQuAD-explorer/)

## Training Procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 12
- eval_batch_size: 8
- seed: 42
- optimizer: AdamW_TORCH with beta=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5

### Training results

- training_loss: 0.6077
- eval_exact_match: 79.508
- eval_f1: 87.7293
- train_runtime: 1:09:34.90
- train_samples_per_second: 106.019
- train_steps_per_second: 8.835

## Intended uses & limitations

This model is intended for English question answering tasks. It performs best on factual questions where the answer is explicitly stated in the provided context. Note that this model was trained on SQuAD v1.1, which means it always tries to find an answer in the context (it cannot handle questions that have no answer).

### Usage Example

```python
from transformers import AutoModelForQuestionAnswering, AutoTokenizer

model = AutoModelForQuestionAnswering.from_pretrained("real-jiakai/bert-base-uncased-finetuned-squad")
tokenizer = AutoTokenizer.from_pretrained("real-jiakai/bert-base-uncased-finetuned-squad")

# Example usage
context = "BERT was developed by Google in 2018."
question = "Who developed BERT?"

inputs = tokenizer(question, context, return_tensors="pt")
outputs = model(**inputs)

answer_start = outputs.start_logits.argmax()
answer_end = outputs.end_logits.argmax()

answer = tokenizer.decode(inputs["input_ids"][0][answer_start:answer_end+1])
print(f"Answer: {answer}")  # Expected output: "google"
```

## Training Infrastructure

- Training Device: Single GPU (NVIDIA 4090 24GB)
- Training Time: ~70 minutes
- Framework: PyTorch
- Training Script: Hugging Face Transformers' `run_qa.py`

### Framework versions

- Transformers 4.47.0.dev0
- Pytorch 2.5.1+cu124
- Datasets 3.1.0
- Tokenizers 0.20.3
