roberta-base-finetuned-squad-v1

This model is a fine-tuned version of roberta-base on the squad dataset.

Model description

Given a context / content, the model answers to a question by searching the content and extracting the relavant information.

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 16
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 3
  • mixed_precision_training: Native AMP

Training results

  • training loss: 0.77257

Framework versions

  • Transformers 4.27.4
  • Pytorch 1.13.1+cu116
  • Datasets 2.11.0
  • Tokenizers 0.13.3
Downloads last month
131
Safetensors
Model size
124M params
Tensor type
I64
F32
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.

Model tree for sooolee/roberta-base-finetuned-squad-v1

Finetuned
(1639)
this model

Evaluation results