readme: ---
  license: other
  tags:
  - generated_from_trainer
  model-index:
  - name: distilroberta-current
    results: []
  ---
  
  # distilroberta-current
  
  This model classifies articles as current (covering or discussing current events) or not current (not relating to current events).
  
  The model is a fine-tuned version of [distilroberta-base](https://huggingface.co/distilroberta-base) on a dataset of articles labeled using weak-supervision and manual labeling
  
  It achieves the following results on the evaluation set:
  - Loss: 0.1745
  - Acc: 0.9355
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 2e-05
  - train_batch_size: 8
  - eval_batch_size: 8
  - seed: 12345
  - gradient_accumulation_steps: 4
  - total_train_batch_size: 32
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - lr_scheduler_warmup_steps: 16
  - num_epochs: 20
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step | Validation Loss | Acc    |
  |:-------------:|:-----:|:----:|:---------------:|:------:|
  | No log        | 1.0   | 11   | 0.6559          | 0.7097 |
  | 0.6762        | 2.0   | 22   | 0.5627          | 0.7097 |
  | 0.5432        | 3.0   | 33   | 0.4606          | 0.7097 |
  | 0.5432        | 4.0   | 44   | 0.3651          | 0.8065 |
  | 0.411         | 5.0   | 55   | 0.2512          | 0.9194 |
  | 0.269         | 6.0   | 66   | 0.2774          | 0.9355 |
  | 0.269         | 7.0   | 77   | 0.2062          | 0.8710 |
  | 0.2294        | 8.0   | 88   | 0.2598          | 0.9355 |
  | 0.1761        | 9.0   | 99   | 0.1745          | 0.9355 |
  
  
  ### Framework versions
  
  - Transformers 4.11.3
  - Pytorch 1.10.1
  - Datasets 1.17.0
  - Tokenizers 0.10.3
  