readme: ---
  tags:
  - generated_from_trainer
  metrics:
  - accuracy
  model-index:
  - name: legal-german-roberta-base
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # legal-german-roberta-base
  
  This model was trained from scratch on an unknown dataset.
  It achieves the following results on the evaluation set:
  - Loss: 0.7080
  - Accuracy: 0.8387
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 0.0001
  - train_batch_size: 1024
  - eval_batch_size: 512
  - seed: 42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: cosine
  - lr_scheduler_warmup_ratio: 0.05
  - training_steps: 1000000
  
  ### Training results
  
  | Training Loss | Epoch | Step    | Accuracy | Validation Loss |
  |:-------------:|:-----:|:-------:|:--------:|:---------------:|
  | 2.1008        | 0.05  | 50000   | 0.6533   | 2.0523          |
  | 1.5248        | 0.1   | 100000  | 0.7661   | 1.1575          |
  | 1.3152        | 0.15  | 150000  | 0.7674   | 1.1281          |
  | 1.1239        | 0.2   | 200000  | 0.7971   | 0.9458          |
  | 0.9472        | 0.25  | 250000  | 0.7876   | 0.9979          |
  | 0.961         | 0.3   | 300000  | 0.8075   | 0.8798          |
  | 1.0179        | 0.35  | 350000  | 0.8018   | 0.9102          |
  | 1.037         | 0.4   | 400000  | 0.8195   | 0.8107          |
  | 1.1206        | 0.45  | 450000  | 0.8152   | 0.8323          |
  | 1.0865        | 0.5   | 500000  | 0.8242   | 0.7829          |
  | 0.9616        | 0.55  | 550000  | 0.8224   | 0.7895          |
  | 0.7727        | 0.6   | 600000  | 0.8285   | 0.7585          |
  | 0.9871        | 1.04  | 650000  | 0.8320   | 0.7391          |
  | 1.0679        | 1.09  | 700000  | 0.8311   | 0.7436          |
  | 0.9203        | 1.14  | 750000  | 0.8355   | 0.7187          |
  | 0.9626        | 1.19  | 800000  | 0.8353   | 0.7242          |
  | 0.7263        | 1.24  | 850000  | 0.7094   | 0.8378          |
  | 0.8578        | 1.29  | 900000  | 0.7140   | 0.8368          |
  | 0.7693        | 1.34  | 950000  | 0.7091   | 0.8377          |
  | 1.0488        | 1.39  | 1000000 | 0.7080   | 0.8387          |
  
  
  ### Framework versions
  
  - Transformers 4.20.1
  - Pytorch 1.10.0+cu113
  - Datasets 2.8.0
  - Tokenizers 0.12.1
  