readme: ---
  license: apache-2.0
  tags:
  - generated_from_trainer
  metrics:
  - accuracy
  model-index:
  - name: output
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # output
  
  ## Model description
  
  This model is a fine-tuned version of [EleutherAI/gpt-neo-2.7B](https://huggingface.co/EleutherAI/gpt-neo-2.7B) on the Lila-IID-train/dev set from the [Lila dataset](https://github.com/allenai/Lila).
  
  ## Intended uses & limitations
  
  If you use this model, please cite our work.
  ```
  @INPROCEEDINGS{Mishra2022Lila,
    author = {
      Swaroop Mishra 
        and Matthew Finlayson 
        and Pan Lu 
        and Leonard Tang 
        and Sean Welleck 
        and Chitta Baral 
        and Tanmay Rajpurohit 
        and Oyvind Tafjord 
        and Ashish Sabharwal 
        and Peter Clark 
        and Ashwin Kalyan},
    title = {Lila: A Unified Benchmark for Mathematical Reasoning},
    booktitle = {Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
    year = {2022}
  }
  ```
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 5e-05
  - train_batch_size: 4
  - eval_batch_size: 4
  - seed: 42
  - distributed_type: multi-GPU
  - num_devices: 2
  - total_train_batch_size: 8
  - total_eval_batch_size: 8
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - num_epochs: 10.0
  
  ### Training results
  
  | Training Loss | Epoch | Step  | Validation Loss | Accuracy |
  |:-------------:|:-----:|:-----:|:---------------:|:--------:|
  | No log        | 0.06  | 100   | 0.7930          | 0.8214   |
  | No log        | 0.11  | 200   | 0.7544          | 0.8290   |
  | No log        | 0.17  | 300   | 0.7358          | 0.8328   |
  | No log        | 0.23  | 400   | 0.7192          | 0.8357   |
  | 0.8156        | 0.28  | 500   | 0.7012          | 0.8397   |
  | 0.8156        | 0.34  | 600   | 0.6904          | 0.8419   |
  | 0.8156        | 0.4   | 700   | 0.6802          | 0.8440   |
  | 0.8156        | 0.45  | 800   | 0.6670          | 0.8465   |
  | 0.8156        | 0.51  | 900   | 0.6572          | 0.8486   |
  | 0.7219        | 0.57  | 1000  | 0.6499          | 0.8500   |
  | 0.7219        | 0.62  | 1100  | 0.6411          | 0.8522   |
  | 0.7219        | 0.68  | 1200  | 0.6343          | 0.8537   |
  | 0.7219        | 0.74  | 1300  | 0.6299          | 0.8546   |
  | 0.7219        | 0.79  | 1400  | 0.6221          | 0.8561   |
  | 0.662         | 0.85  | 1500  | 0.6157          | 0.8574   |
  | 0.662         | 0.91  | 1600  | 0.6138          | 0.8579   |
  | 0.662         | 0.96  | 1700  | 0.6055          | 0.8595   |
  | 0.662         | 1.02  | 1800  | 0.6143          | 0.8598   |
  | 0.662         | 1.08  | 1900  | 0.6191          | 0.8599   |
  | 0.5707        | 1.14  | 2000  | 0.6118          | 0.8607   |
  | 0.5707        | 1.19  | 2100  | 0.6123          | 0.8611   |
  | 0.5707        | 1.25  | 2200  | 0.6089          | 0.8617   |
  | 0.5707        | 1.31  | 2300  | 0.6064          | 0.8619   |
  | 0.5707        | 1.36  | 2400  | 0.6079          | 0.8625   |
  | 0.4923        | 1.42  | 2500  | 0.6040          | 0.8625   |
  | 0.4923        | 1.48  | 2600  | 0.6030          | 0.8630   |
  | 0.4923        | 1.53  | 2700  | 0.6021          | 0.8636   |
  | 0.4923        | 1.59  | 2800  | 0.6001          | 0.8643   |
  | 0.4923        | 1.65  | 2900  | 0.5981          | 0.8644   |
  | 0.4909        | 1.7   | 3000  | 0.5942          | 0.8648   |
  | 0.4909        | 1.76  | 3100  | 0.5918          | 0.8650   |
  | 0.4909        | 1.82  | 3200  | 0.5923          | 0.8659   |
  | 0.4909        | 1.87  | 3300  | 0.5884          | 0.8664   |
  | 0.4909        | 1.93  | 3400  | 0.5884          | 0.8663   |
  | 0.4964        | 1.99  | 3500  | 0.5903          | 0.8669   |
  | 0.4964        | 2.04  | 3600  | 0.6421          | 0.8655   |
  | 0.4964        | 2.1   | 3700  | 0.6401          | 0.8651   |
  | 0.4964        | 2.16  | 3800  | 0.6411          | 0.8649   |
  | 0.4964        | 2.21  | 3900  | 0.6387          | 0.8645   |
  | 0.345         | 2.27  | 4000  | 0.6362          | 0.8654   |
  | 0.345         | 2.33  | 4100  | 0.6362          | 0.8654   |
  | 0.345         | 2.38  | 4200  | 0.6362          | 0.8654   |
  | 0.345         | 2.44  | 4300  | 0.6357          | 0.8655   |
  | 0.345         | 2.5   | 4400  | 0.6362          | 0.8656   |
  | 0.3463        | 2.55  | 4500  | 0.6377          | 0.8658   |
  | 0.3463        | 2.61  | 4600  | 0.6357          | 0.8660   |
  | 0.3463        | 2.67  | 4700  | 0.6294          | 0.8665   |
  | 0.3463        | 2.72  | 4800  | 0.6333          | 0.8665   |
  | 0.3463        | 2.78  | 4900  | 0.6362          | 0.8662   |
  | 0.3508        | 2.84  | 5000  | 0.6357          | 0.8666   |
  | 0.3508        | 2.89  | 5100  | 0.6299          | 0.8673   |
  | 0.3508        | 2.95  | 5200  | 0.6313          | 0.8668   |
  | 0.3508        | 3.01  | 5300  | 0.7188