How much GPU memory is needed to run run_squad.py
## ❓ Questions & Help

How much GPU memory is needed to run `run_squad.py`, I tried on `GTX 1050ti (4gb)` with the following setting and I am getting out of memory error

```
$ python3 examples/run_squad.py \
  --model_type bert \
  --model_name_or_path bert-base-cased \
  --do_train \
  --do_eval \
  --do_lower_case \
  --train_file $SQUAD_DIR/train-v1.1.json \
  --predict_file $SQUAD_DIR/dev-v1.1.json \
  --per_gpu_train_batch_size 12 \
  --learning_rate 3e-5 \
  --num_train_epochs 2.0 \
  --max_seq_length 384 \
  --doc_stride 128 \
  --output_dir /tmp/debug_squad/ \
  --save_steps 1000
```
With 4 GB you're bound to issues with a batch size of 12. You could figure out the total memory usage of the model + calculate the memory footprints of tensors to determine the biggest batch size that would fit on your GPU. 

Specifying a smaller batch size (like 1 or 2) would let you run the script, though.<|||||>> With 4 GB you're bound to issues with a batch size of 12. You could figure out the total memory usage of the model + calculate the memory footprints of tensors to determine the biggest batch size that would fit on your GPU.
> 
> Specifying a smaller batch size (like 1 or 2) would let you run the script, though.

Hello @LysandreJik I am able to run the code with `batch size = 1`

