I have issue training the longformer on custom dataset, even on a small batch number, it says CUDA out of memory,
RuntimeError                              Traceback (most recent call last)
 in ()
----> 1 trainer.train()
18 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in _pad(input, pad, mode, value)
3550     assert len(pad) // 2 <= input.dim(), ‘Padding length too large’
3551     if mode == ‘constant’:
-> 3552         return _VF.constant_pad_nd(input, pad, value)
3553     else:
3554         assert value == 0, ‘Padding mode “{}”" doesn’t take in value argument’.format(mode)
RuntimeError: CUDA out of memory. Tried to allocate 1.13 GiB (GPU 0; 15.90 GiB total capacity; 11.40 GiB already allocated; 659.81 MiB free; 14.39 GiB reserved in total by PyTorch)