Longformer inference time
The longformer-base-4096 model should be faster than bert-base-cased ,However I find that the former one takes more time to inference
hi @cmdllx , I don't think longformer will  be necessarily faster than bert-base-cased. The goal of longformer is to make memory complexity of attention layer liner w.r.t to seq length instead of quadratic, so it saves memory, not compute. More the seq-length , more will be inference time.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
