I reduce the longformer's attention window from 512 to 256, but train speed not changed 
# ❓ Questions & Help

According to the Longformer's design, it should be sensitive for training speed and memory consuming when the local window size is changed. However, I find nothing happened when I half the attention window size (from [512]*12 to [256]*12). Is this phenomenon under expectations? or there is something wrong?
Hey @xixiaoyao, the main gain is memory savings for longformer - so I'm not really surprised if you say that training speed does not change when halving the attention window...<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
