[GPT-Neo] Simplify local attention
# What does this PR do?

Co-authored-by: finetuneanon <finetuneanon@users.noreply.github.com>

This PR is a continuation of #11630 which simplifies GPT Neo's local attention implementation. All credit to @finetuneanon for finding this issue, providing the fix, and a detailed explanation. Thanks a lot for working on this :) 

The issue is described in #11320 and  performance evaluation results are available here:
[#12106 (comment)](https://github.com/huggingface/transformers/pull/12106#discussion_r650008259)

This PR does some cleanup and updates tests on top of @finetuneanon changes.

Fixes #11320, Fixes #11787, Fixes #12964, Fixes #11096
I had error in running:
```
  File "src\transformers\models\gpt_neo\configuration_gpt_neo.py", line 218, in __init__
    from .modeling_gpt_neo import GPTNeoAttentionMixin
ImportError: cannot import name 'GPTNeoAttentionMixin'
```
Need a patch for this PR.