why this implementation didn't apply residual and layer norm?
## ❓ Questions & Help

In ALBERT implementation code `modeling_albert.py`, i can't find applying skip-connection and layer normalizing after multi-head attention layer. I didn't read about this tequnique. 
Is there a special reason about it?

One more, i saw the argument `inner_group_num` in `AlbertLayerGroup` class, is it introduced in original ABERT paper?

Thank you for good implementing.

The layer normalization happening after the attention is visible [here](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_albert.py#L255).

The `inner_group_num` is used to better understand how many layers are in a specific group.  It is set to 1 in all the configurations that google-research has output as they all have a single repeating layer, but it would be necessary to increase the number of inner groups if you pre-trained an ALBERT model that used more than one repeating layer.

You can see the `inner_group_num` in the official configuration files, for example the [xxlarge-v3](https://tfhub.dev/google/albert_xxlarge/3).