Why the positional embeddings in bert are not inplemented by sin/cos as the original paper said? Are these embeddings trainable?
https://github.com/huggingface/transformers/blob/948b730f9777174335812cf76de2a9dd9e4cf20e/src/transformers/models/bert/modeling_bert.py#L172
BERT uses absolute position embeddings by default. The sin/cos embeddings are from the original Transformer paper IRRC. <|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.