In most if not all built-in cases (e.g. bert-base-cased), the original paper implementations are ported to the transformers architecture. (you can have a look at conversion scripts, e.g. https://github.com/huggingface/transformers/blob/77cd0e13d2d09f60d2f6d8fb8b08f493d7ca51fe/src/transformers/convert_pytorch_checkpoint_to_tf2.py 2, https://github.com/huggingface/transformers/blob/d155b38d6ea70fef3dec2e1f678269e713672bb7/src/transformers/commands/convert.py)
User models (e.g. username/mymodel-uncased) may have been trained in other ways or ported to the transformers architecture manually, with custom scripts, or they are trained by using the transformers library directly.