Which files are essential when customize and modify a specific pre-trained model？
For some reason, I want to customize and modify a specific pre-trained model to adapt my experiments, e.g. BERT.  

However, there are so many unrelated files in the  `src/transformers`, and it really confuses me a lot. 

I only want to extract the essential source files to support my experiments without installing `transformers`.

**So I would like to know Which files are essential when customize and modify a specific pre-trained model.**

For example, for the BERT model, I think the files related to BERT are in the `src/transformers/models/bert`. 
Besides that, are there any source files essential?

Could you help me at your convenience? @LysandreJik  @patrickvonplaten @qqaatw 
For example, I find https://github.com/microsoft/unilm/tree/master/unilm-v1/src/pytorch_pretrained_bert  use the `pytorch_pretrained_bert` without installing `transformers`. The file tree is very clean and clear. I also want to do this too, without unnecessary irrelevant source files.
![image](https://user-images.githubusercontent.com/26213546/131616444-d4fe68bf-5406-40ce-9914-03b525621678.png)<|||||>Hi,

Basically, it's not readily to extract `BertModel` or `BertTokenizer` as an independent module because they're inheriting some base classes resided at other folders, intending to provide some easy-to-use utilities such as `from_pretrained` method.

IMO, I think you can have `transformers` as a git submodule placed in your repository and modify the features you want accordingly.  This way you can still pull the latest features from upstream `transformers` and have your custom BERT (or other models).<|||||>> Hi,
> 
> Basically, it's not readily to extract `BertModel` or `BertTokenizer` as an independent module because they're inheriting some base classes resided at other folders, intending to provide some easy-to-use utilities such as `from_pretrained` method.
> 
> IMO, I think you can have `transformers` as a git submodule placed in your repository and modify the features you want accordingly. This way you can still pull the latest features from upstream `transformers` and have your custom BERT (or other models).

Thanks for your reply.
I plan to inherit the origin class and modify it. Maybe this is a compromise solution for me.

Do you think so? @qqaatw 

<|||||>Yeah, I think this way is comparatively easy to achieve your goal.<|||||>Thanks.