How can I use only one layer transformer via this repository?
## ❓ Questions & Help
I want to use only one layer transformer on the head of some backbone model. Can I use this repository in a simple way?
This repository is especially useful if you're looking to use a pre-trained transformer of the same architecture than that of BERT, GPT, GPT-2, XLM, XLNet or TransfoXL.

If you're looking at using a simple transformer of your own making, how about using the newly released [torch.nn.Transformer](https://pytorch.org/docs/stable/nn.html?highlight=transformer#torch.nn.Transformer)?<|||||>Thanks! But since I use pytorch v1.0.1 with cuda8.0, it's not convenient to upgrade to v1.2.0 and use torch.nn.Transformer.<|||||>If you're looking to use an existing architecture and modifying a few things (like the number of layers, or embedding size), you can always do so by specifying these values in a config file.

As you were saying you would like to use a one-layer transformer on top of some backbone model, you could create a config and specify `num_hidden_layers = 1` and `num_attention_heads = 1` to have a very simple one-layer single-headed transformer.

The documentation for the `BertConfig` file can be found [here](https://huggingface.co/pytorch-transformers/model_doc/bert.html#bertconfig). Each model has its own configuration file.

If you're looking to build a Transformer from scratch, something you could do is re-use some of our model's logic to create your own transformer. For example if you want to use our Attention for GPT-2, you could always import it like this:

```python
from pytorch_transformers.modeling_gpt2 import Attention
```

You can then re-use it as a part of your code, building your own Transformer architecture.

Hope that helps.<|||||>Thanks for your patience! I will try what you told me. It really helps me a lot!
Thanks again.