[How to] Carefully designing the head of a Transformer model?
# ❓ Questions & Help

While using the pre-trained any transformer model, what are the main things we normally should consider while designing the head? Simply like,

```python
distill = transformer.distilbert.... <----- slicing the first position
x  = Dense(n, activation = ' ')(distill) <-------- simple classifer head
``` 
Is it really necessary to design an additional head? (I'm using `tensorflow` backend.)