I am using the huggingface pytorch-transformers for knowledge distilation,
How can I, for example, extract 8 layers from the 12 BertLayers of the bert-base-uncased to form a new model? I want to use the embedding and pooler layer of orginal model, but use only a portion of the encoder layers.