Note that this model has the weights of the encoder and the decoder tied, so if you want to use the existing weights, you probably want to just mask the indices of the tokens you don’t want to use in your predictions.
Otherwise you can try to replace the last layer, but you will need to adapt the code in modeling_gpt2.py to do this.