include the output layer in the model using the pretrained weights
This is to be able to load the final output layer (bert.output_layer) from the TensorFlow pre-trained model.
In particular, it is a fully connected layer that is used to map the final hidden layer to the vocabulary size, to then apply the softmax, as follows:

logits = bert.output_layer(sequence_output)
log_softmax = nn.LogSoftmax(dim=-1)
log_probs = log_softmax(logits)

Thanks for that. I've ended up taking a more modular approach in the first pip release of the library.