Why not directly fine-tune the full BERT model with a classification layer on top? That way you wouldn’t have to store the features.
If you go that way, I would suggest to drop the 3-layer DNN and just use a linear layer since BERT has a huge capacity, you likely don’t need 3 layers on top.
And if you go that way… well you can directly use the  BertForSequenceClassification model for PyTorch BERT 27 since that’s exactly what is this model