I did work with the BERT model very recently, but I’m not allowed to share the code publicly.
This time did a more thorough pass through the code.  You are clearing the gradients (with optim.zero_grad() before applying them (optim.step()). Basically, the model is not learning anything. Put the optim.zero_grad() call at the beginning of the for loop.
loss.backward()
# update weights
optim.zero_grad() <- gradients are zeroed out after this call
torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)
optim.step()