What are you currently plotting? Is it the loss of your train and validation set?
If so, could you plot the log of these values?
It looks like both are going down (to zero) pretty quickly. Why are you assuming your model suffers from vanishing gradients?
You could try to print out the norm of the gradients in a specific layer using torch.Tensor.register_hook 4.