As long as the validation set loss is low enough for the target task you need, there is no problem. It is certainly possible for MSE loss to be explode and that usually means the network has a problem learning the underlying data distribution. However, here, your validation loss tells a different story.
First thing, initialization of parameters with torch.nn.init.kaiming_uniform_ or something other than random which is what you have. Second, check your source code; are you printing correctly (i.e. did you initialize variables correctly, etc)? If the network is performing poorly on your testset, post the train code here – it may be a small oversight/bug in the train/val loops.