The fluctuating loss behavior might come from your hyperparameters, not from a code bug.
Did the model architecture work in the past with your kind of data?
Your model is currently quite deep, so if you started right away with this kind of deep model, the behavior might be expected. I’m usually the fan of reusing a known good architecture or “growing” my model, i.e. I start with quite a shallow model until it converges, and add some layers to it as long as nothing breaks.
Also, a good Idea is to try out a few weight initialization schemes.
Here is a dummy code:
def weight_init(m):
    if isinstance(m, nn.Conv2d) or isinstance(m, nn.ConvTranspose2d):
        nn.init.xavier_uniform_(m.weight, gain=nn.init.calculate_gain('tanh'))
        nn.init.zeros_(m.bias)

model.apply(weight_init)

Another approach would be to scale down your data a bit and just use very few samples for training to see, if there are other code bugs.