Where are these values or tensors coming from?
            l1_part += float(l1_weight * weights_norm_1 / float(loss) * 100)
            l2_part += float(l2_weight * weights_norm_2 / float(loss) * 100)

Could you check if some of them are attached to a computation graph and would thus have a valid .grad_fn?
If so, detach() them before accumulating.



 Nemfor:

Can any things be in the impelemnted model class that can slows the perofmance?


Yes, if your model increases the workload in each iteration e.g. by using longer sequences or by backpropagating through all previous iterations, you would see a slowdown.