@ptrblck Thank you for the response. It gave me the idea to include an additional check on the gradient plotting function that does not include gradients if they are None. After doing so the plot still contains the “wc_module.lin.weight” gradient, and printing it explicitly showed me that there are values there, they are just incredibly small. My mistake for not checking more thoroughly, it’s just odd to me that those parameters have such small updates compared to the rest of them.