Giuseppe_Castiglione:

-initialize the diagonal elements of the weight matrix to be zero, and then set requires_grad = False for all diagonal elements


This won’t work. There is no notion of part of a tensor requires_grad or not requires_grad.



 Giuseppe_Castiglione:

-create a constant variable that’s a mask of 1s and 0s and use that to multiply an adjustable variable.  Then use this product as the weight matrix


This could work.
Lastly, another way that might be easier is just to register a backward hook on the weight parameters. In the hook you can zero out all the entries on the diagonals of the gradient.