I have seen similar questions but without well defined solutions, but I want to force a layer’s weight matrix to be diagonal. Is there a way I can do this?
One workaround could be to simply set the off-diagonals to zero before each forward pass through the network. Would this work? I’m uncertain if it will cause issues with the backprop.