Hi @ptrblck, what if instead of using nn.Parameter in the below code in the init() function, I use nn.utils.weight_norm?
Will that first reparameterize the weights then followed by L2-normalization before each forward pass?

init()
weights = nn.Parameter(torch.Tensor(in_size, out_size))

forward
forward(x):
w = F.normalize(weights)
out = torch.mm(x, w)
return out
Looking for help with PyTorch. Thanks!