weight_mat = weight_mat.contiguous().view(height, -1)

Was just a way to stop error messages but it probably doesn t solve the problem. If you did not encounter problems with that line it might just be because of your Pytorch version (I am on 0.4.0 so maybe they made some changes in 0.4.1 which solve this).
Anyway I solved this using another SN implementation found on Github but @SimonW solution looks nice as well! Thank you for that Simon.