It seems if I use what you suggested,
layer.weight.data.copy_(ones(layer.weight.data.size()))
then it works but if I use
layer.weight.data.copy_(sign(randn(layer.weight.data.size())))
then the backprop returns NaN.