Yes the name is definitely a bit confusing.



 agarret7:

it should be applied sample-wise following dense layers


Are you saying that you want to normalize over all but the batch dimension, i.e. each sample is normalized with mean and std computed from that sample? If so, you should use Layer Normalization, which will become available in next release. To achieve the same effects in 0.3.1, you can feed input.unsqueeze(0) into a BN layer.