Hi. I’m using a unit batch-size and following this discussion, [Error: Expected more than 1 value per channel when training - #2 by ptrblck] (Error: Expected more than 1 value per channel when training - #2 by ptrblck)  I can switch to InstanceNorm.
But this is the error i get when using
x =  torch.randn(1, 512)
batch_norm = nn.BatchNorm1d(512)
inst_norm = nn.InstancceNorm1d(512)
batch_norm(x)
>> ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 512])
inst_norm(x)
>>  InstanceNorm1d returns 0-filled tensor to 2D tensor.This is because InstanceNorm1d reshapes inputs to(1, N * C, ...) from (N, C,...) and this makesvariances 0.
x = torch.randn(32, 512)
out = batch_norm(x)
out.shape
>> torch.Size([32, 512])

     File "/miniconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/miniconda3/lib/python3.9/site-packages/torch/nn/modules/instancenorm.py", line 56, in forward
    self._check_input_dim(input)
  File "//miniconda3/lib/python3.9/site-packages/torch/nn/modules/instancenorm.py", line 132, in _check_input_dim
    raise ValueError(
ValueError: InstanceNorm1d returns 0-filled tensor to 2D tensor.This is because InstanceNorm1d reshapes inputs to(1, N * C, ...) from (N, C,...) and this makesvariances 0.