I don’t quite understand your code, since you are averaging the batch dimension in:
x_i = x_i.mean(dim=0)

and are creating a new tensor in:
out = torch.cat(out, dim=1)

which would have shape of [50, *] where * is created by the temporal dimension in the input.
The error you are seeing is then raised in self.bn(out) so could you explain your use case a bit more, please?