Hi @e_sh, apoligies for the long delay, and thank you so much for providing the repro.  Your repro helped us find a bug in PyTorch.  The issue was that for per-channel observers, replication was not working properly in some cases in nn.DataParallel, specifically when the scale and zero_point buffers were not initialized.  We have a fix upcoming in fix unflatten_dense_tensor when there is empty tensor inside by zhaojuanmao · Pull Request #50321 · pytorch/pytorch · GitHub 3.  A short term workaround you could do before the fix lands is to run an input through the network before replicating it - this will populate all the quantization buffers.
As an aside, I’d recommend looking into using nn.DistributedDataParallel, it is generally recommended over nn.DataParallel.