I guess you didn’t set your model to evaluation before generating the samples. Since you are using some BatchNorm layers, the running statistics will be calculated and updated using the current batch. Try to call model.eval before the inference step. This will make sure to use the approximate running mean and var.
Sometimes it might help to pass some validation inputs in training mode without backpropagating to update the BatchNorm layers.
Let me know, if my assumption was right.