Could the output be somehow “normalized” such that the gray color would be visualized as white?
The background should have the color [126, 126, 126], so I’m unsure why it’s shown as white.
I’m getting a gray background here:
x = torch.randint(0, 255, (5, 3, 1080, 1920)).float()
y = F.interpolate(x, (234, 416))
res = torch.ones(5, 3, 416, 416) * 126.
res[:, :, 91:325, :] = y
plt.imshow(res[0].permute(1, 2, 0).byte())

Assuming that a “tall” image should be resizes as [416, 234], you might need to change the code to:
if tall:
    res[:, :, :, 91:325] = y
else:
    res[:, :, 91:325, :] = y

or
if tall:
    idx = 3
else:
    idx = 2
res.index_copy_(idx, torch.arange(91, 325), y)