The memory_format seems to be respected in this code snippet:
x = torch.empty(2, 3, 4, 5, memory_format=torch.contiguous_format)
print(x.is_contiguous(memory_format=torch.contiguous_format))
print(x.is_contiguous(memory_format=torch.channels_last))

y = x.to('cuda')
print(y.is_contiguous(memory_format=torch.contiguous_format))
print(y.is_contiguous(memory_format=torch.channels_last))


x = torch.empty(2, 3, 4, 5, memory_format=torch.channels_last)
print(x.is_contiguous(memory_format=torch.contiguous_format))
print(x.is_contiguous(memory_format=torch.channels_last))

y = x.to('cuda')
print(y.is_contiguous(memory_format=torch.contiguous_format))
print(y.is_contiguous(memory_format=torch.channels_last))

Could you post a code snippet, which would break it, please?