It seems to work for me in 1.7.0.dev20200830:
x, y, z, channels = 2, 3, 4, 5
input = torch.randn((x, y, z, channels))
lin = nn.Linear(channels, 6)
out = lin(input)
print(out.shape)
> torch.Size([2, 3, 4, 6])