You could permute the dimensions and just apply a linear layer:
x = torch.randn(2, 8, 16)
x = x.permute(0, 2, 1).contiguous()
lin = nn.Linear(8, 4)
output = lin(x)

The docs 145 explain the behavior of the input and output shapes.