Are the posted numbers representing the number of samples for the images and targets? If so, are 10 images corresponding to a single label?
I still don’t know how the output would be treated, but one approach would be to use a linear layer with out_features=1700 and reshape the output to [batch_size, 17, 100]. However, it depends on your actual use case to know, if this would be the right approach.
The out_features are passed as the second argument in the nn.Linear as described in the docs.

To permute the input from [16540, 500, 500, 3] to a channels-first format you could use x = x.permute(0, 3, 1, 2).