The fourth dimension stores the x, y input pixel locations which are used to compute output.
Yes, as the doc 37 states, grid_sample computes the output using input pixel locations from the grid, i.e. using x and y in your case.
Downsampling of a non rectangular size should also work.
Your code runs on my machine using PyTorch 0.4.0.
Which version are you using?