Advanced indexing 5 (links to the NumPy reference, but PyTorch wants to be compatible here) should do the trick:
values = torch.arange(2 * 3 * 5 * 5).view(2, 3, 5, 5)
idx_x = torch.randint(0, 5, (4,))
idx_y = torch.randint(0, 5, (4,))
values[:, :, idx_x, idx_y]

Best regards
Thomas