So, I had some time to play around with this and the code makes perfect sense. However, during debugging I noticed something strange. The output values from the “grid” tensor don’t match up to the new opencv Mat pointer from the tensor. What makes it even more mysterious is that if the grid tensor is either a 1 or 3 channel tensor then the values match. Here is the code:
torch::Tensor _x = torch::arange(10).view({ 1, -1 }).expand({ 10, -1 });
torch::Tensor _y = torch::arange(10).view({ -1, 1 }).expand({ -1, 10 });
torch::Tensor grid = torch::stack({ _x,_y }, 0);
grid = grid.unsqueeze(0).expand({ 1, -1, -1, -1});

cv::Mat gridMat(cv::Size(10, 10), CV_32FC2, grid.data_ptr());

for (int j = 0; j < 10; j++) {
        for (int i = 0; i < 10; i++) {
                 //these bellow should match
	         std::cout << gridMat.at<cv::Vec2f>(j, i)[0] << ", " << grid[0][0][j][i] << "\n";
	}
}