There is a weird way (I soon will try it) to do so. Make a x,y grid transform it to PIL image and apply reverse transform for that grids and vuala you get mapping from old coordinates to new.
        w, h = img.width, img.height

        assert w == h

        grid_x = torch.arange(w).repeat(h, 1).view([h, w]).type(torch.FloatTensor) / w
        grid_y = torch.arange(h).repeat(w, 1).t().view([h, w]).type(torch.FloatTensor) / h
        dummy = torch.zeros((h, w))
        grid_xy = torch.stack([grid_x, dummy, grid_y])
        grid_xy = transforms.ToPILImage()(grid_xy)

        rot = 30
        img = img.rotate(rot, Image.NEAREST, expand=0)
        grid_xy = grid_xy.rotate(-rot, Image.NEAREST, expand=0)

        grid_xy.show()
        img.show()

        grid_xy = transforms.ToTensor()(grid_xy)
        x_orig = label[:, 2] * w
        y_orig = label[:, 3] * h

        label[:, 2] = grid_xy[0, y_orig, x_orig]
        label[:, 3] = grid_xy[2, y_orig, x_orig]

The most weird thing here is that you may apply ant number of transformations whats so ever without any geometry headache  the only thing you care about is keep transforming grid_xy in reverse order
Enjoy!