Even with the identical input still the network outputs different results. here are the input samples from python and c++ with their respective outputs :
python
input :
img.shape: torch.Size([1, 3, 101, 180])
img_tensored: tensor([[
        [[0.3555, 0.3555, 0.3555],
          [0.3633, 0.3633, 0.3555],
          [0.3398, 0.3320, 0.3320]],

         [[0.8398, 0.8398, 0.8242],
          [0.8320, 0.8320, 0.8242],
          [0.7930, 0.7852, 0.7773]],

         [[0.9414, 0.9414, 0.9336],
          [0.9180, 0.9180, 0.9102],
          [0.8711, 0.8711, 0.8633]]]])

output :

offsets.shape: (1, 4, 46, 85)
offsets: [[[[-0.03554668 -0.03690651]
   [-0.01305734 -0.00945055]]

  [[-0.01119633 -0.01438548]
   [ 0.01255823  0.01120286]]

  [[-0.01207605  0.00594437]
   [ 0.01896726  0.02646127]]

  [[ 0.21539722  0.2312614 ]
   [ 0.24871898  0.24893966]]]]
probs.shape: (46, 85)
probs: [[0.00016916 0.00012743]
 [0.00051409 0.00051995]]

cpp:
input :
img.shape: [1, 3, 101, 180]
img: (1,1,.,.) =
  0.9414  0.9414  0.9336
  0.9180  0.9180  0.9102
  0.8711  0.8711  0.8633

(1,2,.,.) =
  0.8398  0.8398  0.8242
  0.8320  0.8320  0.8242
  0.7930  0.7852  0.7773

(1,3,.,.) =
  0.3555  0.3555  0.3555
  0.3633  0.3633  0.3555
  0.3398  0.3320  0.3320
[ CPUFloatType{1,3,3,3} ]

output :
offsets.shape: [1, 4, 46, 85]
probs.shape: [46, 85]
offsets: (1,1,.,.) =
 0.01 *
  0.1006  1.2322
  -2.9587 -2.2280

(1,2,.,.) =
 0.01 *
  1.3772  1.3971
  -1.2813 -0.8563

(1,3,.,.) =
 0.01 *
  6.2367  9.2561
   3.5719  5.4744

(1,4,.,.) =
  0.2901  0.2963
  0.2618  0.2771
[ CPUFloatType{1,4,2,2} ]
probs: 0.0001 *
 1.4593  1.0351
  6.6782  4.9104
[ CPUFloatType{2,2} ]

Note that the cv_image channels seem to be locked when converted into tensor, that is no matter what I do , i.e. use
cv::cvtColor(image, image, cv::COLOR_BGR2RGB); or not, the channel order that you see displayed in the output wont change!