Do you need the neural network just for tracking the bbox of the target object on the ground?
If you already know the drone XYZ from the ground and  yaw, pitch and roll and the camera is calibrated you can project the bbox in pixel on the ground plane.