I don’t know about any public dataset on this so you need to collect your own dataset with different holes, weather etc…
You can try to find a good collection of depth estimatation papers in:

  
      

      GitHub
  

  
    

GitHub - scott89/awesome-depth: A curated list of publication for depth...

  A curated list of publication for depth estimation - GitHub - scott89/awesome-depth: A curated list of publication for depth estimation


  

  
    
    
  

  


An alternative you could try to adapt many 3d human pose estimation network to your escavator pose dataset.
E.g.:

  
      

      Kapernikov
  

  
    

Pose estimation – Deep learning for pose estimation – Kapernikov

  Pose estimation is second nature to Kapernikov. We apply our expertise in computer vision and machine learning to track machine movements.

  
    Est. reading time: 3 minutes
  

  

  
    
    
  

  


If you can use fiducials you can explore something like:

  

      mdpi-res.com
  

  
    

sensors-21-04478.pdf

  3.81 MB