  Majority of the perception methods in robotics require depth information
provided by RGB-D cameras. However, standard 3D sensors fail to capture depth
of transparent objects due to refraction and absorption of light. In this
paper, we introduce a new approach for depth completion of transparent objects
from a single RGB-D image. Key to our approach is a local implicit neural
representation built on ray-voxel pairs that allows our method to generalize to
unseen objects and achieve fast inference speed. Based on this representation,
we present a novel framework that can complete missing depth given noisy RGB-D
input. We further improve the depth estimation iteratively using a
self-correcting refinement model. To train the whole pipeline, we build a large
scale synthetic dataset with transparent objects. Experiments demonstrate that
our method performs significantly better than the current state-of-the-art
methods on both synthetic and real world data. In addition, our approach
improves the inference speed by a factor of 20 compared to the previous best
method, ClearGrasp. Code and dataset will be released at
https://research.nvidia.com/publication/2021-03_RGB-D-Local-Implicit.

Computer Vision and Pattern Recognition;Robotics