  Medical endoscopy remains a challenging application for simultaneous
localization and mapping (SLAM) due to the sparsity of image features and size
constraints that prevent direct depth-sensing. We present a SLAM approach that
incorporates depth predictions made by an adversarially-trained convolutional
neural network (CNN) applied to monocular endoscopy images. The depth network
is trained with synthetic images of a simple colon model, and then fine-tuned
with domain-randomized, photorealistic images rendered from computed tomography
measurements of human colons. Each image is paired with an error-free depth map
for supervised adversarial learning. Monocular RGB images are then fused with
corresponding depth predictions, enabling dense reconstruction and mosaicing as
an endoscope is advanced through the gastrointestinal tract. Our preliminary
results demonstrate that incorporating monocular depth estimation into a SLAM
architecture can enable dense reconstruction of endoscopic scenes.

Image and Video Processing;Computer Vision and Pattern Recognition;Robotics