## DIODE: A Dense Indoor and Outdoor DEpth Dataset

Figure 1: Samples from our DIODE dataset. Black represents no valid depth or no valid normals.

![image](https://ac.marcodsn.me/arxiv-markdown-images/1908.00463v2-figure-fa04fb51-668e-4d98-821b-313d3ac3a65e.jpg)

Igor Vasiljevic Nick Kolkin Shanyi Zhang * glyph[sharp] Ruotian Luo Haochen Wang † Falcon Z. Dai Andrea F. Daniele Mohammadreza Mostajabi Steven Basart † Matthew R. Walter Gregory Shakhnarovich

TTI-Chicago, † University of Chicago, glyph[sharp] Beihang University

{ ivas,nick.kolkin,syzhang,rluo,whc,dai,afdaniele,mostajabi,steven,mwalter,greg @ttic.edu }

## Abstract

## 1. Introduction

We introduce DIODE (Dense Indoor/Outdoor DEpth), a dataset that contains thousands of diverse, high-resolution color images with accurate, dense, long-range depth measurements. DIODE is the first public dataset to include RGBD images of indoor and outdoor scenes obtained with one sensor suite. This is in contrast to existing datasets that involve just one domain/scene type and employ different sensors, making generalization across domains difficult. The dataset is available for download at diode-dataset.org .

* Most of the work was performed when the author was at TTI-Chicago.

Many of the most dramatic successes of deep learning in computer vision have been for recognition tasks, and have relied upon large, diverse, manually labeled datasets such as ImageNet [7], Places [35] and COCO [21]. In contrast, RGBD datasets that pair images and depth cannot be created with crowd-sourced annotation, and instead rely on 3D range sensors that are noisy, sparse, expensive, and often all of the above. Some popular range sensors are restricted to indoor scenes due to range limits and sensing technology. Other types of sensors are typically deployed only outdoors. As a result, available RGBD datasets [13, 27, 30, 25] primarily include only one of these scene types. Furthermore, RGBD datasets tend to be fairly homogeneous, particularly

for outdoor scenes, where the dataset is usually collected with autonomous driving in mind [13]. While there have been many recent advances in 2.5D and 3D vision, we believe progress has been hindered by the lack of large and diverse real-world datasets comparable to ImageNet and COCO for semantic object recognition.

Depth information is integral to many problems in robotics, including mapping, localization and obstacle avoidance for terrestrial and aerial vehicles, and in computer vision, including augmented and virtual reality [23]. Compared to depth sensors, monocular cameras are inexpensive and ubiquitous, and would provide a compelling alternative if coupled with a predictive model that can accurately estimate depth. Unfortunately, no public dataset exists that would allow fitting the parameters of such a model using depth measurements taken by the same sensor in both indoor and outdoor settings. Even if one's focus is on unsupervised learning of depth perception [14], it is important to have an extensive, diverse dataset with depth ground-truth for evaluation of models.

Indoor RGBD datasets are usually collected using structured light cameras, which provide dense, but noisy, depth maps up to approximately 10 m, limiting their application to small indoor environments (e.g., home and office environments). Outdoor datasets are typically collected with a specific application in mind (e.g., self-driving vehicles), and generally acquired with customized sensor arrays consisting of monocular cameras and LiDAR scanners. Typical LiDAR scanners have a high sample rate, but relatively low spatial resolution. Consequently, the characteristics of available indoor and outdoor depth maps are quite different (see Table 1), and networks trained on one kind of data typically generalize poorly to another [12]. Confronting this challenge has attracted recent attention, motivating the CVPR 2018 Robust Vision Challenge workshop.

This paper presents the DIODE (Dense Indoor/Outdoor DEpth) dataset in an effort to address the aforementioned limitations of existing RGBD datasets. DIODE is a largescale dataset of diverse indoor and outdoor scenes collected using a survey-grade laser scanner (FARO Focus S350 [1]). Figure 1 presents a few representative examples from DIODE, illustrating the diversity of the scenes and the quality of the 3D measurements. This quality allows us to produce not only depth maps of unprecedented density and resolution, but also to derive surface normals with a level of accuracy not possible with existing datasets. The most important feature of DIODE is that it is the first dataset that covers both indoor and outdoor scenes in the same sensing and imaging setup .

## 2. Related Work

A variety of RGBD datasets in which images (RGB) are paired with associated depth maps (D) have been proposed through the years. Most exclusively consist of either indoor or outdoor scenes, and many are tied to a specific task (e.g., residential interior modeling or autonomous driving).

## 2.1. Outdoor scenes

Perhaps the best known RGBD dataset is KITTI [13]. It was collected using a vehicle equipped with a sparse Velodyne VLP-64 LiDAR scanner and RGB cameras, and features street scenes in and around the German city of Karlsruhe. The primary application of KITTI involves perception tasks in the context of self-driving. Thus, the diversity of outdoor scenes is much lower than that of DIODE, but the extent of the street scenes makes it complementary.

Cityscapes [5] similarly provides a dataset of street scenes, albeit with more diversity than KITTI. With a focus on semantic scene understanding, Cityscapes only includes depth obtained from a stereo camera and has no ground truth. Synthia [25] is another street scene dataset with depth maps of comparable density to DIODE, but consists of synthetic data, requiring domain adaptation to apply to realworld settings. Sintel [24] is another synthetic dataset that includes outdoor scenes. Megadepth [20] is a large-scale dataset of outdoor internet images, with depth maps reconstructed using structure-from-motion techniques, but also lacking in ground truth depth and scale.

Make3D [27] provides RGB and depth information for outdoor scenes that are similar in nature to our dataset. Like DIODE, it contains diverse outdoor scenes that are not limited to street views. Make3D was an early RGBD dataset that spurred the development of monocular depth estimation techniques, but the depth maps are very low-resolution (see Table 1). Our dataset can be considered a successor to Make3D, collected using a much higher resolution scanner and including many more diverse scenes.

More recently, the ETH3D dataset [29] is similar to DIODE in terms of sensing modality and diversity. It uses the FARO X330 laser scanner (we use the FARO S350) to record 360 ° panoramic scans along with high-resolution DSLR images for the purpose of benchmarking multi-view stereo algorithms. Like DIODE, ETH3D contains indoor and outdoor scenes. However, the dataset is intended for benchmarking rather than training, and is an order of magnitude smaller than DIODE. Tanks and Temples [17] is a similar dataset for benchmarking 3D reconstructions, with acccurate ground truth obtained by a laser scanner but a comparatively small number of scans.

Recently, the 3D Movies dataset [19] was introduced, utilizing the depth information that can be obtained from stereoscopic movies in order to create a large and diverse dataset. This dataset can be seem as complementary to ours given that the depth is approximate and lacks scale, but has a large number of frames with dynamic objects and diverse scenes.

## 2.2. Indoor scenes

The NYUv2 dataset [30] is widely used for monocular depth estimation in indoor environments. The data was collected with a Kinect RGBD camera, which provides sparse and noisy depth returns. These returns are generally inpainted and smoothed before they are used for monocular depth estimation tasks. As a result, while the dataset includes sufficient samples to train modern machine learning pipelines, the 'ground-truth' depth does not necessarily correspond to true scene depth. Our dataset complements NYUv2 by providing very high-resolution, low-noise depth maps of both indoor and outdoor scenes. Another indoor dataset that relies on SfM is SUN3D [34, 32], which provides approximate depth without scale.

Meanwhile, the recent Matterport3D [4] and ScanNet [6] datasets offer a large number of dense depth images of indoor scenes. The datasets were rendered from multiple views using a SLAM pipeline. As a result, the depth maps are much noisier and of lower resolution than DIODE, and are intended for semantic tasks like 3D segmentation rather than accurate 3D reconstruction or depth estimation.

To summarize, compared to existing RGBD datasets, DIODE offers larger scene variety; higher image and depth map resolution; higher density and accuracy of depth measurements; and most importantly, the ability to reason over depth perception in both indoor and outdoor environments in a truly unified framework.

## 2.3. Monocular depth estimation

Depth estimation is a crucial step towards inferring scene geometry from 2D images. There is an extensive literature on estimating depth from stereo images; most of these methods rely on point-matching between left and right images, typically based on hand-crafted or learned features [31, 28, 10]. The goal in monocular depth estimation is to predict the depth value of each pixel, given only a single RGB image as input. Make3D [27] was an early approach that leveraged supervised learning for monocular depth estimation, and more recent work has applied deep neural networks to the task [8, 18, 26, 22, 11, 11].

We use the DenseDepth [2] architecture, which provides near-state-of-the-art results on both the NYUv2 and KITTI datasets and thus serves as a simple baseline to test the performance of neural networks on our indoor+outdoor dataset.

## 3. The DIODE Dataset

We designed and acquired the DIODE dataset with three primary desiderata in mind. First, the dataset should include a diverse set of indoor (e.g., homes, offices, lecture halls, and communal spaces) and outdoor (e.g., city streets, parking lots, parks, forests, and river banks) scenes. Sec- ond, the dataset should provide dense depth maps, with accurate short-, mid-, and long-range depth measurements for a large fraction of image pixels. Third, the depth measurements should be highly accurate.

## 3.1. Data Acquisition

The aforementioned qualities preclude measuring depth using structured light cameras, and instead requires using a LiDAR. We collected our dataset using a FARO Focus S350 scanner. The FARO is an actuated survey-grade phase-shift laser scanner for both indoor and outdoor environments that provides highly accurate depth measurements over a large depth FOV (between 0.6 m and 350 m with error as low as 1 mm), and at high angular resolution (0.009°). The FARO includes a color camera mounted coaxially with the depth laser, and produces a high-resolution panorama that is automatically aligned with the FARO's depth returns. These attributes give the FARO a variety of advantages over the more frequently used Velodyne LiDAR with a separate RGB camera, or Kinect depth cameras:

- · the scanner is equally well suited for in indoor and outdoor scanning;
- · the point clouds are orders of magnitude more dense;
- · the RGB camera is placed very close to the sensor, so there is virtually no baseline between the detector and the camera.

Scanning parameters The FARO allows for the customization of various parameters that govern the scanning process. These include the resolution of the resulting depth scan (i.e., the number of points), the color resolution of the RGB panorama (i.e., standard or high definition), and the quality of the scan (i.e., the integration time of each range measurement). We chose the following scanning settings:

- · 1 × quality: single scanning pass for every azimuth;
- · 360 degree horizontal FOV, 150 degree vertical FOV;
- · 1 2 / resolution: ≈ 170M points;
- · 3 × HDR: low exposure, regular, high exposure bracketing for RGB.

These settings result in a scan time of approximately 11 minutes. The intermediate output of a scan is a 20700 × 8534 (approximately) RGB panorama and a corresponding point cloud, with each 3D point associated with a pixel in the panorama (and thus endowed with color). As with other LiDAR sensors, highly specular objects as well as those that are farther than 350 m (including the sky) do not have an associated depth measurement. Another limitation of the scanner for RGBD data collection is that the LiDAR 'sees' through glass or in darkness, resulting in detailed depth maps for image regions that lack the corresponding appearance information.

Scanning Locations We chose scan locations to ensure diversity in the dataset as well a similar number of indoor and outdoor scenes. The scenes include small student offices, large residential buildings, hiking trails, meeting halls, parks, city streets, and parking lots, among others. The scenes were drawn from three different cities. Given the relatively long time required for each scan (approximately 11 min) and the nature of the scanning process, we acquired scans when we could avoid excessive motion and dynamic changes in the scene. However, occasional movement through the scenes is impossible to completely avoid.

The resulting scans exhibit diversity not just between the scenes themselves, but also in the scene composition. Some outdoor scans include a large number of nearby objects (compared to KITTI, where the majority of street scans have few objects near the car), while some indoor scenes include distant objects (e.g., as in the case of large meeting halls and office buildings with large atria), in contrast to scenes in other indoor datasets collected with comparatively short-range sensors.

## 3.2. Data Curation and Processing

Image Extraction We process the scans to produce a set of rectified RGB images (henceforth referred to as 'crops') at a resolution of 768 × 1024 . The crops correspond to a grid of viewing directions, at four elevation angles ( -20 ◦ , -10 ◦ , 0 ◦ , 10 ◦ , 20 ◦ , and 30 ◦ ), and at regular 10 ◦ azimuth intervals, yielding 216 viewing directions. We rectify each crop corresponding to 45 ◦ (vertical) × 60 ◦ (horizontal) FOV. . 1

Curved sections of the panorama corresponding to each viewing frustum must be undistorted to form each rectified crop, i.e., a rectangular image with the correct perspective. To accomplish this we associate each pixel in the rectified crop with a ray (3D vector) in the canonical coordinate frame of the scanner. We use this information to map from panorama pixels and the 3D point cloud to crop pixels.

For each pixel p ij in the desired 768 × 1024 crop, let the ray passing through the pixel be r ij . We assign the RGB value of each pixel p ij to the average of the RGB values of the nearest five pixels in terms of the angular distance between their rays and r ij .

We employ a similar procedure to generate a rectified depth map. For each ray r ij , we find in the pointcloud the set of 3D points X ij whose rays are nearest to r ij in angular distance.

We discard points with angular distance to r ij greater than 0 5 . ◦ . We then set the depth of pixel p ij to the robust mean of the depth of points in X ij , using the median 80% of depth values.

1 In the CVPR2019 Workshop version of the paper, we described extracting crops for 67 5 . ◦ (vertical) × 90 ◦ (horizontal) FOV. That version of the dataset is now deprecated, but available upon request.

In the event that the set X ij is empty we record p ij as having no return (coded as depth 0 ).

To compute normals for each crop we begin by associating each pointcloud point with a spatial index in the panorama. Then for each spatial index ( i, j ) of the panorama we take the set of 3d points ˆ X ij indexed by the 11x11 grid centered on ( i, j ) , and find a plane using RANSAC [9] which passes through the median of the ˆ X ij , and for which at least 40% of the points in ˆ X ij have a residual less than 0 1 . cm. We define the normal at position ( i, j ) to be the vector normal to this plane that faces towards the pointcloud's origin. Finally for each crop we rotate these normals according to the camera vector, and rectify them via the same procedure used for the depth map.

Crop selection The scanner acquires the full 3D pointcloud before capturing RGB images. This, together with the relatively long scan duration can result in mismatches between certain RGB image regions and the corresponding depth values for dynamic elements of the scene (e.g., when a car present and static during the 3D acquisition moves before the RGB images of its location are acquired). Additionally, some crops might have almost no returns (e.g., an all-sky crop for an outdoor scan). We manually curated the dataset to remove such crops, as well as those dominated by flat, featureless regions (e.g., a bare wall surface close to the scanner).

Masking Though the depth returns are highly accurate and dense, the scanner has some of the same limitations as many LiDAR-based scanners-i.e. erroneous returns on specular objects, 'seeing through' glass and darkness causing inconsistencies between RGB and depth, etc.

To ameliorate issues caused by spurious returns, for every crop we create an automated 'validity mask' using a robust median filter that rejects depth returns that are too far from the median of a small neighborhood. We provide the raw depth returns to allow users to implement alternative masking or inpainting schemes (e.g. [30]). In addition, for the validation set we manually mask regions with spurious depth or inconsistencies between RGB and depth.

Standard Split Weestablish a train/validation/test split in order to ensure the reproducibility of our results as well as to make it easy to track progress of methods using DIODE. The validation set consists of curated crops from 10 indoor and 10 outdoor scans, while the test set consists of crops from 20 indoor and 20 outdoor scans.

When curating scans in the validation and test partitions, we do not allow the fields-of-view of the selected crops to overlap by more than 20 ◦ in azimuth for validation scans, and 40 ◦ for test scans. No such restriction is used when selecting train crops.

|                            | DIODE       | NYUv2   | KITTI         | MAKE3d     |
|----------------------------|-------------|---------|---------------|------------|
| Return Density (Empirical) | 99.6%/66.9% | 68%     | 16%           | 0.38%      |
| # Images Indoor/Outdoor    | 8574/16884  | 1449/0  | 0/94000       | 0/534      |
| Sensor Depth Precision     | ± 1 mm      | ± 1 cm  | ± 2 cm        | ± 3 . 5 cm |
| Sensor Angular Resolution  | 0.009°      | 0.09°   | 0.08°H, 0.4°V | 0.25°      |
| Sensor Max Range           | 350 m       | 5 m     | 120 m         | 80 m       |
| Sensor Min Range           | 0.6 m       | 0.5 m   | 0.9 m         | 1 m        |

Table 1: Statistics of DIODE compared to other popular RGBD datasets. Separate indoor and outdoor density percentages are provided for DIODE.

## 3.3. Dataset Statistics

Table 1 compares the statistics of DIODE to other widely used RGBD datasets. Note the return density of the data, i.e., the ratio of color pixels with depth measurements to all color pixels; the captured point cloud has a higher resolution than our projected depth maps and thus we have returns for most pixels, missing returns on either very far regions (e.g. sky) or specular regions in indoor images. The depth precision allows for the capture of fine depth edges as well as thin objects.

Figure 2 compares the distribution of values in the depth maps in popular datasets to DIODE (values beyond 100 m are only found in DIODE and thus we clip the figures for DIODE for ease of comparison). Note that given that there are often objects both near and far from the camera in outdoor scans, the distribution of depth values is more diffuse in DIODE/outdoor than in KITTI. Only the much smaller and lower resolution Make3D is close to matching the diversity of DIODE depth values.

## 4. Experiments

In this section, we provide a baseline for monocular depth estimation on the DIODE dataset, and highlight the challenge of predicting high-resolution depth with current methods. We use the simple architecture of Alhashim et al. [3] (DenseDepth) in all of our experiments since

Figure 2: Distribution of measured depth values for DIODE and other popular RGBD datasets.

![image](https://ac.marcodsn.me/arxiv-markdown-images/1908.00463v2-figure-65d2e1e0-ab39-402b-b65f-a66594490bf4.jpg)

it achieves near-state-of-the-art results on both the KITTI and NYUv2 datasets. Their architecture uses a DenseNet169 [15] pretrained on ImageNet as an encoder as well as a simple decoder with no batch normalization.

## 4.1. Model

We train three models on the indoor (DIODE/Indoor) and outdoor (DIODE/Outdoor) subsets of DIODE, as well as the entire dataset (DIODE/All). During training, all networks are trained with the batch size of 4 for 30 epochs using Adam [16]. We start with a learning rate of 0 0001 . and decrease it by one-tenth after 20 epochs. The CNN is fed with a full-resolution image ( 1024 × 768 ) and outputs the predicted depth at half of the resolution ( 512 × 384 ). We employ random horizontal flips and random channel swaps for data augmentation.

Weuse the same objective as in previous work [3], which consists of L1 loss, depth gradient, and structural similarity (SSIM) [33]. The weight on each loss term is set as the same as that in the original DenseDepth model. We set the maximum depth to be 350 m. Note that we do not finetune the model on DIODE/Indoor or DIODE/Outdoor after training on DIODE/All.

## 4.2. Evaluation

During final evaluation, we apply 2 × upsampling to the prediction to match the size of the ground truth. Other settings are identical to the original DenseDepth model [3].

We evaluate the performance of the model on the validation set using standard pixel-wise error metrics [8]:

- · average absolute difference between predicted and ground-truth depth (mae)
- · absolute difference scaled by the reciprocal of the ground-truth depth (abs rel)
- · square root of the average squared error (rmse)
- · rmse and mae between the log of predicted depth and log of ground-truth depth (rmse log 10 and mae log 10 )
- · percentage of depth predictions d within thr relative to ground-truth depth d ∗ , i.e., δ = max( d d ∗ , d ∗ d ) &lt; thr .

Table 2: Baseline performance for different training and validation sets, where δ i indicates δ &lt; 1 25 . i .

| Experimental Setting   | Experimental Setting   | lower is better   | lower is better   | lower is better   | lower is better   | lower is better   | higher is better   | higher is better   | higher is better   |
|------------------------|------------------------|-------------------|-------------------|-------------------|-------------------|-------------------|--------------------|--------------------|--------------------|
| Train                  | Validation             | mae               | rmse              | abs rel           | mae log 10        | rmse log 10       | δ 1                | δ 2                | δ 3                |
| DIODE/Indoor           | DIODE/Indoor           | 1.5016            | 1.6948            | 0.3306            | 0.1577            | 0.1775            | 0.4919             | 0.7159             | 0.8256             |
| DIODE/Indoor           | DIODE/Outdoor          | 12.1237           | 15.9203           | 0.6691            | 0.6141            | 0.6758            | 0.1077             | 0.1812             | 0.2559             |
| DIODE/Indoor           | DIODE/All              | 7.6462            | 9.9238            | 0.5264            | 0.4217            | 0.4658            | 0.2697             | 0.4066             | 0.4961             |
| DIODE/Outdoor          | DIODE/Indoor           | 2.2836            | 3.2810            | 0.8428            | 0.2910            | 0.3547            | 0.2456             | 0.4399             | 0.5900             |
| DIODE/Outdoor          | DIODE/Outdoor          | 5.0366            | 8.8323            | 0.3636            | 0.1879            | 0.3149            | 0.5368             | 0.7558             | 0.8505             |
| DIODE/Outdoor          | DIODE/All              | 3.8761            | 6.4922            | 0.5656            | 0.2314            | 0.3317            | 0.4140             | 0.6226             | 0.7407             |
| DIODE/All              | DIODE/Indoor           | 1.1425            | 1.4779            | 0.3343            | 0.1233            | 0.1506            | 0.5510             | 0.7816             | 0.8989             |
| DIODE/All              | DIODE/Outdoor          | 5.4865            | 9.2781            | 0.3870            | 0.1972            | 0.3141            | 0.4781             | 0.7236             | 0.8360             |
| DIODE/All              | DIODE/All              | 3.6554            | 5.9900            | 0.3648            | 0.1660            | 0.2452            | 0.5088             | 0.7481             | 0.8625             |

## 4.3. Analysis

Table 2 presents the results of the experiment. The model trained on the entire dataset (DIODE/All) outperforms the model trained on DIODE/Indoor on indoor validation. This may be explained by the larger size (roughly 2 × the images) of the outdoor dataset as well as the fact that outdoor scans capture many objects at a wide range of distances (including near the scanner). The performance slightly degrades on the outdoor validation when training on DIODE/All, this may be because most of the objects in a typical indoor scene are well within ∼ 50 m of the camera.

The model trained on the entire dataset (DIODE/All) performs better on the entire validation set than models trained on the indoor and outdoor subsets.

## 5. Conclusion

We expect the unique characteristics of DIODE, in particular the density and accuracy of depth data and above all the unified framework for indoor and outdoor scenes, to enable more realistic evaluation of depth prediction methods and facilitate progress towards general depth estimation methods. We plan to continue acquiring additional data to expand DIODE, including more locations and additional variety in weather and season.

## References

- [1] FARO™ S350 scanner. https://www.faro. com/products/construction-bim-cim/ faro-focus/ . Accessed: 2019-03-20.
- [2] I. Alhashim and P. Wonka. High quality monocular depth estimation via transfer learning. arXiv preprint arXiv:1812.11941 , 2018.
- [3] I. Alhashim and P. Wonka. High quality monocular depth estimation via transfer learning. arXiv preprint arXiv:1812.11941 , 2018.
- [4] A. Chang, A. Dai, T. Funkhouser, M. Halber, M. Nießner, M. Savva, S. Song, A. Zeng, and Y. Zhang. Matterport3D: Learning from RGBD data in indoor environments. arXiv preprint arXiv:1709.06158 , 2017.
- [5] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele. The Cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2016.
- [6] A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. Nießner. ScanNet: Richlyannotated 3D reconstructions of indoor scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2017.
- [7] J. Deng, W. Dong, R. Socher, L. Li, K. Li, and F. Li. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2009.
- [8] D. Eigen, C. Puhrsch, and R. Fergus. Depth map prediction from a single image using a multi-scale deep network. In Advances in Neural Information Processing Systems (NeurIPS) , 2014.
- [9] M. Fischler and R. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM , 24(6):381-395, June 1981.
- [10] J. Flynn, I. Neulander, J. Philbin, and N. Snavely. Deep stereo: Learning to predict new views from the world's imagery. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2016.
- [11] H. Fu, M. Gong, C. Wang, K. Batmanghelich, and D. Tao. Deep ordinal regression network for monocular depth estimation. In Proceedings of the IEEE Con-

- ference on Computer Vision and Pattern Recognition (CVPR) , 2018.
- [12] R. Garg, V. K. B.G., G. Carneiro, and I. Reid. Unsupervised CNN for single view depth estimation: Geometry to the rescue. In Proceedings of the European Conference on Computer Vision (ECCV) , 2016.
- [13] A. Geiger, P. Lenz, C. Stiller, and R. Urtasun. Vision meets robotics: The KITTI dataset. International Journal of Robotics Research , 2013.
- [14] V. Guizilini, R. Ambrus, S. Pillai, and A. Gaidon. Packnet-sfm: 3d packing for self-supervised monocular depth estimation. arXiv preprint arXiv:1905.02693 , 2019.
- [15] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition , 2017.
- [16] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.
- [17] A. Knapitsch, J. Park, Q.-Y. Zhou, and V. Koltun. Tanks and temples: Benchmarking large-scale scene reconstruction. ACMTransactions on Graphics (ToG) , 36(4):78, 2017.
- [18] I. Laina, C. Rupprecht, V. Belagiannis, F. Tombari, and N. Navab. Deeper depth prediction with fully convolutional residual networks. In Proceedings of the International Conference on 3D Vision (3DV) , 2016.
- [19] K. Lasinger, R. Ranftl, K. Schindler, and V. Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. arXiv preprint arXiv:1907.01341 , 2019.
- [20] Z. Li and N. Snavely. Megadepth: Learning singleview depth prediction from internet photos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2018.
- [21] T. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll´ ar, and C. L. Zitnick. Microsoft COCO: Common objects in context. In Proceedings of the European Conference on Computer Vision (ECCV) , 2014.
- [22] F. Liu, C. Shen, G. Lin, and I. Reid. Learning depth from single monocular images using deep convolutional neural fields. IEEE Transactions on Pattern Analysis and Machine Intelligence , 38(10):20242039, 2016.
- [23] E. Marchand, H. Uchiyama, and F. Spindler. Pose estimation for augmented reality: A hands-on survey. IEEE Transactions on Visualization and Computer Graphics , 22(12):2633-2651, 2016.
- [24] N. Mayer, E. Ilg, P. Hausser, P. Fischer, D. Cremers, A. Dosovitskiy, and T. Brox. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2016.
- [25] G. Ros, L. Sellart, J. Materzynska, D. Vazquez, and A. M. Lopez. The SYNTHIA dataset: A large collection of synthetic images for semantic segmentation of urban scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2016.
- [26] A. Roy and S. Todorovic. Monocular depth estimation using neural regression forest. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2016.
- [27] A. Saxena, M. Sun, and A. Y. Ng. Make3D: Depth perception from a single still image. In Proceedings of the National Conference on Artificial Intelligence (AAAI) , 2008.
- [28] D. Scharstein and R. Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal on Computer Vision , 47(1-3):7-42, 2002.
- [29] T. Schops, J. L. Schonberger, S. Galliani, T. Sattler, K. Schindler, M. Pollefeys, and A. Geiger. A multiview stereo benchmark with high-resolution images and multi-camera videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2017.
- [30] N. Silberman, D. Hoiem, P. Kohli, and R. Fergus. Indoor segmentation and support inference from RGBD images. In Proceedings of the European Conference on Computer Vision (ECCV) , 2012.
- [31] N. Smolyanskiy, A. Kamenev, and S. Birchfield. On the importance of stereo for accurate depth estimation: An efficient semi-supervised deep neural network approach. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops , 2018.
- [32] X. Sun, Y. Xie, P. Luo, and L. Wang. A dataset for benchmarking image-based localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2017.
- [33] Z. Wang, E. P. Simoncelli, and A. C. Bovik. Multiscale structural similarity for image quality assessment. In The Thrity-Seventh Asilomar Conference on Signals, Systems &amp; Computers, 2003 , volume 2, pages 1398-1402. Ieee, 2003.
- [34] J. Xiao, A. Owens, and A. Torralba. Sun3d: A database of big spaces reconstructed using sfm and ob-

ject labels. In Proceedings of the International Conference on Computer Vision (ICCV) , 2013.

- [35] B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva. Learning deep features for scene recognition using places database. In Advances in Neural Information Processing Systems (NeurIPS) , 2014.

## Appendix: Significant changes between versions of the paper

- v1: Initial version, coinciding with initial public release of DIODE (RGB and depth data)
- v2: Added link to the dataset website, improved depth visualization scheme, added baseline experiments.
Igor Vasiljevic Nick Kolkin Shanyi Zhang Ruotian Luo Haochen Wang Falcon Z. Dai Andrea F. Daniele Mohammadreza Mostajabi Steven Basart Matthew R. Walter Gregory Shakhnarovich
DIODE: A Dense Indoor and Outdoor DEpth Dataset
We introduce DIODE, a dataset that contains thousands of diverse high
resolution color images with accurate, dense, long-range depth measurements.
DIODE (Dense Indoor/Outdoor DEpth) is the first public dataset to include RGBD
images of indoor and outdoor scenes obtained with one sensor suite. This is in
contrast to existing datasets that focus on just one domain/scene type and
employ different sensors, making generalization across domains difficult. The
dataset is available for download at http://diode-dataset.org