An Error Detection and Correction Framework for Connectomics
We define and study error detection and correction tasks that are useful for 3D reconstruction of neurons from electron microscopic imagery, and for image segmentation more generally. Both tasks take as input the raw image and a binary mask representing a candidate object. For the error detection task, the desired output is a map of split and merge errors in the object. For the error correction task, the desired output is the true object. We call this object mask pruning, because the candidate object mask is assumed to be a superset of the true object. We train multiscale 3D convolutional networks to perform both tasks. We find that the error-detecting net can achieve high accuracy. The accuracy of the error-correcting net is enhanced if its input object mask is "advice" (union of erroneous objects) from the error-detecting net.
The paper tackles the important, but relatively unexplored, area of error detection and targeted error correction in connectomics. There is a clear need for this type of methods in the field, primarily in order to optimize the allocation of human proofreading time. The authors make the observation that to a human segmentation errors are much easier to detect than to correct, and propose to automate this process.
Specifically, they propose to build an error detector module in the form a multi-scale 3d CNN, taking as input a binary object mask and predicting whether it is equal to an object in the ground truth segmentation. At inference time, the network is applied on overlapping windows distributed over a grid to identify and localize errors in the segmentation.
The paper also proposes an error correction module -- a 3d CNN reconstructing the object containing the central pixel, similarly to flood-filling networks (FFNs), which the authors cite as related work. Instead of predicting a binary mask like in FFNs, The authors propose to predict a k-dimensional vector for each point of the output, so voxels of the same object have a similar vector, and different objects have not. This vector field is then transformed into a binary mask with an exponential transform. The stated goal of this approach is to soften the dependency on the precise location of the central object. The authors should consider including some additional information about why they chose this particular form of the transform to generate the binary mask, and whether other forms were considered; what value of k was used in the experiments, as well as any experimental data showing that this approach indeed improves the results compared to a direct binary encoding of the output.
The paper is missing information on which losses were used to train the network, which seems particularly important for the error correction module where the k-dimensional vectors are arbitrary and presumably cannot be fully specified based on the training data.
In section 5, the confidence threshold should be explained in more detail -- how was this confidence computed, and what threshold was used? The text also states that the termination condition was an error-free state as predicted by the error detector, or two corrections per iteration. Why was this particular condition used? Would applying the network more than two times result in a better segmentation? Would such a process converge to a stationary state of the segmentation?
It should also be explained explicitly, whether the error corrector deals with both split and merge errors, or only split errors. If merge errors are handled, it should be mentioned what happens in case they are at the supervoxel graph level, and within the supervoxels itself.
In section 6.3, the parameters of the Adam optimizer should be explicitly mentioned.
The paper presents a detailed analysis of performance of the presented systems. The authors should be commended for computing per-object VI scores to make the metrics easier to understand.
Technical comments about the text:
- Fig. 1 is hard to understand, and it is unclear what the nodes represent. The caption mentions horizontal layers, but it does not look like the data flows through the network vertically.
- Line 48 states that the "overhead of agglomeration prohibits the use of deep learning". This does not seem to be the case, as deep nets have been used for agglomeration, see e.g. https://papers.nips.cc/paper/6595-combinatorial-energy-learning-for-image-segmentation.pdf for a recent example.
- Appendix A, Table 4: Why are layer IDs not consecutive, and if this is intentional, what do the IDs mean?