  Deep learning methods are developing rapidly in coded aperture snapshot
spectral imaging (CASSI). The number of parameters and FLOPs of existing
state-of-the-art methods (SOTA) continues to increase, but the reconstruction
accuracy improves slowly. Current methods still face two problems: 1) The
performance of the spatial light modulator (SLM) is not fully developed due to
the limitation of fixed Mask coding. 2) The single input limits the network
performance. In this paper we present a dynamic-mask-based dual camera system,
which consists of an RGB camera and a CASSI system running in parallel. First,
the system learns the spatial feature distribution of the scene based on the
RGB images, then instructs the SLM to encode each scene, and finally sends both
RGB and CASSI images to the network for reconstruction. We further designed the
DMDC-net, which consists of two separate networks, a small-scale CNN-based
dynamic mask network for dynamic adjustment of the mask and a multimodal
reconstruction network for reconstruction using RGB and CASSI measurements.
Extensive experiments on multiple datasets show that our method achieves more
than 9 dB improvement in PSNR over the SOTA.
(https://github.com/caizeyu1992/DMDC)

Image and Video Processing;Computer Vision and Pattern Recognition