IB-AdCSCNet:Adaptive Convolutional Sparse Coding Network Driven by Information Bottleneck
<|reference_start|>IB-AdCSCNet:Adaptive Convolutional Sparse Coding Network Driven by Information Bottleneck: In the realm of neural network models, the perpetual challenge remains in retaining task-relevant information while effectively discarding redundant data during propagation. In this paper, we introduce IB-AdCSCNet, a deep learning model grounded in information bottleneck theory. IB-AdCSCNet seamlessly integrates the information bottleneck trade-off strategy into deep networks by dynamically adjusting the trade-off hyperparameter $\lambda$ through gradient descent, updating it within the FISTA(Fast Iterative Shrinkage-Thresholding Algorithm ) framework. By optimizing the compressive excitation loss function induced by the information bottleneck principle, IB-AdCSCNet achieves an optimal balance between compression and fitting at a global level, approximating the globally optimal representation feature. This information bottleneck trade-off strategy driven by downstream tasks not only helps to learn effective features of the data, but also improves the generalization of the model. This study's contribution lies in presenting a model with consistent performance and offering a fresh perspective on merging deep learning with sparse representation theory, grounded in the information bottleneck concept. Experimental results on CIFAR-10 and CIFAR-100 datasets demonstrate that IB-AdCSCNet not only matches the performance of deep residual convolutional networks but also outperforms them when handling corrupted data. Through the inference of the IB trade-off, the model's robustness is notably enhanced.<|reference_end|>
@article{zou2024ib-adcscnet:adaptive,
    title={IB-AdCSCNet:Adaptive Convolutional Sparse Coding Network Driven by
  Information Bottleneck},
    author={He Zou, Meng'en Qin, Yu Song, Xiaohui Yang},
    journal={arXiv preprint arXiv:2405.14192},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.14192},
    primaryClass={cs.CV}
}