Extraction Propagation
<|reference_start|>Extraction Propagation: We consider the problem of learning to map large instances, such as sequences and images, to outputs. Since training one large neural network end to end with backpropagation is plagued by vanishing gradients and degradation, we develop a novel neural network architecture called Extraction propagation, which works by training, in parallel, many small neural networks which interact with one another. We note that the performance of Extraction propagation is only conjectured as we have yet to implement it. We do, however, back the algorithm with some theory. A previous version of this paper was entitled "Fusion encoder networks" and detailed a slightly different architecture.<|reference_end|>
@article{pasteris2024extraction,
    title={Extraction Propagation},
    author={Stephen Pasteris, Chris Hicks, Vasilios Mavroudis},
    journal={arXiv preprint arXiv:2402.15883},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.15883},
    primaryClass={cs.LG}
}