Self-Contrastive Forward-Forward Algorithm
<|reference_start|>Self-Contrastive Forward-Forward Algorithm: The Forward-Forward (FF) algorithm is a recent, purely forward-mode learning method, that updates weights locally and layer-wise and supports supervised as well as unsupervised learning. These features make it ideal for applications such as brain-inspired learning, low-power hardware neural networks, and distributed learning in large models. However, while FF has shown promise on written digit recognition tasks, its performance on natural images and time-series remains a challenge. A key limitation is the need to generate high-quality negative examples for contrastive learning, especially in unsupervised tasks, where versatile solutions are currently lacking. To address this, we introduce the Self-Contrastive Forward-Forward (SCFF) method, inspired by self-supervised contrastive learning. SCFF generates positive and negative examples applicable across different datasets, surpassing existing local forward algorithms for unsupervised classification accuracy on MNIST (MLP: 98.7%), CIFAR-10 (CNN: 80.75%), and STL-10 (CNN: 77.3%). Additionally, SCFF is the first to enable FF training of recurrent neural networks, opening the door to more complex tasks and continuous-time video and text processing.<|reference_end|>
@article{chen2024self-contrastive,
    title={Self-Contrastive Forward-Forward Algorithm},
    author={Xing Chen, Dongshu Liu, Jeremie Laydevant, Julie Grollier},
    journal={arXiv preprint arXiv:2409.11593},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.11593},
    primaryClass={cs.LG cs.AI cs.CV cs.ET cs.NE}
}