Big Cooperative Learning
<|reference_start|>Big Cooperative Learning: Cooperation plays a pivotal role in the evolution of human intelligence; moreover, it also underlies the recent revolutionary advancement of artificial intelligence (AI) that is driven by foundation models. Specifically, we reveal that the training of foundation models can be interpreted as a form of big cooperative learning (\textit{abbr.} big learning), where massive learning individuals/tasks \emph{cooperate} to approach the unique essence of data from diverse perspectives of data prediction, leveraging a universal model. The presented big learning therefore unifies most training objectives of foundation models within a consistent framework, where their underlying assumptions are exposed simultaneously. We design tailored simulations to demonstrate the principle of big learning, based on which we provide learning-perspective justifications for the successes of foundation models, with interesting side-products. Furthermore, we reveal that big learning is a new dimension for upgrading conventional machine learning paradigms, valuable for endowing reinvigorations to associated applications; as an illustrative example, we propose the BigLearn-GAN, which is a novel adversarially-trained foundation model with versatile data sampling capabilities. Code is available at \texttt{https://github.com/YulaiCong/BigCooperativeLearning}.<|reference_end|>
@article{cong2024big,
    title={Big Cooperative Learning},
    author={Yulai Cong},
    journal={arXiv preprint arXiv:2407.21319},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.21319},
    primaryClass={cs.LG cs.AI}
}