Scaling Law Hypothesis for Multimodal Model
<|reference_start|>Scaling Law Hypothesis for Multimodal Model: We propose a scaling law hypothesis for multimodal models processing text, audio, images, and video within a shared token and embedding space. Our framework predicts model performance based on modality-specific compression and tokenization efficiency, extending established scaling laws from text-based decoder models to mixed-modality systems. We explore whether leveraging more training data in multiple modalities can reduce the size of the multimodal model, enabling efficient deployment on resource-constrained devices.<|reference_end|>
@article{sun2024scaling,
    title={Scaling Law Hypothesis for Multimodal Model},
    author={Qingyun Sun, Zhen Guo, PIN AI Team},
    journal={arXiv preprint arXiv:2409.06754},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.06754},
    primaryClass={cs.LG cs.AI}
}