InFiConD: Interactive No-code Fine-tuning with Concept-based Knowledge Distillation
<|reference_start|>InFiConD: Interactive No-code Fine-tuning with Concept-based Knowledge Distillation: The emergence of large-scale pre-trained models has heightened their application in various downstream tasks, yet deployment is a challenge in environments with limited computational resources. Knowledge distillation has emerged as a solution in such scenarios, whereby knowledge from large teacher models is transferred into smaller student' models, but this is a non-trivial process that traditionally requires technical expertise in AI/ML. To address these challenges, this paper presents InFiConD, a novel framework that leverages visual concepts to implement the knowledge distillation process and enable subsequent no-code fine-tuning of student models. We develop a novel knowledge distillation pipeline based on extracting text-aligned visual concepts from a concept corpus using multimodal models, and construct highly interpretable linear student models based on visual concepts that mimic a teacher model in a response-based manner. InFiConD's interface allows users to interactively fine-tune the student model by manipulating concept influences directly in the user interface. We validate InFiConD via a robust usage scenario and user study. Our findings indicate that InFiConD's human-in-the-loop and visualization-driven approach enables users to effectively create and analyze student models, understand how knowledge is transferred, and efficiently perform fine-tuning operations. We discuss how this work highlights the potential of interactive and visual methods in making knowledge distillation and subsequent no-code fine-tuning more accessible and adaptable to a wider range of users with domain-specific demands.<|reference_end|>
@article{huang2024inficond:,
    title={InFiConD: Interactive No-code Fine-tuning with Concept-based Knowledge
  Distillation},
    author={Jinbin Huang, Wenbin He, Liang Gou, Liu Ren, Chris Bryan},
    journal={arXiv preprint arXiv:2406.17838},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.17838},
    primaryClass={cs.LG cs.AI cs.HC}
}