Language-oriented Semantic Communication for Image Transmission with Fine-Tuned Diffusion Model
<|reference_start|>Language-oriented Semantic Communication for Image Transmission with Fine-Tuned Diffusion Model: Ubiquitous image transmission in emerging applications brings huge overheads to limited wireless resources. Since that text has the characteristic of conveying a large amount of information with very little data, the transmission of the descriptive text of an image can reduce the amount of transmitted data. In this context, this paper develops a novel semantic communication framework based on a text-2-image generative model (Gen-SC). In particular, a transmitter converts the input image to textual modality data. Then the text is transmitted through a noisy channel to the receiver. The receiver then uses the received text to generate images. Additionally, to improve the robustness of text transmission over noisy channels, we designed a transformer-based text transmission codec model. Moreover, we obtained a personalized knowledge base by fine-tuning the diffusion model to meet the requirements of task-oriented transmission scenarios. Simulation results show that the proposed framework can achieve high perceptual quality with reducing the transmitted data volume by up to 99% and is robust to wireless channel noise in terms of portrait image transmission.<|reference_end|>
@article{wei2024language-oriented,
    title={Language-oriented Semantic Communication for Image Transmission with
  Fine-Tuned Diffusion Model},
    author={Xinfeng Wei, Haonan Tong, Nuocheng Yang, and Changchuan Yin},
    journal={arXiv preprint arXiv:2409.17104},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.17104},
    primaryClass={cs.MM}
}