Igea: a Decoder-Only Language Model for Biomedical Text Generation in Italian
<|reference_start|>Igea: a Decoder-Only Language Model for Biomedical Text Generation in Italian: The development of domain-specific language models has significantly advanced natural language processing applications in various specialized fields, particularly in biomedicine. However, the focus has largely been on English-language models, leaving a gap for less-resourced languages such as Italian. This paper introduces Igea, the first decoder-only language model designed explicitly for biomedical text generation in Italian. Built on the Minerva model and continually pretrained on a diverse corpus of Italian medical texts, Igea is available in three model sizes: 350 million, 1 billion, and 3 billion parameters. The models aim to balance computational efficiency and performance, addressing the challenges of managing the peculiarities of medical terminology in Italian. We evaluate Igea using a mix of in-domain biomedical corpora and general-purpose benchmarks, highlighting its efficacy and retention of general knowledge even after the domain-specific training. This paper discusses the model's development and evaluation, providing a foundation for future advancements in Italian biomedical NLP.<|reference_end|>
@article{buonocore2024igea:,
    title={Igea: a Decoder-Only Language Model for Biomedical Text Generation in
  Italian},
    author={Tommaso Mario Buonocore, Simone Rancati and Enea Parimbelli},
    journal={arXiv preprint arXiv:2407.06011},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.06011},
    primaryClass={cs.CL cs.AI}
}