Transformer and Hybrid Deep Learning Based Models for Machine-Generated Text Detection
<|reference_start|>Transformer and Hybrid Deep Learning Based Models for Machine-Generated Text Detection: This paper describes the approach of the UniBuc - NLP team in tackling the SemEval 2024 Task 8: Multigenerator, Multidomain, and Multilingual Black-Box Machine-Generated Text Detection. We explored transformer-based and hybrid deep learning architectures. For subtask B, our transformer-based model achieved a strong \textbf{second-place} out of $77$ teams with an accuracy of \textbf{86.95\%}, demonstrating the architecture's suitability for this task. However, our models showed overfitting in subtask A which could potentially be fixed with less fine-tunning and increasing maximum sequence length. For subtask C (token-level classification), our hybrid model overfit during training, hindering its ability to detect transitions between human and machine-generated text.<|reference_end|>
@article{marchitan2024transformer,
    title={Transformer and Hybrid Deep Learning Based Models for Machine-Generated
  Text Detection},
    author={Teodor-George Marchitan, Claudiu Creanga, Liviu P. Dinu},
    journal={arXiv preprint arXiv:2405.17964},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.17964},
    primaryClass={cs.CL}
}