Neural Architecture Search for Sentence Classification with BERT
<|reference_start|>Neural Architecture Search for Sentence Classification with BERT: Pre training of language models on large text corpora is common practice in Natural Language Processing. Following, fine tuning of these models is performed to achieve the best results on a variety of tasks. In this paper we question the common practice of only adding a single output layer as a classification head on top of the network. We perform an AutoML search to find architectures that outperform the current single layer at only a small compute cost. We validate our classification architecture on a variety of NLP benchmarks from the GLUE dataset.<|reference_end|>
@article{kenneweg2024neural,
    title={Neural Architecture Search for Sentence Classification with BERT},
    author={Philip Kenneweg, Sarah Schr"oder, Barbara Hammer},
    journal={arXiv preprint arXiv:2403.18547},
    year={2024},
    doi={10.14428/esann/2022.ES2022-45},
    archivePrefix={arXiv},
    eprint={2403.18547},
    primaryClass={cs.AI}
}