BERTer: The Efficient One
<|reference_start|>BERTer: The Efficient One: We explore advanced fine-tuning techniques to boost BERT's performance in sentiment analysis, paraphrase detection, and semantic textual similarity. Our approach leverages SMART regularization to combat overfitting, improves hyperparameter choices, employs a cross-embedding Siamese architecture for improved sentence embeddings, and introduces innovative early exiting methods. Our fine-tuning findings currently reveal substantial improvements in model efficiency and effectiveness when combining multiple fine-tuning architectures, achieving a state-of-the-art performance score of on the test set, surpassing current benchmarks and highlighting BERT's adaptability in multifaceted linguistic tasks.<|reference_end|>
@article{saligram2024berter:,
    title={BERTer: The Efficient One},
    author={Pradyumna Saligram, Andrew Lanpouthakoun},
    journal={arXiv preprint arXiv:2407.14039},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.14039},
    primaryClass={cs.CL cs.LG}
}