  Relation extraction (RE) consists in identifying and structuring
automatically relations of interest from texts. Recently, BERT improved the top
performances for several NLP tasks, including RE. However, the best way to use
BERT, within a machine learning architecture, and within a transfer learning
strategy is still an open question since it is highly dependent on each
specific task and domain. Here, we explore various BERT-based architectures and
transfer learning strategies (i.e., frozen or fine-tuned) for the task of
biomedical RE on two corpora. Among tested architectures and strategies, our
*BERT-segMCNN with finetuning reaches performances higher than the
state-of-the-art on the two corpora (1.73 % and 32.77 % absolute improvement on
ChemProt and PGxCorpus corpora respectively). More generally, our experiments
illustrate the expected interest of fine-tuning with BERT, but also the
unexplored advantage of using structural information (with sentence
segmentation), in addition to the context classically leveraged by BERT.

Computation and Language;Machine Learning