KEML: A Knowledge-Enriched Meta-Learning Framework for Lexical Relation Classification

Lexical relations describe how concepts are semantically related, in the form of relation triples. The accurate prediction of lexical relations between concepts is challenging, due to the sparsity of patterns indicating the existence of such relations. We propose the Knowledge-Enriched Meta-Learning (KEML) framework to address the task of lexical relation classification. In KEML, the LKB-BERT (Lexical Knowledge Base-BERT) model is presented to learn concept representations from massive text corpora, with rich lexical knowledge injected by distant supervision. A probabilistic distribution of auxiliary tasks is defined to increase the model's ability to recognize different types of lexical relations. We further combine a meta-learning process over the auxiliary task distribution and supervised learning to train the neural lexical relation classifier. Experiments over multiple datasets show that KEML outperforms state-of-the-art methods.

Introduction
As an important type of linguistic resources, lexical relations describe semantic associations between concepts. Such resources are organized as backbones in lexicons (Miller, 1995), semantic networks (Speer et al., 2017), etc. The explicit usage of such resources has benefited a variety of NLP tasks, including relation extraction (Shen et al., 2018), question answering (Yang et al., 2017) and machine translation (Thompson et al., 2019).
To accumulate such knowledge, Lexical Relation Classification (LRC) is a basic NLP task to classify concepts into a finite set of lexical relations. In the literature, pattern-based and distributional methods are two major types of LRC models Wang et al., 2017a). However, compared to the classification of factual relations for knowledge graph population (Liu et al., 2017;Wu and He, 2019), the accurate classification of lexical relations has more challenges. i) Most lexical relations represent the common sense of human knowledge, not frequently expressed in texts explicitly 1 . Apart from Hearst patterns (Hearst, 1992) for hypernymy ("is-a") extraction, textual patterns that indicate the existence of other types of lexical relations remain few, leading to the "pattern sparsity" problem Washio and Kato, 2018a). ii) Distributional models assume concepts with similar contexts have similar embeddings (Mikolov et al., 2013;Bojanowski et al., 2017). Representations of a concept pair learned by traditional word embedding models are not sufficient to distinguish different types of lexical relations (Glavas and Vulic, 2018;Ponti et al., 2019). iii) Many LRC datasets are highly imbalanced w.r.t. training instances of different lexical relations, and may contain randomly paired concepts. It is difficult for models to distinguish whether a concept pair has a particular type of lexical relation, or has very weak or no semantic relatedness.
In this work, the Knowledge-Enriched Meta-Learning (KEML) framework is presented to address these challenges for LRC, consisting of three modules: Knowledge Encoder, Auxiliary Task Generator and Relation Leaner. In Knowledge Encoder, we propose the LKB-BERT (Lexical Knowledge Base-BERT) model to learn relation-sensitive concept representations. LKB-BERT is built upon BERT (Devlin et al., 2019) and trained via new distant supervised learning tasks over lexical knowledge bases, which encodes both language patterns and relational lexical knowledge into the model. In Auxiliary Task Generator, we treat recognizing single type of lexical relations as auxiliary tasks.
Based on meta-learning (Finn et al., 2017), a probabilistic task distribution is properly defined for the model to optimize, which addresses the imbalanced property and the existence of random relations in LRC datasets. In Relation Leaner, we combine a gradient-based meta-learning process over the auxiliary task distribution and supervised learning to train the final neural relation classifier. Especially, a relation recognition cell is designed and integrated into the neural network for the purpose.
This paper makes the following contributions: • We propose LKB-BERT to learn concept representations for LRC, considering unstructured texts and relational lexical knowledge. • A meta-learning process with auxiliary tasks for single relation recognition is proposed to improve the performance of LRC.

Related Work
In this section, we overview related work on LRC, pre-trained language models and meta-learning. As summarized in , Lexical Relation Classification (LRC) models are categorized into two major types: pattern-based and distributional. Pattern-based approaches extract patterns w.r.t. a concept pair from texts as features to predict its lexical relation. For hypernymy relations, Hearst patterns (Hearst, 1992) are most influential, often used for the construction of largescale taxonomies (Wu et al., 2012). To learn patterns representations,  exploit LSTM-based RNNs to encode dependency paths of patterns. Roller et al. (2018);  calculate Hearst pattern-based statistics from texts and design hypernymy measures to predict the degrees of hypernymy between concepts. For other types of relations, LexNET  extends the network architecture  for multi-way classification of lexical relations. Nguyen et al. (2016Nguyen et al. ( , 2017 design path-based neural networks to distinguish antonymy and synonymy. However, these methods may suffer from the lack of patterns and the occurrence of concept pairs in texts (Washio and Kato, 2018a).
With the rapid development of deep neural language models, distributional models attract more interest. While traditional methods directly leverage the two concepts' embeddings as features for classifier training (Weeds et al., 2014;Vylomova et al., 2016;Roller and Erk, 2016), they may suffer from the "lexical memorization" problem (Levy et al., 2015). Recently, more complicated neural networks are proposed to encode the semantics of lexical relations. Attia et al. (2016) formulate LRC as a multi-task learning task and propose a convolutional neural network for LRC. Mrksic et al. (2017) propose the Attract-Repe model to learn the semantic specialization of word embeddings. Glavas and Vulic (2018) introduce the Specialization Tensor Model, which learns multiple relationsensitive specializations of concept embeddings. SphereRE (Wang et al., 2019) encodes concept pairs in the hyperspherical embedding space and achieves state-of-the-art results. There exist some models to learn word-pair representations for other NLP tasks (Washio and Kato, 2018b;Joshi et al., 2019;Camacho-Collados et al., 2019). KEML is also distributional, further improving LRC by training meta-learners over neural language models.
Pre-trained language models have gained attention from the NLP community. ELMo (Peters et al., 2018) learns context-sensitive embeddings for each token form both left-to-right and rightto-left. BERT (Devlin et al., 2019) is a notable work, employing layers of transformer encoders to learn language representations. Follow-up works include Transformer-XL , XL-Net , ALBERT (Lan et al., 2019) and many more Yet another direction is to fuse additional knowledge sources into BERT-like models. ERNIE  incorporates the rich semantics of entities in the model. KG-BERT (Yao et al., 2019) and K-BERT (Liu et al., 2019) employ relation prediction objectives in knowledge graphs as additional learning tasks. In our work, we leverage the conceptual facts in lexical knowledge bases to improve the representation learning for LRC.
Meta-learning is a learning paradigm to train models that can adapt to a variety of different tasks with little training data (Vanschoren, 2018), mostly applied to few-shot learning. In NLP, metalearning algorithms have not been extensively employed, mostly due to the large numbers of training examples required to train the model for different NLP tasks. Existing models mostly focus on training meta-learners for single applications, such as link prediction , dialog systems (Madotto et al., 2019) and semantic parsing (Guo et al., 2019). Dou et al. (2019) leverage meta-learning for various low-resource natural language understanding tasks. KEML is one of the early attempts to improve LRC via gradient-based meta-learning (Finn et al., 2017). Since the mechanism of meta-learning is not our major research focus, we do not further elaborate.

The KEML Framework
In this section, we formally describe the KEML framework for LRC. A brief technical flow is presented, followed by its algorithmic details.

A Brief Overview of KEML
We first overview the LRC task briefly. Denote (x i , y i ) as an arbitrary concept pair. The goal of LRC is to learn a classifier f to predict the lexical relation r i ∈ R between x i and y i , based on a labeled, training set D = {(x i , y i , r i )}. Here, R is the collection of all pre-defined lexical relation types (e.g., hypernymy, synonymy), (possibly) including a special relation type RAN ("random"), depending on different task settings. It means that x i and y i are randomly paired, without clear association with any lexical relations.
The framework of KEML is illustrated in Figure 1, with three modules introduced below: Knowledge Encoder. Representation learning for LRC is significantly different from learning traditional word embeddings. This is because some lexical concepts are naturally Multiword Expressions (e.g., card game, orange juice), in which the entire sequences of tokens should be encoded in the embedding space (Cordeiro et al., 2016). Additionally, these models are insufficient to capture the lexical relations between concepts, due to the pattern sparsity issue (Washio and Kato, 2018a). Hence, the semantics of concepts should be encoded from a larger corpus and rich language resources. Inspired by BERT (Devlin et al., 2019), and its extensions, we propose the LKB-BERT (Lexical Knowledge Base-BERT) model to encode the semantics of concepts from massive text corpora and lexical knowledge bases. LKB-BERT employs the neural architecture and pre-trained parameters of BERT (Devlin et al., 2019) for token encoding, and imposes two new distant supervised learning objectives over lexical knowledge bases (such as WordNet (Miller, 1995)) as fine-tuning tasks. After model training, each concept x i receives the em-beddings of the last transformer encoding layer of LKB-BERT as the its representation. Denote the embeddings of x i as x i , with the dimension as d.
Auxiliary Task Generator. As discussed, training concept embedding based classifiers directly may produce sub-optimal results due to the highly imbalanced nature of LRC training sets and the existence of RAN relations. Inspired by the design philosophy of meta-learning (Finn et al., 2017(Finn et al., , 2018 and its NLP applications Madotto et al., 2019), we regard the relation classifier f as the meta-learner, and design a series of auxiliary tasks to update model parameters. Each task aims at distinguishing between concept pairs that have a particular relation r ∈ R and randomly paired concepts. The training sets for auxiliary tasks are sampled from subsets of D. Denote the collection of all auxiliary tasks as T . The metalearner f is optimized over a probabilistic distributions of tasks p(T ). By designing p(T ) properly, the relation classifier f is capable of alleviating the imbalanced classification and the RAN relation problems of LRC at the same time.
Relation Leaner. Finally, we design a two-stage algorithm to train the neural relation classifier f : i) meta-learning and ii) supervised learning. In the meta-learning stage, the adapted model parameters of neural networks are iteratively learned over the distribution p(T ). Therefore, the neural network learns how to recognize specific lexical relations, with the guidance of the underlying lexical knowledge base. Here, a special cell, i.e., Single Relation Recognition Cell is designed and integrated into the neural network. In the supervised learning stage, we fine-tune meta-learned parameters to obtain the multi-way classification model for LRC over D.

Knowledge Encoder
We consider the training of LKB-BERT as a variant of the fine-tune process of BERT (Devlin et al., 2019). In original BERT, the inputs are arbitrary spans of token sequences. To encode the semantics of concept pairs, we combine a concept pair (x i , y i ) to form a sequence of tokens, separated by a special token "[SEP]" as the input for LKB-BERT (see Figure 1). We first initialize all the model parameters of transformer encoders to be the same as BERT's pre-training results. Different from any standard fine-tuning tasks in BERT (Devlin et al., 2019) or KG-BERT (Yao et al., 2019), to address the RAN problem, LKB-BERT learns to classify a concept LKB-BERT Lexical KB [CLS] Tok 1  Figure 1: The high-level framework of KEML (best viewed in color).
pair (x i , y i ) into the lexical relation collection R (including the special relation type RAN).
Let KB be the collection of labeled concept pairs in lexical knowledge bases. For each concept pair with its label (x i , y i , r i ) ∈ KB, we compute τ r (x i , y i ) as the predicted score w.r.t. the lexical relation r by LKB-BERT's transformer encoders (we have ∀r ∈ R, τ r (x i , y i ) ∈ [0, 1] and r∈R τ r (x i , y i ) = 1). The first loss, i.e., the multiway relation classification loss L (1) KB is defined as: 3 where 1(·) is the indicator function that returns 1 if the input expression is true; and 0 otherwise.
To improve LKB-BERT's ability to recognize concept pairs without any lexical relations, we add a binary cross-entropy loss for LKB-BERT to optimize. We only need LKB-BERT to learn whether a concept pair (x i , y i ) are randomly paired. Let ¬ RAN be any non-random lexical relation types in R. The complete objective of LKB-BERT is: In KEML, we regard lexical relations sampled from WordNet (Miller, 1995) and in training sets as sources of KB. As for the neural network structure, LKB-BERT has two sets of classification outputs.
Refer to the C 1 and C 2 units of LKB-BERT.

Auxiliary Task Generator
Although LKB-BERT is capable of learning deep concept representations, using such features for classifier training is insufficient. The reasons are twofold. i) Direct classification can suffer from "lexical memorization" (Levy et al., 2015), meaning that the relation classifier f only learns the individual characteristics of two concepts alone. ii) The LRC datasets are highly imbalanced. For example, in the widely used dataset EVALution (Santus et al., 2015), the numbers of training instances w.r.t. several lexical relation types are very few. Hence, the learning bias of the classifier trained by naive approaches is almost unavoidable. Finn et al. (2017) observe that meta-learning achieves better parameter initialization for few-shot learning, compared to multi-task learning across all the tasks. In KEML, we propose a series of auxiliary tasks T , where each task (named Single Relation Recognition) T r ∈ T corresponds to a specific type of lexical relation r ∈ R (excluding RAN). The goal is to distinguish concept pairs with the lexical relation type r and randomly paired concepts. Let S r and S RAN be the collection of concept pairs with lexical relations as r and RAN, respectively, randomly sampled from the training set D. The goal of learning auxiliary task T r is to minimize the following loss function L(T r ): where q r (x i , y i ) is the predicted probability of the concept pair (x i , y i ) having the lexical relation r.
A remaining problem is the design of the probabilistic distribution of auxiliary tasks p(T ). We need to consider two issues. i) The semantics of all types of lexical relations should be fully learned. ii) Assume the batch sizes for all tasks are the same, i.e., ∀r p , r q ∈ R \ {RAN}, |S rp | = |S rq |. Tasks related to lexical relations with more training instances should be learned more frequently by the meta-learner. Let D r be the subset of the training set D with the lexical relation as r. ∀r p , r q ∈ R \ {RAN}, if |D rp | > |D rq |, we have the sampling probability p(T rp ) > p(T rq ). Hence, we define p(T rp ) empirically as follows: where γ > 0 is the smoothing factor. The expectation of all the losses of auxiliary tasks (represented as L(T )) is: E(L(T )) = r * ∈R\{RAN} p(T r ) · L(T r ), which is the real learning objective that these auxiliary tasks aim to optimize.

Relation Learner
In this part, we introduce the meta-learning algorithm for LRC. Assume the relation classifier f is parameterized by θ, with learning and metalearning rates as α and . Relation Learner has two stages: i) meta-learning and ii) supervised learning. For each iteration in meta-learning, we sample N auxiliary tasks from p(T ). For each auxiliary task T r , we learn adapted parameters based on two sampled subsets: S r and S RAN , to make the model to recognize one specific type of lexical relations. After that, the adapted parameters on each task T r are averaged and updated to θ. We simplify the meta-update step by only taking firstorder derivatives (Nichol et al., 2018) to avoid the time-consuming second-order derivative computation. For supervised learning, we fine-tune the parameters θ of the classifier f to obtain the multiway LRC model over the entire training set D. The algorithmic description is shown in Algorithm 1.
Algorithm 1 Meta-Learning Algorithm for LRC 1: Initialize model parameters θ; 2: while not converge do 3: Sample N auxiliary tasks Tr 1 , Tr 2 , · · · , Tr N from the task distribution p(T ); 4: for each auxiliary task Tr do 5: Sample a batch (positive samples Sr and negative samples S RAN ) from the training set D; 6: Update adapted parameters: θr ← θ − α∇L(Tr) based on Sr and S RAN ; 7: end for 8: Update meta-parameters: θ ← θ − ∇ Tr L(Tr); 9: end while 10: Fine-tune θ over D by standard supervised learning LRC; Concept 1 (x i ) Concept 2 (y i )  Finally, we describe the neural network structure for LRC. In this network, the Single Relation Recognition Cell (SRR Cell) is deigned for learning auxiliary tasks and enabling knowledge injection, with the structure illustrated in Figure 2. For each lexical relation r ∈ R \ {RAN}, we extract the relation prototype r proto from the lexical knowledge base KB by averaging all the embedding offsets of concept pairs (x i , y i ) with relation r:

Dense Layers
We use x i − y i as features because the Diff model is effective for representing semantic relations (Fu et al., 2014;Vylomova et al., 2016;Wang et al., 2019). Consider the SRR Cell structure in Figure 2. Given the inputs x i , y i and r proto , we compute the d-dimensional hidden states U 1 and U 2 by: where W 1 ∈ R 2d×d , W 2 ∈ R 2d×d , b 1 ∈ R 1×d and b 2 ∈ R 1×d are the weights and biases of these states. This can be interpreted as inferring the embeddings of relation objects or subjects, given the relation prototype and subjects/objects as inputs, similar to knowledge graph completion (Wang et al., 2017b). Next, we compute the offsets U 1 − y i and U 2 − x i and two new d-dimensional hidden states U 3 and U 4 , with W 3 ∈ R d×d , W 4 ∈ R d×d , b 3 ∈ R 1×d and b 4 ∈ R 1×d as learnable parameters: We can see that if x i and y i actually have the lexical relation r, U 3 and U 4 should be good indicators of the existence of such relations. For example, one way to interpret U 1 and U 3 is that U 1 tries to infer the relation object given x i and r proto as inputs, and U 3 makes the judgment by comparing U 1 and the true relation object embedding y i . Hence, the network learns whether r proto is a good fit for the concept pair (x i , y i ). The functionalities of U 2 and U 4 are similar, only with directions reversed. After that, we concatenate U 3 and U 4 as part of the inputs for the next layer.
Re-consider the entire network structure in Figure 1. For each concept pair (x i , y i ), we compare x i and y i with all relation prototypes (treated as constants in the network). The results are represented by 2(|R|−1) vectors of hidden states. After that, a dense layer and multiple output layers are connected. During the meta-learning stage, we train |R| − 1 binary meta-classifiers to minimize L(T r ) (T r ∈ T ), with meta-parameters updated. In the supervised learning stage, we discard all the output layers of meta-classifiers, and train the final LRC model. This is because the numbers of output units of meta-classifiers and the final classifier are different. The parameters of the last layer can not be re-used. We also need to note that additional skip connections between x i , y i and the dense layer are added, in order to improve the effect of back propagation during training (He et al., 2016).
Discussion. KEML employs a "divide-andconquer" strategy for LRC. In mate-learning, each SRR Cell learns the semantics of single lexical relation type, and also handles the RAN problem. Hence, the supervised learning process of the relation classifier can be improved with better parameter initializations. Unlike traditional metalearning (Finn et al., 2017(Finn et al., , 2018, KEML does not contain meta-testing steps (since LRC is not a fewshot learning problem), but takes advantages of both meta-learning and supervised learning.

Experiments
We conduct extensive experiments to evaluate KEML over multiple benchmark datasets, and compare it with state-of-the-art methods.

Datasets and Experimental Settings
We employ Google's pre-trained BERT model 4 to initialize the parameters of LKB-BERT. The lexical knowledge base KB contains 16.7K relation triples 5 . Following Wang et al. (2019) (which produced state-of-the-art results for LRC previously), we use the five public benchmark datasets for multiway classification of lexical relations to evaluate KEML, namely, K&H+N (Necsulescu et al., 2015), BLESS Lenci, 2011), ROOT09 (Santus et al., 2016b), EVALution (Santus et al., 2015) and CogALex-V Subtask 2 (Santus et al., 2016a). Due to space limitation, we do not introduce all the datasets here. Refer to Wang et al. (2019) for the statistical summarization of all the datasets. K&H+N, BLESS, ROOT09 and EVALution are partitioned into training, validation and testing sets, following the extract same settings as in . The CogALex-V dataset has training and testing sets only, with no validation sets provided (Santus et al., 2016a). Hence, we randomly sample 80% of the training set to learn the parameters, and use the rest for parameter tuning.
The default hyper-parameter settings of KEML are as follows: N = |R| − 1 6 , γ = 1 and α = = 10 −3 . We use tanh as the activation function, and Adam as the optimizer to train the neural network. All the model parameters are l 2 -regularized, with the hyper-parameter λ = 10 −3 . The batch size is set as 256. The dimension of hidden layers is set as the same of d (768 for the base BERT model). The number of parameters of the final neural classifier is around 7M to 24M, depending on the number of classes. The algorithms are implemented with TensorFlow and trained with NVIDIA Tesla P100 GPU. For evaluation, we use Precision, Recall and F1 as metrics, reported as the average of all the classes, weighted by the support.

General Experimental Results
We follow the experimental steps (Wang et al., 2019) to evaluate KEML over K&H+N, BLESS, ROOT09 and EVALution. Since EVALution does not contain RAN relations, during the meta-learning process of auxiliary task T r , we randomly sample relation triples from D that do not have the relation r, and take them as S RAN . We manually tune the regularization hyperparameter λ from 10 −2 to 10 −4 using the validation set (based on F1) and report the performance over the testing set. As for baselines, we consider traditional distributional models Concat (Baroni et al., 2012) and Diff (Weeds et al., 2014), pattern-based models NPB , LexNET , NPB+Aug and LexNET+Aug (Washio and Kato, 2018a), and the state-of-the-art model SphereRE (Wang et al., 2019). We refer readers to the following papers Washio and Kato, 2018a;Wang et al., 2019) for the detailed descriptions of these baselines. Additionally, we implement two variants of our approach: i) LKB-BERT (using trained concept representations to predict lexical relations) and ii) KEML-S (KEML without the meta-learning stage). The results of KEML and all the baselines are summarized in Table 1. As shown, KEML outperforms all baselines, especially over BLESS, ROOT09 and EVALution. As for K&H+N, KEML produces a slightly better F1 score (0.3%) than the strongest baseline SphereRE (Wang et al., 2019). A probable cause is that K&H+N is an "easy" dataset (99% F1 by SphereRE), leaving little room for improvement. Comparing KEML against LKB-BERT and KEML-S, we can conclude that, the knowledge enrichment technique for concept representation learning and the meta-learning algorithm are highly beneficial for accurate prediction of lexical relations.

Results of CogALex-V Shared Task
We evaluate KEML over the CogALex-V Shared Task (Subtask 2) (Santus et al., 2016a). This dataset is most challenging as it contains a large number of random word pairs and disables "lexical memorization". The organizer requires participants to  discard the results of the random class from average, and report the F1 scores for each type of lexical relations. We consider two top systems reported in this task (i.e., GHHH (Attia et al., 2016) and LexNET ), as well as two recent models that have been evaluated over the shared task (i.e., STM (Glavas and Vulic, 2018) and SphereRE (Wang et al., 2019)) as strong competitors. Because the training set contains an overwhelming number of random word pairs, during the training process of KEML-S and KEML, we randomly discard 70% (manually tuned) of the random pairs in each epoch 7 . Results are reported in Table 2, showing that KEML achieves the highest F1 score of 50.0%. It also has highest scores on three types of lexical relations: synonymy (SYN), hypernymy (HYP) and meronymy (MER).

Detailed Analysis of KEML
To facilitate deeper understanding, we conduct additional experiments to analyze KEML's components. We first study how knowledge-enriched concept representation learning benefits LRC. We implement three models:    described previously. We take the two concepts' representations as features ( x i and y i ) to train relation classifiers for LRC and report the results over the validation sets. For fair comparison, we use neural networks with one hidden layer (with the dimension d, and the activation function tanh) as classifiers in all the experiments and present the results in Table 4. We can see that the improvement of using L KB = L (2) KB as the objective function is consistent across all the datasets (which ranges from 0.8% to 1.9% in terms of F1). Particularly, LKB-BERT outperforms the strongest baseline SphereRE in a few cases (for example, the dataset ROOT09, as shown in Table 1 and Table 4). Hence, LKB-BERT is capable of encoding knowledge in patterns and lexical knowledge bases to represent the semantics of lexical relations.
Next, we look into the meta-learning process in KEML. We test whether SRR Cells can distinguish a specific type of lexical relations from random concept pairs. In each iteration of meta-learning, we sample another batch of positive and negative samples from D and compute the accuracy of single relation recognition. Figure 3 illustrates how accuracies changes through time in K&H+N, BLESS and ROOT09. Within 100 iterations, our models can achieve desirable performance efficiently, achieving good parameter initializations for LRC. This experiment also explains why KEML produces better results than KEML-S.

Error Analysis
We analyze prediction errors produced by KEML. Because the inputs of our task are very simple and the interpretation of deep neural language models is still challenging, the error analysis process is rather difficult. Here, we analyze the causes of errors from a linguistic point of view, with some cases presented in Table 3. As seen, some types of lexical relations are very similar in semantics. For instance, concept pairs with the synonymy relation and the co-hyponymy relation are usually mapped similar positions in the embedding space. Hence, it is difficult for models to distinguish the differences between the two relations without rich contextual information available. Another problem is that some of the relations are "blurry" in semantics, making KEML hard to discriminate between these relations and random relations.

Conclusion and Future Work
In this paper, we present the Knowledge-Enriched Meta-Learning (KEML) framework to distinguish different lexical relations. Experimental results confirm that KEML outperforms state-of-the-art approaches. Future work includes: i) improving concept representation learning with deep neural language models; ii) integrating richer linguistic and commonsense knowledge into KEML; and iii) extending KEML to other similar semantics-intensive NLP tasks, such as natural language inference.


extfieldsofstudy:
Physics
Computer Science

text:
Smart Home with Biometric System Recognition

In the last decade, smart homes have become applicable and widely developed where all the electronic devices and household appliances are controlled from a centralized control unit. However, the rise of misuses such as the security aspect and interoperability mechanism is proving them to be less reliable. Moreover, the traditional home automation system is complex, expensive and they are inconvenient for the aging population or disable persons. Besides, most smart home technologies forcing users to use their hands in order to control electronic devices. Therefore, the smart home with biometric system recognition is proposed to increase security, efficiency and decrease manual labor effort. This project used biometric of the human voice as an input command. The scope of this project is only focused on fan and lamp as an output of home appliances. The proposed system is divided into two main parts which are speech recognition (software development) and electrical home appliance control system (hardware development). In the speech recognition, the process contains two main modules which are feature extraction and pattern matching. Meanwhile, in hardware development, the microcontroller board is used to process the home appliance system such that the system can be operated automatically. It is expected that this project allows the user to conveniently control the home appliances by using voice. The experiment shows that the highest performance of the system achieves an accuracy of 83.25% with the number of k, training, and testing is 1, 20, and 40 respectively.

Introduction
A smart home is a residence that equipped with technologies including internal network, intelligent control, and home automation. It is anticipated and responded to the needs of the users or occupants. In addition, it becomes applicable and widely developed where all the electronic devices and household appliances are controlled from a centralized control unit [1][2][3]. Nevertheless, the rise of misuses such as the security aspect and interoperability mechanism is proving them to be less reliable. [4][5][6].
In order to overcome the mentioned problems, the smart home with biometric system recognition has been introduced and widely employed over the years. This new system can work automatically with the aid from biometric recognition for authentication [7,8]. It can identify or verify a user based on a specific behavioral or physiological feature a user possessed. It is considered the best alternative to the traditional smart home system as it increases security, efficiency, and decrease manual labor efforts.
The application of biometric recognition system based on the image (fingerprint, finger vein and face) and signal (voice and speech) have been widely used in the smart home system. Both of the systems are well-performed in term of accuracy and time processing. However, it is found that the biometric system based on an image is more expensive compares to the signal. This is due to the cost of equipment and complexity development of the system [7,8]. In contrast, the biometric system based on signal processing is more convenient, less complex and low cost. Moreover, the voice is easy to be produced whether the user is a healthy or sore throat. Motivated with the effectiveness of this system, this study proposed the smart home system that used biometric of the human voice as an input command to control the appliances.
The system is divided into two main parts which are speech recognition (software development) and electrical home appliance control system (hardware development). For speech recognition, the data is based on the online and offline system. The speech data is firstly acquired from subjects of male and female between the ages of 22 and 24 years old. The speech data is recorded in a clean environment with a different type of voice condition such as normal, fever and sore throat. The collected data is then stored in a database before the identification system is implemented to evaluate the speech signal of the online user. The Matlab program is applied to develop the biometric system for smart home appliances.
Meanwhile, for hardware development, the Arduino UNO is used as a microcontroller board to process the home appliance system. The development of a smart home is a focus for home appliances such as fan and lamp located in the living room, kitchen and bedroom spaces. The overall architecture of this project is illustrated in Figure 1. The system is divided into three phases which are the input voice produced by the user, speech recognition process by Matlab programming, and hardware interface.

Methodology
In the smart home with the biometric system, the process is divided into two main parts which are software and hardware development process. The software development involves data acquisition, audio signal processing, and Graphic User Interface (GUI) development. Meanwhile, hardware development involves prototype design and circuit construction.

Data Acquisition
The human voices are collected from 20 subjects which are ten males and ten females between the ages of 22 and 24 years old. The total of 1200 speech data is recorded, with 60 speech data collected from each subject. The speech data is acquired in different types of voice condition such as normal, sore throat, and fever. For each type of voice condition, 10 speech data of 'on' words and 10 speech data of 'off' words are recorded. This means each subject has a total of 20 speech data for each condition. The recordings are carried out by using Sony Stereo IC Recorder ICD-AX412F supported by a Sony electric condenser microphone 32-bit, and 32kHz sampling frequency with the mp3 format. The length for each recorded speech data is approximated 2s. Fig. 2 shows an example of recorded data.

Audio Signal Processing
In audio signal processing, each of the collected speech data undergoes a series of signal processing which includes pre-processing, signal segmentation, feature extraction and classification. The preprocessing involves the process of digitization, filtering, windowing and framing [9]. Subsequently, the signal segmentation is employed to separate the undesired and desired signals. Here, time domain techniques which are energy (E) and zero crossing rate (ZCR) are used [10,11]. In E technique, it assumes that speech is louder than the background noise. Hence, the energy is high-energy frames to speech and low-energy frames to noise. Meanwhile, the ZCR is the rate where the signal changes from the positive to negative values. Therefore, when the voice is detected, the value of ZCR is relatively low as the higher one is detected as an unvoiced. Fig. 3 shows an example of E and ZCR technique. The red line is indicated the starting point and ending point of E and ZCR. It can be seen, the combination of these techniques able to remove the undesired signal such that the sequencing process is only focused on the main part of the signal. After the speech signal is segmented, the data is extracted to obtain the important data and this process is called feature extraction. In this process, the well-known technique name mel-frequency cepstral coefficients (MFCC) is applied MFCC has selected due to the features are robust to noise which is suitable to be implemented in an outdoor environment that contains interference of background noises [12]. Moreover, calculation of MFCC is based on the human auditory system aiming for artificial implementation of the ear physiology assuming that the human ear can be good speaker recognizer too. The operation of MFCC is shown in Fig. 4. There are 12 mel cepstrum coefficients, one log energy coefficient and three delta coefficients per frame have been set in the experiments [14].

Figure 4. Typical MFCC process
In order to determine the performance of results, k-nearest neighbor (kNN) is used in the pattern machine process [13]. In this process, the query point is assigned to the class by calculating the distances of the training samples from the query point. the distance between the query point and the training samples is written as in (1).
where d(y, x j ) is the Euclidean distance, x j is the training sample and y is the query point. The distances are consequently arranged in ascending order and the kNN is selected based on the training sample that has the smallest distance to the query point.

Graphic User Interface Development
The Graphic User Interface (GUI) is applied to display the online speech recognition systems that identify/verify the real-time user and platform to control the home appliances. It acts as a control or monitoring system that displays the user speech signal, result of identification/verification, and status of homes appliance (on/off) as shown in Fig. 5.
There are two main push buttons namely record and identify the button. Each of this button has its own functions and executes specific processes when activated by the user. When the record push button is pressed, the microphone recorded the user's speech for 5 seconds and then it stops. After the speech is recorded, the signal is extracted and displayed in the GUI box. Then, the identify push button is pressed to display the result of the user's identification or the verification as well as the status of home appliances (on/off).

Hardware Development
The smart home hardware system will be working automatically and will be responding to the change of the input condition. In order to develop the system, there are few electrical components have been used i.e. Arduino UNO as microcontroller, infrared (IR) sensor, direct current (DC) motor, light emitting diode (LED), and Xbee in order to develop the system. Fig. 6 shows the full circuit of hardware components and Fig. 7 shows the workflow of the circuit operation.

Experimental Results
The experiments are implemented using Matlab R2016(b) and have been testing in Intel Core i5, 2.1GHz CPU, 2G RAM and Window 8 operating system. The feature dimension size was fixed at 4096 (64×64). For the classification, the number of k is set to 5.

Performance Evaluation for Offline System
In order to evaluate the performance based on signal segmentation, the analysis test of the signal without segmentation and signal with segmentation is carried out. The number of training and testing is fixed to 20 and 40 respectively. The performance is evaluated based on classification accuracy (CA). The evaluation measures are based on the CA such that: where N c is the correct identified number of samples and N A is the total number of test samples. The results in Table 1 show the combination of E and ZCR outperforms than manual segmentation with total CA of 87.17% and 85.83% for ON and OFF words. The results also indicate the CA is outperformed during normal condition. This is because, during the segmentation process for both sore throat and fever, the E and ZCR techniques are mistakenly segmented the noise part and the background noise is slightly higher than the desired speech.
Consequently, the performance of the speech recognition system based on a different number of testing was investigated. In this experiment, the number of testing were set with 20, 30, and 40. The number of nearest neighbors, k and training are fixed to 1 and 20 respectively. The performances accuracy of a different number of testing is shown in Table 2. From the result, it shows that 40 number of testing outperforms with CA 85.63%, better than others.

Experimental Results for Online
In the online system, 20 users have been tested where the speech from four users have been processed earlier and saved as the training data. Meanwhile, the other user is considered as untrained user where there was no record of this user earlier. To start the identification process, the user was asked to record the voice instruction (input) by pressing the record button. The recorded signal voice is displayed in the GUI as in Fig. 8. The recorded signal voice is considered as testing data and will be segmented and extracted before the identification process is employed. If the user is successfully identified, the identity of the user will be displayed as shown in Fig. 9. The green colour of the output fan and lamp in GUI indicates that the smart house is activated. However, if the system does not recognize the voice signal of the user, it will prompt an error window appears as shown in Fig. 10. The error dialog indicated the intruder is detected. In this case, the smart house will be deactivated. To reduce energy consumption, the system will be OFF if the IR sensor does not detect the presence of the user. Then, the message box will be displayed to notify that no user is detected as shown in Fig. 11. Here, the red colour of output fan and lamp in GUI indicate that it is in turn OFF state.

Conclusion
In this study, the smart home with the biometric system has successfully developed a smart home with biometric system recognition. The system is divided into software and hardware development. In software development, the speech data undergoes a series of signal processing which includes preprocessing, signal segmentation, feature extraction, and classification. By acquiring speech data in different conditions such as normal, sore throat and fever, the identification process has been obtained with the accuracy is more than 85% for the offline system. on the other hands, the GUI is developed and integrated with hardware prototype. The results show that the online system is able to identify the registered user or intruder. The hardware is successfully operated based on the result of the online system.


extfieldsofstudy:
Political Science
Medicine

text:
How to build an effective research network: lessons from two decades of the GARNet plant science community

Abstract Successful collaborative research is dependent on excellent ideas and innovative experimental approaches, as well as the provision of appropriate support networks. Collaboration requires venues, infrastructures, training facilities, and, perhaps most importantly, a sustained commitment to work together as a community. These activities do not occur without significant effort, yet can be facilitated and overseen by the leadership of a research network that has a clearly defined role to help build resources for their community. Over the past 20 years, this is a role that the UKRI-BBSRC-funded GARNet network has played in the support of the UK curiosity-driven, discovery-led plant science research community. This article reviews the lessons learnt by GARNet in the hope that they can inform the practical implementation of current and future research networks.

The origin and development of the Genomic Arabidopsis Resource Network (GARNet) The Arabidopsis thaliana genome was published in December 2000, bringing plant science research into the genomics era and strengthening the foothold of this unassuming weed as the primary plant model organism (Arabidopsis Genome Initiative, 2000). To take advantage of the rapidly increasing set of molecular tools that were emerging for use with Arabidopsis, Ottoline Leyser, then based at the University of York, led a successful bid to the BBSRC Investigating Gene Function (IGF) initiative to establish the Genomic Arabidopsis Resource Network (GARNet). GARNet was to provide reliable, userdriven, and publicly available functional genomic resources for Arabidopsis researchers. The 4 year project was led from York, coordinated by Ottoline and Karin Van de Sande, whilst the activities were overseen by an advisory group of academics.
During this first phase of funding, GARNet facilitated the set-up of transcriptomic, bioinformatic (both at Nottingham), proteomic (Cambridge), and metabolomic (Rothamsted) facilities as well as provision for the generation of mutant populations and clones at the John Innes Centre. In addition, GARNet hosted a successful annual meeting, the first of which was attended by 250 delegates in 2000. Since its inception, GARNet has had close links with the Nottingham Arabidopsis Stock Centre (NASC). This facility was established in 1990 and helped place the UK as a global leader within the Arabidopsis research community. GARNet has played an advisory role in the ongoing evolution of NASC activities, and NASC Director Sean May has contributed to the GARNet advisory committee since its inception.
After four productive years of facilitating the adoption of functional genomics, the role of GARNet was revised to support the emergence of systems biology as a tool for network analysis and gene discovery. From 2004 to 2009, the Arabidopsis and wider plant community obtained over £31 million investment in plant systems biology throughout the UK. During this period, GARNet activities were led from the University of Edinburgh by Andrew Millar as principal investigator (PI) and Ruth Bastow as the full-time coordinator.
The foresight to be involved with emerging technologies such as systems biology was again demonstrated as GARNet was funded by BBSRC for another 5 years (2010)(2011)(2012)(2013)(2014), with Jim Beynon at the University of Warwick as PI and Ruth Bastow, Irene Lavagi, Charis Cook, and Lisa Martin as the coordination team. On this occasion, the proposal explored the use of synthetic biology in plant science, helped expand the uptake of systems biology approaches, promoted translational research, and supported the international community through administration of the Multinational Arabidopsis Steering Committee (MASC). The success of this funding also allowed GARNet to play a major role in shaping the wider UK plant science community as they led in the formation of the UK Plant Science Federation.
The linkages between GARNet and the UK plant science community were expanded during a fourth round of BBSRC funding between 2015 and 2020. In this period, GARNet promoted the use of new technologies, many developed and validated in Arabidopsis that were relevant for all plant scientists to facilitate the translation of ideas from models to crops. These activities had particular focuses on the emerging field of gene editing and on establishing the software and hardware infrastructures needed to deal with big data. This final UKRI-BBSRC-funded iteration of GARNet activities was led by Jim Murray at Cardiff University with Geraint Parry as the full-time coordinator, with support from Ruth Bastow and Lisa Martin.
From July 2020, the GARNet leadership team was unable to obtain further UKRI funding to continue activities. However, the GARNet advisory committee maintains that UK plant science requires a community-facing network that can integrate researchers through broad knowledge exchange, highlighting of and training provision in new technologies that reach between experimental systems. In general, the GARNet advisory committee proposes that a key role for a future research network is to encourage the interaction between researchers and extant research infrastructures, such as multiomic or microscopy facilities, and ensure that previous capital investments are maximized and supported by world class expertise.
One challenge in supporting research networks is to define the mechanism through which funding is provided for their activities. Since 2004, GARNet was funded through responsive mode funding calls, in which it directly competed with conventional research proposals. Hence, although a successful network will add value to a wide set of researchers, the network in essence drew funding away from the very researchers it aims to support. Hopefully the successes of the GARNet network will be a motivation for funders to develop schemes through which successful community-enhancing networks can obtain longer term support.

The changing landscape of UK plant science
The past two decades have brought a radical transformation in the ways in which plant science research is conducted, including the increasingly pivotal role of data science skills, the integration of molecular analysis with developmental, evolutionary, and environmental insights, and the translational advances in applied crop science. Over the same period, many, but not all, exclusively botanical-or plant science-focused degree programmes have disappeared from UK higher education (Drea, 2011). Nevertheless, it appears that the UK plant science research community has at the very least remained at a similar size if not expanded. Figure 1 shows a comparison between 2004 and 2019 in the number of plant science-focused researchers at UK universities and research institutes.
Collaborative plant science projects have also benefited from the emergence of formal regional research networks. These include the N8 Agrifood network and the Great Western 4 (GW4) consortium which have been successful in bringing together researchers to pool expertise and expand access to research infrastructures. These relationships have succeeded in supporting shared infrastructures or obtaining funding for graduate training networks. These achievements are greater than might be possible by individual institutions acting alone.
GARNet has been active throughout this time, with an overarching remit to add value to the investments made in the UK plant science community, both from UKRI and from other sources. This was initially achieved by providing access to facilities for emerging omic technologies and then through promotion and training in the new technologies needed for the UK plant community to be internationally competitive. The UK remains a world leader in plant science research, as measured by both citation rate and H-index of publications (Scimagojr, 2020). Much of the research in these publications has been underpinned by access to resources initially funded by GARNet as well as through its many training activities. Over the past 5 years alone, this has included organization of 14 conferences and training events.
The management structure that links the GARNet leadership team and its advisory committee to the wider community has allowed its activities to stay ahead of the technology curve and take a leading role in the training of plant scientists in cutting-edge research techniques.
Many of the lessons learnt from 20 years of GARNet activities are discussed below (Box 1). We hope that they will provide valuable advice for those planning to establish new initiatives or expand existing networks.

Initially incentivize community participation
Network projects usually arise from the motivation of a small group of like-minded individuals. However, over the longer term, a network will only succeed if it has more extensive buy-in from the wider community. The wheels of this process can be greased by providing financial or access-driven incentives to draw people into the network. If there is research or technology development funding associated with the network then this will motivate academics and others to become involved with the process.
GARNet initially benefitted from the BBSRC IGF programme that provided direct funding to help set up omic resources that had an Arabidopsis focus. This attracted those were taken from Edition 1 of the GARNish newsletter (https://garnetcommunity.org.uk/sites/ default/files/newsltr/garnish_july04.pdf) and may exclude PIs from departments that focus on more applied research. Data for 2019 were taken from prior knowledge and online searches. At the very least, this suggests that there has not been a decline in the number of PIs whose primary research focus is in some area of plant science. Box 1. Advice for community-focused research networks 1. Initially incentivize community participation 2. Stay ahead of the technology curve 3. Build an advisory group who are invested in the network 4. Add value by securing additional funding to support community-facing activities 5. Integrate with the wider community 6. Engage the next generation 7. Employ a Project Manager academics in the position to lead these community facilities. A similar strategy is now employed in 2020 by both the PhenomUK program (PhenomUK, 2020), which is funded through the UKRI-BBSRC Technology Touching Life scheme, and multiple 'Networks in Industrial Biotechnology and Bioenergy' (NIBB) schemes (UKRI-BBSRC, 2020). These offer up to £50 000 in Proof of Concept funding for either technology development in plant phenotyping or for development of academic-industrial interactions, respectively. Despite this funding being relatively small, it is clearly welcomed as an available route to support research projects. This type of interaction will hopefully ensure that these networks can expand to become part of the wider research landscape.
In GARNet's early years, the community was also incentivized through the scheduling of an annual meeting that brought together UK-based Arabidopsis researchers and international experts. Financial support from GARNet allowed these meetings to be inexpensive and developed a sense of community, particularly for PhD students and postdocs, which encouraged delegates to participate in other GARNet activities.
Establishing an annual meeting meant that each year could focus on a novel technology or ongoing area of research in order to keep the community informed about new developments. Although this began as an Arabidopsis-focused event, it soon evolved to promote technologies that were relevant to plant scientists irrespective of their organism of research. It is clear from this example that hosting a regular gathering of network members further engages the community and helps establish its activities and shape priorities.

Stay ahead of the technology curve
The 20 year success of GARNet was built around the incentivized awareness, engagement, and understanding of technology. Any research network must be continuously forward looking, assessing new technologies and techniques as they emerge. This was facilitated by the ability of the GARNet advisory group to predict the importance of new technologies and by GARNet's capability to respond to community training needs. Specifically, the initial GARNet proposal was funded in response to the imminent release of the Arabidopsis genome sequence, whilst subsequent GARNet proposals coincided with the emergence of systems and/or synthetic biology approaches as important parts of the research portfolio. An important caveat to this advice is to not lose sight of the reason for the technology, which ultimately should be used to support scientific inquiry, as highlighted in 2004 by then GARNet chairman Ian Furner in the December 2004 edition of the GARNish newsletter; 'GARNet has been very much about technology but the real point of all the technology is to find out fun stuff about biology' (GARNish Newsletter, 2004).
The advice for future networks is to ensure that their leadership team has the foresight to plan to provide the wider community with something they might not even know they need. From a GARNet perspective, this was exemplified in 2013 by promotion of 'Opportunities in Plant Synthetic Biology' that included introducing novel genome assembly techniques and heterologous expression systems (Cook et al., 2014). It is of course not possible to predict all approaching advances in technology, especially if a grant period extends for 5 years. Therefore, where possible, it is important that the funding model has the agility to respond to emerging technologies by allocating funds to areas not directly earmarked in the successful proposal. This ability to respond to emerging challenges was highlighted during the GARNet2020 grant period. The initial proposal was submitted in early 2014, at which time the potential of CRISPR (clustered regularly interspaced palindromic repeats)-mediated gene editing had not fully emerged. However, through the allocation of funding for unspecified future events, the details of which would be decided by the GARNet advisory committee, it allowed the flexibility to organize events that focused on gene editing technologies that reach across the entire plant science community (Parry et al., 2016;Parry and Jose, 2018;Parry and Harrison, 2019). Therefore, important advice for any new network is to ensure that during the funding period there is reasonable flexibility to enable it to respond to unexpected technological developments.
Despite the clear importance to provide training in new technologies, it is critical that there is simultaneous support for the infrastructure that underpins the outputs of these technologies. Over the past 20 years, the uptake of new technologies has been characterized by an enormous increase in the generation of big data. This has driven the necessity for the integration of experimental outputs with the digital infrastructure. Therefore, any new research network should be involved with both ends of the data journey, from training in the techniques to generate the data and in the infrastructure to manage and allow reuse of the data.

Build an advisory group who are invested in the network
Academics are busy people and are pulled in many directions. However, the success of any academic-focused research network will rely upon the recruitment of members who are committed to contribute to its activities. These network advocates greatly enhance the chance of the network succeeding. Conversely, it is important that the network leadership team does not place onerous demands on an advisory committee to prevent membership becoming a chore.
From the second GARNet grant period in 2004, the advisory committee comprised at least nine members who were elected for a period of 3 years. The location of GARNet advisory committee meetings moved around the UK so as to even out the burden of travel time. An annual rotation of three members ensured that new ideas were regularly introduced to the committee. The wider UK plant science community was engaged with this process as the 'GARNet electoral register' is comprised of the entirety of UK-based plant science academics (currently >600 members).
A fresh impetus of new ideas is achieved not only by the annual rotation of committee membership but also by an excellent gender balance, variety of experience levels, and geographic distribution, which ensures that a mix of voices can input their advice. This was achieved through a two-step election process in which UK-based plant science PIs nominated and then voted for their peers who would become part of the GARNet advisory committee. As such, we hope that any unconscious biases from the advisory committee were removed by having the community make decisions on their behalf. GARNet made it clear that any eligible PI was welcome to join the committee and, over recent years, this was reflected by election of PIs whose primary research does not use Arabidopsis, thus expanding the areas of expertise of the advisory committee. Table 1 shows that 37% of the 51 academics who have sat on the GARNet advisory committee are female. Although this is not perfect, we feel that over 20 years it represents a commitment to equality. These PIs are distributed throughout the UK, increasing the likelihood that resulting GARNet activities are shared around the country. Future networks should look to replicate this wide geographic distribution to ensure they operate as a true national network.
GARNet's commitment to the future of UK plant science is confirmed by the regular involvement of early career faculty members on the advisory committee. Since 2014, eight of the 15 elected members have been within the first 8 years in an academic position (Henderson, Spoel, Gibbs, Harrison, Harper, McKim, Kaiserli, and Benitez-Alfonso). Academics who participate on the GARNet advisory committee will naturally have a community outlook and so GARNet has been able to expand its wider network through the external interactions of advisory committee members. Over recent years, these have included organizational roles with the Society of Experimental Biology (SEB), the British Society of Plant Pathology, and the UK Plant Science Federation.
Providing a clear benefit to advisory committee members for their contribution will ensure that they remain invested in promoting the network. Over the past few years the GARNet coordinator worked with committee members to organize symposia in their areas of research interest including 'Integrating Large Data into Plant Science: from Big Data to Discovery' (Leonelli et al., 2017), 'Natural Variation as a Tool for Gene Discovery and Crop Improvement' (Henderson and Salt, 2017), and 'From Proteome to Phenotype: Role of Post-translational Modifications' (Spoel, 2018).
GARNet took the additional step to ensure that it was fully connected to its community by inviting a representative from its funder (UKRI-BBSRC) to sit on the advisory committee. This is not so that the funder would influence GARNet activities but rather that they were made aware of matters arising from within the community. Over the past decade, this relationship has included extremely useful discussions regarding the funding landscape, for which the BBSRC shared non-public information that GARNet could then report back to the community. In general, this collaborative relationship between a grant recipient and its funder(s) provides a two-way dialogue that can benefit both parties. This transparency is further demonstrated by the online publication of the minutes from each advisory committee meeting. This allows the community to be fully aware of network activities and provides an open conduit between the community and network.

Add value by securing additional funding to support community-facing activities
For a network to be successful, it must both integrate into the existing community and use its resources to benefit that community. Even the most community-minded PI has limited time to be involved with external training events and conferences. Therefore, a network should look to add value for its community by gaining support from outside organizations that might have resources to fund relevant activities. This may involve applying for conference funding from learned societies, arranging travel grants for early career researchers, bringing together academics to apply for collaborative grants, or leading journal-supported community initiatives. For example, the SEB has Animal, Cell, and Plant Sections for which it organizes meeting and workshops. GARNet has a longstanding relationship with the SEB and over the past 5 years this has resulted in GARNet obtaining over £50 000 in support from the SEB for meetings and training events. This enabled GARNet to direct both SEB funding and the work of the SEB directorate toward providing events specifically for the plant science community it represents. This takes the burden away from academics and others to commit their time to being involved with these type of funding applications.
GARNet has used its community connections to be involved in a range of collaborative grants aimed at building UK research infrastructure. In this regard, the network coordinator can heavily input toward time-consuming writing of proposals. This was a successful strategy during the third grant period in which GARNet was involved with the CyVerseUK/iPlant project that aimed to federate hardware and software resources under the US CyVerse infrastructure (UK Research and Innovation, 2020). This project linked the Earlham Institute with the Universities of Nottingham, Liverpool, and Warwick. Furthermore, GARNet members contributed to the COPO (COpenPlantOmics) project that facilitated the management of complex plant science datasets and linked the Earlham Institute, EMBL-EBI, and the Universities of Oxford, Warwick, and York (COPO Project, 2020).

Integrate with the wider community
Although GARNet began with the remit to provide funding for resources that focused on Arabidopsis research, over the second half of its lifetime the activities of the leadership team, advisory committee, and coordinator were integrated with a wider group of UK and global plant scientists. This included a BBSRC-funded administrative role with the Multinational Arabidopsis Steering Committee (2020), a leadership role with the INDEPTH Cost Action (INDEPTH, 2020), and knowledge exchange relationships with other UK community research networks. This strategy demonstrates that a network should have value that extends beyond its initial objectives so as to demonstrate to funders that their investment is reaching a wider community than originally planned.
In 2010, the leadership team at GARNet identified that the UK plant science community lacked a single unified voice so, in collaboration with the leaders of Brassica, Monogram, and Solanaceae networks, came together to establish a 'Federation of UK Plant Science Communities' that was supported by a GARNet-managed website (Leonelli et al., 2012). The permanent staff supported by GARNet funding managed these resources for the benefit of the wider community. Early successes of this network included winning funding from the SEB, the Gatsby Charitable Foundation, the Biochemical Society, and the British Society of Plant Pathology to support a national 'UK PlantSci meeting' and produce a widely circulated report on 'Current Status and Future Challenges of the UK plant Science Community' (Royal Society of Biology, 2020a). Over past 5 years, the now renamed UK Plant Science Federation (UKPSF) has evolved to be a special interest group of the Royal Society of Biology and, with extensive input from the GARNet Advisory Committee, recently produced the 'Growing the Future' report (Royal Society of Biology, 2020b) on the future of UK plant science. UKPSF activities are now managed by the Royal Society of Biology and, as it does not rely on grant funding to maintain its activities, it will hopefully ensure UKPSF's longevity.
Any nascent network can benefit from widening its influence and using the strength of its funding to support other communities. This will benefit a network over the longer term via the connections it can make throughout the community.
As well as developing connections within its research area, any network will benefit by expanding to establish relationships with multidisciplinary groups. GARNet worked with CPIB at the University of Nottingham to initiate the Mathematics and Plant Science Study Groups. These events invited a group of mathematicians to tackle previously 'unmodelled' problems in plant science. These events lasted 7 years and directly resulted in (at least) two peer-reviewed publications (Antoniou-Kourounioti et al., 2012;Nelson et al., 2012;GARNet, 2020).
GARNet also interacted with the history and philosophy of science research community through the invitation to Sabina Leonelli to join the advisory committee as an ex-offico member. Leonelli investigates the policy implications and technical mechanisms that allow the effective management of big data so that they are findable and reusable. This provided GARNet with an expert perspective and clear voice Name Institution Starting year on GARNet Advisory Committee within discussions on the future of mechanisms to promote data management, sharing, and reuse (Bastow and Leonelli, 2010;Leonelli et al., 2013Leonelli et al., , 2017. The focus on data usage and availability was an important part of the successful proposal that secured the fourth round of GARNet funding in 2014. Engaging a multidisciplinary group of collaborators during advisory committee meetings and event organization demonstrated that GARNet had a willingness to embrace new ideas and interactions. Any new network would benefit from looking at the options to expand its circle of interactions. This strategy requires the leadership team and advisory committee to have the necessary vision to move network activities into unexpected areas, even if they may at times be outside the advisory committee's 'comfort zone'.

Engage the next generation
The success of any network relies on engaging not only with established PIs but also with the next generation of scientists. GARNet's knowledge exchanges activities catered to early career researchers by offering affordable opportunities to attend and participate in conferences, provide information about available job and research opportunities, interact with other organizations to facilitate funding for overseas conference travel, and, most importantly, provide training in new technologies. Each of these activities aims to build a general appreciation of research network activities and that these younger scientists will continue to see this value as their careers progress.
Community engagement now relies not only on routine E-mail correspondences or occasional newsletters but also on dedicated use of multimedia outputs and social media for the circulation of new findings, events, and opportunities. These multi-media interactions are surprisingly time-consuming so it is important that sufficient time is given in order to maximize their impact. Finally, it is important to ensure that a network chooses a social media profile that is closely connected to its activities so that it is easy to find. This will work better than the use of a more enigmatic ID such as @GARNetweets! Despite this, the @GARNetweets twitter account gains ~30 followers a week with a high engagement rate of 1.7%.

Employ a project manager
Despite the best intentions of the leaders of any network, its effective management will probably be more timeconsuming than they might imagine. Therefore, irrespective of the size of the network, it is important that a project manager is employed to direct its everyday activities. This might be a full-time or part-time role but is critical to liaise with the advisory committee, apply for external funding, organize events, deal with grant administration, and disseminate outputs. GARNet has historically employed an experienced coordinator who becomes an active member of the advisory committee, yet a new network might choose a less integrated manager. Overall, a project manager adds value both by overseeing the network and also by allowing the academic leadership to focus their own research programmes, which after all has brought them to the position to take the lead in a community research network.

Conclusion: GARNet perspectives
The landscape of UK plant science has changed significantly over the past two decades. The revolution in plant genomics was led by Arabidopsis researchers but now has been followed by the establishment of equivalent tools that will allow discovery-led research in crop plants with more complex genomes (Adamski et al., 2020). Nevertheless, models such as Arabidopsis remain the key test beds for developing new techniques and tools, and translating emerging technologies from outside the plant field that can subsequently be utilized by the wider plant science community.
UKRI-BBSRC remains the primary individual grant funder of world-leading UK plant science research. Each year since 2014, UKRI-BBSRC has funded an average of 38 responsive mode grants that have some focus on plant science; for an approximate value of £19 million. This compares with an approximate value of €7 million per year awarded through European Research Council funding for individual plant science projects. In addition, UKRI provides institutional support for plant science research at the John Innes Centre, IBERS, The Earlham Institute, the Quadram Institute, and Rothamsted Research. Over this time period, the Gatsby Foundation has provided significant support for The Sainsbury laboratories in Norwich and Cambridge as well as for numerous PhD studentships.
It is impossible to fully assess the change in the number of UK-based PIs over the last 15 years who conduct plant science research, but Fig. 1 suggests that at the very least this number has remained stable. However, it is likely that there has been a shift from discovery-led research to more applied research. Indeed, over the past 5 years, the number of grant proposals that plan to use Arabidopsis to answer fundamental questions in plant science has declined and often been replaced by proposals across research areas that will use a variety of other plant species, including crops, trees, and newer model organisms such as Nicotiana benthamiana. An additional and unfortunate consequence of broadening the funded portfolio of UK plant science is that the overall network remains fragmented between researchers involved in 'fundamental' or 'applied' research. This increases the need for flexible research networks that can bring together researchers who work in different areas.
Through its two decades, GARNet has demonstrated that adaptability is necessary for longevity. It has adapted its activities to move from supporting the use of Arabidopsis-focused research infrastructures, through promotion of new research areas such as systems and synthetic biology, through to the more recent focus on providing training opportunities in new technologies. With the support of the GARNet advisory committee, the GARNet PIs have gained BBSRC-responsive mode funding in preference to more conventional research proposals. Hopefully, the lessons learnt from 20 years of community involvement can make GARNet an exemplar for future research networks.