As a broad and major category in machine reading comprehension (MRC), the generalized goal of discriminative MRC is answer prediction from the given materials. However, the focuses of various discriminative MRC tasks may be diverse enough: multi-choice MRC requires model to highlight and integrate all potential critical evidence globally; while extractive MRC focuses on higher local boundary preciseness for answer extraction. Among previous works, there lacks a unified design with pertinence for the overall discriminative MRC tasks. To fill in above gap, we propose a lightweight PO S-Enhanced I terative Co-Attention Net work ( POI-Net ) as the first attempt of unified modeling with pertinence, to handle diverse discriminative MRC tasks synchronously. Nearly without introducing more parameters, our lite unified design brings model significant improvement with both encoder and decoder components. The evaluation results on four discriminative MRC benchmarks consistently indicate the general effectiveness and applicability of our model, and the code is available at https://github. com/Yilin1111/poi-net . Machine reading comprehension (MRC) as a challenging branch in NLU, has two major categories: generative MRC which emphasizes on answer generation (Kocisk et al., 2018), and discriminative MRC which focuses on answer prediction from given contexts (Baradaran et al., 2020). Among them, discriminative MRC is in great attention of researchers due to its plentiful application scenarios, such as extractive and multi-choice MRC two major subcategories. Given a question with corresponding passage, extractive MRC asks for precise answer span extraction in passage (Joshi et al.,  Corresponding author. Multi-choice MRC Example ... In addition, Lynn's pioneering efforts also provide public educational forums via Green Scenes  a series of three hour events , each focusing on specific topics teaching Hoosiers how to lead greener lifestyles . ... Q: What can we learn about Green Scenes ? A. It is a scene set in a three-hour film . B. It is a series of events focusing on green life . ( Golden ) C. It is a film set in Central Indiana. D. It is a forum focusing on green lifestyle . Extractive MRC Example ... Early versions were in use by 1851, but the most successful indicator was developed for the high speed engine inventor and manufacturer Charles Porter by Charles Richard and exhibited at London Exhibition in 1862 . ... Q: Where was the Charles Porter steam engine indicator shown? Golden Answer: London Exhibition Imprecise Answer 1: London Exhibition in 1862 Imprecise Answer 2: exhibited at London Exhibition Table 1: Different focuses of multi-choice MRC task (RACE) and extractive MRC task (SQuAD 2.0). 2017; Trischler et al., 2017; Yang et al., 2018), while multi-choice MRC requires suitable answer selection among given candidates (Huang et al., 2019; Khashabi et al., 2018). Except for the only common goal shared by different discriminative MRCs, the focuses of extractive and multi-choice MRC are different to a large extent due to the diversity in the styles of predicted answers: multi-choice MRC usually requires to highlight and integrate all potential critical information among the whole passage; while extractive MRC pays more attention to precise span boundary extraction at local level, since the rough scope of answer span can be located relatively easily, shown in Table 1. In MRC field, several previous works perform general-purpose language modeling with considerable computing cost at encoding aspect (Devlin et al., 2019; Clark et al., 2020; Zhang et al., 2020c), or splice texts among diverse MRC tasks simply to expand training dataset (Khashabi et al., 2020), without delicate and specialized design for sub-8682 E P0 POS-Enhanced Embedding Layer P0 P1 PN  Q0 Q1 QN  E P1 EPN  E Q0  E Q1 EQN Preliminary Interaction Layer E P0 E P0 1 E P1 E P1 1 EPNEPN 1 E Q0 E Q0 1 E Q1 E Q1 1 EQNEQN 1   T x Iterative Interaction Layer E P0 E P0 TE P1 E P1 TEPNEPNTE Q0 E Q0 TE Q1 E Q1 TEQNEQNT   Attention Integration Layer Output Layer E P0  E P1 EPN  MPC C CS S S x E P0 E P1 EPN x x  E P0 E P0 1 E P1 E P1 1 EPNEPN 1 E Q1  E Q0 EQN  x x x  E Q1 E Q1 1 E Q0 E Q0 1 EQNEQN 1 MPE Q1 E Q0 EQNC C CS S S   MPC C CS S S x E P0 E P1 EPN x x  E P0 E P0 E P1 E P1 EPNEPN  x x x  E Q1 E Q1 E Q0 E Q0 EQNEQNMPC C CS S S    t t t E P0 E P1 EPN t-1 t-1 t-1 EQNE Q1 E Q0 t-1 t-1 t-1 t t t E P0  E P1 EPNE Q0  E Q1 EQN   Encoding Interaction Integration Output Figure 1: Overview of POI-Net . categories in discriminative MRC. Others utilize excessively detailed design for one special MRC subcategory at decoding aspect (Sun et al., 2019b; Zhang et al., 2020a), lacking the universality for overall discriminative MRC. To fill in above gap in unified modeling for different discriminative MRCs, based on core focuses of extractive and multi-choice MRC, we design two complementary reading strategies at both encoding and decoding aspects. The encoding design enhances token linguistic representation at local level, which is especially effective for extractive MRC. The explicit possession of word part-of-speech (POS) attribute of human leads to precise answer extraction. In the extractive sample from Table 1, human extracts golden answer span precisely because  London Exhibition  is a proper noun (NNP) corresponding to interrogative qualifier (WDT)  Where  in the question, while imprecise words like  1862  (cardinal number, CD) and  exhibited  (past tense verb, VBD) predicted by machines will not be retained. Thus, we inject word POS attribute explicitly in embedding form. The decoding design simulates human reconsideration and integration abilities at global level, with especial effect for multi-choice MRC. To handle compound questions with limited attention, human will highlight critical information in turns, and update recognition and attention distribution iteratively. Inspired by above reconsideration strategy, we design Iterative Co-Attention Mechanism with no additional parameter, which iteratively executes the interaction between passage and question-option ( Q  O ) pair globally in turns. In the multi-choice example from Table 1, during the first interaction, model may only focus on texts related to rough impression of Q  O pair such as  Green Scenes , ignoring plentiful but scattered critical information. But with sufficient iterative interaction, model can ultimately collect all detailed evidence (bold in Table 1). Furthermore, we explore a series of attention integration strategies for captured evidence among interaction turns. We combine two above methods and propose a novel model called POI-Net ( PO S-Enhanced I terative Co-Attention Net work), to alleviate the gap between machines and humans on discriminative MRC. We evaluate our model on two multi-choice MRC benchmarks, RACE (Lai et al., 2017) and DREAM (Sun et al., 2019a); and two extractive MRC benchmarks, SQuAD 1.1 (Rajpurkar et al., 2016) and SQuAD 2.0 (Rajpurkar et al., 2018), obtaining consistent and significant improvements, with nearly zero additional parameters. We aim to design a lightweight, universal and effective model architecture for various subcategories of discriminative MRC, and the overview of our model is shown in Figure 1, which consists of four main processes: Encoding (2.1), Interaction (2.2), Integration (2.3) and Output (2.4). Based on pre-trained contextualized encoder ALBERT (Lan et al., 2020), we encode input tokens with an additional POS embedding layer, as Figure 2 shows. Since the input sequence will be tokenized into subwords in the contextualized encoder, we tokenize sequences in word-level with nltk tokenizer (Bird et al., 2009) additionally and implement POS-Enhanced Encoder , where each subword in a complete word will share the same POS tag. In detail, input sequences are fed into nltk POS tagger to obtain the POS tag of each word such as JJ. Subject to Penn Treebank style, our adopted POS tagger has 36 POS tag types. Considering on the specific scenarios in discriminative MRC, we add additional SP E tag for special tokens (i.e., [ CLS ] , [ SEP ] ), P AD tag for padding tokens and ERR tag for potential unrecognized tokens. Appendix A shows detailed description of POS tags. The input embedding in our model is the normalized sum of Subword Embedding and POS Embedding . Following the basic design in embedding layers of BERT-style models, we retain Token Embedding E t , Segmentation Embedding E s and Position Embedding E p in subword-level, constituting Subword Embedding . For POS Embedding EPOS , we implement another embedding layer with the same embedding size to Subword Embedding , guaranteeing all above indicator embeddings are in the same vector space. Formulaically, the input embedding E can be represented as: E = Norm ( E t + E s + E p + EPOS ) , where Norm () is a layer normalization function (Ba et al., 2016). POI-Net employs a lightweight Iterative CoAttention module to simulate human inner reconsidering process, with no additional parameter. POI-Net splits all N input token embeddings into passage domain ( P ) and question (or Q  O pair) domain ( Q ) to start P  Q interactive process. To generate the overall impression of the given passage or question like humans, POI-Net concentrates all embeddings in corresponding domain into one Concentrated Embedding by max pooling: CE 1 P = MaxP ooling ( EP 0 , ..., EPN )  RH , CE 1 Q = MaxP ooling ( EQ 0 , ..., EQN )  RH , where H is the hidden size, P N/QN is the token amount of P/Q domain. Then POI-Net calculates the similarity between each token in EP /E Q and CE 1 Q /CE 1 P , to generate attention score s for each token contributing to the P  Q pair. In detail, we use cosine similarly for calculation: s 1 P 0 , ..., s 1 PN = Cosine ([ EP 0 , ..., EPN ] , CE 1 Q ) , s 1 Q 0 , ..., s 1 QN = Cosine ([ EQ 0 , ..., EQN ] , CE 1 P ) . We normalize these scores to [0 , 1] by min-max scaling, then execute dot product with corresponding input embeddings: E 1 Pi =  s 1 Pi  E Pi , E 1 Qi =  s 1 Qi  E Qi , where  s Pi is the normalized attention score of i th passage token embedding, E 1 Pi is the attention-enhanced embedding of i -th passage token after preliminary interaction (the 1 -st turn interaction). To model human reconsideration ability between passage and question in turns, we add iterable modules with co-attention mechanism, as the Iterative Interaction Layer in Figure 1. Detailed processes in the t -th turn interaction are similar to preliminary interaction: CE tQ = MaxP ooling ( E t  1 Q 0 , ..., E t  1 QN )  RH , CE tP = MaxP ooling ( E t  1 P 0 , ..., E t  1 PN )  RH , 8684 s tP 0 , ..., s tPN = Cosine ([ EP 0 , ..., EPN ] , CE tQ ) , s tQ 0 , ..., s tQN = Cosine ([ EQ 0 , ..., EQN ] , CE tP ) , E tPi =  s tPi  E Pi , E tQi =  s tQi  E Qi . Note that, during all iteration turns, we calculate attention scores with the original input embedding E instead of attention-enhanced embedding E t  1 from the ( t -1)-th turn, due to: on base size model), comparing to adopted method; 2) With the same embedding E , attention integration in 2.3 can be optimized into attention score integration, which is computationally efficient with no additional embedding storage 1 . Human recommends to integrate all critical information from multiple turns for a comprehensive conclusion, instead of discarding all findings from previous consideration. In line with above thought, POI-Net returns attention-enhanced embedding E t =  s t  E of each turn (we only store  s t in an optimized method), and integrates them with specific strategies. We design four integration strategies according to the contribution proportion of each turn and adopt Forgetting Strategy ultimately. Average Strategy : The attention network treats normalized attention score  s t of each turn equally, and produces the ultimate representation vector R with average value of  s t : R = 1 TT (cid:88) t =1  s t  E  RN  H , where T is the total amount of iteration turns. Weighted Strategy : The attention network treats  s t with two normalized weighted coefficients  tP ,  tQ , which measure the contribution of the t -th turn calculation: R = (cid:80) Tt =1  tP  s tP (cid:80) Tt =1  tP  EP + (cid:80) Tt =1  tQ  s tQ (cid:80) Tt =1  tQ  EQ ,   t P = Max ( s t  1 Q 0 , ..., s t  1 QN ) ,   tQ = Max ( s t  1 P 0 , ..., s t  1 PN ) , 1 Approximate 15.3% training time is saved on average for each iteration turn. where s 0 Pi = s 0 Qi = 1 . 0 . The design motivation for  tP ,  tQ is intuitive: when Concentrated Embedding CE tQ /CE tP (calculating attention score at the t -th turn) has higher confidence (behaving as higher maximum value in s t  1 Q /s t  1 P due to max pooling calculation), system should pay more attention to input embedding E tP /E tQ at the t -th turn 2 . Forgetting Strategy : Since human will partly forget knowledge from previous consideration and focus on findings at current turn, we execute normalization operation of attention scores from two most previous turns iteratively: During the iterative normalization, the ultimate proportion of attention scores from previous turns will be diluted gradually, which simulates the effect of forgetting strategy 3 . Intuition Strategy : In some cases, human can solve simple questions in intuition without excessive consideration, thus we introduce two attenuation coefficients  tP ,  tQ for attention scores from the t -th turn, which decrease gradually as the turn of iteration increases: Setting  tP / tQ as learnable parameters cannot bring further improvement since the contribution proportion of each turn varies with the specific circumstance of input samples. 3 Method of activation functions in LSTM (Hochreiter and Schmidhuber, 1997) may filter out information completely in one single-turn calculation, which cannot bring consistent improvement in our experiments. The input sequence for multi-choice MRC is [ CLS ] P [ SEP ] Q + O i [ SEP ] , where + denotes concatenation, O i denotes the i -th answer options. In Output Layer , the representation vector R  RN  H is fed into a max pooling operation to generate general representation: R = MaxP ooling ( R )  RH . Then a linear softmax layer is employed to calculate probabilities of options, and standard Cross Entropy Loss is employed as the total loss. Option with the largest probability is determined as the predicted answer. The input sequence for extractive MRC can be represented as [ CLS ] P [ SEP ] Q [ SEP ] , and we use a linear softmax layer to calculate start and end token probabilities in Output Layer . The training object is the sum of Cross Entropy Losses for the start and end token probabilities: L = y s  log ( s ) + y e  log ( e ) , s, e = softmax ( Linear ( R ))  RN , where s/e are the start/end probabilities for all tokens and y s /y e are the start/end targets. score ij = s i + e j , 0  i  j  N, then the span with the maximum score score has is the predicted answer. The score of null answer is: score no = s 0 + e 0 , where the 0 -th token is [ CLS ] . The final score is calculated as score has  score no , and a threshold  is set to determine whether the question is answerable, which is heuristically computed in linear time. POI-Net predicts the span with the maximum score if the final score is above the threshold, and null answer otherwise. The experiments are run on 8 NVIDIA Tesla P40 GPUs and the implementation of POI-Net is based on the Pytorch implementation of ALBERT (Paszke et al., 2019). We set the maximum iteration turns in Iterative Co-Attention as 3 . Table 2 shows the hyper-parameters of POI-Net achieving reported results. As a supplement, the warmup rate is 0.1 for all tasks. We evaluate POI-Net on two multi-choice MRC benchmarks: RACE (Lai et al., 2017), DREAM (Sun et al., 2019a), and two extractive MRC benchmarks: SQuAD 1.1 (Rajpurkar et al., 2016) and SQuAD 2.0 (Rajpurkar et al., 2018). The detailed introduction is shown as following: RACE is a large-scale multi-choice MRC task collected from English examinations which contains nearly 100K questions. The passages are in the form of articles and most questions need contextual reasoning, and the domains of passages are diversified. DREAM is a dialogue-based dataset for multi-choice MRC, containing more than 10K questions. The challenge of the dataset is that more than 80% of the questions are non-extractive and require reasoning from multi-turn dialogues. SQuAD 1.1 is a widely used large-scale extractive MRC benchmark with more than 107K passage-question pairs, which are produced from Wikipedia. Models are asked to extract precise word span from the Wikipedia passage as the answer of the given passage. SQuAD 2.0 retains the questions in SQuAD 1.1 with over 53K unanswerable questions, which are similar to answerable ones. For SQuAD 2.0, models must not only answer questions when possible, but also abstain from answering when the question is unanswerable with the paragraph. We take accuracy as evaluation criteria for multi-choice benchmarks, while exact match (EM) and 4 Due to the test sets of SQuAD 1.1 and SQuAD 2.0 are not open for free evaluation with different random seeds, we report the results on development set instead. a softer metric F1 score for extractive benchmarks. The average results of three random seeds are shown in Table 3, where we only display several BERT-style models with comparable parameters. Appendix B reports the complete comparison results with other public works on each benchmark. The results show that, for multi-choice benchmarks, our model outperforms most baselines and comparison works, and passes the significance test (Zhang et al., 2021) with p  value < 0 . 01 in DREAM (2.0% average improvement) and RACE (1.7% average improvement). And for extractive benchmarks, though the performance of baseline ALBERT is strong, our model still boosts it essentially (1.3% average improvement on EM for SQuAD 1.1 and 2.3% for SQuAD 2.0). Furthermore, we report the parameter scale and train-ing/inference time costs in 4.4. In this section, we implement POI-Net on ALBERT base for further discussions, and such settings have the similar quantitative tendency to POI-Net on ALBERT xxlarge . To evaluate the contribution of each component in POI-Net , we perform ablation studies on RACE and SQuAD 1.1 development sets and report the average results of three random seeds in Table 4. The results indicate that, both POS Embedding and Iterative Co-Attention Mechanism provide considerable contributions to POI-Net , but in different roles for certain MRC subcategory. For multi-choice MRC like RACE, Iterative CoAttention Mechanism contributes much more than POS Embedding (3.86% v.s. 1.14%), since multi-choice MRC requires to highlight and integrate critical information in passages comprehensively. Therefore, potential omission of critical evidence may be fatal for answer prediction, which is guaranteed by Iterative Co-Attention Mechanism , while precise evidence span boundary and POS attributes are not as important as the former. On the contrary, simple POS Embedding even brings a little more improvement than the well-designed Iterative Co-Attention (0.99% v.s. 0.85% on EM) for extractive MRC. In these tasks, model focuses on answer span extraction with precise boundaries, and requires to discard interference words which not exactly match questions, such as redundant verbs, prepositions and infinitives ( politically and socially unstable  instead of  to be politically and socially unstable ), or partial interception of proper nouns ( Seljuk Turks  instead of  Turks ). With the POS attribute of each word, POI-Net locates the boundaries of answer spans precisely 5 . Since extractive MRC does not require comprehensive information integration like multi-5 Note that, the improvement of POI-Net on EM score is consistently higher than F1 score, as corroboration. is less significant. Besides, we also implement POI-Net on other contextualized encoders like BERT, and achieve significant improvements as Table 4 shows. The consistent and significant improvements over various baselines verify the universal effectiveness of POI-Net . To study how POS Embedding enhances token representation, we make a series of statistics on SQuAD 1.1 development set about: 1) POS type of boundary words from predicted spans, as Table 5 shows; 2) error POS classification of POI-Net and its baseline ALBERT base , as Figure 3 shows. The statistical results show, with POS Embedding , the overall distribution of the POS types of answer boundary words predicted by POI-Net is more similar to golden answer, compared with its baseline; and the amount of error POS classification cases by POI-Net also reduces significantly. And there are also two further findings: 1) The correction proportion of error POS classification (8.09%) is much higher than correction proportion of overall error predictions (1.82%) in POI-Net , which indicates the correction of POS classification benefits mostly from the perception of word POS attributes by POS Embedding , instead of the improvement on overall accuracy. 2) Though answers in SQuAD 1.1 incline to distribute in several specific POS types (NN, CD, NNS and JJ), POS Embedding prompts model to consider words in each POS type more equally than the baseline, and the predicted proportions of words in rarer POS type (IN, VBN, RB, VBG and so on) increase. Robustness is one of the important indicators to measure model performance, when there is numerous rough data or resource in applied tasks. To measure the anti-interference of POS Embedding , we randomly modify part of POS tags from nltk POS tagger to error tags, and the results on SQuAD 1.1 development set are shown in Table 6. The results indicate that, POI-Net possesses satisfactory POS Embedding robustness, and the improvement brought by POS Embedding will not suffer a lot with a slight disturbance (5%). We argue that the robustness of POI-Net may benefit from the integration with other contextualized embeddings, such as Token Embedding E t which encodes the contextual meaning of current word or subword. Though more violent interference (20%) may further hurt token representations, existing mature POS taggers achieve 97% + accuracy, which can prevent the occurrence of above situations. To explore the most suitable integration strategy and maximum iteration turn in Iterative CoAttention Mechanism , we implement our proposed strategies with different maximum iteration turns, together with a baseline replacing Iterative CoAttention mechanism by a widely used Multihead Co-Attention mechanism (Devlin et al., 2019; Zhang et al., 2020a, 2021) for comparison in Figure 4. We take RACE as the evaluated benchmark due to the significant effect of attention mechanism to multi-choice MRC. As the figure shows, forgetting strategy leads to the best performance, with slight improvement than weighted strategy. Both these two strategies are in line with the logical evidence integration in human reconsidering process, while average strategy and intuition strategy may work against common human logic. From the trends of four strategies in multiple iterations, we conclude that 2 or 3 iteration turns for Iterative Co-Attention lead to an appropriate result, due to: 1) Fewer iteration turns may lead to inadequate interaction between passage and question, and model may focus on rough cognition instead of exhaustive critical information; 2) Excessive iteration turns may lead to over-integration of information, declining the contribution by real critical evidence. Compared to the typical Multi-head CoAttention mechanism, our proposed Iterative CoAttention mechanism obtains higher performance with more iterations, indicating it has stronger iterative reconsideration ability. Besides, Iterative Co-Attention defeats Multihead Co-Attention on both parameter size and training time cost. As the parameter comparison in Table 7 shows, POI-Net basically brings no additional parameter except an linear embedding layer for POS Embedding . Multi-head Co-Attention mechanism and models based on it (like DUMA in Table 3) introduces much more parameters, with slightly Model Parameters ALBERT base (Lan et al., 2020) 12M ALBERT base (rerun) 11.14M Multi-head Co-Attention on ALBERT base 17.94M POI-Net on ALBERT base 11.15M ALBERT xxlarge (Lan et al., 2020) 235M ALBERT xxlarge (rerun) 212.29M Multi-head Co-Attention on ALBERT xxlarge 404.50M POI-Net on ALBERT xxlarge 212.30M Table 7: Training parameters in POI-Net and baselines. lower performance. We also record time costs on RACE for one training epoch on ALBERT base , Iterative Co-Attention costs 54 , 62 , 72 , 83 , 96 minutes from 0 -turn iteration to 4 -turn iterations, while Multi-head Co-Attention costs 54 , 65 , 76 , 89 , 109 minutes instead, with 8 . 3% increase on average. We perform a visualization display for discriminative MRC examples in Table 1, as Figure 5 shows. For the extractive example, benefited from POS Embedding , POI-Net predicts the precise answer span, based on the interrogative qualifier  where  and POS attributes of controversial boundary tokens exhibited, at, London, Exhibition, 1862 . And for the multi-choice example, without proposed Iterative Co-Attention Mechanism , the overall distribution of attention is more scattered. The baseline can only notice special tokens like [ CLS ] at the 0 -th turn, and even interrogative qualifier  how  due to the similar usage to  what  in the question. With the execution of Iterative Co-Attention , POI-Net pays more attention on discrete critical words like  Green Scenes  and  events  at the 1 -st turn,  series  and  focusing  at the 2 -nd turn and  greener lifestyle  at the 3 -rd turn. After the integration of all above critical evidence, POI-Net predicts the golden option ultimately. To cope with challenging MRC tasks, numerous powerful pre-trained language models (PLMs) have been proposed (Devlin et al., 2019; Lewis et al., 2020; Raffel et al., 2020). Though advanced PLMs demonstrate strong ability in contextual representation, the lack of explicit semantic and linguistic clues leads to the bottleneck of previous works. Benefited from the development of semantic role labeling (Li et al., 2018) and dependency syntactic parsing (Zhou and Zhao, 2019), some researchers focus on enhancing semantic representations. Zhang et al. (2020b) strengthen token representation by fusing semantic role labels, while Zhang et al. (2020c) and Bai et al. (2021) implement additional self attention layers to encode syntactic dependency. Furthermore, Mihaylov and Frank (2019) employ multiple discourse-aware semantic annotations for MRC on narrative texts. Instead of semantic information, we pay attention to more accessible part-of-speech (POS) information, which has been widely used into non-MRC fields, such as open domain QA (Chen et al., 2017), with much lower pre-processing calculation consumption but higher accuracy (Bohnet et al., 2018; Strubell et al., 2018; Zhou et al., 2020). However, previous application of POS attributes mostly stays in primitive and rough embedding methods (Huang et al., 2018), leading to much slighter improvement than proposed POI-Net . In discriminative MRC field, various attention mechanisms (Raffel and Ellis, 2015; Seo et al., 2017; Wang et al., 2017; Vaswani et al., 2017) play increasingly important roles. Initially, attention mechanism is mainly adopted on extractive MRC (Yu et al., 2018; Cui et al., 2021), such as multiple polishing of answer spans (Xiong et al., 2017) and multi-granularity representations generation (Zheng et al., 2020; Chen et al., 2020). Recently, researchers notice its special effect for multi-choice MRC. Zhang et al. (2020a) model domains bidirectionally with dual co-matching network, Jin et al. (2020) use multi-step attention as classifier, and Zhu et al. (2020) design multi-head co-attentions for collaborative interactions. We thus propose a universal Iterative CoAttention mechanism, which performs interaction between paired input domains iteratively, to hopefully enhance discriminative MRC. Unlike other works introducing numerous parameters by complicated attention network (Zhang et al., 2020a), our POI-Net is more effective and efficient with almost no introduction of additional parameters. In this work, we propose PO S-Enhanced I terative Co-Attention Net work ( POI-Net ), as a lightweight unified modeling for multiple subcategories of discriminative MRC. POI-Net utilizes POS Embedding to encode POS attributes for the preciseness of answer boundary, and Iterative Co-Attention Mechanism with integration strategy is employed to highlight and integrate critical information at decoding aspect, with almost no additional parameter. As the first effective and unified modeling with pertinence for different types of discriminative MRC, evaluation results on four extractive and multi-choice MRC benchmarks consistently indicate the general effectiveness and applicability of our model.
abstain abstain abstain objective result other abstain abstain abstain abstain objective objective objective abstain objective objective objective objective objective objective objective objective objective objective abstain other other abstain abstain method abstain abstain abstain method abstain abstain method abstain abstain objective objective result abstain abstain method abstain method method abstain method method method abstain abstain abstain other abstain method method method other method abstain abstain abstain method abstain abstain other abstain abstain abstain method abstain abstain abstain method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain result abstain result abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain objective abstain objective