In this paper, we propose a neural model EPT-X (Expression-Pointer Transformer with Ex-planations), which utilizes natural language explanations to solve an algebraic word problem. To enhance the explainability of the encoding process of a neural model, EPT-X adopts the concepts of plausibility and faithfulness which are drawn from math word problem solving strategies by humans. A plausible explanation is one that includes contextual information for the numbers and variables that appear in a given math word problem. A faithful explanation is one that accurately represents the reasoning process behind the model's solution equation. The EPT-X model yields an average baseline performance of 69.59% on our PEN dataset and produces explanations with quality that is comparable to human output. The contribution of this work is two-fold. (1) EPT-X model: An explainable neural model that sets a baseline for algebraic word problem solving task, in terms of model's correctness, plausibility, and faithfulness. (2) New dataset: We release a novel dataset PEN (Problems with Explanations for Numbers), which expands the existing datasets by attaching explanations to each number/variable. Algebraic word problem solving is a challenging task for understanding natural language. As shown in Table 1, a model needs to interpret a word problem into a solution equation to solve the problem. Recent neural approaches have employed encoder-decoder architecture to tackle this task and achieved remarkable answer correctness (Huang et al., 2018; Chiang and Chen, 2019; Amini et al., 2019; Kim et al., 2020; Ki et al., 2020): ranging from 65% to 84% depending on datasets. So, as the model delivers a plausible answer, researchers have a firm belief that an encoder component of a neural model can comprehend the problem correctly. However, this belief has less been verified due to Q. Tom has 12 coins in quarters and nickels. Our novel model, Expression-Pointer Transformer with Explanations (EPT-X), is inspired by some pedagogical studies about human strategies on understanding an algebraic word problem (Conway and Polya, 1985; Jitendra et al., 2007; Montague, 2008; Jitendra and Star, 2012). In classrooms, teachers ask students to make an explanation or a diagram that depicts the role of each number written in the problem. Then, students use these explanations for numbers to build a correct equation. That is, understanding a problem produces explanations that satisfy the following two criteria. (1) plausibility : A plausible explanation is one that includes contextual information for the numbers and variables that appear in a given math word problem. Especially, as humans recognize each number/variable individually, the explanations should reveal what each number or variable represents in the context of the given problem. (2) faithfulness : A faithful explanation is one that accurately represents the reasoning process behind the model's solution equation (Jacovi and Goldberg, 2020). In other words, they should imply a reason behind selecting operators or operands. To reflect these two criteria in EPT-X, we adopt a two-phase architecture: (1) explaining num-bers/variables and (2) building solution equations. Though Ling et al. (2017) attempted to generate explanations, this work is different from ours in that their model focused on explaining the decoding process. So, they have less explored the above two criteria. In contrast, this paper attempts to explain how the model understands the given word problem by modifying an encoder component of a neural model. As humans successfully solve word problems by explaining their understanding, we expect our EPT-X model to achieve a good performance in terms of three criteria: correctness of equations, plausibility of explanations, and faithfulness of explanations. Through several analyses, our paper shows the following two contributions: 1. EPT-X model: We propose a baseline model that can generate explanations and solve algebraic word problems, in terms of correctness, plausibility, and faithfulness. 2. New dataset: We release a novel dataset PEN (Problems with Explanations for Numbers), which expands the existing datasets by attaching explanations to each number/variable. Correctness: Researchers have attempted to build a model that solves word problems. Early attempts used hand-crafted features collected by experts to make a model understand a word problem (Kushman et al., 2014; Roy and Roth, 2015; Koncel-Kedziorski et al., 2015; Zhou et al., 2015; Upadhyay et al., 2016; Roy and Roth, 2017). Although researchers can interpret these models using the features, extending these studies to other datasets is limited as designing features is labor-intensive. On the other hand, recent studies have employed neural models (Wang et al., 2017; Huang et al., 2018; Chiang and Chen, 2019; Amini et al., 2019; Kim et al., 2020; Ki et al., 2020) and achieved answer correctness ranging from 62% to 84%. Though their extensibility is better than handcrafted features, it becomes harder to interpret how a neural model understands a word problem. Plausibility: To make a neural model that explains its reasoning process, Ling et al. (2017) built a model that outputs both a computation process and a rationale behind the process. Though their model generated a natural language phrase that explains a computation step in advance, the model is not enough to meet the plausibility criterion because of two issues. First, it is not guaranteed whether their model explains all numbers and variables required to solve the problem. As they focused more on explaining the model's computation, their model often skips explaining its understanding of numbers and variables stated in a problem. Second, it is not confirmed whether their model generates rationale comparable to that of humans. Though they measured their quality of rationale using BLEU-4 (Papineni et al., 2002), the reported score of 27.2 is somewhat low and not compared with any human-level performance. We suspect that this low-quality explanation affected the low correctness of their model: 36.4%. Therefore, it is worthwhile to build a new model that fulfills the plausibility criterion. Faithfulness: As far as we know, studies on solving algebraic word problems have not measured the faithfulness of a generated explanation. Existing studies so far measured the quality of explanations using plausibility only. Following Jacovi and Goldberg (2020), we define faithful explanation as one that accurately represents the reasoning process behind the model's solution equation. Humans expect an explanation to be faithful. However, a model can generate an explanation that may not be related to the equations (Jacovi and Goldberg, 2020); it can generate random plausible sentences independently from the process of generating solution equations. Therefore, measuring faithfulness is meaningful in that a highly faithful explanation reflects a solution equation generation process that is expected by human problem solvers. The proposed model, Expression-Pointer Transformer with Explanations (EPT-X) 1 , is a variant of Expression-Pointer Transformer (EPT; Kim et al. 2020), which is state-of-the-art correctness model. Figure 1 depicts the two phases EPT-X model. (1) Plausibility : In phase 1, EPT-X receives a problem text as an input and generates explanations for each number/variable. The number of variables is also predicted in this phase. (2) Faithfulness : In phase 2, EPT-X receives both the original problem and the generated explanations as inputs and then builds an equation using EPT. To jointly train these two phases, we add up the loss functions for the number 1 http://github.com/snucclab/ept-x 4443 Figure 1: The two-phase pipeline of generating explanations and equations in our EPT-X model. of variables, explanations, and equations; all three use smoothed cross-entropy (Szegedy et al., 2016) with  = 0 . 01 . Phase 1 is a three-step procedure for generating explanations as shown in the top part of Figure 1. Phase 1 contains two components: text encoder and explanation decoder. Step 1-1. Compute problem text vectors The text encoder receives a natural language problem as an input and computes problem context vectors. To utilize world knowledge in the computation process, we used ELECTRA (Clark et al., 2020), a pre-trained language model. After applying the text encoder, we obtain the problem context vector w s for each token w s in the given problem. Step 1-2. Predict the number of variables Using the problem context vectors, EPT-X predicts the number of required variables N to solve the given problem. Using the first token's problem context vector w 0 , we compute the probability distribution of N as follows: P ( N ) = softmax (FF n 1 (ReLU (FF n 2 ( w 0 )))) , where FF(  ) indicates the feed-forward layer. Step 1-3. Generating plausible explanations The explanation decoder then produces explanations using problem context vectors as memories. The decoder uses a Transformer (Vaswani et al., 2017) decoder and a pointer-generator network (See et al., 2017). Before predicting the next explanation token x t +1 , the Transformer decoder computes a hidden state h t based on the problem context vectors w s and previously generated explanation tokens x 1 ,    , x t . To utilize world knowledge in generating explanations, we adopt Rothe et al. (2020) and use ELECTRA (Clark et al., 2020) as the initial weight. The pointer-generator head receives the computed h t and predicts the next token. Let p g , P v , and P c be the probability of using the generated word, the probability of generating from the vocabulary, and the probability of copying from the problem, respectively. Then, the next token x t +1 is predicted as follows: x t +1 = arg max  p g P v (  ) + (1  p g ) P c (  ) , p g =  (FF g ( w  t  h t  E( x t  1 ))) , P v (  ) = softmax (FF v ( h t )) , P c (  ) = (cid:80) w s : w s =  attn( w s , h t ) , w  t = (cid:80) w s attn( w s , h t ) , where  (  ) , E(  ) , and attn(  ) indicate the sigmoid, embedding, and single-head attention scoring function, respectively. And  indicates concatenation of vectors. Plausibility of explanation is implemented during this stage by generating an explanation for each number/variable separately. We use unique initial input values for all numbers and variables. This method has been used in other studies to bind the decoder to a specific context (Raffel et al., 2020; 4444 Keskar et al., 2019). For numbers, instead of using the initial input value  [CLS] ' of the Transformer decoder, we use the input  [CLS] explain: context [SEP] , where the context part depends on the number or variable. For the numbers, we use a window of tokens that are near the given number token. For example, if the window size is three, we use three tokens placed before and after the given token. For variables, we use the variable index because variables do not appear in the problem. So, for example, the initial input value of the n th variable becomes  [CLS] explain: variable n [SEP] . 3.2 Phase 2. Building solution equations Phase 2 is a three-step procedure for producing equations as shown in the bottom part of Figure 1. Phase 2 uses the same text encoder from Phase 1. Step 2-1. Recombine explanations Inspired by human paraphrasing strategies (Conway and Polya, 1985; Gagnon and Maccini, 2001; Montague, 2008), EPT-X paraphrases the original problem by recombining its understanding. First, the model places each explanation and the corresponding number token value into a sentence:  explanation is a number value . for numbers and What is explanation ? for variables. Then, EPT-X creates a recombined problem by concatenating these paraphrased sentences. We randomly recombined one of the reference explanations in the training process as EPT-X may not generate explanations ideally. Step 2-2. Compute recombined context vectors The text encoder once again receives both the original problem and the recombined problem as inputs and computes the recombined context vectors r i for each input token r i . We designed EPT-X to use both problems for two reasons. First, using the original problem can avoid information loss. Second, using the recombined problem can make the equation decoder exploit the information of explanations. We arrange these two problems into the text encoder as follows:  [CLS] original [SEP] recombined [SEP] . Step 2-3. Generate equations faithfully The equation decoder then produces equations using the recombined context vectors as memories. Following the EPT model (Kim et al., 2020), the decoder produces equations using expression tokens, each of which is a tuple of an operator and relevant operands. So, the equation decoder predicts the next j th expression as follows. First, the decoder receives expression tokens generated so far and converts them into embedding vectors v k ( k = 0 ,    , j  1) . Then, using these embedding vectors v k and recombined context vectors r i , the decoder builds an equation context vector q j for the next expression. Lastly, the decoder simultaneously predicts the next operator and its required operands using q j . Thus, when we translate expressions into an equation, we can compute an answer to a problem. The faithfulness of explanation is implemented during this stage by using explanations as the input data source. We change the input format of numbers and variables in EPT's equation decoder to use explanations. Originally, EPT used different types of vectors to input them: the encoder's hidden state for each known number and the decoder's hidden state for each unknown variable. However, in EPT-X, we guide the model to utilize the information from the explanation when writing an equation. As all numbers and variables appear in the recombined problem, EPT-X uses the vector r i corresponding to each number/variable. We release Problems with Explanations for Num-bers' (PEN) 2 , an algebraic word problem dataset with problem texts, equations, and explanations of numbers/variables for each problem to train and evaluate EPT-X. As existing datasets for algebraic word problems do not contain explanations, we provided explanations on the existing three benchmark datasets on solving algebraic word problems 3 : ALG514 (Kushman et al., 2014), DRAW-1K (Upadhyay and Chang, 2017), and MAWPS (Koncel-Kedziorski et al., 2016). The following sections introduce the two stages of building PEN: preparation for correcting errors and annotation for collecting explanations. We corrected the errors and organized the data in three steps. In the first step, we revised the problems' typos, grammatical errors, and logical flaws. For example, we found a problem asking 2 http://github.com/snucclab/pen 3 Though we considered using AQuA-RAT (Ling et al., 2017), which includes rationale about computation, we found that using it is intractable since we have to re-collect explanations for numbers and variables in most problems. about Senators' after telling a story about the House of Representatives.' So we replaced the out-of-context term with the other one. Second, we extracted numeric words from the modified text using WordNet (Fellbaum, 1998); Arabic numerals, fractions, ordinals, and their synonyms were extracted. Third, to normalize equations, we re-formulated them according to nine source formulas organized by Mayer (1981) and four formulas organized by Carpenter et al. (1996). Among 3,886 problems from the three datasets, we corrected 3,581 problems in the PEN dataset. We excluded 305 problems because they are (1) exact duplicates of others (303 problems) 4 or (2) not an algebra problem (2 problems) 5 . After excluding 305 problems, we further revised incorrect equations: 62 of the 3,581 problems (1.73%). When collecting natural language explanations, the explanations can be irrelevant to the given problem without any guidelines. Thus, we instructed our workers to follow eight rules, including Use at least one word appearing in the problem text when writing an explanation. Moreover, we make workers obey the rules consistently using a web-based system. Details about all eight rules and the web-based system are illustrated in Appendix A. Fourteen skilled workers provided explanations for numbers and variables in a problem. Before assigning workloads, we split the entire dataset into training (80%), development (10%), and test (10%) sets. Then, we collected multiple explanations for each problem; 3 for training set and 4 for the other. Table 2 shows the statistics of the PEN dataset. 4 Since we manually corrected errors and flaws in each problem and combined three different datasets, some problems become exact duplicates of other problems. 5 These problems cannot be solved with a multivariate equation alone: problems about least common multiples or counting the number of cases. To verify whether the EPT-X model can solve an algebraic problem correctly while generating plausible and faithful explanations, we conduct three types of analyses: model performance analysis, quantitative error analysis, and qualitative output analysis. This section illustrates each analysis and further implementation details. The model performance analysis measures the model's correctness, which is the percentage of correctly answered problems on the PEN dataset. We regard an answer to be correct only if the answer values of all the variables in the problem are paired and solved correctly. For example, Table 1 shows that a correct answer contains two variables and answer values of x = 8 and y = 4 . Existing studies regarded x = 4 , y = 8 to be a correct solution(Kushman et al., 2014; Kim et al., 2020; Lee and Gweon, 2020). However, in the context of generating explanations along with solutions, different explanations are generated with different variables (Conway and Polya, 1985; Montague, 2008). Therefore, we enforce a stricter constraint that requires that a variable should be matched with a correct answer value. Using the correctness, we compare the EPT-X model with two previous inexplainable models (EPT (Kim et al., 2020), GEO (Lee and Gweon, 2020)) and human performance. The EPT is a model that generates one expression at a time and uses pointers instead of classifiers, and it achieved state-of-the-art accuracy on MAWPS and DRAW datasets. The GEO is a model that mixes encoder and decoder outputs before predicting a token, and it achieved state-of-the-art accuracy on DRAW and ALG514 datasets. To establish a human performance baseline, our research team manually checked for the answer correctness of the original datasets of ALG514, DRAW, and MAWPS. We found that 62 of the 3581 problems were incorrectly solved, thus yielding a human baseline performance of 98%. We conduct four types of error analyses to understand the possible cause of EPT-X solution errors: plausibility test, faithfulness test, faithfulness control test, and error propagation test. Through these four types of analyses, we show how the generated explanations by the EPT-X model can be used to understand the equation generation process. To test the plausibility of explanations, we used BLEU-4 (Papineni et al., 2002), ROUGE-L (Lin, 2004), CIDEr (Vedantam et al., 2015), and BLEURT (Sellam et al., 2020). These metrics can measure the extent of similarity between a generated explanation and the reference explanation in the dataset. Using the above four metrics, we compare the EPT-X model with a baseline model and human performance. First, we compare EPT-X with a baseline model which only contains Phase 1 (P1-only). This model can generate an explanation for each number/variable but cannot solve a word problem. Second, we compared EPT-X with human performance. While collecting explanations for each number/variable, we collected four sets of explanations. Of these four, one set is randomly set aside to serve as a hypothesis sentence and the other three as reference sentences when measuring human performance. To measure the faithfulness of EPT-X, we used two metrics: sufficiency and comprehensiveness (DeYoung et al., 2020). First, in our context, comprehensiveness means were explanations (Step 13) needed to produce the solution equation (Step 2-3)?. Figure 2 (a) shows the measurement setup for comprehensiveness. Specifically, we examined the amount of change between the two output solution equations: the equation from the original Phase 2 setup and the equation that is generated with only the problem text for the input of Phase 2. Since the output equation is not a single prediction as in DeYoung et al. (2020), we measured the change in the solution equations using tree edit distance (Zhang and Shasha, 1989). Secondly, in our context, sufficiency means do explanations (Step 1-3) contain enough information to produce solution equation (Step 2-3)?. Figure 2 (b) shows the measurement setup for sufficiency. Here, we examined the amount of change between the two output solution equations: the equation from the original Phase 2 setup and the equation that is generated with only the generated explanation. Similar to the comprehensiveness measure, the difference in equations was also computed using tree edit distance. To provide a statistical baseline for interpreting the two metrics of comprehensiveness and sufficiency, we adopted a bootstrapping method (Koehn, 2004). We sampled 500 bootstrapped samples (each sample has 50 problems) to estimate the population distribution of each metric. After the estimation, we conducted hypothesis testing for each metric. For comprehensiveness C , we set the following hypothesis HA : C > 1 as we expect to observe changes in equations when using only the problem input compared to using both problem and explanation input. For sufficiency S , we set the following hypothesis HA : S < 1 as we expect to observe no change in equations when only using the explanation input. To examine a trade-off relationship between correctness and faithfulness, we train and analyze two variants of EPT-X, whose faithfulness is controlled. The first model is an inherently faithful model (EPT-XF) that uses only the explanation, but not the original problem, as an input to Phase 2. As EPT-XF entirely depends on the explanation to generate a solution equation, the model passes the test of faithfulness by definition. The second model is an inherently unfaithful model (EPT-XU) that uses only the original problem, but not the explanation from phase 1, as an input to Phase 2. As EPT-XU ignores the explanation input, the model fails the test of faithfulness by definition. Implementation details on these two models are explained in Appendix B. 4447 5.2.4 Error propagation test To examine how the quality of explanation affects the model's correctness, we used two models, EPT-X and EPT-XF. Both models employ a two-phase architecture, thus they are prone to errors in both phases. For the error propagation test, we examine how the performance of Phase 1 (plausibility) affects the end-task performance (correctness). Note that testing EPT-X may not reveal the errors that are solely propagated from the generated explanation because EPT-X also uses the original problem as an input. Therefore, EPT-XF performance was also measured in order to examine the impact of errors from the generated explanation only. We measured the amount of error propagation in the two models, EPT-X and EPT-XF, by comparing correctness under two conditions: control and experiment. Under the control condition, the models build solution equations based on explanations generated by themselves. On the other hand, under the experiment condition, they build solution equations based on the gold standard explanations. Then, we measure the change of correctness between these two conditions for each model. Here, we expect that the change to reveal the proportion of problems affected by errors that are propagated from Phase 1. 5.3 Qualitative output analysis The explanations generated by EPT-X were analyzed qualitatively using two methods. First, to measure the quality of the generated explanation itself, we manually labeled the quality in the PEN's development set using two criteria: (1) plausibility and (2) faithfulness . Human coders were asked to label an explanation to be plausible when the explanation and the original problem text are coherent in meaning. And for faithfulness, we asked human coders to build a solution equation using only the explanation produced from the EPT-X model. If the generated solution equation is identical to the EPT-X generated solution equation, the explanation is labeled to be faithful. Second, to find the primary cause of errors when generating an explanation, we manually classified errors in EPT-X's explanations. The errors were categorized by comparing the generated explanation with the gold-standard explanation. We also used the PEN development set for this analysis. We describe three major implementation details used for training EPT-X: encoder, optimizer, and training epochs. For the text encoder component, EPT-X uses the base version of ELECTRA (Clark et al., 2020). We fixed its embedding and tied the embedding with the weights of FF v in the explanation decoder to preserve the world knowledge in the embedding and to stabilize the training procedure. For the optimizer, we used LAMB (You et al., 2020) with a learning rate of 0.00176, which was found from a grid search on the development set. Finally, for the training epochs, we trained EPT-X for 500 epochs. Appendix C lists additional details of the model, including hardware, software, libraries, hyper-parameters, and random seeds. The result of three analyses reveals that the EPT-X can generate an equation correctly based on a plausible and faithful explanation. First, Section 6.1 presents the result of the model performance analysis, which shows that EPT-X can achieve correctness 5% lower than previous inexplainable models. Second, Section 6.2 shows the result of error analysis, which reveals that many of EPT-X's errors are due to insufficient explanations. And lastly, Section 6.3 shows the result of qualitative analysis, which reveals three types of errors found in the explanation generation process of EPT-X. The model performance analysis shows that EPT-X generates equations with 69.59% accuracy on the PEN dataset, despite being a two-phase model. Table 3 shows that adding the explanation generation functionality decreases the accuracy by approximately 5%, compared to state-of-the-art model EPT 6 . We suspect that this performance 6 The results on the whole dataset is reported in this section, whereas results on each subset are reported in Appendix D. 4448 BLEU ROUGE CIDEr BLEURT Dev: Human 57.16 78.66 343.0 71.44 P1-Only 60.26 78.02 346.7 69.11 EPT-X 60.07 77.99 347.1 69.61 Test: Human 55.69 78.28 347.3 71.51 P1-Only 59.32 77.99 342.9 69.69 EPT-X 60.49 78.34 341.5 69.59 Table 4: Plausibility of EPT-X on PEN dataset Dev. drop is due to propagation of the errors in the generated explanations. Regardless, the results of EPT-X are meaningful since the model automatically generates explanations of problems without too much decrease in correctness. The difference of 5% is quite promising compared to that of the previous explainable model (Ling et al., 2017), about 40%, although a direct comparison is not possible due to differences in datasets. EPT-X achieved plausibility scores that are comparable to humans and the P1-only model, as shown in Table 4. The differences in plausibility scores between EPT-X and the other two baselines range between 1 to 2 points. This result indicates that EPT-X can select proper words to generate an explanation. In fact, BLEU-4 score of 60 is promising compared to Ling et al. (2017) (27.2). Given that EPT-X achieved human-level performance in terms of plausibility, but not for correctness, we explored the faithfulness metric to examine additional causes for the low model performance. The results of the faithfulness test showed two characteristics of the explanation output of EPT-X. In terms of comprehensiveness, the generated explanation contains some information required to generate a solution equation, as evidenced by Table 2. Here, we observe that EPT-X passed the comprehensiveness test for the 99% confidence Dev. level. That is, compared to using both inputs in Phase 2, forcing EPT-X to use only the original problem input made EPT-X generate a different solution equation. This result suggests that the generated explanation provides information, which is not provided by the original problem but contributes to generating a solution equation. Meanwhile, in terms of sufficiency, the generated explanation may not provide sufficient numeric information to generate a solution equation. Table 2 shows that EPT-X failed the sufficiency test under the confidence level of 95%. That is, EPT-X generates different solution equations when it only receives the generated explanation as input in Phase 2. So, the explanation generated in Phase 1 does not contain sufficient information, which is contained in the original problem, to generate a solution equation. As the generated explanation fails to capture some information from the original problem, the correctness may change when we control the faithfulness of a model. Specifically, the control test shows that there is a trade-off between faithfulness and correctness; as faithfulness increases, the correctness decreases. Table 6 shows that the most faithful model EPT-XF achieves the lowest correctness score, which is 6% lower than EPT-X. Conversely, the most unfaithful model EPT-XU achieves the highest correctness score, which is 4% greater than EPT-X. Thus, the results of the faithfulness test and the faithfulness control test imply that in order to achieve a higher correctness score, we should verify whether the explanation contains sufficient" information to build a correct equation. 6.2.4 Error propagation analysis The error propagation test shows that the generated explanation does not contain sufficient information to build a correct equation for some problems. Table 7 shows that both EPT-X and EPT-XF can outperform the EPT model by 8% when using a gold standard explanation as an input. However, 4449 Generated Gold Change Dev.: EPT-XF 66.03 83.29 +17.26 EPT-X 72.88 86.03 +13.15 Test: EPT-XF 62.19 85.20 +23.01 EPT-X 69.59 85.21 +15.62 Table 7: Result of error propagation test of explanation using explanations generated by the EPT-X model may decrease the correctness by more than 15%. That is, more than 15% of errors are due to information loss in Phase 1. 6.3 Qualitative output analysis Quality of explanations: The qualitative analysis showed that the quality of the generated explanations could be improved given that information required for solving word problems is missing. When we manually labeled the generated explanations for plausibility, 167 of 365 problems (45.8%) were labeled as plausible. Thus, the majority of the explanations are insufficient or contain incorrect information to generate a correct equation. Similarly, when we manually labeled the generated explanations for faithfulness, 201 of 365 problems (55.1%) were labeled as faithful. That is, when the same yet insufficient explanations were used to generate equations, the equations generated by EPT-X and humans were different. Three categories of errors: Additional qualitative analysis found three possible causes for the EPT-X errors. Here, we will briefly discuss the causes, and the detailed examples are illustrated in Appendix E. First, when a problem mentions several entities with similar properties (e.g., Heather's weight and Emily's weight), the difference between entities is ignored in the encoder (69 of 118 incorrect problems; 58.5%). This error implies that the context window used in Step 1-3 may not be big enough to distinguish two different entities. Second, if a problem provides multiple situations related to an entity (e.g., outward trip versus return trip), assigning a corresponding number to the correct situation fails in the encoding process. Detailed explanations of situations of a word problem were often omitted in the encoder (57 of 118 problems; 48.3%). Third, when a problem contains some irrelevant numbers, which are not used in solving the problem (e.g., year), sometimes an explanation for an irrelevant number was generated instead of the relevant one in the encoding process (32 of 118 problems; 27.1%). The second and third error types imply that sharing the encoder in Phases 1 and 2 might have caused confusion. The goal of the encoder in Phase 1 was to provide a detailed explanation of a given number, whereas the goal in Phase 2 was to build an equation, which involves ignoring some details to build an abstraction in the form of an equation. 7 Conclusion This study proposed a novel neural model EPT-X, Expression Pointer Transformer with Explanations, which generates explanations along with solution equations. The EPT-X model was designed to address two criteria of plausibility and faithfulness when generating an explanation. To address plausibility, the model generates explanations for each number/variable in the solution equation separately. And to address faithfulness, the model produces equations based on the information in the generated explanation. In addition to EPT-X, we release a new dataset, Problem with Explanations for Numbers (PEN), which extends existing three algebraic word problem datasets by augmenting explanations for numbers/variables. Using the PEN dataset, we conducted three analyses. The model performance analysis revealed that EPT-X could produce a correct equation with 69.59% accuracy. The quantitative error analysis showed that the EPT-X model could produce a plausible albeit insufficient explanation. Lastly, the qualitative output analysis identified three categories of errors made when generating explanations. Despite the insufficiency of explanations generated by the EPT-X model, our work is significant in that we demonstrated the possibility of generating explanations while solving an algebraic word problem. For future work, we plan to improve the correctness and faithfulness of EPT-X to enhance the existing state-of-the-art model. Acknowledgements This work was supported by the National Research Foundation of Korea (NRF) grant (No. 2020R1C1C1010162) and the Institute for Information & communications Technology Promotion (IITP) grant (No. 2021-0-02146), both funded by the Korean government (MSIT). Also, we thank S. Oh, Y. Lee, J. An, J. Kim, and H. Rhim who helped in building the PEN dataset. 4450 References Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Ha-jishirzi. 2019. MathQA: Towards interpretable math word problem solving with operation-based formalisms. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages 23572367, Minneapolis, Minnesota. Association for Computational Linguistics. Thomas P Carpenter, Elizabeth Fennema, and Megan L Franke. 1996. Cognitively guided instruction: A knowledge base for reform in primary mathematics instruction. The elementary school journal. , 97(1):320. Ting-Rui Chiang and Yun-Nung Chen. 2019. Semantically-aligned equation generation for solving and reasoning math word problems. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages 26562668, Minneapolis, Minnesota. Association for Computational Linguistics. Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. Electra: Pretraining text encoders as discriminators rather than generators. In International Conference on Learning Representations . John H Conway and G. Polya. 1985. How to solve it , volume 85. Princeton university press. Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C. Wallace. 2020. ERASER: A benchmark to evaluate rationalized NLP models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 44434458, Online. Association for Computational Linguistics. Christine Fellbaum. 1998. WordNet: an electronic lexical database . MIT Press. Joseph Calvin Gagnon and Paula Maccini. 2001. Preparing students with disabilities for algebra. TEACHING Exceptional Children , 34(1):815. Danqing Huang, Jing Liu, Chin-Yew Lin, and Jian Yin. 2018. Neural math word problem solver with reinforcement learning. In Proceedings of the 27th International Conference on Computational Linguistics , pages 213223, Santa Fe, New Mexico, USA. Association for Computational Linguistics. Alon Jacovi and Yoav Goldberg. 2020. Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 41984205, Online. Association for Computational Linguistics. Asha K. Jitendra, Edward Sczesniak, Cynthia C. Grif-fin, and Andria Deatline-Buchman. 2007. Mathematical word problem solving in third-grade classrooms. The Journal of Educational Research , 100(5):283302. Asha K. Jitendra and Jon R. Star. 2012. An exploratory study contrasting highand low-achieving students' percent word problem solving. Learning and Individual Differences , 22(1):151158. Nitish Shirish Keskar, Bryan McCann, Lav R. Varsh-ney, Caiming Xiong, and Richard Socher. 2019. Ctrl: A conditional transformer language model for controllable generation. Kyung Seo Ki, Donggeon Lee, Bugeun Kim, and Gahgene Gweon. 2020. Generating equation by utilizing operators : GEO model. In Proceedings of the 28th International Conference on Computational Linguistics , pages 426436, Barcelona, Spain (On-line). International Committee on Computational Linguistics. Bugeun Kim, Kyung Seo Ki, Donggeon Lee, and Gahgene Gweon. 2020. Point to the Expression: Solving Algebraic Word Problems using the Expression-Pointer Transformer Model. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 37683779, Online. Association for Computational Linguistics. Philipp Koehn. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing , pages 388395, Barcelona, Spain. Association for Computational Linguistics. Rik Koncel-Kedziorski, Hannaneh Hajishirzi, Ashish Sabharwal, Oren Etzioni, and Siena Dumas Ang. 2015. Parsing algebraic word problems into equations. Transactions of the Association for Computational Linguistics , 3:585597. Rik Koncel-Kedziorski, Subhro Roy, Aida Amini, Nate Kushman, and Hannaneh Hajishirzi. 2016. MAWPS: A math word problem repository. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages 11521157, San Diego, California. Association for Computational Linguistics. Nate Kushman, Yoav Artzi, Luke Zettlemoyer, and Regina Barzilay. 2014. Learning to automatically solve algebra word problems. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 271281, Baltimore, Maryland. Association for Computational Linguistics. D. Lee and G. Gweon. 2020. Solving arithmetic word problems with a templatebased multi-task deep neural network (t-mtdnn). In 2020 IEEE International 4451 Conference on Big Data and Smart Computing (Big-Comp) , pages 271274. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out , pages 7481, Barcelona, Spain. Association for Computational Linguistics. Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. 2017. Program induction by rationale generation: Learning to solve and explain algebraic word problems. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 158167, Vancouver, Canada. Association for Computational Linguistics. Richard E. Mayer. 1981. Frequency norms and structural analysis of algebra story problems into families, categories, and templates. Instructional Science , 10(2):135175. Marjorie Montague. 2008. Self-regulation strategies to improve mathematical problem solving for students with learning disabilities. Learning Disability Quarterly , 31(1):3744. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics , pages 311318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research , 21(140):167. Sascha Rothe, Shashi Narayan, and Aliaksei Severyn. 2020. Leveraging pre-trained checkpoints for sequence generation tasks. Transactions of the Association for Computational Linguistics , 8:264280. Subhro Roy and Dan Roth. 2015. Solving general arithmetic word problems. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing , pages 17431752, Lisbon, Portugal. Association for Computational Linguistics. Subhro Roy and Dan Roth. 2017. Unit dependency graph and its application to arithmetic word problem solving. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence , AAAI'17, page 30823088. AAAI Press. Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 1073 1083, Vancouver, Canada. Association for Computational Linguistics. Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020. BLEURT: Learning robust metrics for text generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 78817892, Online. Association for Computational Linguistics. Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Shyam Upadhyay and Ming-Wei Chang. 2017. Annotating derivations: A new evaluation strategy and dataset for algebra word problems. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers , pages 494504, Valencia, Spain. Association for Computational Linguistics. Shyam Upadhyay, Ming-Wei Chang, Kai-Wei Chang, and Wen-tau Yih. 2016. Learning from explicit and implicit supervision jointly for algebra word problems. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing , pages 297306, Austin, Texas. Association for Computational Linguistics. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems , pages 59986008. R. Vedantam, C. L. Zitnick, and D. Parikh. 2015. Cider: Consensus-based image description evaluation. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 45664575. Yan Wang, Xiaojiang Liu, and Shuming Shi. 2017. Deep neural solver for math word problems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing , pages 845854, Copenhagen, Denmark. Association for Computational Linguistics. Yang You, Jing Li, Sashank Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Demmel, Kurt Keutzer, and Cho-Jui Hsieh. 2020. Large batch optimization for deep learning: Training bert in 76 minutes. In International Conference on Learning Representations . Kaizhong Zhang and Dennis Shasha. 1989. Simple fast algorithms for the editing distance between trees and related problems. SIAM Journal on Computing , 18(6):12451262. Lipu Zhou, Shuaixiang Dai, and Liwei Chen. 2015. Learn to solve algebra word problems using quadratic programming. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing , pages 817822, Lisbon, Portugal. Association for Computational Linguistics. 4452 Figure 3: A screenshot of the system used for annotating explanations on the PEN dataset A Annotating explanations for PEN dataset This section describes the detailed process of annotating explanations. Using a web-based system shown in Figure 3, coders inputted a natural language explanation for each number/variable. To provide situational information for each num-ber/variable, we highlighted text snippets and equations related to the target number/variable. Based on the given information, the coder needed to complete the following sentence:  number means ..."
objective abstain abstain abstain method objective abstain objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain objective objective objective objective other other other other other other other other other other other abstain other objective other method other other other abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain other abstain abstain method abstain method method method method abstain abstain abstain abstain method abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain method abstain method abstain method abstain method abstain abstain abstain abstain other method method method abstain abstain abstain abstain method method abstain method method abstain method abstain method abstain abstain method abstain abstain abstain method method abstain abstain method method method abstain method method abstain method method abstain method method abstain method method abstain abstain method method abstain abstain method abstain method method method method method method abstain abstain method abstain method abstain abstain method abstain abstain method method method abstain method abstain method abstain method method abstain method method method abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other