Hypothesis testing is an important cognitive process that supports human reasoning. In this paper, we introduce a new computational hypothesis testing framework that is based on memory augmented neural networks. Our approach involves a hypothesis testing loop that reconsiders and progressively refines a previously formed hypothesis in order to generate new hypotheses to test. We applied the proposed approach to language comprehension task by using Neural Semantic Encoders (NSE). Our NSE models achieved the state-of-the-art results showing an absolute improvement of 1.2% to 2.6% accuracy over previous results obtained by single and ensemble systems on standard machine comprehension benchmarks such as the Children’s Book Test (CBT) and Who-Did-What (WDW) news article datasets.
1 INTRODUCTION
Formulating new hypotheses and testing them is a cognitive process that supports human reasoning and intelligence. This hypothesis testing process involves selective attention, working memory and cognitive control (Just and Carpenter, 1992; Polk and Seifert, 2002). Attention and working memory are engaged in order to maintain, manipulate, and update new hypotheses. Cognitive control is required to inspect and ignore incorrect hypotheses. Inspired by the hypothesis testing process in the human brain and to support dynamic reasoning of machines, in this work, we introduce a reasoning approach that is based on memory augmented neural networks (MANN). A new hypothesis is formed by regressing the original statement (i.e. query in context of QA). Then the hypothesis is tested against reality (i.e. data or document story). If the model is satisfied with the current test response or the hypothesis is true, the reasoning process is halted and the answer is found. Otherwise, another hypothesis is formulated by refining the previous one and the process is repeated until the answer is found.
While the idea of modeling hypothesis testing with MANN remains a generic reasoning framework and is applicable to several AI tasks, we apply this approach to cloze-type QA by using Neural Semantic Encoders (NSE). NSE is a flexible MANN architecture and have shown a notable success on several language understanding tasks ranging from sentence classification to language inference and machine translation (Munkhdalai and Yu, 2016). NSE has read, compose and write modules to manipulate external memories and it has introduced a concept of shared and multiple memory accesses, which has shown to be effective for sequence transduction problems.
Cloze-type question answering (QA) is a clever way to assess the ability of human and machine to comprehend natural language. This type of tasks are attractive for the natural language processing (NLP) community because the test sets or datasets can be generated without requiring expert’s supervision, in an automatic manner, which is typically useful in training and testing artificial intelligent (AI) systems that can understand human language. In cloze-type QA setup, the machine is first presented with a text document containing a fact set and then it is asked to output answer for a query related to the document.
With the recent development of large-scale Cloze-type QA datasets (Hermann et al., 2015; Hill et al., 2015; Onishi et al., 2016) and deep neural network methods, remarkable advances have been made in order to solve the problem in an end-to-end fashion. The existing neural network approaches can
be broadly divided into single-step or multi-step comprehension depending on how documenting reading and answer inference processes are modeled. By mimicking human readers for a deeper reasoning, the multi-step comprehension systems have shown promising results (Hermann et al., 2015; Hill et al., 2015; Trischler et al., 2016; Sordoni et al., 2016). However, the current multiturn models are designed with the predefined number of computational hops for inference while the difficulty of document and query pairs can vary. Some of the query-document pairs require a shallow reasoning like word or sentence level matching, but for some of them a deeper document level reasoning or a complex semantic understanding is crucial.
The proposed NSE comprehension models perform a reasoning process we called hypothesis-test loop. In each step, a new hypothesis for the correct answer is formed by a query regression. Then the hypothesis is checked and if it is true, the model halts the reasoning process (or the hypothesistest loop) to give the correct answer. To this end, unlike previous methods with fixed computation our models introduce halting procedure in the hypothesis-test loop. When trained with classic back-propagation algorithm, our NSE models show consistent improvements over state-of-the-art baselines on two cloze-type QA datasets.
2 RELATED WORK
Recently, several large-scale datasets for machine comprehension have been introduced, including the cloze-type QA (Hermann et al., 2015; Hill et al., 2015; Onishi et al., 2016). Consequently there is an increasing interest in developing neural network approaches to solve the problem in an end-toend fashion. The existing models for cloze-type QA can be categorized into single-step or multi-step approach depending on their comprehension process.
2.1 SINGLE-STEP COMPREHENSION
Singe-step comprehension methods read input document once with a single computational hop to make answer prediction. The reading process mainly involves context modeling with bi-directional recurrent neural networks (RNN) and selective focusing with attention mechanism. Hermann et al. (2015) introduced a CNN/Daily News QA task along with a set of baseline models such as Attentive Reader and Impatient Reader. The Attentive Reader model reads the document and the query with bi-directional LSTM (BiLSTM) networks and selects a query-relevant context by attending over the document. Chen et al. (2016) re-designed the Attentive Reader model (so called Stanford Attentive Reader) and examined the CNN/Daily News QA task. They found out that roughly 25% of all queries in Hermann et al. (2015)’s dataset is unanswerable and the recent neural network approaches have obtained the ceiling performance on this task. Kadlec et al. (2016) proposed an Attention Sum (AS) reader model that first attends over the document and then aggregates all the attention score for the same candidate answer to select the highest scoring candidate as correct answer.
However, for complex document and query pairs with deeper semantic association machine reading only once may not be enough and multiple reading and checking is crucial to perform deeper reasoning.
2.2 MULTI-STEP COMPREHENSION
Similar to a human reader, multi-step comprehension methods read the document and the query before making the final prediction. This kind of comprehension is mostly achieved by implementing an external memory and an attention mechanism to retrieve the query-relevant information from the memory throughout time steps. Hermann et al. (2015)’s Impatient Reader revisits the document states with the attention mechanism whenever it reads the next query word in each time scale. Hill et al. (2015) extended multi-hop Memory Networks (Sukhbaatar et al., 2015) with self-supervision to retrieve supporting memory. EpiReader performs a two-stage computation (Trischler et al., 2016). First it chooses a set of most probable answers with the AS model and forms new queries by replacing an answer placeholder in the original query with the candidate answer words. Second EpiReader runs an entailment estimation between the document and the new query pairs to predict the correct answer. Gated Attention (GA) reader extends the AS model with document-gating and iterative reading of the document and the query (Dhingra et al., 2016). The query representation is used as gating for the document and the iterative reading is accomplished by having separate bi-directional
gated recurrent unit (GRU) networks for each computational hop. Iterative Alternative Attention (IAA) reader (Sordoni et al., 2016) is a multi-step comprehension model which uses a GRU network to search for correct answers from a document. In this model, the GRU network is expected to collect evidence from the document and the query that assists prediction in the last time step. Cui et al. (2016) introduced attention-over-attention loss by computing a word level query-document matching matrix. This model provides a fine-grained word-level supervision signal which seems to help model training.
Our proposed model performs multiple computational steps for deeper reasoning. Unlike the previous work, in our model the number of steps to revise the document is not predefined and it is dynamically adapted for a particular document and query pair. Furthermore, we define novel ways to substitute query words with a word chosen from the document (i.e. regression process) and to check a hypothesis that the selected document word actually compliments the query (i.e. check process). When the hypothesis is true, our model halts the reading process and outputs the word chosen from the document as the correct answer. NSE is used as a controller for the whole process throughout the reasoning steps.
Among the aforementioned models, EpiReader seems to be the most relevant one to our language comprehension models. However, having an entailment estimation introduces a constraint in EpiReader which limits its application. Our model is generic and can be useful in different tasks other than machine comprehension such as language-based conversational tasks, knowledge inference and link prediction. As EpiReader has tightly integrated two-stage neural network modules, the performance directly depends on the first stage. If the first module misses out or fails to choose enough candidates, no correct answer can be found. Our model has no such issue and is not constrained in forming new queries. More recently, the idea of dynamic termination in the context of language comprehension is proposed by Shen et al. (2016) independently. While they use reinforcement learning, we explore two different approaches: adaptive computation and query gating to be fully trained by end-to-end back-propagation.
3 PROPOSED APPROACH
Our dataset consists of tuples (D,Q,A, a), where D is the document (i.e. passage) serving as a fact for the query Q, A is a set of candidate answers and a ∈ A is the true answer. The document and the query are sequences of tokens drawn from a vocabulary V . We train a model to predict the correct answer a from the candidate set A for given a pair of query and document.
The main components of our proposed model are shown in Figure 1. First the query and the document memory are initialized via context embedding (omitted from the figure). The memories are then processed with memory read and write operations throughout the hypothesis-test loop. In each step of the loop, the read module formulates a new hypothesis by updating the query memory with relevant content from the document memory. Then the write module tests whether the new hypothesis is true by inspecting the current query and the document states. Selecting relevant content from the document to regress the old query is essentially an inference (i.e. prediction) in our model. Intuitively the input query is regressed toward (becoming) a complete query containing the correct answer word within it. If the write module thinks that the query is complete and the correct answer is found, it halts the hypothesis-test loop. The write module also supervises the query state transitions and retains the right to roll back the new query changes during the reasoning process. To avoid an overconfident prediction and to halt the reasoning process, we explore two different strategies: (a) query gating and (b) adaptive computation.
Like a human reader, the model reads the document multiple times, formulates a hypothetical answer for the query and tests it against the story of the document throughout the hypothesis-test steps. Once satisfied with the response of the current hypothesis, the model outputs the response as the correct answer.
3.1 MEMORY INITIALIZATION
Instead of using raw word embeddings, we initialize the document and the query memories via context embedding in order to inform the memory slots of contextual information from text passages.
Two BiLSTM networks are applied to the document and the query sequences separately, as:
Mq0 = BiLSTM q(Q) (1)
Md = BiLSTMd(D) (2)
where Mq0 ∈ Rk×|Q| and Md ∈ Rk×|D| are the memory representations. k is the size of BiLSTM hidden layer. The query memory is evolved over time while the document memory is not updated and rather serves as a fact set for the query.
Note that in our model the context embedding network can be any network accepting word embeddings as input, such as multilayer perceptron (MLP) or convolutional neural network. We choose BiLSTM because it is able to learn the word-centered context representation effectively by reading text with LSTMs from both left and right and concatenating resulting hidden vectors.
3.2 HYPOTHESIS TESTING
The query and the document memories are further processed through an iterative process called hypothesis-test loop. In each step of the loop, the query memory Mq is updated with content from the document memory to form a new query (i.e. hypothesis formulation). The new query is then checked against the document facts and used to make an answer prediction (i.e. hypothesis testing). The NSE read, compose and write modules collectively perform the following overall process.
Read: this module takes in the query and the document states sqt−1, sdt−1 ∈ Rk from the previous time step t − 1 as input and computes the alignment vectors lqt ∈ R|Q|, ldt ∈ R|D|, the new query and the document states sqt , s d t ∈ Rk and the query memory key vector z q t ∈ R|Q| as follows 1:
rt = read LSTM ([sqt−1; s d t−1]) (3)
lqt = r > t M q t−1 (4)
sqt = softmax(l q t ) >Mqt−1 (5)
zqt = sigmoid(l q t ) (6)
ldt = s q t > Md (7)
sdt = softmax(l d t ) >Md (8)
Intuitively depending on the previously retrieved document content as well as the previous query states, the read module retrieves from the document a word to be relocated to the query and computes the positions of this word in the query. Since the document word can be located in multiple different positions in the query, sigmoid function is used to normalize the alignment vector lqt . The document state sdt represents the newly retrieved word and the key vector z q t defines its new position in the query memory. Because such a decision is made sequentially in every step, we equip the read module with a LSTM network (i.e. readLSTM ). We initialize the document and the query states sd0, s q 0 with the last hidden states of the query and the document BiLSTM networks.
Compose: the compose module combines the current query and document states sqt , sdt and the current hidden state of the read module rt ∈ Rk as:
ct = compose MLP (sqt , s d t , rt) (9)
The resulting single vector ct ∈ Rk is passed to the write module for subsequent process. The compose module can be viewed as a feature extractor from the current document and query pair. By taking in the hidden state rt, the compose module also informs the write module of the read module’s current decision.
Write: this module accepts the outputs of the read module and updates the query memory as follows:
Mqt = M q t−1z q t + s d t (1− z q t ) (10)
where 1 is a matrix of ones. Note that the values of the key vector zqt ranges from zero to one and zero (or near zero) values indicate the query position where the new document word is written.
The write module is also responsible for checking the new hypothesis in order to decide whether to halt the hypothesis-test loop for the final answer or to continue. We explore two different strategies to be discussed below. Both methods take in the output of the compose module and employs the LSTM to make the sequential decision.
3.2.1 QUERY GATING
Figure 1 (a) shows the overall architecture of our model with query gating mechanism. In this model instead of making a hard decision on halting the loop (i.e. stop reading), the write module performs a word-level query gating as:
wt = write LSTM (ct) (11)
gqt = sigmoid(w > t M q t−1) (12)
Mqt = M q t (1− g q t ) +M q t−1g q t (13)
where 1 is a matrix of ones. The key part in the above equation is obtaining the gating weights. This is accomplished by comparing each memory slot with the hidden vector wt ∈ Rk and normalizing
1For brevity, we omit the broadcasting operations from the equations.
resulting scores with sigmoid function. Therefore, the gating vector gqt ∈ R|Q| have ones for preserving and zeros for erasing content of the old query.
This can be seen as a memory gating process which prevents the model from forgetting the old query information. Note that even if the query memory is updated with the document content given by the read module, the write module makes the final decision based on features extracted by the compose module. In other words if the write module decides to keep the old query information, the changes in the new query are simply ignored and the same query from the previous time step is passed along to the next step. The number of steps T in the hypothesis-test loop is a hyperparameter in this model. Therefore, in this setup the write module is expected to lock the query state with its gating mechanism as soon as the hypothesis is true.
3.2.2 ADAPTIVE COMPUTATION
In this model, the write module is equipped with a termination head as shown in Figure 1 (b). Particularly, the write module with the termination head decides its willingness to continue or finish the computation in each step.
We define a probabilistic framework for halting. Our approach is similar to the input and output handling mechanism of Neural Random-Access Machines (Kurach et al., 2016). In each time step, the write module outputs a termination score et as follows:
wt = write LSTM (ct) (14)
et = sigmoid(o >wt) (15)
where o ∈ Rk is a trainable vector that projects the hidden state wt to a scalar value. Then the probability to halt the hypothesis-test loop after t steps is
pt = et t−1∏ i=1 (1− ei) (16)
We also introduce a hyperparameter T for the maximum number of permitted steps. If the model runs out of the time without halting the process (after T steps), we force the model to output the final answer in step T . In this case, the probability to stop reading is
pT = 1− T−1∑ i=1 pi (17)
3.3 ANSWER PREDICTION
In step t, the query-to-document alignment score ldt is used to compute the probability P (a|Q,D) that the answer a is correct given the document and the query. In particular, we adapt the pointer sum attention mechanism (Kadlec et al., 2016) as
Pt(a|Q,D) = v>softmax(ldt ) (18)
where v ∈ R|D| is a mask denoting the positions of the answer token a in the document (ones for the token and zeros otherwise).
For the query gating model, we use the probability PT (a|Q,D) produced in the last step T to choose the correct answer. For the second model, we incorporate the termination score pt and redefine the probability as
P (a|Q,D) = T∑
i=1
(pi · Pi(a|Q,D)) (19)
We then train the models to minimize cross-entropy loss between the predicted probabilities and correct answers.
4 EXPERIMENTS
We evaluated our models on two large-scale datasets: Childrens Book Test (CBT) (Hill et al., 2015) and Who-Did-What (WDW) (Onishi et al., 2016). We focused on these tasks because there is still a large gap between the human and the machine performances on CBT and WDW, which is in contrast to the CNN/Daily News QA datasets covered in Section 2. The CBT dataset was constructed from the children book domain whereas the WDW corpus was built from the news article domain (the English Gigaword corpus); therefore we think that the two datasets are quite representative for evaluation of our models. Furthermore the CBT dataset comes with two difficult tasks depending on the type of answer words to be predicted: named entity (CBT-NE) and common nouns (CBT-CN). The WDW training set has two different setups with strict and relaxed baseline suppression. Table 1 summarizes some important statistics of the datasets.
4.1 TRAINING DETAILS
We chose one-layer LSTM networks for the read and the write modules and an MLP with singlelayer for the composition module. We used stochastic gradient descent with an Adam optimizer to train the models. The initial learning rate (lr) was set to 0.0005 for CBT-CN or 0.001 for other tasks. A pre-trained 300-D Glove 840B vectors (Pennington et al., 2014) were used to initialize the word embedding layer2; therefore the embedding layer size is 300. The hidden layer size of the context embedding BiLSTM nets k = 436. The embeddings for out-of-vocabulary words and the model parameters were randomly initialized from the uniform distribution over [-0.1, 0.1). The gradient
2http://nlp.stanford.edu/projects/glove/
clipping threshold was set to 15. The models were regularized by applying 20% dropouts to the embedding layer3. We used the batch size n = 32 for the CBT dataset and n = 25 for the WDW dataset and early stopping with a patience of 1 epoch. For the WDW dataset, we anonymized the answer candidates by following the work of Onishi et al. (2016) and Hermann et al. (2015).
We run a hyperparameter search over k = { 256, 368, 436, 512 } , lr = { 0.0005, 0.001 } and l2
decay = { 0.0001, 0.00005, 0.00001 } on the CBT dev sets to come up with the current setting for training. Among these parameters, the l2 weight decay regularizer did not seem to help and thus it was not applied. We did not tune the dropouts.
We used the following batching heuristic in order to speedup the training. We created a temporary example pool by randomly sampling from the training set and sorted them according to the length of the document. Then the first n examples (n = 32 or n = 25) of the ordered pool were put into the same batch, without replacement, and the rest of the pool was replaced back in the training set. This is performed until there is not enough training examples to create a new pool. Finally the documents in the same batch are padded with special symbol <pad> to the same length. The batches were regenerated in every epoch to prevent the model from learning a simple mapping function. We also padded queries in the same batch with the special symbol to an equal length.
4.2 RESULTS
In Tables 2 and 3 we compared the performance of our models with all published models (as baselines) including their ensemble variations. We report the performance of our query gating model at the varying number of hypothesis-test steps T = { 1, 2, 6, 9, 12 } . T for our adaptive computation model was set to 2 or 12. When T = 1, both models update the memory only once and do not have enough time to read it back and thus they are reduced to the same model called NSE (i.e. NSE T = 1). Typically when T is greater than one, the proposed halting strategies result in two different models.
Our query gating model achieves 72.6% and 72.0% accuracy on the CBT tasks outperforming all the previous baselines. The performance varies for the different number of steps. For the larger number of allowed steps, the query gating model tends to overfit on the CBT dataset as the performance gap between the dev and test sets increases. Our NSE Adaptive Computation model sets the best score on CBT-NE task with 73.2% accuracy. Overall our NSE models bring modest improvements of 1.2% on CBT-NE and 2.6% on CBT-CN tasks and now the performance differences between the human and the machines are only 8.4% and 9.6%.
On the WDW task our single model, NSE with adaptive computation, scores 2.2% and 1.7% higher than the previous best result of Stanford Attentive Reader. The NSE Query Gating model obtains its best result when T = 9. Comparing to the proposed variations of NSE, our Adaptive Computation
3More detail on hyperparameters can be found in our code: https://bitbucket.org/tsendeemts/nse-rc
model is more robust and sets new state-of-the-art performance on three out of four tasks by effectively deciding when to halt the reasoning processing with the termination head. Memory locking (to prevent from forgetting) did not show to be as effective as the termination-based approach on this task. In Appendix A, we visualize query regression process in both models and observe that while new queries are generated, query words relevant to the correct answer are not overwritten by the memory write module.
The performance of the NSE model with T = 1 matches that of the previous single systems. In general given a small number of permitted steps, the proposed models tend to less overfit but the final performance is not high. As the number of permitted steps increases both dev and test accuracy improves yielding an overall higher performance. However, this holds up to a certain point and with a large permitted steps we no longer observe a significant performance improvement in terms of testing accuracy. For example, NSE Query Gating model with T = 15 (not included in the table) achieved 79.2% dev accuracy and 71.4% test accuracy on CBT-NE task, showing the highest accuracy on the dev set yet massively overfitting on the test set. Furthermore it becomes expensive to train a model with a large number of allowed steps.
It is worth noting that even if the proposed models in this work were trained using classic end-toend back-propagation, they can easily be trained with reinforcement learning methods, such as the REINFORCE algorithm (Williams, 1992), for further evaluation. Such an adaptation is particularly straightforward for NSE Adaptive Computation because this model has already incorporated a probabilistic termination head in it.
5 CONCLUSION
Inspired by cognitive process of hypothesis testing in human brain, we proposed a reasoning approach based memory augmented neural networks and applied it to language comprehension. Our proposed NSE models with dynamic reasoning have achieved the state-of-the-art results on two machine comprehension tasks. In order to halt reasoning process we explored two different strategies to be fully trained with classic back-propagation algorithm: query memory gating which prevents from forgetting old query and adaptive computation with termination head. The NSE adaptive computation model has shown to be effective in the experiments. Our proposed models can be trained using reinforcement learning. We plan to apply our approach to other AI tasks, such language-based conversational tasks, link prediction and knowledge inference.
ACKNOWLEDGMENTS
We would like to thank Abhyuday Jagannatha, Jesse Lingeman and the reviewers for their insightful comments and suggestions. This work was supported in part by the grant HL125089 from the National Institutes of Health and by the grant 1I01HX001457-01 supported by the Health Services Research & Development of the US Department of Veterans Affairs Investigator Initiated Research. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect those of the sponsor.
A QUERY REGRESSION ANALYSIS
We analyzed how input queries are updated through memory read, write and gating operations in order for the model to make a prediction for the correct answer.
A.1 QUERY GATING
Figure 2 depicts the memory key (zqt ) and the query gate (g q t ) states of the NSE Query Gating model for the input document shown on the top. As noted previously, the model updates the query memory at a position that has a lower memory key value (i.e. the values in the memory key closer to zero indicate the update positions) but rolls back or ignores the new query change if the gating value is one at the same position. The top attended words resulted by the memory read operation define the information for the update. We listed the top-3 attended words in Figure 2, which already includes the correct answer.
Note how the the memory key and the gating states are changed during the regression steps. The memory key values are varied across the query positions. Overall the query tokens ’He’ and ’Meadows’ are pointed to be updated. As proceeding with the query regression, the model effectively adjusts its gates. During the first two steps, the model is willing to accept new updates occurred at certain positions such as ’He’ and ’Meadows’. However, the gating value reaches one and the gates are closed in the later steps. The model no longer accepts new query updates in step 6 and is now ready to output the correct answer.
Our analysis showed that the model rarely updates the query at the place holder position (i.e. ’XXXXX’ in the query). This is desirable in such iterative query regression process. Here the placeholder information remains unchanged in order to be used in the subsequent steps and to inform the model of what part of the query needs to be completed.
A.2 ADAPTIVE COMPUTATION
In Figure 3, we showed the memory key and the termination head states for an input pair from the CBT dev set. The model actively performs the query regression during the first two steps and then it halts the process as the termination head reaches one in the third step. Interestingly, the memory key points to the most of the query positions for update in the first step and then its values gradually increase in the later steps implying no information exchange between the document and the query. The answer cue word ’Johnny’ and the place holder information are not overwritten until the loop is halted.

Thie paper proposed an iterative memory updating model for cloze-style question-answering task. The approach is interesting, and result is good. For the paper, I have some comments:
1. Actually the model in the paper is not single model, it proposed two models. One consists of "reading", "writing", "adaptive computation" and " Answer module 2", the other one is "reading", "composing", "writing", "gate querying" and "Answer module 1". Based on the method section and the experiment, it seems the "adaptive computation" model is simpler and performs better. And without two time memory update in single iteration and composing module, the model is similar to neural turing machine.
2. What is the MLP setting in the composing module?
3. This paper tested different size of hidden state:[256, 368, 436, 512], I do not find any relation between those numbers, how could you find 436? Is there any tricks helping you find those numbers?
4. It needs more ablation study about using different T such as T=1,2..
5. According to my understanding, for the adaptive computation,  it would stop when the P_T <0. So what is the distribution of T in the testing data?


LEARNING INVARIANT REPRESENTATIONS OF PLANAR CURVES
We propose a metric learning framework for the construction of invariant geometric functions of planar curves for the Euclidean and Similarity group of transformations. We leverage on the representational power of convolutional neural networks to compute these geometric quantities. In comparison with axiomatic constructions, we show that the invariants approximated by the learning architectures have better numerical qualities such as robustness to noise, resiliency to sampling, as well as the ability to adapt to occlusion and partiality. Finally, we develop a novel multi-scale representation in a similarity metric learning paradigm.
1 INTRODUCTION
The discussion on invariance is a strong component of the solutions to many classical problems in numerical differential geometry. A typical example is that of planar shape analysis where one desires to have a local function of the contour which is invariant to rotations, translations and reflections like the Euclidean curvature. This representation can be used to obtain correspondence between the shapes and also to compare and classify them. However, the numerical construction of such functions from discrete sampled data is non-trivial and requires robust numerical techniques for their stable and efficient computation.
Convolutional neural networks have been very successful in recent years in solving problems in image processing, recognition and classification. Efficient architectures have been studied and developed to extract semantic features from images invariant to a certain class or category of transformations. Coupled with efficient optimization routines and more importantly, a large amount of data, a convolutional neural network can be trained to construct invariant representations and semantically significant features of images as well as other types of data such as speech and language. It is widely acknowledged that such networks have superior representational power compared to more principled methods with more handcrafted features such as wavelets, Fourier methods, kernels etc. which are not optimal for more semantic data processing tasks.
In this paper we connect two seemingly different fields: convolutional neural network based metric learning methods and numerical differential geometry. The results we present are the outcome of investigating the question: ”Can metric learning methods be used to construct invariant geometric quantities?” By training with a Siamese configuration involving only positive and negative examples of Euclidean transformations, we show that the network is able to train for an invariant geometric function of the curve which can be contrasted with a theoretical quantity: Euclidean curvature. An example of each can be seen Figure 1. We compare the learned invariant functions with axiomatic counterparts and provide a discussion on their relationship. Analogous to principled constructions like curvature-scale space methods and integral invariants, we develop a multi-scale representation using a data-dependent learning based approach. We show that network models are able to construct geometric invariants that are numerically more stable and robust than these more principled approaches. We contrast the computational work-flow of a typical numerical geometry pipeline with that of the convolutional neural network model and develop a relationship among them highlighting important geometric ideas.
In Section 2 we begin by giving a brief summary of the theory and history of invariant curve representations. In Section 3 we explain our main contribution of casting the problem into the form which
enables training a convolutional neural network for generating invariant signatures to the Euclidean and Similarity group transformations. Section 4 provides a discussion on developing a multi-scale representation followed by the experiments and discussion in Section 5.
2 BACKGROUND
An invariant representation of a curve is the set of signature functions assigned to every point of the curve which does not change despite the action of a certain type of transformation. A powerful theorem from E. Cartan (Cartan (1983)) and Sophus Lie (Ackerman (1976)) characterizes the space of these invariant signatures. It begins with the concept of arc-length which is a generalized notion of the length along a curve. Given a type of transformation, one can construct an intrinsic arclength that is independent of the parameterization of the curve, and compute the curvature with respect to this arc-length. The fundamental invariants of the curve, known as differential invariants (Bruckstein & Netravali (1995), Calabi et al. (1998)) are the set of functions comprising of the curvature and its successive derivatives with respect to the invariant arc-length. These differential invariants are unique in a sense that two curves are related by the group transformation if and only if their differential invariant signatures are identical. Moreover, every invariant of the curve is a
function of these fundamental differential invariants. Consider C(p) = [ x(p) y(p) ] : a planar curve with
coordinates x and y parameterized by some parameter p. The Euclidean arc-length, is given by
s(p) = ∫ p 0 |Cp| dp = ∫ p 0 √ x2p + y 2 p dp, (1)
where xp = dxdp , and yp = dy dp and the principal invariant signature, that is the Euclidean curvature is given by
κ(p) = det(Cp, Cpp) |Cp|3 = xpypp − ypxpp
(x2p + y 2 p)
3 2
. (2)
Thus, we have the Euclidean differential invariant signatures given by the set {κ, κs, κss ...} for every point on the curve. Cartan’s theorem provides an axiomatic construction of invariant signatures and the uniqueness property of the theorem guarantees their theoretical validity. Their importance is highlighted from the fact that any invariant is a function of the fundamental differential invariants.
The difficulty with differential invariants is their stable numerical computation. Equations 1 and 2, involve non-linear functions of derivatives of the curve and this poses serious numerical issues for their practical implementation where noise and poor sampling techniques are involved. Apart from methods like Pajdla & Van Gool (1995) and Weiss (1993), numerical considerations motivated the development of multi-scale representations. These methods used alternative constructions of invariant signatures which were robust to noise. More importantly, they allowed a hierarchical representation, in which the strongest and the most global components of variation in the contour of the curve are encoded in signatures of higher scale, and as we go lower, the more localized and rapid changes get injected into the representation. Two principal methods in this category are scale-space methods and integral invariants. In scale-space methods (Mokhtarian & Mackworth (1992); Sapiro & Tannenbaum (1995); Bruckstein et al. (1996)), the curve is subjected to an invariant evolution process where it can be evolved to different levels of abstraction. See Figure 5. The curvature function
at each evolved time t is then recorded as an invariant. For example, {κ(s, t), κs(s, t), κss(s, t)...} would be the Euclidean-invariant representations at scale t.
Integral invariants (Manay et al. (2004); Fidler et al. (2008); Pottmann et al. (2009); Hong & Soatto (2015)) are invariant signatures which compute integral measures along the curve. For example, for each point on the contour, the integral area invariant computes the area of the region obtained from the intersection of a ball of radius r placed at that point and the interior of the contour. The integral nature of the computation gives the signature robustness to noise and by adjusting different radii of the ball r one can associate a scale-space of responses for this invariant. Fidler et al. (2008) and Pottmann et al. (2009) provide a detailed treatise on different types of integral invariants and their properties.
It is easy to observe that differential and integral invariants can be thought of as being obtained from non-linear operations of convolution filters. The construction of differential invariants employ filters for which the action is equivalent to numerical differentiation (high pass filtering) whereas integral invariants use filters which act like numerical integrators (low pass filtering) for stabilizing the invariant. This provides a motivation to adopt a learning based approach and we demonstrate that the process of estimating these filters and functions can be outsourced to a learning framework. We use the Siamese configuration for implementing this idea. Such configurations have been used in signature verification (Bromley et al. (1993)), face-verification and recognition(Sun et al. (2014); Taigman et al. (2014); Hu et al. (2014)), metric learning (Chopra et al. (2005)), image descriptors (Carlevaris-Bianco & Eustice (2014)), dimensionality reduction (Hadsell et al. (2006)) and also for generating 3D shape descriptors for correspondence and retrieval (Masci et al. (2015); Xie et al. (2015)). In these papers, the goal was to learn the descriptor and hence the similarity metric from data using notions of only positive and negative examples. We use the same framework for estimation of geometric invariants. However, in contrast to these methods, we contribute an analysis of the output descriptor and provide a geometric context to the learning process. The contrastive loss function driving the training ensures that the network chooses filters which push and pull different features of the curve into the invariant by balancing a mix of robustness and fidelity.
3 TRAINING FOR INVARIANCE
A planar curve can be represented either explicitly by sampling points on the curve or using an implicit representation such as level sets (Kimmel (2012)). We work with an explicit representation of simple curves (open or closed) with random variable sampling of the points along the curve. Thus, every curve is a N × 2 array denoting the X and Y coordinates of the N points. We build a convolutional neural network which inputs a curve and outputs a representation or signature for every point on the curve. We can interpret this architecture as an algorithmic scheme of representing a function over the curve. However feeding in a single curve is insufficient and instead we run this convolutional architecture in a Siamese configuration (Figure 2) that accepts a curve and a
transformed version (positive) of the curve or an unrelated curve (negative). By using two identical copies of the same network sharing weights to process these two curves we are able to extract geometric invariance by using a loss function to require that the two arms of the Siamese configuration must produce values that are minimally different for curves which are related by Euclidean transformations representing positive examples and maximally different for carefully constructed negative examples. To fully enable training of our network we build a large dataset comprising of positive and negative examples of the relevant transformations from a database of curves. We choose to minimize the contrastive loss between the two outputs of the Siamese network as this directs the network architecture to model a function over the curve which is invariant to the transformation.
3.1 LOSS FUNCTION
We employ the contrastive loss function (Chopra et al. (2005); LeCun et al. (2006)) for training our network. The Siamese configuration comprises of two identical networks of Figure 3 computing signatures for two separate inputs of data. Associated to each input pair is a label which indicates whether or not that pair is a positive (λ = 1) or a negative (λ = 0) example (Figure 2). Let C1i and C2i be the curves imputed to first and second arm of the configuration for the ith example of the data with label λi. Let SΘ(C) denote the output of the network for a given set of weights Θ for input curve C. The contrastive loss function is given by:
C(Θ) = 1 N { i=N∑ i=1 λi || SΘ(C1i)−SΘ(C2i) || + (1−λi) max( 0, µ − || SΘ(C1i)−SΘ(C2i) || ) } , (3) where µ is a cross validated hyper-parameter known as margin which defines the metric threshold beyond which negative examples are penalized.
3.2 ARCHITECTURE
The network inputs a N × 2 array representing the coordinates of N points along the curve. The sequential nature of the curves and the mostly 1D-convolution operations can also be looked at from the point of view of temporal signals using recurrent neural network architectures. Here however we choose instead to use a multistage CNN pipeline. The network, given by one arm of the Siamese configuration, comprises of three stages that use layer units which are typically considered the basic building blocks of modern CNN architectures. Each stage contains two sequential batches of convolutions appended with rectified linear units (ReLU) and ending with a max unit. The convolutional unit comprises of convolutions with 15 filters of width 5 as depicted in Figure 3. The max unit computes the maximum of 15 responses per point to yield an intermediate output after each stage. The final stage is followed by a linear layer which linearly combines the responses to yield the final output. Since, every iteration of convolution results in a reduction of the sequence length, sufficient padding is provided on both ends of the curve. This ensures that the value of the signature at a point is the result of the response of the computation resulting from the filter centered around that point.
3.3 BUILDING REPRESENTATIVE DATASETS AND IMPLEMENTATION
In order to train for invariance, we need to build a dataset with two major attributes: First, it needs to contain a large number of examples of the transformation and second, the curves involved in the training need to have sufficient richness in terms of different patterns of sharp edges, corners, smoothness, noise and sampling factors to ensure sufficient generalizability of the model. To sufficiently span the space of Euclidean transformations, we generate random two dimensional rotations by uniformly sampling angles from [−π, π]. The curves are normalized by removing the mean and dividing by the standard deviation thereby achieving invariance to translations and uniform scaling. The contours are extracted from the shapes of the MPEG7 Database (Latecki et al. (2000)) as shown in first part of Figure 4. It comprises a total of 1400 shapes containing 70 different categories of objects. 700 of the total were used for training and 350 each for testing and validation. The positive examples are constructed by taking a curve and randomly transforming it by a rotation, translation and reflection and pairing them together. The negative examples are obtained by pairing curves which are deemed dissimilar as explained in Section 4. The contours are extracted and each contour is sub-sampled to 500 points. We build the training dataset of 10, 000 examples with approximately 50% each for the positive and negative examples. The network and training is performed using the Torch library Collobert et al. (2002). We trained using Adagrad Duchi et al. (2011) at a learning rate of 5 × 10−4 and a batch size of 10. We set the contrastive loss hyperparameter margin µ = 1 and Figure 4 shows the error plot for training and the convergence of the loss to a minimum. The rest of this work describes how we can observe and extend the efficacy of the trained network on new data.
4 MULTI-SCALE REPRESENTATIONS
Invariant representations at varying levels of abstraction have a theoretical interest as well as practical importance to them. Enumeration at different scales enables a hierarchical method of analysis which is useful when there is noise and hence stability is desired in the invariant. As mentioned in Section 2, the invariants constructed from scale-space methods and integral invariants, naturally allow for such a decomposition by construction.
A valuable insight for multi-scale representations is provided in the theorems of Gage, Hamilton and Grayson (Gage & Hamilton (1986); Grayson (1987)). It says that if we evolve any smooth nonintersecting planar curve with mean curvature flow, which is invariant to Euclidean transformations, it will ultimately converge into a circle before vanishing into a point. The curvature corresponding to this evolution follows a profile as shown in Figure 5, going from a possibly noisy descriptive feature to a constant function. In our framework, we observe an analogous behavior in a data-dependent setting. The positive part of the loss function (λ = 1) forces the network to push the outputs of the positive examples closer, whereas the negative part (λ = 0) forces the weights of network to push the outputs of the negative examples apart, beyond the distance barrier of µ. If the training data does not contain any negative example, it is easy to see that the weights of the network will converge to a point which will yield a constant output that trivially minimizes the loss function in Equation 3.
Curvature: κ
Figure 5: Curve evolution and the corresponding curvature profile.
1 2 3 4 5
1 2 3 4 5
Figure 6: Experiments with multi-scale representations. Each signature is the output of a network trained on a dataset with training examples formed as per the rows of Table 1. Index1 indicates low and 5 indicates a higher level of abstraction.
This is analogous to that point in curvature flow which yields a circle and therefore has a constant curvature.
Designing the negative examples of the training data provides the means to obtain a multi-scale representation. Since we are training for a local descriptor of a curve, that is, a function whose value at a point depends only on its local neighborhood, a negative example must pair curves such that corresponding points on each curve must have different local neighborhoods. One such possibility is to construct negative examples which pair curves with their smoothed or evolved versions as in Table 1. Minimizing the loss function in equation 3 would lead to an action which pushes apart the signatures of the curve and its evolved or smoothed counterpart, thereby injecting the signature with fidelity and descriptiveness. We construct separate data-sets where the negative examples are drawn as shown in the rows of Table1 and train a network model for each of them using the loss function 3. In our experiments we perform smoothing by using a local polynomial regression with weighted linear least squares for obtaining the evolved contour. Figure 6 shows the outputs of these different networks which demonstrate a scale-space like behavior.
5 EXPERIMENTS AND DISCUSSION
Ability to handle low signal to noise ratios and efficiency of computation are typical qualities desired in a geometric invariant. To test the numerical stability and robustness of the invariant signatures
we designed two experiments. In the first experiment, we add increasing levels of zero-mean Gaussian noise to the curve and compare the three types of signatures: differential (Euclidean curvature), integral (integral area invariant) and the output of our network (henceforth termed as network invariant) as shown in Figure 7. Apart from adding noise, we also rotate the curve to obtain a better assessment of the Euclidean invariance property. In Figure 8, we test descriptiveness of the signature under noisy conditions in a shape retrieval task for a set of 30 shapes with 6 different categories. For every curve, we generate 5 signatures at different scales for the integral and the network invariant and use them as a representation for that shape. We use the Hausdorff distance as a distance measure (Bronstein et al. (2008)) between the two sets of signatures to rank the shapes for retrieval. Figure 7 and 8 demonstrate the robustness of the network especially at high noise levels.
In the second experiment, we decimate a high resolution contour at successive resolutions by randomly sub-sampling and redistributing a set of its points (marked blue in Figure 9) and observe the signatures at certain fixed points (marked red in Figure 9) on the curve. Figure 9 shows that the network is able to handle these changes in sampling and compares well with the integral invariant. Figures 7 and Figure 9 represent behavior of geometric signatures for two different tests: large noise for a moderate strength of signal and low signal for a moderate level of noise.
6 CONCLUSION
We have demonstrated a method to learn geometric invariants of planar curves. Using just positive and negative examples of Euclidean transformations, we showed that a convolutional neural network
70% 50% 30%
20% 10% 5%
0 10 20 30 40 50 60
is able to effectively discover and encode transform-invariant properties of curves while remaining numerically robust in the face of noise. By using a geometric context to the training process we were able to develop novel multi-scale representations from a learning based approach without explicitly
enforcing such behavior. As compared to a more axiomatic framework of modeling with differential geometry and engineering with numerical analysis, we demonstrated a way of replacing this pipeline with a deep learning framework which combines both these aspects. The non-specific nature of this framework can be seen as providing the groundwork for future deep learning data based problems in differential geometry.
ACKNOWLEDGMENTS
This project has received funding from the European Research Council (ERC) under the European Unions Horizon 2020 research and innovation program (grant agreement No 664800)

Authors show that a contrastive loss for a Siamese architecture can be used for learning representations for planar curves. With the proposed framework, authors are able to learn a representation which is comparable to traditional differential or integral invariants, as evaluated on few toy examples.
The paper is generally well written and shows an interesting application of the Siamese architecture. However, the experimental evaluation and the results show that these are rather preliminary results as not many of the choices are validated. My biggest concern is in the choice of the negative samples, as the network basically learns only to distinguish between shapes at different scales, instead of recognizing different shapes. It is well known fact that in order to achieve a good performance with the contrastive loss, one has to be careful about the hard negative sampling, as using too easy negatives may lead to inferior results. Thus, this may be the underlying reason for such choice of the negatives? Unfortunately, this is not discussed in the paper.
Furthermore the paper misses a more thorough quantitative evaluation and concentrates more on showing particular examples, instead of measuring more robust statistics over multiple curves (invariance to noise and sampling artifacts).
In general, the paper shows interesting first steps in this direction, however it is not clear whether the experimental section is strong and thorough enough for the ICLR conference. Also the novelty of the proposed idea is limited as Siamese networks are used for many years and this work only shows that they can be applied to a different task.


We propose a learning method to quantify human intention. Generally, a human being will imagine several potential actions for a given scene, but only one of these actions will subsequently be taken. This makes it difficult to quantify human intentions. To solve this problem, we apply competitive learning to human behavior prediction as supervised learning. In our approach, competitive learning generates several outputs that are then associated with several potential situations imagined by a human. We applied the proposed method to human driving behavior and extracted three potential driving patterns. Results showed a squared error is reduced to 1/25 that of a conventional method . We also found that competitive learning can distinguish valid data from disturbance data in order to train a model.
We propose a learning method to quantify human intention. Generally, a human being will imagine several potential actions for a given scene, but only one of these actions will subsequently be taken. This makes it difficult to quantify human intentions.
To solve this problem, we apply competitive learning to human behavior prediction as supervised learning. In our approach, competitive learning generates several outputs that are then associated with several potential situations imagined by a human. We applied the proposed method to human driving behavior and extracted three potential driving patterns. Results showed a squared error is reduced to 1/25 that of a conventional method . We also found that competitive learning can distinguish valid data from disturbance data in order to train a model.
1 INTRODUCTION
Progress in advanced driving assistance systems (ADASs) has led to an autonomous driving function applicable for highway driving . The vehicle control logic of ADAS is typically developed by hand. There are a few related works on controlling vehicles comfortably , but the complex nature of vehicle environments prevents development by hand. Therefore, recent research has focused on the application of machine learning to autonomous driving systems. The convolutional neural network (CNN) shows particular promise as a core algorithm (Krizhevsky et al. (2012)). C. Chen & Xiao (2015) estimates affordance for driving directly from a front camera image. They train a CNN to generate key perception indicators from images to easily control a vehicle. Similarly, Bojarski et al. (2016) predict the desired steering command directly from front camera images. They train a CNN with a human command as training data and drive in traffic on local roads with or without lane marking and on highways.
Our objective is to provide drivers with a comfortable trip featuring automatic vehicle control . The above methods provide a uniform control for various scenarios, but drivers have an assortment of preference behaviors, and individuals may drive in different ways even if the scenario or situation is the same. For example, on the highway, one person might drive hard to arrive at a destination quickly while another might drive relatively slowly to arrive at a destination safely. Therefore, we want to provide autonomous driving adapted to each individual to improve ease of driving and to generate control targets that imitate individual behavior. To imitate individual behavior, we predict future vehicle states resulting from an individualfs decision.
There are many studies on the imitation of human behavior within the context of driving. Ma & Andréasson (2006) proposed a vehicle interaction model that predicts future acceleration on the basis of current acceleration, velocity, relative velocity, and relative distance of the preceding vehicle. Moon & Choi (2011) proposed a method to predict human steering behavior. Their model considers path planning, feed-forward steering, and feedback steering. While these methods consider only acceleration or steering, Gindele et al. (2010) proposed a more complex model, the dynamic Bayesian network model, to estimate driver behavior and vehicle trajectory. This model considers vehicle model, trajectory, driver decision, and situation context. Wada et al. (2007) pro-
posed a method to predict braking behavior. This method monitors the size of a preceding vehicle by means of front camera images to detect collision risk.
These methods provide one output for one scene. However, we assume that one output might represent just one part of a driverfs behavior. We assume a driver will imagine several potential situations for one scene and have several potential actions to take associated with each situation. This means the driver is making decisions all the time. Therefore, we want to extract several of the driver’s candidate actions or intentions for the autonomous control of a vehicle while considering various potential situations.
In this paper, we propose a method to extract a driver’s potential intention by utilizing competitive learning (Ahalt et al. (1990), Shinozaki & Naruse (2013), Osoba & Kosko (2013)). Originally, competitive learning was used for unsupervised neural network algorithms and functioned as clustering. In such competitive learning, neural networks have several output layers and only one path that outputs the smallest loss to be trained . Consequently, neural networks predict the cluster with the most activated units .
Ahalt et al. (1990) compare several competitive learning algorithms. In this paper, competitive learning algorithms is dealt as a kind of vector quantization algorithms. Shinozaki & Naruse (2013) utilize competitive learning for pre-training of deep neural networks and they also use competitive learning as unsupervised learning. Osoba & Kosko (2013) consider supervised competitive learning. However, they use also competitive learning as a kind of clustering methods. Therefore, in our best knowledge, competitive learning is dealt as a kind of unsupervised learning algorithms or clustering algorithms.
We apply competitive learning to time series supervised learning, especially, regression task. And we train neural networks to output several patterns associated with a driver’s potential intentions. In this paper, we describe how to model the driving behavior with competitive learning architecture . Additionally, we show the experimental results of tests performed with an actual vehicle.
2 DRIVER BEHAVIOR MODEL
Here, we describe how to model driving behavior and how to implement the neural network architecture.
Our driving behavior model is a dynamic system that includes both a driver and a vehicle system. In this model, the driver observes the environment and then decides on a driving action, e.g., steering or pedal operation. The vehicle system then changes its state depending on the driving action and the state of the vehicle at a previous time. Finally, we observe the various vehicle states. For example, the environment observed by the driver is the same as that shown by the front camera and the vehicle states include travel speed, accelerations, gas pedal position, brake pedal position, engine speed, engine load, engine temperature, fuel level, fuel temperature, cooling water temperature, etc.
A graphical model of driving behavior is shown in Fig. 1. Here, e stands for the environment observed by the driver, d stands for the driving action decided by the driver, s stands for vehicle states,
and x stands for observed vehicle states. Subscripts t − 1 and t refer to time steps. Environment e and parts of vehicle states x are observed. Driving action d and whole vehicle states s are latent variables. We assume a front camera view for environment e. Observed variables x include travel speed, gas pedal position, engine speed, and engine load.
To ensure comfortable driving, we estimate the driver’s intention by predicting vehicle states in k step future . The relations between each variable are shown in the following equations.
dt = fe(et; θe) (1)
st = fs(st−1; θs) + fd(, dt; θd) (2)
xt = fx(st; θx) (3)
xt+k = fx+k(st; θx+k) (4)
Additionally, we compensate for latent variable st by observable variable xt. The compensated st are shown in Eq. 5:
st = fs(st−1; θs) + fd(, dt; θd) + f −1 x (xt; θ −1 x (5)
We design the neural network architecture in accordance with the driving behavior model shown in Fig. 1. The designed architecture is shown in Fig. 2. First, we select convolutional neural network (CNN) layers to approximate driving action fe, as the driver decides which action to take depending on the front view and CNN is the best layer to process images. The front camera images are resized to 256×256 with RGB channels. Second, we select a recurrent neural network (RNN) layer to approximate vehicle states fs, as our driver model is a dynamic system (Graves (2013)). Finally, we select a fully connected layer for other layers fd, f−1x , fx+k because of its usability. This model predicts observed variables and we train the driver behavior model by loss function. We select squared error as loss function L(xt+k, x̂t+k) and update parameters by backpropagation (Hecht-NielsenWerbos (1990)).
L(xt+k, x̂t+k) = |xt+k − x̂t+k|2 (6)
The CNN consists of three layers. In the first layer, kernel size is 11×11 with 64 channels and stride 4 . In the second layer, kernel size is 5×5 with 128 channels. In the third layer, kernel size is 3×3 with 128 channels. After all CNN layers, we apply batch normalization, ReLU, and max pooling (Ioffe & Szegedy (2015)). The RNN layer has 1024 units. We use the ADAM algorithm for optimization (Kingma & Ba (2014)).
The architecture shown in Fig. 2 is a baseline architecture and cannot extract potential driver intentions. In the next section, we introduce a competitive learning architecture that can extract potential intentions and discuss how to train it.
3 COMPETITIVE LEARNING FOR DRIVER BEHAVIOR MODEL
3.1 COMPETITIVE LEARNING ARCHITECTURE
The competitive learning architecture is shown in Fig. 3. It has No output layers. Additionally, we set up No RNN layers, as the driver’s potential intentions are separated depending on the driving actions . Therefore, a separated RNN will be affected by separated intentions. The i-th RNN i and the output layer generate i-th prediction x̂it+k at time t+k. We calculate i-th loss using Eq. 7 and decide the loss for backpropagation using Eq. 9. Finally, we update the parameters depending on this loss for backpropagation.
li = L(xt+k, x̂ i t+k) (7)
L = [l1, ...li, ...lNo ] (8)
libp =  0i ̸= arg min j (L)
lii = arg min j
(L) (9)
Backpropagation in the competitive learning architecture is shown in Fig. 4. In the competitive learning architecture, we train only the computation path that output minimum loss. In Fig. 4, the computation path to be trained is indicated by solid lines and the layer whose parameters are not updated is indicated by dashed lines. In this case, the i-th output layer has minimum loss and is trained. The other output layers (i.e., whose losses are bigger than i-th) are not trained and the losses are compensated as zero. Since each output layer is trained by different data, each prediction reflects different intentions. Therefore, competitive learning can extract several potential intentions.
3.2 PRE-TRAINING FOR COMPETITIVE LEARNING
Here, we discuss the pre-training for competitive learning. In our experiments, competitive layers with simple initialization are difficult to train, and many experiments result in only one output layer being trained. Since only one computation path is trained at a time in competitive learning, how the trained path is decided depends on the initialization. Therefore, we need to ensure appropriate initialization for competitive layers. To this end , we adopt a pre-training technique for competitive learning (Erhan et al. (2010), Erhan et al.). Pre-training is an initialization technique for neural networks and is typically used for stacking neural networks. In this work, we apply pre-training to initialize the parallel competitive layer, as shown in Fig. 5. First, we train only one output layer by means of the ordinal loss function in Eq. 6. Then, we initialize competitive layers by copying pre-trained parameters. Finally, we train each output layer as fine-tuning.
4 EXPERIMENTAL RESULTS
In order to evaluate the proposed method, we record the front camera view using a smartphone and collect actual vehicle data using OBD2 (Birnbaum & Truglia (2000), International (2003)). The data on vehicle states are whitened by removing means and scaled variances. Front camera images and vehicle data are resampled to 2 Hz. Six steps (equivalent to three seconds in the future) are predicted. We collect 80 minutes of data and use 40 minutes to train the neural networks. We set three competitive layers and train 300 iterations for pre-training and 2000 iterations for fine-tuning. We train neural networks with TITAN Z and use the Chainer framework (Tokui et al. (2015)).
4.1 COMPARISON OF COMPETITIVE LEARNING ARCHITECTURE WITH BASELINE ARCHITECTURE
Before we show the competitive learning results, we present the results of the baseline architecture shown in Fig. 6 . These data are predictions of vehicle speed over ten minutes. The left figure is the prediction on training data and the right one is on test data. Red line indicates the measured speed and blue line indicates the prediction by baseline architecture. In the training data, the baseline architecture could predict accurately in the low speed band under 30 km/h. However, the prediction error became bigger in the middle-high speed band over 30 km/h. In the higher speed band, the driverfs action had several variations even when the vehicle was operating in the same environment. This variation made prediction difficult. In the test data prediction, prediction error became bigger in not only the high speed band but also the very low speed band around 0 km/h . Additionally, the timing of the acceleration or deceleration shifted later, too. This is because the baseline architecture cannot extract potential driving intentions.
The competitive learning results are shown in Fig. 7. This figure is written up the same way as Fig. 6, except the blue line refers to integrated predictions by the competitive learning architecture. These predictions are a combination of three outputs derived by selecting minimum loss. In the training data prediction, the competitive learning architecture predicted a value quite close to the
measured vehicle speed before three seconds . In the test data prediction, the competitive learning architecture could also predict the speed accurately. Additionally, the timing shift of acceleration or deceleration became smaller. The loss summation in both the training and test data is listed in Table 1. Competitive learning architecture loss was about 1/25 smaller in the training data and about 1/3 smaller in the test data compared with the baseline architecture.
4.2 COMPETITIVE LEARNING PREDICTION ASSOCIATED WITH POTENTIAL INTENTIONS
Here, we show the driving intentions extracted using the competitive learning architecture. The intentions extracted from the training data are shown in Fig. 8. Four time series data are included: upper left is the integrated prediction, upper right is the first output layer’s prediction (indicated by green line), lower left is the second output prediction (blue line), and lower right is the third output prediction (purple line). The red lines are the measured vehicle speed. The upper left figure depicts the winning output prediction, where the minimum error prediction is output for each time and the line color refers to the winning output layer . For example, the purple line in the upper left figure is the prediction by the third output layer . We can see that each output layer is affected by other driving intentions. The first output layer accurately predicted the stop timing associated with the stop intention, the second output layer accurately predicted the acceleration or high speed band associated with a rapid intention, and the third output layer accurately predicted the deceleration timing associated with a careful intention. We also show the results of the test data in Fig. 9, which are written the same as the training data in Fig. 8. Trained intentions are also valid in test data because the first output was accurate at stop time, the second output was accurate at acceleration or the high speed band, and the third output was accurate at the time of deceleration. Therefore, extracting these potential intentions enables accurate prediction.
4.3 EFFECT OF PRE-TRAINING
The competitive learning result without pre-training is shown in Fig. 10. We can train ”only one” output layer to predict driver intention even if we have three output layers. Potential intentions of the driver have similar trends. Therefore, we require the initialization of several of the same output layers . We can extract potential intentions by training each output layer using other data.
We show the summation loss in Table 2. Surprisingly, competitive learning without pre-training significantly improved its losses by about 1/2 for training data and about 1/3 for test data. We consider the second and third output layers to play an important role, even though they cannot learn potential intentions. There is usually a lot of inadequate data to train, and these data disrupt the training of the model. However, in most cases we cannot distinguish useful data from disturbance data. In the competitive learning architecture, each output layer automatically selects data to train its computation path. This selection might distinguish useful data from disturbance data, and the first output layer can be trained using just the useful data, even though other output layers are trained using inadequate data. In this way, competitive learning can train the model such that it is robust against noisy data.
5 CONCLUSION
We proposed supervised competitive learning to imitate a driver’s potential intentions. Competitive learning was applied to supervised learning and the squared error was reduced to 1/25 that of a conventional method. We also demonstrated that competitive learning can distinguish valid data from disturbance data to train a model.

This paper proposes a neural network architecture for car state prediction while driving based on competitive learning. Competitive learning creates several duplicates of the baseline neural architecture and during training only updates the architecture with minimum loss. The experiments compare the competitive learning approach to a single baseline architecture on a driving benchmark task. The paper is understandable but could benefit from some copy editing.
The competitive learning approach seems rather adhoc and this paper feels quite incomplete without significant discussion and comparisons to ensembling. Much recent work has shown that duplicating and ensembling neural architectures can produce gains, and it’s not clear why competitive learning is better than ensembling, it seems less theoretically sound to me.
There is a huge confound in the experiments due to the competitive learning architecture having many more free parameters than the baseline architecture. Again I think comparing to ensembling with the same number of architectures duplicated and perhaps comparing to a single baseline with larger hidden layers to make the total number of free parameters comparable is critical to validating the proposed approach.
The graphical model of the driving process depicted in figure 1 seems nonsensical. If e is observed then all variables are known given the dependencies shown. Further, it is at best very poor notation to say that the driving action d decided at time t affects the vehicle state s at that same time. It should be that s_t depends on d_(t-1). Also, according to this figure the driving decision d does not depend on the observed vehicle state x which also seems invalid.
Odd to have a paragraph break in abstract
Figure 1 caption should include a brief explanation of the variables shown