One of the fundamental tasks in understanding genomics is the problem of predicting Transcription Factor Binding Sites (TFBSs). With more than hundreds of Transcription Factors (TFs) as labels, genomic-sequence based TFBS prediction is a challenging multi-label classification task. There are two major biological mechanisms for TF binding: (1) sequence-specific binding patterns on genomes known as "motifs" and (2) interactions among TFs known as co-binding effects. In this paper, we propose a novel deep architecture, the Prototype Matching Network (PMN) to mimic the TF binding mechanisms. Our PMN model automatically extracts prototypes ("motif"-like features) for each TF through a novel prototypematching loss. Borrowing ideas from few-shot matching models, we use the notion of support set of prototypes and an LSTM to learn how TFs interact and bind to genomic sequences. On a reference TFBS dataset with 2.1 million genomic sequences, the PMN significantly outperforms baselines and validates our design choices empirically. To our knowledge, this is the first deep learning architecture that introduces prototype learning and considers TF-TF interactions for large scale TFBS prediction. Not only is the proposed architecture accurate, but it also models the underlying biology.
Summary
This paper proposes a prototype matching network (PMN) to model transcription factor (TF) binding motifs and TF-TF interactions for large scale transcription factor binding site prediction task. They utilize the idea of having a support set of prototypes (motif-like features) and an LSTM from the few shot learning framework to develop this prototype matching network. The input is genomic sequences from 14% of the human genome, each sequence in the dataset is bound by at least one TF. First a Convolutional Neural Network with three convolutional layers is trained to predict single/multiple TF binding. The output of the last hidden layer before sigmoid transformation is used as the LSTM input. A weighted sum of similarity score (sigmoid of cosine similarity, similar to attention mechanism of LSTMs) along with prototype vectors are used to update the read vector. The final output is a sigmoid of the final hidden state concatenated with the read vector. The loss function used is difference of a cross-entropy loss function and a lambda weighted prototype loss function. The latter is the mean square error between the output label and the similarity score.  The authors compare the PMN with different lambda values with CNN with single/multi-label and see marginal improvement in auROC, auPR and Recall at 50% FDR with the PWM. To test that PWN finds biologically relevant TF interactions, the authors perform hierarchical clustering on the prototypes of 86 TFs and compare the clusters found to the known co-regulators from the TRRUST database and find 6 significant clusters. 
Pros:
1. The authors utilize the idea of prototypes and few shot learning to the task of TF-binding and cooperation. 
2. Attention LSTMs are used to model label interactions. 
Just like CNN can be related to discriminative training of PSSM or PWM, the above points demonstrate nicely how ideas/concepts from the recent developments in DL can be adopted/relate (and possibly improve on) to  similar generative modeling approaches used in the past  for learning cooperative TF binding.
Cons:
1. Authors do not compare their model’s performance to the previously published TF binding prediction algorithms (DeepBind, DeepSEA). 
2. The authors miss important context and make some inaccurate statements: TF do not just “control if a gene is expressed or not” (p.1). It’s not true that previous DL works did not consider co-binding. Works such as DeepSea combined many filters which can capture cooperative binding to define which sequence is “regulated”. It is true this or DeepBind did not construct a structure a structure over those as learned by an LSTM. The authors do point out a model that does add LSTM (Quang and Xie) but then do not compare to it and make a vague claim about it modeling interactions between features but not labels (p. 6 top). Comparing to it and directly to DeepSee/Bind seems crucial to claim improvements on previous works. Furthermore, the authors acknowledge the existence of vast literature on this specific problem but completely discard it as “loose connection to our TFBS formulation”. In reality though, many works in that area are highly relevant and should be discussed in the context of what the authors are trying to achieve. For example, numerous works by Prof. Saurabh Sinha have focused specifically on joint TF modeling (e.g. Kazemian NAR 2011, He Plos One 2009, Ivan Gen Bio 2008, MORPH Plos Comp Bio 2007). In general, trying to lay claims about significant contributions to a problem, as stated here by the authors, while completely disregarding previous work simply because it’s not in a DL framework (which the authors are clearly more familiar with) can easily alienate reviewers and readers alike.  
3. The learning setup seems problematic:
3a. The model may overfit for the genomic sequences that contain TF binding sites as it has never seen genomic sequences without TF binding sites (the genomic sequences that don’t have CHIP peaks are discarded from the dataset). Performance for genome wide scans should definitely include those to assess accuracy.
3b. The train/validation/test are defined by chromosome. There does not seem to be any screening for sequence similarity (e.g. repetitive sequences, paralogs). This may inflate performance, especially for more complicated models which may be able to “memorize” sequences better. 
4. The paper claims to have 4 major contributions. The details of second claim that the prototype matching loss learns motif like features is not explained anywhere in the paper. If we look at the actual loss function equation (12), it penalizes the difference between the label and the similarity score but the prototypes are not updated. The fourth claim about the biological relevance of the network is not sufficiently explored. The authors show that it learns co-bindings already known in the literature which is a good sanity check but does not offer any new biological insight.  The actual motifs or the structure of their relations is not shown or explored.
5. PWN offers only marginal improvement over the CNN networks