Intelligible Language Modeling with Input Switched Affine Networks
The computational mechanisms by which nonlinear recurrent neural networks (RNNs) achieve their goals remains an open question. There exist many problem domains where intelligibility of the network model is crucial for deployment. Here we introduce a recurrent architecture composed of input-switched affine transformations, in other words an RNN without any nonlinearity and with one set of weights per input. We show that this architecture achieves near identical performance to traditional architectures on language modeling of Wikipedia text, for the same number of model parameters. It can obtain this performance with the potential for computational speedup compared to existing methods, by precomputing the composed affine transformations corresponding to longer input sequences. As our architecture is affine, we are able to understand the mechanisms by which it functions using linear methods. For example, we show how the network linearly combines contributions from the past to make predictions at the current time step. We show how representations for words can be combined in order to understand how context is transferred across word boundaries. Finally, we demonstrate how the system can be executed and analyzed in arbitrary bases to aid understanding.
Summary: The authors propose an input switched affine network to do character-level language modeling, a kind of RNN without pointwise nonlinearity, but with switching the transition matrix & bias based on the input character. This is motivated by intelligibility, since it allows decomposition of output contribution into these kappa_s^t terms, and use of basic linear algebra to probe the network. Regarding myself as a reviewer, I am quite sure I understood the main ideas and arguments of this paper, but am not an expert on RNN language models or intelligibility/interpretability in ML. I did not read any papers with a similar premise - closest related work I*m familiar with would be deconvnet for insight into vision-CNNs. PRO: I think this is original and novel work. This work is high quality, well written, and clearly is the result of a lot of work. I found section 4.5 about projecting into readout subspace vs *computational* subspace most interesting and meaningful. CON: + The main hesitation I have is that the results on both parts (ISAN model, and analysis of it) are not entirely convincing: (1) ISAN is only trained on small task (text8), not clear whether it can be a strong char-LM on larger scale tasks, (2) nor do the analysis sections provide all that much real insight in the learned network. (1b) Other caveat towards ISAN architecture: this model in its proposed form is really only fit for small-vocabulary (i.e. character-based) language modeling, not a general RNN with large-vocab discrete input nor continuous input. (2a) For analysis: many cute plots and fun ideas of quantities to look at, but not much concrete insights. (2b) Not very clear which analysis is specific to the ISAN model, and which ideas will generalize to general nonlinear RNNs. (2c) Re sec 4.2 - 4.3: It seems that the quantity kappa_s^t on which analysis rests, isn*t all that meaningful. Elaborating a bit on what I wrote in the question: For example: Fig 2, for input letter *u* in revenue, there*s a red spot where *_* character massively positively impacts the logit of *e*. This seems quite meaningless, what would be the meaning of influence of *_* character? So it looks ot me that the switching matrix W_u (and prior W_n W_e etc) are using previous state in an interesting way to produce that following e. So that metric kappa_s^t just doesn*t seem very meaningful. This remark relates to the last paragraph of Sec4.2. Even though the list of cons here is longer than pro*s, I recommend accept; specifically because the originality of this work will in any case make it more vulnerable to critiques. This work is well-motivated, very well-executed, and can inspire many more interesting investigations along these lines.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Intelligible Language Modeling with Input Switched Affine Networks
The computational mechanisms by which nonlinear recurrent neural networks (RNNs) achieve their goals remains an open question. There exist many problem domains where intelligibility of the network model is crucial for deployment. Here we introduce a recurrent architecture composed of input-switched affine transformations, in other words an RNN without any nonlinearity and with one set of weights per input. We show that this architecture achieves near identical performance to traditional architectures on language modeling of Wikipedia text, for the same number of model parameters. It can obtain this performance with the potential for computational speedup compared to existing methods, by precomputing the composed affine transformations corresponding to longer input sequences. As our architecture is affine, we are able to understand the mechanisms by which it functions using linear methods. For example, we show how the network linearly combines contributions from the past to make predictions at the current time step. We show how representations for words can be combined in order to understand how context is transferred across word boundaries. Finally, we demonstrate how the system can be executed and analyzed in arbitrary bases to aid understanding.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



Summary: The authors propose an input switched affine network to do character-level language modeling, a kind of RNN without pointwise nonlinearity, but with switching the transition matrix & bias based on the input character. This is motivated by intelligibility, since it allows decomposition of output contribution into these kappa_s^t terms, and use of basic linear algebra to probe the network. Regarding myself as a reviewer, I am quite sure I understood the main ideas and arguments of this paper, but am not an expert on RNN language models or intelligibility/interpretability in ML. I did not read any papers with a similar premise - closest related work I*m familiar with would be deconvnet for insight into vision-CNNs. PRO: I think this is original and novel work. This work is high quality, well written, and clearly is the result of a lot of work. I found section 4.5 about projecting into readout subspace vs *computational* subspace most interesting and meaningful. CON: + The main hesitation I have is that the results on both parts (ISAN model, and analysis of it) are not entirely convincing: (1) ISAN is only trained on small task (text8), not clear whether it can be a strong char-LM on larger scale tasks, (2) nor do the analysis sections provide all that much real insight in the learned network. (1b) Other caveat towards ISAN architecture: this model in its proposed form is really only fit for small-vocabulary (i.e. character-based) language modeling, not a general RNN with large-vocab discrete input nor continuous input. (2a) For analysis: many cute plots and fun ideas of quantities to look at, but not much concrete insights. (2b) Not very clear which analysis is specific to the ISAN model, and which ideas will generalize to general nonlinear RNNs. (2c) Re sec 4.2 - 4.3: It seems that the quantity kappa_s^t on which analysis rests, isn*t all that meaningful. Elaborating a bit on what I wrote in the question: For example: Fig 2, for input letter *u* in revenue, there*s a red spot where *_* character massively positively impacts the logit of *e*. This seems quite meaningless, what would be the meaning of influence of *_* character? So it looks ot me that the switching matrix W_u (and prior W_n W_e etc) are using previous state in an interesting way to produce that following e. So that metric kappa_s^t just doesn*t seem very meaningful. This remark relates to the last paragraph of Sec4.2. Even though the list of cons here is longer than pro*s, I recommend accept; specifically because the originality of this work will in any case make it more vulnerable to critiques. This work is well-motivated, very well-executed, and can inspire many more interesting investigations along these lines.
assistant
Summary: The authors propose an input switched affine network to do character-level language modeling, a kind of RNN without pointwise nonlinearity, but with switching the transition matrix & bias based on the input character. This is motivated by intelligibility, since it allows decomposition of output contribution into these kappa_s^t terms, and use of basic linear algebra to probe the network. Regarding myself as a reviewer, I am quite sure I understood the main ideas and arguments of this paper, but am not an expert on RNN language models or intelligibility/interpretability in ML. I did not read any papers with a similar premise - closest related work I*m familiar with would be deconvnet for insight into vision-CNNs. PRO: I think this is original and novel work. This work is high quality, well written, and clearly is the result of a lot of work. I found section 4.5 about projecting into readout subspace vs *computational* subspace most interesting and meaningful. CON: + The main hesitation I have is that the results on both parts (ISAN model, and analysis of it) are not entirely convincing: (1) ISAN is only trained on small task (text8), not clear whether it can be a strong char-LM on larger scale tasks, (2) nor do the analysis sections provide all that much real insight in the learned network. (1b) Other caveat towards ISAN architecture: this model in its proposed form is really only fit for small-vocabulary (i.e. character-based) language modeling, not a general RNN with large-vocab discrete input nor continuous input. (2a) For analysis: many cute plots and fun ideas of quantities to look at, but not much concrete insights. (2b) Not very clear which analysis is specific to the ISAN model, and which ideas will generalize to general nonlinear RNNs. (2c) Re sec 4.2 - 4.3: It seems that the quantity kappa_s^t on which analysis rests, isn*t all that meaningful. Elaborating a bit on what I wrote in the question: For example: Fig 2, for input letter *u* in revenue, there*s a red spot where *_* character massively positively impacts the logit of *e*. This seems quite meaningless, what would be the meaning of influence of *_* character? So it looks ot me that the switching matrix W_u (and prior W_n W_e etc) are using previous state in an interesting way to produce that following e. So that metric kappa_s^t just doesn*t seem very meaningful. This remark relates to the last paragraph of Sec4.2. Even though the list of cons here is longer than pro*s, I recommend accept; specifically because the originality of this work will in any case make it more vulnerable to critiques. This work is well-motivated, very well-executed, and can inspire many more interesting investigations along these lines.