Published as a conference paper at ICLR 2018 SPARSE PERSISTENT RNNS: SQUEEZING LARGE RECURRENT NETWORKS ON- CHIP
Recurrent Neural Networks (RNNs) are powerful tools for solving sequence-based problems, but their efficacy and execution time are dependent on the size of the network. Following recent work in simplifying these networks with model pruning and a novel mapping of work onto GPUs, we design an efficient implementation for sparse RNNs. We investigate several optimizations and tradeoffs: Lamport timestamps, wide memory loads, and a bank-aware weight layout. With these optimizations, we achieve speedups of over 6× over the next best algorithm for a hidden layer of size 2304, batch size of 4, and a density of 30%. Further, our technique allows for models of over 5× the size to fit on a GPU for a speedup of 2×, enabling larger networks to help advance the state-of-the-art. We perform case studies on NMT and speech recognition tasks in the appendix, accelerating their recurrent layers by up to 3×.
This paper introduces sparse persistent RNNs, a mechanism to add pruning to the existing work of stashing RNN weights on a chip. The paper describes the use additional mechanisms for synchronization and memory loading. 
The evaluation in the main paper is largely on synthetic workloads (i.e. large layers with artificial sparsity).  With evaluation largely over layers instead of applications, I was left wondering whether there is an actual benefit on real workloads. Furthermore, the benefit over dense persistent RNNs for OpenNMT application (of absolute 0.3-0.5s over dense persistent rnns?) did not appear significant unless you can convince me otherwise. 
Storing weights persistent on chip should give a sharp benefit when all weights fit on the chip. One suggestion I have to strengthen the paper is to claim that due to pruning, now you can support a larger number of methods or method configurations and to provide examples of those.
To summarize, the paper adds the ability to support pruning over persistent RNNs. However, Narang et. al., 2017 already explore this idea, although briefly. Furthermore, the gains from the sparsity appear rather limited over real applications. I would encourage the authors to put the NMT evaluation in the main paper (and perhaps add other workloads). Furthermore, a host of techniques are discussed (Lamport timestamps, memory layouts) and implementing them on GPUs is not trivial. However, these are well known and the novelty or even the experience of implementing these on GPUs should be emphasized.