Long Short-Term Memory (LSTM) units have the ability to memorise and use long-term dependencies between inputs to generate predictions on time series data. We introduce the concept of modifying the cell state (memory) of LSTMs using rotation matrices parametrised by a new set of trainable weights. This addition shows significant increases of performance on some of the tasks from the bAbI dataset.
The paper proposes to add a rotation operation in long short-term memory (LSTM) cells. It performs experiments on bAbI tasks and showed that the results are better than the simple baselines with original LSTM cells. There are a few problems with the paper.
Firstly, the title and abstract discuss "modifying memories", but the content is only about a rotation operation. Perhaps the title should be "Rotation Operation in Long Short-Term Memory"?
Secondly, the motivation of adding the rotation operation is not properly justified. What does it do that a usual LSTM cell could not learn? Does it reduce the excess representational power compared to the LSTM cell that could result in better models? Or does it increase its representational capacity so that some pattern is modeled in the new cell structure that was not possible before? This is not clear at all after reading the paper. Besides, the idea of using a rotation operation in recurrent networks has been explored before [3].
Finally, the task (bAbI) and baseline models (LSTM from a Keras tutorial) are too weak. There have been recent works that nearly solved the bAbI tasks to perfection (e.g., [1][2][4][5], and many others). The paper presented a solution that is weak compared to these recent results.
In a summary, the main idea of adding rotation to LSTM cells is not properly justified in the paper, and the results presented are quite weak for publication in ICLR 2018.
[1] Sainbayar Sukhbaatar, Jason Weston, Rob Fergus. End-to-end memory networks, NIPS 2015
[2] Caiming Xiong, Stephen Merity, Richard Socher. Dynamic Memory Networks for Visual and Textual Question Answering, ICML 2016
[3] Mikael Henaff, Arthur Szlam, Yann LeCun, Recurrent Orthogonal Networks and Long-Memory Tasks, ICML 2016 
[4] Caglar Gulcehre, Sarath Chandar, Kyunghyun Cho, Yoshua Bengio, Dynamic Neural Turing Machine with Soft and Hard Addressing Schemes, ICLR 2017
[5] Mikael Henaff, Jason Weston, Arthur Szlam, Antoine Bordes, Yann LeCun, Tracking the World State with Recurrent Entity Networks, ICLR 2017