  The advantage of recurrent neural networks (RNNs) in learning dependencies
between time-series data has distinguished RNNs from other deep learning
models. Recently, many advances are proposed in this emerging field. However,
there is a lack of comprehensive review on memory models in RNNs in the
literature. This paper provides a fundamental review on RNNs and long short
term memory (LSTM) model. Then, provides a surveys of recent advances in
different memory enhancements and learning techniques for capturing long term
dependencies in RNNs.

Neural and Evolutionary Computing