torch.nn.LSTM don’t support attention mechanism and have to implement attention based on torch.nn.LSTMCell manually. Howerer, torch.nn.LSTMCell doesn’t has a num_layers param. Thus, OpenNMT-py defines a StackedLSTM which supports attention and multi-layers, as a extension to torch.nn.LSTMCell