In the documentation for LSTM 127, for the dropout argument, it states:

introduces a dropout layer on the outputs of each RNN layer except the last layer

I just want to clarify what is meant by “everything except the last layer”.
Below I have an image of two possible options for the meaning.


Option 1:  The final cell is the one that does not have dropout applied for the output.

Option 2:  In a multi-layer LSTM, all the connections between layers have dropout applied, except the very top layer.  So a single layer LSTM would not have any dropout applied.


diagram.png301×521 27.4 KB

I tried looking at the source code for LSTM and RNNBase, but I can’t figure out how it is being applied.