I am looking for a way of applying dropout in between layers for a stacked LSTM.
I have code implementing this using the cell version of the LSTM and running it on each step of the sequence and I can attest a slow down of a factor of 5 (from 40.000 tokens/s to 8.000 tokens/s).