Some paper run experiment on LSTM on case with and without output gate.
For example, this paper Towards Universal Paraphrastic Sentence Embeddings 8 table 5 in section 4.4.
I would like to suggest an option to turn on/off output gate.