The code looks correct to me. If I understand correctly this a cell of a decoder RNN. The input then is exactly s_{i-1} and hidden is h_j.
Disclaimer: I didn’t actually read the paper nor the code.
Btw, if you have already read through the paper and the code, then why don’t you tell us what the notations mean and what these variables represent? Having to read paper+code to answer your question is quite time-consuming.