Can’t be sure without consulting the author, but I think the intent was to treat the initial state as a learned value. But you’re right that the implementation doesn’t do that since init_hidden() is called in forward() (which I missed). What they probably should’ve done is called init_hidden() once inside __build_model() and not reassigned self.hidden.