@ tom
Thank you very much for your answer. This is very well appreciated.
I have one more question to the 3.), the detaching:
In the example above, the weird thing is that they detach the first hidden state that they have newly created and that they create new again every time they call forward. So in that example the detaching is actually redundant, no? (Since a newly created tensor has no history)
So detaching would actually only be required in a situation like you have described in 2.), where the last hidden state is reused as new initial state, right?