Wesley_Neill:

What is the functional difference between “many-to-one” vs a “one-to-one” approaches?


What the Udacity tutorial is doing is not a “one-to-one” but more like a “many-to-many” approach. Note that the value of the k-th ouput depends on all the (k-1) inputs. Or in other words, the k-th output depends on the hidden state at time k which in turn depends on the (k-1) hidden states before.
In fact, you can directly map the approach used in the Udacity tutorial to the basic “many-to-one” approach. Instead of having one (input => target) example such as

[1, 2, 3, 4, 5, 6] => [2, 3, 4, 5, 6, 7]

you can convert this into 6 “many-to-one” examples

[1] => [2]
[1, 2] => [3]
[1, 2, 3] => [4]
[1, 2, 3, 4] => [5]
[1, 2, 3, 4, 5] => [6]
[1, 2, 3, 4, 5, 6] => [7]

From a learning/training perspective, that’s exactly the same. That’s what the reshape(-1, self.hidden_size) in the Udacity tutorial is doing: “internally” creating this artificial batch of, in this case, size 6.