I am trying to reimplement the torch text example 1
The example is simple and clear, however, it doesn’t use the standard torchtext Iterators instead it has it’s own dataloader and batch generator (and it seems for reason).
I tried to reimplement the code with the standard Interator. But this lead to gradient explosion. Then I tried to use BucketIterator that groups similar length string together, and at least, it didn’t explode but didn’t learn too.
Obviously, padding is useless for this architecture, but I didn’t find a way to disable padding in torchtext. On the other hard, why padding works so bad with SGD? When I replace SGD with Adam everything went smoothly.
A lot of questions and no answers.