If you accumulate your Tensor over a given dimension such that it returns a ByteTensor, it will overflow the same way.
I think this issue will be solved when the Scalar types are added to the autograd.