I am working on adversarial attacks with pytorch.
I was initially using floatTensors but my backpropogation was not consistent.
After searching I found https://github.com/pytorch/pytorch/issues/5351 15 . Due to iteratively using thr wrong values my answers differ significantly across experiments.
Then after I moved to DoubleTensors my answers seem to be very consistent but it has become about 3-4 times slower. This is somewhat expected but is there any workaround into this as 3/4 is a really huge factor!
EDIT Followup:
Since autograd differs in pytorch vs luatorch, does torch autograd pass gradcheck in luatorch pass with float operands??