This sounds related to quantization; have you tried cloning your “Delta X” into a separate tensor that you convert to long for your indexing? You can use detach to keep the clone separate from the original float tensor.