copy.deepcopy(batch_spk_centroid[index1].data)

dont use deep copy
tensor has a method called clone
Besides,
batch_spk_centroid[index1] = ((batch_spk_centroid[index1] * self.hp.train.num_utt_per_spk ) - batch_embedding[index1][index2] ) / (self.hp.train.num_utt_per_spk - 1)


I think this is in-place as you are modifying a tensor value based on its own value, you may need to clone that
batch_spk_centroid[index1] = ((batch_spk_centroid[index1].clone() * self.hp.train.num_utt_per_spk ) - batch_embedding[index1][index2] ) / (self.hp.train.num_utt_per_spk - 1)