tom:

BUILD_LITE_INTERPRETER


Thanks a lot. I was using VSCode to search for this “Functions.cpp” file, the reason why I can’t find it is due to the directory is makred in .gitignore file. Now I commented it out from .gitignore file, I can find all the generated files. Again, thanks.
Besides, by reading the source code, for my case, I just figured out why the second case the templated parameters are all double, but not the first.

For the first MulScalar<float, float> call, this happens during addcmul’s backward function. By checking the backward formula in “derivatives.yaml” file for addcmul:

- name: addcmul(Tensor self, Tensor tensor1, Tensor tensor2, *, Scalar value=1) -> Tensor

  self: handle_r_to_c(self.scalar_type(), grad)

  tensor1: handle_r_to_c(tensor1.scalar_type(), grad * (tensor2 * value).conj())

  tensor2: handle_r_to_c(tensor2.scalar_type(), grad * (tensor1 * value).conj())

It didn’t call another implemented backward function, instead, it writes down the calculation formula using already known operators, here it is mul, specifically, for tensor2 * value, it would be a tensor * Scalar, so it will call aten/src/ATen/native/BinaryOps.cpp  mul():
Tensor mul(const Tensor& self, Scalar other) {
  return native::mul(self, wrapped_scalar_tensor(other));
}

Notice that it will call a wrapped_scalar_tensor() API to the Scalar variable first, by checking the source code for this, it will in fact call a set_wrapped_numer() function, which means when calculating the common_types for self(float) and other(double), the other’s type won’t participate in the calculation, so the result is float.
However, if we don’t use wrapped_scalar_tensor() API, the result of common_type would be double. That is the case of what happened when we do a backward for Mul for Scalar. Here in derivatives.yaml, you can find the following:
- name: mul.Scalar(Tensor self, Scalar other) -> Tensor
  self: mul_tensor_backward(grad, at::scalar_to_tensor(other), self.scalar_type())

Notice that in this backward function, it just directly called at::scalar_to_tensor(other), without calling tensor.unsafeGetTensorImpl()->set_wrapped_number(true);. Thus, the common_type of double and float would be double now. But on my platform, I can’t use double on GPU, that’s why it fails for me.