Hi Jianming!



 jianming_TONG:

I have tried both torch.float32 and torch.float64 for weights and input in my own implementation. And both of them demonstrate some differences compared with the official conv2d API.


If I understand correctly, you have used torch.float32 and torch.float64
for your implementation, but only used torch.float32 for the “official
conv2d API.”
My suggestion, in more detail, is:
Calculation 1:
Use torch.float32 for both your implementation and “the official conv2d.”
Measure the discrepancy.  For example, calculate the mean and the
maximum of the absolute values of the differences between the two
results.
Calculation 2:
Use torch.float64 for both your implementation and “the official
conv2d.”  (Ideally perform this computation using the same weight
and sample_input that you used in the torch.float32 calculation,
except converted to torch.float64.)  The point is that you want to
perform both convolution implementations in double precision – that
is, torch.float64.  Measure the discrepancy the same way as you
did for the torch.float32 calculation.  Does using torch.float64
reduce the discrepancy?
Just to be clear, we do not expect the comparison between using
torch.float32 for one implementation and torch.float64 for
the other to reduce the discrepancy.
Best.
K. Frank