Hi Jianming!



 jianming_TONG:

Hi my manual implementation of the pytorch convolution2d will always have some precision difference compared with official pytorch implementation of conv2d.


I have not looked at your code.
However, this kind of discrepancy is common and is usually caused by
floating-point round-off error due to the fact that the order of operations
in your implementation will almost certainly differ from those in Conv2d.
Even though the two sets of operations should be mathematically
equivalent, the results can differ because of round-off error.
An easy way to test for this is to redo the calculation in double precision
(preferably using the same Conv2d weight matrix and random
sample_input tensor from your single-precision calculation, converted
to double precision).  If performing the calculation in double precision
reduces the discrepancy by several orders of magnitude, you would
have strong evidence that your issue is due to round-off error.
Best.
K. Frank