This issue is not specific to cudnn, but to floating point arithmetic 3.
The linked Wikipedia article describes some issues, where the precision is lost.
Basically, neither result might be the “true” value in by small example and both results come close to the mathematically correct answer.