Thanks for the update. Could you post some stats using
print(np.unique(x, return_counts=True))

This will return the different error values and their count in the image.
Maybe most of the errors are just really small and just very few errors are large.