I guess that means that your network requires very precise floats. In general, unless you’re doing very specific things, this should not be the case. You should check that you don’t have any operation that is sensible to numerical precision. Like dividing by something so small, it would be 0 in single precision.