Normalization of Deviance
> <i>We had built a system that generated thousands of lines of logs for every test, with lots of “failures” recorded in them. Things like “tried to initialize FOOBAR_CONTROLLER: FAILED!!!,” but we just ran that code on all machines, even ones without the FOOBAR_CONTROLLER hardware. So no one noticed when another 5 lines of errors popped up in a 2000-line log file.</i><p>This right there is a big red flag. The whole bendy business is bad enough, but here you're actively training people to ignore the wolf cries.<p>Don't allow false failures in tests. The entire test suite needs to be binary: either everything works, or it fails.