Ideally they should, but might be flaky or fail on unknown/new configs.


You could pass the --continue-through-error argument to python run_test.py so that all tests will be executed even if one fails.


See 2 for the first part. Are you interested in contributing to the tests directly or are you working on another feature/fix?