produces failures (meaning we have some undetected coupled tests), use pytest-replay to replay the tests in the same order, which should help with then somehow
reducing that failing sequence to a minimum.
Test order and repetition
It's good to repeat the tests several times, in sequence, randomly, or in sets, to detect any potential
inter-dependency and state-related bugs (tear down). And the straightforward multiple repetition is just good to detect
some problems that get uncovered by randomness of DL.
Repeat tests
pytest-flakefinder:
bash
pip install pytest-flakefinder
And then run every test multiple times (50 by default):
bash
pytest --flake-finder --flake-runs=5 tests/test_failing_test.py
This plugin doesn't work with -n flag from pytest-xdist.
There is another plugin pytest-repeat, but it doesn't work with unittest.
