Is there some way to speed up the repeated execution of pytest? It seems to spend a lot of time collecting tests, even if I specify which files to execute on the command line. I know it isn't a disk speed issue either since running pyflakes across all the .py files is very fast.

The various answers represent different ways pytest can be slow. They helped sometimes, did not in others. I'm adding one more answer that explains a common speed problem. But it's not possible to select "The" answer here.
Pytest imports all modules in the testpaths directories to look for tests. The import itself can be slow. This is the same startup time you'd experience if you ran those tests directly, however, since it imports all of the files it will be a lot longer. It's kind of a worst-case scenario.
This doesn't add time to the whole test run though, as it would need to import those files anyway to execute the tests.
If you narrow down the search on the command line, to specific files or directories, it will only import those ones. This can be a significant speedup while running specific tests.
Speeding up those imports involves modifying those modules. The size of the module, and the transitive imports, slow down the startup. Additionally look for any code that is executed -- code outside of a function. That also needs to be executed during the test collection phase.
How to speed up pytest