For some reason, every time I upload my package to PyPI, it includes a tests.py file that:

Isn't tracked in git
Isn't even in the project directory anymore
Isn't added to the /dist tarball after running sdist
Isn't in the tarball hosted on Github
Isn't in MANIFEST

Where is it picking this file up from? It was part of my package during development but I've done everything I can to remove it before publishing - and it still shows up.
Could not figure out the issue with distutils. Switched to setuptools and everything is working correctly, no tests.py being added to the tarball.
PyPI tarball includes files that aren't in the dist package