Can brew find / manage packages installed by manually binary download + install? I  manually downloaded quite some software on my Mac, and now I want to use brew to manage all my packages.

But I don't seem to be able to find the manually installed software, maybe I missed some steps? How can I let brew manage them?
Assuming what you downloaded manually was with brew, just do brew list, you can see what you've installed using brew. Then just do brew uninstall packageNameHere to get rid of what you don't want.