You can do much better than that: you can use summaries or hashes to get a short version of each file and then compare the hashes for similarity. (Much like using perceptual hashes to find similar images.) A tool that does this is &#x27;binsort&#x27;. It&#x27;s O(N^2) IIRC, because of all the pairwise comparisons, but it&#x27;s usable for up to a few hundred or thousand files.