I was taking the point of &quot;Why is the problem taking so long when it seems like a simple thing to calculate?&quot; which was a little orthonogal from theme of the total post, so I can see why there would be some confusion.<p>The article talked about 30 minutes to run through a &quot;is this set of items in this other set of a bunch of items&quot;.<p>While it would probably make sense to do this efficiently in memory the reality is the company mentioned are probably doing it inefficiently in memory; asking some tool which can do this type of work effectively and is fairly well known was my basic suggestion.