Like I said I have +20M records so potentially tens of thousands of results per query.<p>This would mean that every time a user triggers this API call we have to insert those tens of thousands of documents into Redis and run an operation at the end. Even without taking into account the JSON serialisation &#x2F; deserialisation costs, this would take forever where we need something that runs in around one or two seconds at the maximum.<p>The problem is actually pretty simple and can be summed up by this sentence: &quot;I want to group documents by field X, take one document per group of X and return documents from index OFFSET to index OFFSET + LIMIT&quot;.<p>In MongoDB for example this would be quite easy with a $group + $first operation in the aggregation framework. Sadly MongoDB lacks the nice full-text search features that ElasticSearch has. It&#x27;s highly possible though that going forward we&#x27;ll have to hack a full-text-like search on MongoDB and switch from ElasticSearch to MongoDB since stuff like this doesn&#x27;t seem to possible.