<p dir="auto">I have index with parent "model" and child "sell" documents at one index with millions of documents and heavy concurrent indexing.</p>
<p dir="auto">All nodes works with Oracle Hotspot 8 with 30gb heap, ElasticSearch 1.3.5.</p>
<p dir="auto">I'm execute delete by query like this:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="curl -XDELETE 'http://127.0.0.1:9200/sells/sell/_query' -d '{&quot;query&quot;:{&quot;filtered&quot;:{&quot;query&quot;:{&quot;match_all&quot;:{}},&quot;filter&quot;:{&quot;and&quot;:{&quot;filters&quot;:[{&quot;term&quot;:{&quot;client_id&quot;:123}}]}}}}}]}'"><pre class="notranslate"><code class="notranslate">curl -XDELETE 'http://127.0.0.1:9200/sells/sell/_query' -d '{"query":{"filtered":{"query":{"match_all":{}},"filter":{"and":{"filters":[{"term":{"client_id":123}}]}}}}}]}'
</code></pre></div>
<p dir="auto">and our index is corrupted with this error message and exception stack trace in logs:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="[2014-11-18 12:49:33,650][DEBUG][action.deletebyquery     ] [elastic1] [sells][3], node[hv2718pkQm-5SZq7agcY9g], [P], s[STARTED]: Failed to execute [delete_by_query {[sells][sell], query [{&quot;query&quot;:{&quot;filtered&quot;:{&quot;query&quot;:{&quot;match_all&quot;:{}},&quot;filter&quot;:{&quot;and&quot;:{&quot;filters&quot;:[{&quot;term&quot;:{&quot;client_id&quot;:123}}]}}}}}]}] 
org.elasticsearch.index.engine.RefreshFailedEngineException: [sells][3] Refresh failed 
        at org.elasticsearch.index.engine.internal.InternalEngine.refresh(InternalEngine.java:789) 
        at org.elasticsearch.index.engine.internal.InternalEngine.delete(InternalEngine.java:686) 
        at org.elasticsearch.index.shard.service.InternalIndexShard.deleteByQuery(InternalIndexShard.java:465) 
        at org.elasticsearch.action.deletebyquery.TransportShardDeleteByQueryAction.shardOperationOnPrimary(TransportShardDeleteByQueryAction.java:123) 
        at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:535) 
        at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:434) 
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745) 
Caused by: java.lang.IllegalStateException: parentFilter must return FixedBitSet; got org.apache.lucene.search.BitsFilteredDocIdSet@74ebc73d 
        at org.elasticsearch.index.search.nested.IncludeNestedDocsQuery$IncludeNestedDocsWeight.scorer(IncludeNestedDocsQuery.java:123) 
        at org.apache.lucene.search.QueryWrapperFilter$1.iterator(QueryWrapperFilter.java:59) 
        at org.apache.lucene.index.BufferedUpdatesStream.applyQueryDeletes(BufferedUpdatesStream.java:554) 
        at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:287) 
        at org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3322) 
        at org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3313) 
        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:425) 
        at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:292) 
        at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:267) 
        at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:257) 
        at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:171) 
        at org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:118) 
        at org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:58) 
        at org.apache.lucene.search.ReferenceManager.doMaybeRefresh(ReferenceManager.java:176) 
        at org.apache.lucene.search.ReferenceManager.maybeRefresh(ReferenceManager.java:225) 
        at org.elasticsearch.index.engine.internal.InternalEngine.refresh(InternalEngine.java:779) 
        ... 8 more"><pre class="notranslate"><code class="notranslate">[2014-11-18 12:49:33,650][DEBUG][action.deletebyquery     ] [elastic1] [sells][3], node[hv2718pkQm-5SZq7agcY9g], [P], s[STARTED]: Failed to execute [delete_by_query {[sells][sell], query [{"query":{"filtered":{"query":{"match_all":{}},"filter":{"and":{"filters":[{"term":{"client_id":123}}]}}}}}]}] 
org.elasticsearch.index.engine.RefreshFailedEngineException: [sells][3] Refresh failed 
        at org.elasticsearch.index.engine.internal.InternalEngine.refresh(InternalEngine.java:789) 
        at org.elasticsearch.index.engine.internal.InternalEngine.delete(InternalEngine.java:686) 
        at org.elasticsearch.index.shard.service.InternalIndexShard.deleteByQuery(InternalIndexShard.java:465) 
        at org.elasticsearch.action.deletebyquery.TransportShardDeleteByQueryAction.shardOperationOnPrimary(TransportShardDeleteByQueryAction.java:123) 
        at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:535) 
        at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:434) 
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745) 
Caused by: java.lang.IllegalStateException: parentFilter must return FixedBitSet; got org.apache.lucene.search.BitsFilteredDocIdSet@74ebc73d 
        at org.elasticsearch.index.search.nested.IncludeNestedDocsQuery$IncludeNestedDocsWeight.scorer(IncludeNestedDocsQuery.java:123) 
        at org.apache.lucene.search.QueryWrapperFilter$1.iterator(QueryWrapperFilter.java:59) 
        at org.apache.lucene.index.BufferedUpdatesStream.applyQueryDeletes(BufferedUpdatesStream.java:554) 
        at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:287) 
        at org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3322) 
        at org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3313) 
        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:425) 
        at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:292) 
        at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:267) 
        at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:257) 
        at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:171) 
        at org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:118) 
        at org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:58) 
        at org.apache.lucene.search.ReferenceManager.doMaybeRefresh(ReferenceManager.java:176) 
        at org.apache.lucene.search.ReferenceManager.maybeRefresh(ReferenceManager.java:225) 
        at org.elasticsearch.index.engine.internal.InternalEngine.refresh(InternalEngine.java:779) 
        ... 8 more
</code></pre></div>
<p dir="auto">I cannot reproduce this error on our test stand cluster, sorry. It's seems like a concurrent access error.</p>
<p dir="auto">What's a kind of problem? Does it just bug?</p>
<p dir="auto">The ability to have several types on the same index is causing problems:</p>
<ul dir="auto">
<li>the mappings APIs need to maintain one mapping per type, yet those mappings can't be independent and keeping them synchronized is complicated (see eg. discussions on <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="122943851" data-permission-text="Title is private" data-url="https://github.com/elastic/elasticsearch/issues/15539" data-hovercard-type="pull_request" data-hovercard-url="/elastic/elasticsearch/pull/15539/hovercard" href="https://github.com/elastic/elasticsearch/pull/15539">#15539</a>)</li>
<li>it gives the feeling that the system can easily deal with documents that have very different mappings in the same index, which is not true. This is why in 2.0 we added more restrictions on mappings across types. In addition types encourage sparsity and sparse fields cause Lucene to either be slow when there is a special impl for the sparse case (eg. doc values) or use tremendous memory and disk space in spite of the fact that few documents have a value (eg. norms, because a fixed amount of memory is used for every doc, regardless of whether they have a value for this field or not).</li>
</ul>
<p dir="auto">Migrating existing users is certainly going to be complicated but this would also make the system more honest to new users about the fact that we can't do index-level multi-tenancy efficiently. Also I suspect that the restrictions that we added in 2.0 (that eg. two fields that have the same name in different types) already made lots of users migrate to a single index per data type instead of folding them into different types of the same index.</p>
<p dir="auto">See also <a href="https://www.elastic.co/blog/index-vs-type" rel="nofollow">https://www.elastic.co/blog/index-vs-type</a>.</p>