<p dir="auto">Hi, I am getting the error below. The same query runs without any problems when just a few records are returned by the where clause. I am currently running on a single node "quickstart" configuration.</p>
<p dir="auto">Thank You</p>
<p dir="auto">`2017-09-19 11:03:54 DEBUG DruidDao:414 - SQL: select A, B, C, D, E, G, H, I, L from dataSource where A in ('aValue' ) AND B in ('SomeVALUE' )<br>
[WARNING] /dl<br>
AvaticaClientRuntimeException: Remote driver error: IllegalStateException: DefaultQueryMetrics must not be modified from multiple threads. If it is needed to gather dimension or metric information from multiple threads or from an async thread, this information should explicitly be passed between threads (e. g. using Futures), or this DefaultQueryMetrics's ownerThread should be reassigned explicitly. Error -1 (00000) UNKNOWN</p>
<p dir="auto">java.lang.IllegalStateException: DefaultQueryMetrics must not be modified from multiple threads. If it is needed to gather dimension or metric information from multiple threads or from an async thread, this information should explicitly be passed between threads (e. g. using Futures), or this DefaultQueryMetrics's ownerThread should be reassigned explicitly<br>
at io.druid.query.DefaultQueryMetrics.checkModifiedFromOwnerThread(DefaultQueryMetrics.java:59)<br>
at io.druid.query.DefaultQueryMetrics.reportMetric(DefaultQueryMetrics.java:253)<br>
at io.druid.query.DefaultQueryMetrics.reportCpuTime(DefaultQueryMetrics.java:231)<br>
at io.druid.query.CPUTimeMetricQueryRunner$1.after(CPUTimeMetricQueryRunner.java:86)<br>
at io.druid.java.util.common.guava.WrappingYielder.close(WrappingYielder.java:96)<br>
at io.druid.java.util.common.guava.Yielders$2.close(Yielders.java:73)<br>
at io.druid.java.util.common.guava.ConcatSequence.wrapYielder(ConcatSequence.java:123)<br>
at io.druid.java.util.common.guava.ConcatSequence.access$000(ConcatSequence.java:29)<br>
at io.druid.java.util.common.guava.ConcatSequence$3.next(ConcatSequence.java:143)<br>
at io.druid.sql.avatica.DruidStatement.nextFrame(DruidStatement.java:277)<br>
at io.druid.sql.avatica.DruidMeta.fetch(DruidMeta.java:209)<br>
at org.apache.calcite.avatica.remote.LocalService.apply(LocalService.java:252)<br>
at org.apache.calcite.avatica.remote.Service$FetchRequest.accept(Service.java:1350)<br>
at org.apache.calcite.avatica.remote.Service$FetchRequest.accept(Service.java:1317)<br>
at org.apache.calcite.avatica.remote.AbstractHandler.apply(AbstractHandler.java:95)<br>
at org.apache.calcite.avatica.remote.JsonHandler.apply(JsonHandler.java:52)<br>
at org.apache.calcite.avatica.server.AvaticaJsonHandler.handle(AvaticaJsonHandler.java:129)<br>
at io.druid.sql.avatica.DruidAvaticaHandler.handle(DruidAvaticaHandler.java:63)<br>
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)<br>
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)<br>
at org.eclipse.jetty.server.Server.handle(Server.java:534)<br>
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)<br>
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)<br>
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)<br>
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)<br>
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)<br>
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)<br>
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)<br>
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)<br>
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)<br>
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)<br>
at java.lang.Thread.run(Thread.java:748)<br>
Suppressed: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: DefaultQueryMetrics must not be modified from multiple threads. If it is needed to gather dimension or metric information from multiple threads or from an async thread, this information should explicitly be passed between threads (e. g. using Futures), or this DefaultQueryMetrics's ownerThread should be reassigned explicitly<br>
at com.google.common.base.Throwables.propagate(Throwables.java:160)<br>
at io.druid.sql.avatica.DruidStatement.close(DruidStatement.java:335)<br>
at io.druid.sql.avatica.DruidStatement.nextFrame(DruidStatement.java:290)<br>
... 22 more<br>
Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: DefaultQueryMetrics must not be modified from multiple threads. If it is needed to gather dimension or metric information from multiple threads or from an async thread, this information should explicitly be passed between threads (e. g. using Futures), or this DefaultQueryMetrics's ownerThread should be reassigned explicitly<br>
at java.util.concurrent.FutureTask.report(FutureTask.java:122)<br>
at java.util.concurrent.FutureTask.get(FutureTask.java:192)<br>
at io.druid.sql.avatica.DruidStatement.close(DruidStatement.java:319)<br>
... 23 more<br>
Caused by: java.lang.IllegalStateException: DefaultQueryMetrics must not be modified from multiple threads. If it is needed to gather dimension or metric information from multiple threads or from an async thread, this information should explicitly be passed between threads (e. g. using Futures), or this DefaultQueryMetrics's ownerThread should be reassigned explicitly<br>
at io.druid.query.DefaultQueryMetrics.checkModifiedFromOwnerThread(DefaultQueryMetrics.java:59)<br>
at io.druid.query.DefaultQueryMetrics.reportMetric(DefaultQueryMetrics.java:253)<br>
at io.druid.query.DefaultQueryMetrics.reportCpuTime(DefaultQueryMetrics.java:231)<br>
at io.druid.query.CPUTimeMetricQueryRunner$1.after(CPUTimeMetricQueryRunner.java:86)<br>
at io.druid.java.util.common.guava.WrappingYielder.close(WrappingYielder.java:96)<br>
at io.druid.java.util.common.guava.ConcatSequence$3.close(ConcatSequence.java:157)<br>
at io.druid.java.util.common.guava.ConcatSequence$3.close(ConcatSequence.java:156)<br>
at io.druid.sql.avatica.DruidStatement.lambda$close$1(DruidStatement.java:315)<br>
at java.util.concurrent.FutureTask.run(FutureTask.java:266)<br>
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)<br>
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)<br>
... 1 more</p>
<p dir="auto">at org.apache.calcite.avatica.remote.Service$ErrorResponse.toException(Service.java:2476)<br>
at org.apache.calcite.avatica.remote.JsonService.decode(JsonService.java:55)<br>
at org.apache.calcite.avatica.remote.JsonService.apply(JsonService.java:140)<br>
at org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:309)<br>
at org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:306)<br>
at org.apache.calcite.avatica.AvaticaConnection.invokeWithRetries(AvaticaConnection.java:756)<br>
at org.apache.calcite.avatica.remote.RemoteMeta.fetch(RemoteMeta.java:305)<br>
at org.apache.calcite.avatica.MetaImpl$FetchIterator.moveNext(MetaImpl.java:1606)<br>
at org.apache.calcite.avatica.MetaImpl$FetchIterator.next(MetaImpl.java:1591)<br>
at org.apache.calcite.avatica.util.IteratorCursor.next(IteratorCursor.java:46)<br>
at org.apache.calcite.avatica.AvaticaResultSet.next(AvaticaResultSet.java:239)<br>
at ... (DruidDao.java:1181)<br>
at ... (Download.java:75)<br>
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)<br>
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)<br>
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:833)<br>
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)<br>
at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:206)<br>
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)<br>
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)<br>
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)<br>
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)<br>
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)<br>
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)<br>
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)<br>
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)<br>
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)<br>
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)<br>
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)<br>
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)<br>
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)<br>
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)<br>
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)<br>
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)<br>
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)<br>
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)<br>
at org.eclipse.jetty.server.Server.handle(Server.java:564)<br>
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:317)<br>
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)<br>
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)<br>
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)<br>
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)<br>
at org.eclipse.jetty.util.thread.Invocable.invokePreferred(Invocable.java:128)<br>
at org.eclipse.jetty.util.thread.Invocable$InvocableExecutor.invoke(Invocable.java:222)<br>
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:294)<br>
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:126)<br>
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:673)<br>
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:591)<br>
at java.lang.Thread.run(Thread.java:748)`</p>
<p dir="auto">Today we have 2 ways to create a druid dimension based on a lookup.<br>
The first one (old way) is as an extraction function <code class="notranslate">ExtractionFn</code> and the second is via lookup dimension spec <code class="notranslate">LookupDimensionSpec</code>.<br>
Chronologically speaking the LookupDimensionSpec was introduced after ExtractionDimensionSpec.<br>
The reason behind this is the fact that lookups can offer more functionalities like unApply, unApplyAll, bulk lookups or the fact that one dimension can map to multiple values use case.<br>
IMHO This means it is better for lookup to be independent from extraction function from a design perspective. Now that we ended up with 2 way of doing things, i guess it is time to unify it.<br>
Benefits.</p>
<ul dir="auto">
<li>-Will be able to support the case where we have a one to many  mapping as lookups</li>
<li>Ensure that the logic of the mapping is consistent for both filtering and aggregation. This is really important if we want to support the use case mapping one to many</li>
<li>-Avoid to have duplicate code where issue like this occurs <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="158273807" data-permission-text="Title is private" data-url="https://github.com/apache/druid/issues/3065" data-hovercard-type="pull_request" data-hovercard-url="/apache/druid/pull/3065/hovercard" href="https://github.com/apache/druid/pull/3065">#3065</a></li>
<li>-Support the bulk lookup issues <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="157243320" data-permission-text="Title is private" data-url="https://github.com/apache/druid/issues/3030" data-hovercard-type="issue" data-hovercard-url="/apache/druid/issues/3030/hovercard" href="https://github.com/apache/druid/issues/3030">#3030</a></li>
</ul>
<p dir="auto">To achieve that is to make filters accept dimension spec instead of just dimension as a string and extraction function.<br>
Then make the dimensionSpec use the decorate method since getExtractionFn is deprecated anyway.<br>
Also we have to make all the filters work with either decoration of selector or extraction function.</p>