title: Solr Dismax handler - whitespace and special character behaviour
body: <p>I've got strange results when I have special characters in my query.</p>
  
  <p>Here is my request :</p>
  
  <pre><code>q=histoire-france&amp;start=0&amp;rows=10&amp;sort=score+desc&amp;defType=dismax&amp;qf=any^1.0&amp;mm=100%
  </code></pre>
  
  <p>Parsed query :</p>
  
  <pre><code>&lt;str name="parsedquery_toString"&gt;+((any:histoir any:franc)) ()&lt;/str&gt;
  </code></pre>
  
  <p>I've got 17000 results because Solr is doing an OR (should be AND).</p>
  
  <p>I have no problem when I'm using a whitespace instead of a special char :</p>
  
  <pre><code>q=histoire france&amp;start=0&amp;rows=10&amp;sort=score+desc&amp;defType=dismax&amp;qf=any^1.0&amp;mm=100%
  
  &lt;str name="parsedquery_toString"&gt;+(((any:histoir) (any:franc))~2) ()&lt;/str&gt;
  </code></pre>
  
  <p>2000 results for this query.</p>
  
  <p>Here is my schema.xml (relevant parts) :</p>
  
  <pre><code>&lt;fieldType name="text" class="solr.TextField" positionIncrementGap="100" autoGeneratePhraseQueries="false"&gt;
        &lt;analyzer type="index"&gt;
          &lt;tokenizer class="solr.WhitespaceTokenizerFactory"/&gt;
          &lt;filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="1" preserveOriginal="1"/&gt;
          &lt;filter class="solr.LowerCaseFilterFactory"/&gt;
          &lt;filter class="solr.CommonGramsFilterFactory" words="stopwords_french.txt" ignoreCase="true"/&gt;
          &lt;filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords_french.txt" enablePositionIncrements="true"/&gt;
          &lt;filter class="solr.SnowballPorterFilterFactory" language="French" protected="protwords.txt"/&gt;
          &lt;filter class="solr.RemoveDuplicatesTokenFilterFactory"/&gt;
          &lt;filter class="solr.ASCIIFoldingFilterFactory"/&gt;
        &lt;/analyzer&gt;
        &lt;analyzer type="query"&gt;
          &lt;tokenizer class="solr.WhitespaceTokenizerFactory"/&gt;
          &lt;!--&lt;filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/&gt;--&gt;
          &lt;filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1" preserveOriginal="0"/&gt;
          &lt;filter class="solr.LowerCaseFilterFactory"/&gt;
          &lt;filter class="solr.CommonGramsFilterFactory" words="stopwords_french.txt" ignoreCase="true"/&gt;
          &lt;filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords_french.txt" enablePositionIncrements="true"/&gt;
          &lt;filter class="solr.SnowballPorterFilterFactory" language="French" protected="protwords.txt"/&gt;
          &lt;filter class="solr.RemoveDuplicatesTokenFilterFactory"/&gt;
          &lt;filter class="solr.ASCIIFoldingFilterFactory"/&gt;
        &lt;/analyzer&gt;
      &lt;/fieldType&gt;
  </code></pre>
  
  <p>I even tried with a PatternTokenizerFactory to tokenize on whitespaces &amp; special chars but no change...</p>
  
  <p>My current workaround is to replace all special chars by whitespaces before sending query to Solr, but it is not satisfying.</p>
  
  <p><strong>EDIT :</strong> Even with a charFilter (PatternReplaceCharFilterFactory) to replace special characters by whitespace, it doesn't work...</p>
  
  <p>First line of analysis via solr admin, with verbose output, for query = 'histoire-france' :</p>
  
  <pre><code>org.apache.solr.analysis.PatternReplaceCharFilterFactory {replacement= , pattern=([,;./\\'&amp;-]), luceneMatchVersion=LUCENE_32}
  text    histoire france
  </code></pre>
  
  <p>The '-' is replaced by ' ', then tokenized by WhitespaceTokenizerFactory. However I still have different number of results for 'histoire-france' and 'histoire france'.</p>
  
  <p>Did i miss something ?</p>