title: Lucene Analyzer chain: ShingleFilter without filler tokens
body: <p>In my analyzer chain, ShingleFilter comes after stopword filter. As mentioned in the <a href="http://lucene.apache.org/core/4_4_0/analyzers-common/org/apache/lucene/analysis/shingle/ShingleFilter.html" rel="nofollow">docs</a>, ShingleFilter handles position increments > 1 by inserting filler tokens (tokens with termtext "_").</p>
  
  <pre><code>For example : "please divide this sentence into biword shingles" 
  
  Shingles of size 2 : please divide, divide _, _ sentence, sentence _, _ biword, biword shingles (assuming that "this, "into" are stopwords)
  </code></pre>
  
  <p>I would like to eliminate those shingles with the filler tokens, i.e. my desired output contains only: please divide, biword shingles.</p>
  
  <p>I've a dedicated field for facets with shingles up to 4-grams. Due to these stopwords, all the facet constraints (or values) look useless with those fillers like <strong>"divide _ sentence _"</strong> </p>
  
  <p>Please could you guide me. </p>
  
  <p>Using Solr 4.4.</p>
  
  <p><em>UPDATE</em></p>
  
  <p>I thought of setting enablePositionIncrement to false in StopFilter configuration. Not sure whether that solves the problem or not but Lucene 4.4 doesn't support that anymore.</p>