title: Why is this simple Solr highlighting attempt failing?
body: <p>I've read the Solr highlighting wiki document several times, searched everywhere, but cannot get even basic highlighting to work with my Solr installation. I am running Solr 3.5 on the demo Jetty 6.1 server. </p>
  
  <p>I have indexed 250K documents, and am able to search them just fine. Other than configuring my document field definitions, most of the Solr configuration is "stock," although I have temporarily commented out the solrconfig.xml's "Highlighting defaults" to make sure they aren't causing this problem:</p>
  
  <pre><code>  &lt;!-- Highlighting defaults
     &lt;str name="hl"&gt;on&lt;/str&gt;
     &lt;str name="hl.fl"&gt;title snippet&lt;/str&gt;
     &lt;str name="f.name.hl.fragsize"&gt;0&lt;/str&gt;
     &lt;str name="f.name.hl.alternateField"&gt;name&lt;/str&gt; --&gt;
  </code></pre>
  
  <p>My URL querystring is very simple. I've tried many variations, but here is my latest with it returning the most basic query:</p>
  
  <pre><code>hl=on&amp;hl.fl=title&amp;indent=on&amp;version=2.2&amp;q=toyota&amp;fq=&amp;start=0&amp;rows=1&amp;fl=*%2Cscore
  </code></pre>
  
  <p>Here is the resulting XML:</p>
  
  <pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;
  &lt;response&gt;
  
  &lt;lst name="responseHeader"&gt;
    &lt;int name="status"&gt;0&lt;/int&gt;
    &lt;int name="QTime"&gt;32&lt;/int&gt;
    &lt;lst name="params"&gt;
      &lt;str name="explainOther"/&gt;
      &lt;str name="indent"&gt;on&lt;/str&gt;
      &lt;str name="hl.fl"&gt;title&lt;/str&gt;
      &lt;str name="wt"/&gt;
      &lt;str name="hl"&gt;true&lt;/str&gt;
      &lt;str name="version"&gt;2.2&lt;/str&gt;
      &lt;str name="rows"&gt;1&lt;/str&gt;
      &lt;str name="fl"&gt;*,score&lt;/str&gt;
      &lt;str name="start"&gt;0&lt;/str&gt;
      &lt;str name="q"&gt;toyota&lt;/str&gt;
      &lt;str name="qt"/&gt;
      &lt;str name="fq"/&gt;
    &lt;/lst&gt;
  &lt;/lst&gt;
  &lt;result name="response" numFound="9549" start="0" maxScore="0.9960097"&gt;
    &lt;doc&gt;
      &lt;float name="score"&gt;0.9960097&lt;/float&gt;
      &lt;str name="id"&gt;2-33-200&lt;/str&gt;
      &lt;str name="title"&gt;1992 Toyota Camry 2.2L CV Boots&lt;/str&gt;
    &lt;/doc&gt;
  &lt;/result&gt;
  &lt;lst name="highlighting"&gt;
    &lt;lst name="2-33-200"/&gt;
  &lt;/lst&gt;
  &lt;/response&gt;
  </code></pre>
  
  <p>How can I debug this issue further? Thanks!</p>
  
  <p><strong>Edit</strong> Here is the <code>&lt;highlighting&gt;</code> section from solrconfig.xml. As I stated, it is stock. That could be the issue, but I'm new to Solr and not familiar with the highlighting ins and outs yet (obviously).</p>
  
  <pre><code>    &lt;highlighting&gt;
    &lt;!-- Configure the standard fragmenter --&gt;
    &lt;!-- This could most likely be commented out in the "default" case --&gt;
    &lt;fragmenter name="gap" 
                default="true"
                class="solr.highlight.GapFragmenter"&gt;
      &lt;lst name="defaults"&gt;
        &lt;int name="hl.fragsize"&gt;100&lt;/int&gt;
      &lt;/lst&gt;
    &lt;/fragmenter&gt;
  
    &lt;!-- A regular-expression-based fragmenter 
         (for sentence extraction) 
      --&gt;
    &lt;fragmenter name="regex" 
                class="solr.highlight.RegexFragmenter"&gt;
      &lt;lst name="defaults"&gt;
        &lt;!-- slightly smaller fragsizes work better because of slop --&gt;
        &lt;int name="hl.fragsize"&gt;70&lt;/int&gt;
        &lt;!-- allow 50% slop on fragment sizes --&gt;
        &lt;float name="hl.regex.slop"&gt;0.5&lt;/float&gt;
        &lt;!-- a basic sentence pattern --&gt;
        &lt;str name="hl.regex.pattern"&gt;[-\w ,/\n\&amp;quot;&amp;apos;]{20,200}&lt;/str&gt;
      &lt;/lst&gt;
    &lt;/fragmenter&gt;
  
    &lt;!-- Configure the standard formatter --&gt;
    &lt;formatter name="html" 
               default="true"
               class="solr.highlight.HtmlFormatter"&gt;
      &lt;lst name="defaults"&gt;
        &lt;str name="hl.simple.pre"&gt;&lt;![CDATA[&lt;em&gt;]]&gt;&lt;/str&gt;
        &lt;str name="hl.simple.post"&gt;&lt;![CDATA[&lt;/em&gt;]]&gt;&lt;/str&gt;
      &lt;/lst&gt;
    &lt;/formatter&gt;
  
    &lt;!-- Configure the standard encoder --&gt;
    &lt;encoder name="html" 
             class="solr.highlight.HtmlEncoder" /&gt;
  
    &lt;!-- Configure the standard fragListBuilder --&gt;
    &lt;fragListBuilder name="simple" 
                     default="true"
                     class="solr.highlight.SimpleFragListBuilder"/&gt;
  
    &lt;!-- Configure the single fragListBuilder --&gt;
    &lt;fragListBuilder name="single" 
                     class="solr.highlight.SingleFragListBuilder"/&gt;
  
    &lt;!-- default tag FragmentsBuilder --&gt;
    &lt;fragmentsBuilder name="default" 
                      default="true"
                      class="solr.highlight.ScoreOrderFragmentsBuilder"&gt;
      &lt;!-- 
      &lt;lst name="defaults"&gt;
        &lt;str name="hl.multiValuedSeparatorChar"&gt;/&lt;/str&gt;
      &lt;/lst&gt;
      --&gt;
    &lt;/fragmentsBuilder&gt;
  
    &lt;!-- multi-colored tag FragmentsBuilder --&gt;
    &lt;fragmentsBuilder name="colored" 
                      class="solr.highlight.ScoreOrderFragmentsBuilder"&gt;
      &lt;lst name="defaults"&gt;
        &lt;str name="hl.tag.pre"&gt;&lt;![CDATA[
             &lt;b style="background:yellow"&gt;,&lt;b style="background:lawgreen"&gt;,
             &lt;b style="background:aquamarine"&gt;,&lt;b style="background:magenta"&gt;,
             &lt;b style="background:palegreen"&gt;,&lt;b style="background:coral"&gt;,
             &lt;b style="background:wheat"&gt;,&lt;b style="background:khaki"&gt;,
             &lt;b style="background:lime"&gt;,&lt;b style="background:deepskyblue"&gt;]]&gt;&lt;/str&gt;
        &lt;str name="hl.tag.post"&gt;&lt;![CDATA[&lt;/b&gt;]]&gt;&lt;/str&gt;
      &lt;/lst&gt;
    &lt;/fragmentsBuilder&gt;
  
    &lt;boundaryScanner name="default" 
                     default="true"
                     class="solr.highlight.SimpleBoundaryScanner"&gt;
      &lt;lst name="defaults"&gt;
        &lt;str name="hl.bs.maxScan"&gt;10&lt;/str&gt;
        &lt;str name="hl.bs.chars"&gt;.,!? &amp;#9;&amp;#10;&amp;#13;&lt;/str&gt;
      &lt;/lst&gt;
    &lt;/boundaryScanner&gt;
  
    &lt;boundaryScanner name="breakIterator" 
                     class="solr.highlight.BreakIteratorBoundaryScanner"&gt;
      &lt;lst name="defaults"&gt;
        &lt;!-- type should be one of CHARACTER, WORD(default), LINE and SENTENCE --&gt;
        &lt;str name="hl.bs.type"&gt;WORD&lt;/str&gt;
        &lt;!-- language and country are used when constructing Locale object.  --&gt;
        &lt;!-- And the Locale object will be used when getting instance of BreakIterator --&gt;
        &lt;str name="hl.bs.language"&gt;en&lt;/str&gt;
        &lt;str name="hl.bs.country"&gt;US&lt;/str&gt;
      &lt;/lst&gt;
    &lt;/boundaryScanner&gt;
  &lt;/highlighting&gt;
  </code></pre>
  
  <p><strong>Edit</strong> Although initially my "title" field was set to indexed="false" I have since tested setting it to true (no change / no highlighting still), and also termVectors="true" termPositions="true" termOffsets="true"... still no effect. (I tried these based on reading <a href="https://stackoverflow.com/questions/3547779/solr-highlighting-problem">this post to SO</a>.)</p>
  
  <p>And here is my "title" field definition as of now:</p>
  
  <pre><code>&lt;field name="title" type="string" indexed="true" stored="true" required="true" termVectors="true" termPositions="true" termOffsets="true" /&gt;
  </code></pre>
  
  <p>Initially I started with:</p>
  
  <pre><code>&lt;field name="title" type="string" indexed="false" stored="true" required="true" /&gt;
  </code></pre>
  
  <p><strong>Edit</strong> I've now also tried this definition:</p>
  
  <pre><code>&lt;field name="title" type="text_general" indexed="true" stored="true" required="true" termVectors="true" termPositions="true" termOffsets="true" /&gt;
  </code></pre>
  
  <p>and no change in highlighting, still not working. My text_general definition is the default one that comes with Solr's demo:</p>
  
  <pre><code> &lt;!-- A general text field that has reasonable, generic
          cross-language defaults: it tokenizes with StandardTokenizer,
   removes stop words from case-insensitive "stopwords.txt"
   (empty by default), and down cases.  At query time only, it
   also applies synonyms. --&gt;
   &lt;fieldType name="text_general" class="solr.TextField" positionIncrementGap="100"&gt;
      &lt;analyzer type="index"&gt;
        &lt;tokenizer class="solr.StandardTokenizerFactory"/&gt;
        &lt;filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" /&gt;
        &lt;!-- in this example, we will only use synonyms at query time
        &lt;filter class="solr.SynonymFilterFactory" synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/&gt;
        --&gt;
        &lt;filter class="solr.LowerCaseFilterFactory"/&gt;
      &lt;/analyzer&gt;
      &lt;analyzer type="query"&gt;
        &lt;tokenizer class="solr.StandardTokenizerFactory"/&gt;
        &lt;filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" /&gt;
        &lt;filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/&gt;
        &lt;filter class="solr.LowerCaseFilterFactory"/&gt;
      &lt;/analyzer&gt;
   &lt;/fieldType&gt;
  </code></pre>
  
  <p><strong>Edit</strong> I've now also tried re-indexing title with the text_en_splitting fieldtype, which uses WhitespaceTokenizerFactory instead of StandardTokenizerFactory, and still no highlighting. For what it's worth, I am using the standard query parser, which according to debugQuery=on is the LuceneQParser.</p>
  
  <p><strong>FINALLY!</strong> Thanks to @javanna for the help. I've done a lot of experimenting, and the two key takeaways are: </p>
  
  <ol>
  <li>You must use a tokenizing field type. The string fieldtype won't work. It doesn't seem necessary to have indexed=true or termVectors=true, but the field type must be tokenized.</li>
  <li>You must be careful to refer to your fields with the proper case. In addition to screwing up the tokenizing, I had also changed the case on my fields during development, and forgot to change the case on the hl.fl (highlighted field) definition -- preventing highlighting from working. </li>
  <li>Make sure you re-index between each configuration change. To be safe, I was deleting all documents from the index, and rebuilding it from scratch, but that may not be necessary.</li>
  </ol>
  
  <p>My definition now appears as:</p>
  
  <pre><code>&lt;field name="Title" type="text_general" indexed="false" stored="true" required="true" /&gt;
  </code></pre>
  
  <p>And my solrconfig.xml has this set:</p>
  
  <pre><code>&lt;str name="hl"&gt;on&lt;/str&gt;
  &lt;str name="hl.fl"&gt;Title&lt;/str&gt;
  </code></pre>