title: logstash grok filter for custom logs
body: <p>I have two related questions.  First is how best to grok logs that have "messy" spacing and so on, and the second, which I'll ask separately, is how to deal with logs that have arbitrary attribute-value pairs. (See: <a href="https://stackoverflow.com/questions/31884204/logstash-grok-filter-for-logs-with-arbitrary-attribute-value-pairs">logstash grok filter for logs with arbitrary attribute-value pairs</a> )</p>
  
  <p>So for the first question, I have a log line that looks like this:</p>
  
  <pre><code>14:46:16.603 [http-nio-8080-exec-4] INFO  METERING - msg=93e6dd5e-c009-46b3-b9eb-f753ee3b889a CREATE_JOB job=a820018e-7ad7-481a-97b0-bd705c3280ad data=71b1652e-16c8-4b33-9a57-f5fcb3d5de92
  </code></pre>
  
  <p>Using <a href="http://grokdebug.herokuapp.com/" rel="nofollow noreferrer">http://grokdebug.herokuapp.com/</a> I was able to eventually come up with the following grok pattern that works for this line:</p>
  
  <pre><code>%{TIME:timestamp} %{NOTSPACE:http} %{WORD:loglevel}%{SPACE}%{WORD:logtype} - msg=%{NOTSPACE:msg}%{SPACE}%{WORD:action}%{SPACE}job=%{NOTSPACE:job}%{SPACE}data=%{NOTSPACE:data}
  </code></pre>
  
  <p>With the following config file:</p>
  
  <pre><code>input {
          file {
                  path =&gt; "/home/robyn/testlogs/trimmed_logs.txt"
                  start_position =&gt; beginning
                  sincedb_path =&gt; "/dev/null" # for testing; allows reparsing
          }
  }
  filter {
          grok {
                  match =&gt; {"message" =&gt; "%{TIME:timestamp} %{NOTSPACE:http} %{WORD:loglevel}%{SPACE}%{WORD:logtype} - msg=%{NOTSPACE:msg}%{SPACE}%{WORD:action}%{SPACE}job=%{NOTSPACE:job}%{SPACE}data=%{NOTSPACE:data}" }
          }
  }
  output {
          file {
                  path =&gt; "/home/robyn/filteredlogs/trimmed_logs.out.txt"
          }
  }
  </code></pre>
  
  <p>I get the following output:</p>
  
  <pre><code>{"message":"14:46:16.603 [http-nio-8080-exec-4] INFO  METERING - msg=93e6dd5e-c009-46b3-b9eb-f753ee3b889a CREATE_JOB job=a820018e-7ad7-481a-97b0-bd705c3280ad data=71b1652e-16c8-4b33-9a57-f5fcb3d5de92","@version":"1","@timestamp":"2015-08-07 T17:55:16.529Z","host":"hlt-dev","path":"/home/robyn/testlogs/trimmed_logs.txt","timestamp":"14:46:16.603","http":"[http-nio-8080-exec-4]","loglevel":"INFO","logtype":"METERING","msg":"93e6dd5e-c009-46b3-b9eb-f753ee3b889a","action":"CREATE_JOB","job":"a820018e-7ad7-481a-97b0-bd705c3280ad","data":"71b1652e-16c8-4b33-9a57-f5fcb3d5de92"}
  </code></pre>
  
  <p>That's pretty much what I want, but I feel like it's a really kludgy pattern, particularly with the need to use %{SPACE} and %{NOSPACE} so much.  This suggests to me that I'm not really doing this the best possible way.  Should I be creating a more specific pattern for the hex ids?  I think I need the %{SPACE} between loglevel and logtype because of the extra space between INFO and METERING in the log, but that also feels kludgy.  </p>
  
  <p>Also how do I get the log's timestamp to replace the @timestamp that seems to be the time logstash ingested the log, which we don't want/need.</p>
  
  <p>Obviously I'm just getting started with ELK and grok, so pointers to useful resources are also appreciated.</p>