title: logstash filter for Java logs
body: <p>I am trying to write a logstash filter for my Java logs so that I can insert them into my database cleanly.</p>
  
  <p>Below is an example of my log format:</p>
  
  <pre><code>FINE  2016-01-28 22:20:42.614+0000  net.myorg.crypto.CryptoFactory:getInstance:73:v181328
  AppName   : MyApp  AssocAppName:
  Host      : localhost  127.000.000.001  AssocHost:
  Thread    : http-bio-8080-exec-5[23]
  SequenceId: -1
  Logger    : net.myorg.crypto.CryptoFactory
  Message   : ENTRY
  ---
  FINE  2016-01-28 22:20:42.628+0000  net.myorg.crypto.CryptoFactory:getInstance:75:v181328
  AppName   : MyApp  AssocAppName:
  Host      : localhost  127.000.000.001  AssocHost:
  Thread    : http-bio-8080-exec-5[23]
  SequenceId: -1
  Logger    : net.myorg.crypto.CryptoFactory
  Message   : RETURN
  ---
  </code></pre>
  
  <p>My logstash-forwarder is pretty simple.  It just includes all logs in the directory (they all have the same format as above)</p>
  
  <pre><code>"files": [
      {
          "paths":  [ "/opt/logs/*.log" ],
          "fields": { "type": "javaLogs" }
      }
  ]
  </code></pre>
  
  <p>The trouble I'm having is on the logstash side.  How can I write a filter in logstash to match this log format?  </p>
  
  <p>Using something like this, gets me close:</p>
  
  <pre><code>filter {
  
        if [type] == "javaLogs" {
  
            multiline {
                pattern =&gt; "^%{TIMESTAMP_ISO8601}"
                negate =&gt; true
                what =&gt; "previous"
            }
        }
  }
  </code></pre>
  
  <p>But I want to break each line in the log down to its own mapping in logstash.  For example, creating fields like <code>AppName</code>, <code>AssocHost</code>, <code>Host</code>, <code>Thread</code>, etc.</p>
  
  <p>I think the answer is using <code>grok</code>.</p>