Answers: <p>Fixed by creating a mapping for the timestamp field:</p>
  <pre><code>{
    &quot;index_patterns&quot;: &quot;httpd-*&quot;,
    &quot;mappings&quot;: {
      &quot;dynamic&quot;: true,
      &quot;properties&quot;: {
          &quot;request&quot;: {
            &quot;type&quot;: &quot;text&quot;
          },
          &quot;auth&quot;: {
            &quot;type&quot;: &quot;text&quot;
          },
          &quot;ident&quot;: {
            &quot;type&quot;: &quot;text&quot;
          },
          &quot;verb&quot;: {
            &quot;type&quot;: &quot;text&quot;
          },
          &quot;response&quot;: {
            &quot;type&quot;: &quot;text&quot;
          },
          &quot;bytes&quot;: {
            &quot;type&quot;: &quot;integer&quot;
          },
          &quot;clientip&quot;: {
            &quot;type&quot;: &quot;ip&quot;
          },
          &quot;httpversion&quot;: {
            &quot;type&quot;: &quot;text&quot;
          },
          &quot;timestamp&quot;: {
            &quot;type&quot;: &quot;date&quot;,
            &quot;format&quot;: &quot;dd/MMM/yyyy:HH:mm:ss Z&quot;
          }
      }
    }
  </code></pre>
  
Answers: CC BY-SA 4.0
Answers: 2022-04-28T00:39:55.020Z
Answers: 2022-04-28T00:39:55.020Z
Answers: chkm0052
Body: <p>I'm getting a date parse failure when trying to parse a log that has the following format:</p>
  <pre><code>172.20.21.10 - - [29/Mar/2022:12:41:27 -0400] &quot;some invalid request&quot; 400 226
  </code></pre>
  <p>The event structure looks like this:</p>
  <pre><code>{
          &quot;request&quot; =&gt; &quot;/um/&quot;,
             &quot;auth&quot; =&gt; &quot;-&quot;,
            &quot;ident&quot; =&gt; &quot;-&quot;,
        &quot;@metadata&quot; =&gt; {
          &quot;path&quot; =&gt; &quot;C:/net-logs/m4-httpd-logs/wordpress_access.log-20220403&quot;,
          &quot;host&quot; =&gt; &quot;M6&quot;
      },
             &quot;verb&quot; =&gt; &quot;GET&quot;,
          &quot;message&quot; =&gt; &quot;172.20.21.10 - - [29/Mar/2022:12:44:36 -0400] \&quot;GET /um/ HTTP/1.1\&quot; 404 17787&quot;,
             &quot;path&quot; =&gt; &quot;C:/net-logs/m4-httpd-logs/wordpress_access.log-20220403&quot;,
            &quot;bytes&quot; =&gt; &quot;17787&quot;,
         &quot;response&quot; =&gt; &quot;404&quot;,
         &quot;clientip&quot; =&gt; &quot;172.20.21.10&quot;,
         &quot;@version&quot; =&gt; &quot;1&quot;,
             &quot;host&quot; =&gt; &quot;M6&quot;,
      &quot;httpversion&quot; =&gt; &quot;1.1&quot;,
        &quot;timestamp&quot; =&gt; &quot;29/Mar/2022:12:44:36 -0400&quot;
  }
  </code></pre>
  <p>To convert the timestamp field from text to a date I have a filter in the logstash configuration file that looks like this:</p>
  <pre><code>filter {
          grok {
                match =&gt; {&quot;message&quot; =&gt; &quot;%{COMMONAPACHELOG}&quot;}
          }
          mutate {
                  remove_field =&gt; [ &quot;@timestamp&quot;]
          }
          date {
                match =&gt; [&quot;timestamp&quot;, &quot;dd/MM/yyyy:HH:mm:ss Z&quot;]
                target =&gt; [&quot;timestamp&quot;]
          }
  }
  </code></pre>
  <p>I have used the correct syntax for the format according to the <a href="https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html" rel="nofollow noreferrer" title="plugins-filters-date">documentation</a> for the date filter. I'm not sure whats causing the parse failure since the error msg does not show exactly why the parse failed. It looks like this:</p>
  <pre><code>{
          &quot;request&quot; =&gt; &quot;/wp-includes/IXR/login&quot;,
             &quot;auth&quot; =&gt; &quot;-&quot;,
            &quot;ident&quot; =&gt; &quot;-&quot;,
        &quot;@metadata&quot; =&gt; {
          &quot;path&quot; =&gt; &quot;C:/net-logs/m4-httpd-logs/wordpress_access.log-20220403&quot;,
          &quot;host&quot; =&gt; &quot;M6&quot;
      },
             &quot;verb&quot; =&gt; &quot;GET&quot;,
          &quot;message&quot; =&gt; &quot;172.20.21.10 - - [29/Mar/2022:12:47:57 -0400] \&quot;GET /wp-includes/IXR/login HTTP/1.1\&quot; 404 17787&quot;,
             &quot;tags&quot; =&gt; [
          [0] &quot;_dateparsefailure&quot;
      ],
             &quot;path&quot; =&gt; &quot;C:/net-logs/m4-httpd-logs/wordpress_access.log-20220403&quot;,
         &quot;response&quot; =&gt; &quot;404&quot;,
            &quot;bytes&quot; =&gt; &quot;17787&quot;,
         &quot;clientip&quot; =&gt; &quot;172.20.21.10&quot;,
         &quot;@version&quot; =&gt; &quot;1&quot;,
             &quot;host&quot; =&gt; &quot;M6&quot;,
      &quot;httpversion&quot; =&gt; &quot;1.1&quot;,
        &quot;timestamp&quot; =&gt; &quot;29/Mar/2022:12:47:57 -0400&quot;
  }
  </code></pre>
  
Tags: <logstash>
Title: How do I troubleshoot a dateparsefailure when using Logstash's date plugin?
UserDisplayName: chkm0052
ThreadText: # How do I troubleshoot a dateparsefailure when using Logstash's date plugin?
  
  (Asked by: username_0 on 2022-04-27)
  
  I'm getting a date parse failure when trying to parse a log that has the following format:
  
  ```
  172.20.21.10 - - [29/Mar/2022:12:41:27 -0400] "some invalid request" 400 226
  
  ```
  
  The event structure looks like this:
  
  ```
  {
          "request" => "/um/",
             "auth" => "-",
            "ident" => "-",
        "@metadata" => {
          "path" => "C:/net-logs/m4-httpd-logs/wordpress_access.log-20220403",
          "host" => "M6"
      },
             "verb" => "GET",
          "message" => "172.20.21.10 - - [29/Mar/2022:12:44:36 -0400] \"GET /um/ HTTP/1.1\" 404 17787",
             "path" => "C:/net-logs/m4-httpd-logs/wordpress_access.log-20220403",
            "bytes" => "17787",
         "response" => "404",
         "clientip" => "172.20.21.10",
         "@version" => "1",
             "host" => "M6",
      "httpversion" => "1.1",
        "timestamp" => "29/Mar/2022:12:44:36 -0400"
  }
  
  ```
  
  To convert the timestamp field from text to a date I have a filter in the logstash configuration file that looks like this:
  
  ```
  filter {
          grok {
                match => {"message" => "%{COMMONAPACHELOG}"}
          }
          mutate {
                  remove_field => [ "@timestamp"]
          }
          date {
                match => ["timestamp", "dd/MM/yyyy:HH:mm:ss Z"]
                target => ["timestamp"]
          }
  }
  
  ```
  
  I have used the correct syntax for the format according to the [documentation](https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html "plugins-filters-date") for the date filter. I'm not sure whats causing the parse failure since the error msg does not show exactly why the parse failed. It looks like this:
  
  ```
  {
          "request" => "/wp-includes/IXR/login",
             "auth" => "-",
            "ident" => "-",
        "@metadata" => {
          "path" => "C:/net-logs/m4-httpd-logs/wordpress_access.log-20220403",
          "host" => "M6"
      },
             "verb" => "GET",
          "message" => "172.20.21.10 - - [29/Mar/2022:12:47:57 -0400] \"GET /wp-includes/IXR/login HTTP/1.1\" 404 17787",
             "tags" => [
          [0] "_dateparsefailure"
      ],
             "path" => "C:/net-logs/m4-httpd-logs/wordpress_access.log-20220403",
         "response" => "404",
            "bytes" => "17787",
         "clientip" => "172.20.21.10",
         "@version" => "1",
             "host" => "M6",
      "httpversion" => "1.1",
        "timestamp" => "29/Mar/2022:12:47:57 -0400"
  }
  
  ```
  
  **Tags:** logstash
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2022-04-28. Score: 0
  
  Fixed by creating a mapping for the timestamp field:
  
  ```
  {
    "index_patterns": "httpd-*",
    "mappings": {
      "dynamic": true,
      "properties": {
          "request": {
            "type": "text"
          },
          "auth": {
            "type": "text"
          },
          "ident": {
            "type": "text"
          },
          "verb": {
            "type": "text"
          },
          "response": {
            "type": "text"
          },
          "bytes": {
            "type": "integer"
          },
          "clientip": {
            "type": "ip"
          },
          "httpversion": {
            "type": "text"
          },
          "timestamp": {
            "type": "date",
            "format": "dd/MMM/yyyy:HH:mm:ss Z"
          }
      }
    }
  
  ```
  
  
  ---
  
  