title: Hierarchical clustering of different length time series data using scipy and DTW
body: <p>I have a set of time series data having different lengths and I am trying to cluster them using Dynamic Time Warping (DTW). </p>
  
  <p>For the completeness of the question, I am using <a href="https://nbviewer.jupyter.org/github/alexminnaar/time-series-classification-and-clustering/blob/master/Time%20Series%20Classification%20and%20Clustering.ipynb" rel="nofollow noreferrer">this</a> simple implementation of DTW </p>
  
  <pre><code>def DTWDistance(s1, s2):
      DTW={}
  
      for i in range(len(s1)):
          DTW[(i, -1)] = float('inf')
      for i in range(len(s2)):
          DTW[(-1, i)] = float('inf')
      DTW[(-1, -1)] = 0
  
      for i in range(len(s1)):
          for j in range(len(s2)):
              dist= (s1[i]-s2[j])**2
              DTW[(i, j)] = dist + min(DTW[(i-1, j)],DTW[(i, j-1)], DTW[(i-1, j-1)])
  
      return sqrt(DTW[len(s1)-1, len(s2)-1])
  </code></pre>
  
  <p>and I have my time series data of the form </p>
  
  <pre><code>timeseries = [[0, 1, 2, 3, 4, 5, 6, 7, 8],
               [0, 0, 1, 2, 3, 4],
               [6, 7, 8, 9, 10, 11, 12, 13, 14],
               [15, 14, 13, 14, 15, 16, 17, 18, 19]]
  </code></pre>
  
  <p>When I try</p>
  
  <pre><code>import scipy.cluster.hierarchy as hac
  
  Z = hac.linkage(data, method='complete', metric= DTWDistance)
  </code></pre>
  
  <p>I get <code>ValueError: setting an array element with a sequence.</code>, which is understandable because <a href="https://docs.scipy.org/doc/scipy/reference/generated/scipy.cluster.hierarchy.linkage.html#scipy-cluster-hierarchy-linkage" rel="nofollow noreferrer"><code>scipy.cluster.hierarchy.linkage</code></a> documentation says </p>
  
  <blockquote>
    <p>...a collection of <em>m</em> observation vectors in <em>n</em> dimensions may be passed
    as an <em>m</em> by <em>n</em> array. All elements of the condensed distance matrix must
    be finite, i.e. no NaNs or infs.</p>
  </blockquote>
  
  <p>And clearly my input doesn't fulfil this specification. What will be the correct approach to classify different length time series data?</p>
  
  <p><strong>EDIT 1</strong></p>
  
  <p>A simple workaround will be to fill the missing entries for shorter timeseries with 0, so that we can obtain an <em>m</em> x <em>n</em> matrix as required. I am not sure if this will alter the semantics of the timeseries. </p>