title: How to map a json string into object type in sink transformation
body: <p>Using Azure Data Factory and a data transformation flow. I have a csv that contains a column with a json object string, below an example including the header:</p>
  <pre><code>&quot;Id&quot;,&quot;Name&quot;,&quot;Timestamp&quot;,&quot;Value&quot;,&quot;Metadata&quot;
  &quot;99c9347ab7c34733a4fe0623e1496ffd&quot;,&quot;data1&quot;,&quot;2021-03-18 05:53:00.0000000&quot;,&quot;0&quot;,&quot;{&quot;&quot;unit&quot;&quot;:&quot;&quot;%&quot;&quot;}&quot;
  &quot;99c9347ab7c34733a4fe0623e1496ffd&quot;,&quot;data1&quot;,&quot;2021-03-19 05:53:00.0000000&quot;,&quot;4&quot;,&quot;{&quot;&quot;jobName&quot;&quot;:&quot;&quot;RecipeB&quot;&quot;}&quot;
  &quot;99c9347ab7c34733a4fe0623e1496ffd&quot;,&quot;data1&quot;,&quot;2021-03-16 02:12:30.0000000&quot;,&quot;state&quot;,&quot;{&quot;&quot;jobEndState&quot;&quot;:&quot;&quot;negative&quot;&quot;}&quot;
  &quot;99c9347ab7c34733a4fe0623e1496ffd&quot;,&quot;data1&quot;,&quot;2021-03-19 06:33:00.0000000&quot;,&quot;23&quot;,&quot;{&quot;&quot;unit&quot;&quot;:&quot;&quot;kg&quot;&quot;}&quot;
  </code></pre>
  <p>Want to store the data in a json like this:</p>
  <pre><code>{
      &quot;id&quot;: &quot;99c9347ab7c34733a4fe0623e1496ffd&quot;,
      &quot;name&quot;: &quot;data1&quot;,
      &quot;values&quot;: [
          {
              &quot;timestamp&quot;: &quot;2021-03-18 05:53:00.0000000&quot;,
              &quot;value&quot;: &quot;0&quot;,
              &quot;metadata&quot;: {
                  &quot;unit&quot;: &quot;%&quot;
              }
          },
          {
              &quot;timestamp&quot;: &quot;2021-03-19 05:53:00.0000000&quot;,
              &quot;value&quot;: &quot;4&quot;,
              &quot;metadata&quot;: {
                  &quot;jobName&quot;: &quot;RecipeB&quot;
              }
          }
         ....
      ]
  }
  </code></pre>
  <p>The challenge is that metadata has dynamic content, meaning, that it will be always a json object but the content can vary. Therefore I cannot define a schema. Currently the column &quot;metadata&quot; on the sink schema is defined as object, but whenever I run the transformation I run into an exception:</p>
  <pre><code>Conversion from ArrayType(StructType(StructField(timestamp,StringType,false), 
  StructField(value,StringType,false), StructField(metadata,StringType,false)),true) to ArrayType(StructType(StructField(timestamp,StringType,true), 
  StructField(value,StringType,true), StructField(metadata,StructType(StructField(,StringType,true)),true)),false) not defined
  </code></pre>