title: Loading data into Biqquery Partitioned table through Google Dataflow/Beam with write_truncate
body: <p>So the existing setup we had use to create a new table for each day, which worked fine with <a href="https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/google_cloud_platform/bigquery.py#L451" rel="nofollow noreferrer">"WRITE_TRUNCATE"</a> option, however when we updated our code to use partitioned table, though our dataflow job, it wouldn`t work with write_truncate.</p>
  
  <p>It works perfectly fine, with write disposition set as "WRITE_APPEND" (From what i understood, from <a href="https://github.com/apache/beam/blob/2689ca43c634c2f1bc57132db4fb2391489534c4/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java#L1534-L1539" rel="nofollow noreferrer">beam</a>, it maybe tries to delete the table, and then recreate it), since i`m supplying the table decorator it fails to create a new table.</p>
  
  <p>Sample snippet using python code:</p>
  
  <pre><code>beam.io.Write('Write({})'.format(date), beam.io.BigQuerySink(output_table_name + '$' + date, create_disposition=beam.io.BigQueryDisposition.CREATE_NEVER, write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE)
  </code></pre>
  
  <p>which gives the error:</p>
  
  <blockquote>
    <p>Table IDs must be alphanumeric</p>
  </blockquote>
  
  <p>since it tried to recreate the table, and we supply the partition decorator in the argument.</p>
  
  <p>Here are some of the things that i`v tried:</p>
  
  <ol>
  <li>Updating the write_disposition as WRITE_APPEND, although it works, it fails the purpose, since running for the same date again would duplicate data.</li>
  <li>Using </li>
  </ol>
  
  <p><code>bq --apilog /tmp/log.txt load  --replace --source_format=NEWLINE_DELIMITED_JSON 'table.$20160101' sample_json.json</code> </p>
  
  <p>command, to see if i can observe any logs, on how does truncate actually works, based on the <a href="https://cloud.google.com/bigquery/docs/creating-partitioned-tables#restating_data_in_a_partition" rel="nofollow noreferrer">link</a> that i found.</p>
  
  <ol start="3">
  <li>Tried some other <a href="https://stackoverflow.com/questions/38114306/creating-writing-to-parititoned-bigquery-table-via-google-cloud-dataflow">links</a>, but this as well uses WRITE_APPEND.</li>
  </ol>
  
  <p>Is there a way to write to a partitioned table, from a dataflow job using write_truncate method?</p>
  
  <p>Let me know if any additional details are required.
  Thanks</p>