<h3 dir="auto">Apache Airflow version</h3>
<p dir="auto">main (development)</p>
<h3 dir="auto">What happened</h3>
<p dir="auto">I have 16 dags which all update the same dataset.  They're set to finish at the same time (when the seconds on the clock are 00). About three quarters of them behave as expected, but the other quarter fails with errors like:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="[2022-07-21, 06:06:00 UTC] {standard_task_runner.py:97} ERROR - Failed to execute job 8 for task increment_source ((psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint &quot;dataset_dag_run_queue_pkey&quot;
DETAIL:  Key (dataset_id, target_dag_id)=(1, simple_dataset_sink) already exists.

[SQL: INSERT INTO dataset_dag_run_queue (dataset_id, target_dag_id, created_at) VALUES (%(dataset_id)s, %(target_dag_id)s, %(created_at)s)]
[parameters: {'dataset_id': 1, 'target_dag_id': 'simple_dataset_sink', 'created_at': datetime.datetime(2022, 7, 21, 6, 6, 0, 131730, tzinfo=Timezone('UTC'))}]
(Background on this error at: https://sqlalche.me/e/14/gkpj); 375)"><pre class="notranslate"><code class="notranslate">[2022-07-21, 06:06:00 UTC] {standard_task_runner.py:97} ERROR - Failed to execute job 8 for task increment_source ((psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "dataset_dag_run_queue_pkey"
DETAIL:  Key (dataset_id, target_dag_id)=(1, simple_dataset_sink) already exists.

[SQL: INSERT INTO dataset_dag_run_queue (dataset_id, target_dag_id, created_at) VALUES (%(dataset_id)s, %(target_dag_id)s, %(created_at)s)]
[parameters: {'dataset_id': 1, 'target_dag_id': 'simple_dataset_sink', 'created_at': datetime.datetime(2022, 7, 21, 6, 6, 0, 131730, tzinfo=Timezone('UTC'))}]
(Background on this error at: https://sqlalche.me/e/14/gkpj); 375)
</code></pre></div>
<p dir="auto">I've prepaired a gist with the details: <a href="https://gist.github.com/MatrixManAtYrService/b5e58be0949eab9180608d0760288d4d">https://gist.github.com/MatrixManAtYrService/b5e58be0949eab9180608d0760288d4d</a></p>
<h3 dir="auto">What you think should happen instead</h3>
<p dir="auto">All dags should succeed</p>
<h3 dir="auto">How to reproduce</h3>
<p dir="auto">See this gist:  <a href="https://gist.github.com/MatrixManAtYrService/b5e58be0949eab9180608d0760288d4d">https://gist.github.com/MatrixManAtYrService/b5e58be0949eab9180608d0760288d4d</a></p>
<p dir="auto">Summary:  Unpause all of the dags which we expect to collide, wait two minutes.  Some will have collided.</p>
<h3 dir="auto">Operating System</h3>
<p dir="auto">docker/debian</p>
<h3 dir="auto">Versions of Apache Airflow Providers</h3>
<p dir="auto">n/a</p>
<h3 dir="auto">Deployment</h3>
<p dir="auto">Astronomer</p>
<h3 dir="auto">Deployment details</h3>
<p dir="auto"><code class="notranslate">astro dev start</code> targeting commit: <a class="commit-link" data-hovercard-type="commit" data-hovercard-url="https://github.com/apache/airflow/commit/cff7d9194f549d801947f47dfce4b5d6870bfaaa/hovercard" href="https://github.com/apache/airflow/commit/cff7d9194f549d801947f47dfce4b5d6870bfaaa"><tt>cff7d91</tt></a></p>
<p dir="auto">be sure to have <code class="notranslate">pause</code> in requirements.txt</p>
<h3 dir="auto">Anything else</h3>
<p dir="auto"><em>No response</em></p>
<h3 dir="auto">Are you willing to submit PR?</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> Yes I am willing to submit a PR!</li>
</ul>
<h3 dir="auto">Code of Conduct</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I agree to follow this project's <a href="https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md">Code of Conduct</a></li>
</ul>
<h3 dir="auto">Description</h3>
<p dir="auto">Currently google cloud provider restricts the version of google-video-intellegiance library to &gt;=1.7.0,&lt;2.0.0.<br>
However this version old and latest available version is 2.8.1.<br>
This is also mentioend as one of the - <a href="https://github.com/apache/airflow/blob/00fd789258b48d92b16294b598956db271bc7b3d/airflow/providers/google/provider.yaml#L65">ToDos </a></p>
<h3 dir="auto">Use case/motivation</h3>
<p dir="auto">Version &gt;2.0.0 offers features and v2.0.0 is couple of years old.</p>
<h3 dir="auto">Related issues</h3>
<p dir="auto"><em>No response</em></p>
<h3 dir="auto">Are you willing to submit a PR?</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> Yes I am willing to submit a PR!</li>
</ul>
<h3 dir="auto">Code of Conduct</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I agree to follow this project's <a href="https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md">Code of Conduct</a></li>
</ul>