<h1 dir="auto">Checklist</h1>

<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have verified that the issue exists against the <code class="notranslate">master</code> branch of Celery.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> This has already been asked to the <a href="https://groups.google.com/forum/#!forum/celery-users" rel="nofollow">discussion group</a> first.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have read the relevant section in the<br>
<a href="http://docs.celeryproject.org/en/latest/contributing.html#other-bugs" rel="nofollow">contribution guide</a><br>
on reporting bugs.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/issues?q=is%3Aissue+label%3A%22Issue+Type%3A+Bug+Report%22+-label%3A%22Category%3A+Documentation%22">issues list</a><br>
for similar or identical bug reports.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/pulls?q=is%3Apr+label%3A%22PR+Type%3A+Bugfix%22+-label%3A%22Category%3A+Documentation%22">pull requests list</a><br>
for existing proposed fixes.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/commits/master">commit log</a><br>
to find out if the bug was already fixed in the master branch.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included all related issues and possible duplicate issues<br>
in this issue (If there are none, check this box anyway).</li>
</ul>
<h2 dir="auto">Mandatory Debugging Information</h2>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included the output of <code class="notranslate">celery -A proj report</code> in the issue.<br>
(if you are not able to do this, then at least specify the Celery<br>
version affected).</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have verified that the issue exists against the <code class="notranslate">master</code> branch of Celery.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have included the contents of <code class="notranslate">pip freeze</code> in the issue. <strong>N/A</strong></li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have included all the versions of all the external dependencies required<br>
to reproduce this bug. <strong>N/A</strong></li>
</ul>
<h2 dir="auto">Optional Debugging Information</h2>

<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one Python version<br>
and/or implementation.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one message broker and/or<br>
result backend.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one version of the message<br>
broker and/or result backend.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one operating system.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one workers pool.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue with autoscaling, retries,<br>
ETA/Countdown &amp; rate limits disabled.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue after downgrading<br>
and/or upgrading Celery and its dependencies.</li>
</ul>
<h2 dir="auto">Related Issues and Possible Duplicates</h2>

<h4 dir="auto">Related Issues</h4>
<ul dir="auto">
<li><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="313039617" data-permission-text="Title is private" data-url="https://github.com/celery/celery/issues/4653" data-hovercard-type="issue" data-hovercard-url="/celery/celery/issues/4653/hovercard" href="https://github.com/celery/celery/issues/4653">#4653</a></li>
</ul>
<h4 dir="auto">Possible Duplicates</h4>
<ul dir="auto">
<li>None</li>
</ul>
<h2 dir="auto">Environment &amp; Settings</h2>

<p dir="auto"><strong>Celery version</strong>: 4.4.7 (cliffs)</p>

<details>
<summary><b><code class="notranslate">celery report</code> Output:</b></summary>
<p dir="auto">
</p><div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="software -&gt; celery:4.4.7 (cliffs) kombu:4.6.11 py:3.8.2
            billiard:3.6.3.0 py-amqp:2.6.1
platform -&gt; system:Darwin arch:64bit
            kernel version:16.7.0 imp:CPython
loader   -&gt; celery.loaders.app.AppLoader
settings -&gt; transport:amqp results:db+postgresql://&lt;redacted&gt;

include: ['redacted']
accept_content: ['redacted-custom']
database_table_names: {
 'group': 'celery_group', 'task': 'celery_task'}
result_serializer: 'redacted-custom'
task_serializer: 'redacted-custom'
task_track_started: True
broker_url: 'amqp://&lt;redacted&gt;'
result_backend: 'db+postgresql://&lt;redacted&gt;'"><pre class="notranslate"><code class="notranslate">software -&gt; celery:4.4.7 (cliffs) kombu:4.6.11 py:3.8.2
            billiard:3.6.3.0 py-amqp:2.6.1
platform -&gt; system:Darwin arch:64bit
            kernel version:16.7.0 imp:CPython
loader   -&gt; celery.loaders.app.AppLoader
settings -&gt; transport:amqp results:db+postgresql://&lt;redacted&gt;

include: ['redacted']
accept_content: ['redacted-custom']
database_table_names: {
 'group': 'celery_group', 'task': 'celery_task'}
result_serializer: 'redacted-custom'
task_serializer: 'redacted-custom'
task_track_started: True
broker_url: 'amqp://&lt;redacted&gt;'
result_backend: 'db+postgresql://&lt;redacted&gt;'
</code></pre></div>
<p dir="auto"></p>
</details>
<h1 dir="auto">Steps to Reproduce</h1>
<p dir="auto">When celery uses a database result backend, the following line can be called multiple times from different processes:</p>
<p dir="auto"></p><div class="Box Box--condensed my-2">
  <div class="Box-header f6">
    <p class="mb-0 text-bold">
      <a href="https://github.com/celery/celery/blob/9a6c2923e859b6993227605610255bd632c1ae68/celery/backends/database/session.py#L56">celery/celery/backends/database/session.py</a>
    </p>
    <p class="mb-0 color-fg-muted">
         Line 56
      in
      <a data-pjax="true" class="commit-tease-sha" href="/celery/celery/commit/9a6c2923e859b6993227605610255bd632c1ae68">9a6c292</a>
    </p>
  </div>
  <div itemprop="text" class="Box-body p-0 blob-wrapper blob-wrapper-embedded data">
    <table class="highlight tab-size mb-0 js-file-line-container" data-tab-size="8" data-paste-markdown-skip="">

        <tbody><tr class="border-0">
          <td id="L56" class="blob-num border-0 px-3 py-0 color-bg-default" data-line-number="56"></td>
          <td id="LC56" class="blob-code border-0 px-3 py-0 color-bg-default blob-code-inner js-file-line"> <span class="pl-v">ResultModelBase</span>.<span class="pl-s1">metadata</span>.<span class="pl-en">create_all</span>(<span class="pl-s1">engine</span>) </td>
        </tr>
    </tbody></table>
  </div>
</div>
<p></p>
<p dir="auto">This is a race condition because SQLAlchemy first checks if the tables/sequences exist and then tries to create them. It causes errors like this (at least on PostgreSQL):</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Traceback (most recent call last):
  File &quot;/usr/local/lib/python3.7/site-packages/redacted.py&quot;, line 168, in _redacted
    result = async_result.get()
  File &quot;/usr/local/lib/python3.7/site-packages/celery/result.py&quot;, line 226, in get
    self.maybe_throw(callback=callback)
  File &quot;/usr/local/lib/python3.7/site-packages/celery/result.py&quot;, line 342, in maybe_throw
    self.throw(value, self._to_remote_traceback(tb))
  File &quot;/usr/local/lib/python3.7/site-packages/celery/result.py&quot;, line 335, in throw
    self.on_ready.throw(*args, **kwargs)
  File &quot;/usr/local/lib/python3.7/site-packages/vine/promises.py&quot;, line 244, in throw
    reraise(type(exc), exc, tb)
  File &quot;/usr/local/lib/python3.7/site-packages/vine/five.py&quot;, line 195, in reraise
    raise value
Exception: &lt;class 'sqlalchemy.exc.IntegrityError'&gt;(('(psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint &quot;pg_type_typname_nsp_index&quot;\nDETAIL:  Key (typname, typnamespace)=(taskset_id_sequence, 2200) already exists.\n',))"><pre class="notranslate"><code class="notranslate">Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/redacted.py", line 168, in _redacted
    result = async_result.get()
  File "/usr/local/lib/python3.7/site-packages/celery/result.py", line 226, in get
    self.maybe_throw(callback=callback)
  File "/usr/local/lib/python3.7/site-packages/celery/result.py", line 342, in maybe_throw
    self.throw(value, self._to_remote_traceback(tb))
  File "/usr/local/lib/python3.7/site-packages/celery/result.py", line 335, in throw
    self.on_ready.throw(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/vine/promises.py", line 244, in throw
    reraise(type(exc), exc, tb)
  File "/usr/local/lib/python3.7/site-packages/vine/five.py", line 195, in reraise
    raise value
Exception: &lt;class 'sqlalchemy.exc.IntegrityError'&gt;(('(psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "pg_type_typname_nsp_index"\nDETAIL:  Key (typname, typnamespace)=(taskset_id_sequence, 2200) already exists.\n',))
</code></pre></div>
<p dir="auto">One workaround is  to force the table creation ahead of time as was proposed by a user in the issue I linked: <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="313039617" data-permission-text="Title is private" data-url="https://github.com/celery/celery/issues/4653" data-hovercard-type="issue" data-hovercard-url="/celery/celery/issues/4653/hovercard?comment_id=400029147&amp;comment_type=issue_comment" href="https://github.com/celery/celery/issues/4653#issuecomment-400029147">#4653 (comment)</a>.</p>
<p dir="auto">I think Celery should handle this itself. A possible solution would catch <code class="notranslate">IntegrityError</code> and try again until <code class="notranslate">create_all</code> succeeds. (Perhaps with a limited number of retries and with sleeps compared to this snippet):</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="    def prepare_models(self, engine):
        from sqlalchemy.exc import IntegrityError
        if not self.prepared:
            while True:
                try:
                    ResultModelBase.metadata.create_all(engine)
                except IntegrityError:
                    continue
                else:
                    break
            self.prepared = True"><pre class="notranslate">    <span class="pl-k">def</span> <span class="pl-en">prepare_models</span>(<span class="pl-s1">self</span>, <span class="pl-s1">engine</span>):
        <span class="pl-k">from</span> <span class="pl-s1">sqlalchemy</span>.<span class="pl-s1">exc</span> <span class="pl-k">import</span> <span class="pl-v">IntegrityError</span>
        <span class="pl-k">if</span> <span class="pl-c1">not</span> <span class="pl-s1">self</span>.<span class="pl-s1">prepared</span>:
            <span class="pl-k">while</span> <span class="pl-c1">True</span>:
                <span class="pl-k">try</span>:
                    <span class="pl-v">ResultModelBase</span>.<span class="pl-s1">metadata</span>.<span class="pl-en">create_all</span>(<span class="pl-s1">engine</span>)
                <span class="pl-k">except</span> <span class="pl-v">IntegrityError</span>:
                    <span class="pl-k">continue</span>
                <span class="pl-k">else</span>:
                    <span class="pl-k">break</span>
            <span class="pl-s1">self</span>.<span class="pl-s1">prepared</span> <span class="pl-c1">=</span> <span class="pl-c1">True</span></pre></div>
<h2 dir="auto">Minimally Reproducible Test Case</h2>

<p dir="auto">This example doesn't use celery at all, but shows that calling create_all in multiple processes can cause the error. It's a race condition, so you might need to try it multiple times or play around with the number of processes:</p>
<details>
<p dir="auto">
</p><p dir="auto">Requires a local postgres, and this database must be created:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="createdb racetest"><pre class="notranslate"><code class="notranslate">createdb racetest
</code></pre></div>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="from concurrent.futures import ProcessPoolExecutor, as_completed

from sqlalchemy import Column, Integer, Table, MetaData, create_engine

metadata = MetaData()

tbl1 = Table('tbl1', metadata, Column('id', Integer, primary_key=True))


def create_all(url):
    engine = create_engine(url)
    metadata.create_all(bind=engine)


def main():
    url = 'postgresql:///racetest'
    engine = create_engine(url)
    
    # Make sure schema is empty before we start
    metadata.drop_all(bind=engine)

    with ProcessPoolExecutor(max_workers=50) as executor:
        futures = []
        for _ in range(50):
            future = executor.submit(create_all, url)
            futures.append(future)

        for fut in as_completed(futures):
            fut.result()


if __name__ == '__main__':
    main()
"><pre class="notranslate"><span class="pl-k">from</span> <span class="pl-s1">concurrent</span>.<span class="pl-s1">futures</span> <span class="pl-k">import</span> <span class="pl-v">ProcessPoolExecutor</span>, <span class="pl-s1">as_completed</span>

<span class="pl-k">from</span> <span class="pl-s1">sqlalchemy</span> <span class="pl-k">import</span> <span class="pl-v">Column</span>, <span class="pl-v">Integer</span>, <span class="pl-v">Table</span>, <span class="pl-v">MetaData</span>, <span class="pl-s1">create_engine</span>

<span class="pl-s1">metadata</span> <span class="pl-c1">=</span> <span class="pl-v">MetaData</span>()

<span class="pl-s1">tbl1</span> <span class="pl-c1">=</span> <span class="pl-v">Table</span>(<span class="pl-s">'tbl1'</span>, <span class="pl-s1">metadata</span>, <span class="pl-v">Column</span>(<span class="pl-s">'id'</span>, <span class="pl-v">Integer</span>, <span class="pl-s1">primary_key</span><span class="pl-c1">=</span><span class="pl-c1">True</span>))


<span class="pl-k">def</span> <span class="pl-en">create_all</span>(<span class="pl-s1">url</span>):
    <span class="pl-s1">engine</span> <span class="pl-c1">=</span> <span class="pl-en">create_engine</span>(<span class="pl-s1">url</span>)
    <span class="pl-s1">metadata</span>.<span class="pl-en">create_all</span>(<span class="pl-s1">bind</span><span class="pl-c1">=</span><span class="pl-s1">engine</span>)


<span class="pl-k">def</span> <span class="pl-en">main</span>():
    <span class="pl-s1">url</span> <span class="pl-c1">=</span> <span class="pl-s">'postgresql:///racetest'</span>
    <span class="pl-s1">engine</span> <span class="pl-c1">=</span> <span class="pl-en">create_engine</span>(<span class="pl-s1">url</span>)
    
    <span class="pl-c"># Make sure schema is empty before we start</span>
    <span class="pl-s1">metadata</span>.<span class="pl-en">drop_all</span>(<span class="pl-s1">bind</span><span class="pl-c1">=</span><span class="pl-s1">engine</span>)

    <span class="pl-k">with</span> <span class="pl-v">ProcessPoolExecutor</span>(<span class="pl-s1">max_workers</span><span class="pl-c1">=</span><span class="pl-c1">50</span>) <span class="pl-k">as</span> <span class="pl-s1">executor</span>:
        <span class="pl-s1">futures</span> <span class="pl-c1">=</span> []
        <span class="pl-k">for</span> <span class="pl-s1">_</span> <span class="pl-c1">in</span> <span class="pl-en">range</span>(<span class="pl-c1">50</span>):
            <span class="pl-s1">future</span> <span class="pl-c1">=</span> <span class="pl-s1">executor</span>.<span class="pl-en">submit</span>(<span class="pl-s1">create_all</span>, <span class="pl-s1">url</span>)
            <span class="pl-s1">futures</span>.<span class="pl-en">append</span>(<span class="pl-s1">future</span>)

        <span class="pl-k">for</span> <span class="pl-s1">fut</span> <span class="pl-c1">in</span> <span class="pl-en">as_completed</span>(<span class="pl-s1">futures</span>):
            <span class="pl-s1">fut</span>.<span class="pl-en">result</span>()


<span class="pl-k">if</span> <span class="pl-s1">__name__</span> <span class="pl-c1">==</span> <span class="pl-s">'__main__'</span>:
    <span class="pl-en">main</span>()</pre></div>
<p dir="auto"></p>
</details>
<h1 dir="auto">Checklist</h1>

<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/issues?q=is%3Aissue+label%3A%22Issue+Type%3A+Enhancement%22+-label%3A%22Category%3A+Documentation%22">issues list</a><br>
for similar or identical enhancement to an existing feature.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/pulls?q=is%3Apr+label%3A%22Issue+Type%3A+Enhancement%22+-label%3A%22Category%3A+Documentation%22">pull requests list</a><br>
for existing proposed enhancements.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/commits/main">commit log</a><br>
to find out if the if the same enhancement was already implemented in the<br>
main branch.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included all related issues and possible duplicate issues in this issue<br>
(If there are none, check this box anyway).</li>
</ul>
<h2 dir="auto">Related Issues and Possible Duplicates</h2>

<h4 dir="auto">Related Issues</h4>
<ul dir="auto">
<li>None</li>
</ul>
<h4 dir="auto">Possible Duplicates</h4>
<ul dir="auto">
<li>None</li>
</ul>
<h1 dir="auto">Brief Summary</h1>

<p dir="auto">Considering <code class="notranslate">pytz</code> is deprecating in Django and will be removed in 5.0, suggest follow approaches in Django</p>
<ul dir="auto">
<li>use <code class="notranslate">zoneinfo</code> to replace <code class="notranslate">pytz</code></li>
<li>backport by <code class="notranslate">backports.zoneinfo</code></li>
</ul>
<p dir="auto">Ref: <a href="https://docs.djangoproject.com/en/4.1/releases/4.0/" rel="nofollow">https://docs.djangoproject.com/en/4.1/releases/4.0/</a></p>
<h1 dir="auto">Design</h1>
<h2 dir="auto">Architectural Considerations</h2>

<p dir="auto">None</p>
<h2 dir="auto">Proposed Behavior</h2>

<p dir="auto">No changes</p>
<h2 dir="auto">Proposed UI/UX</h2>

<p dir="auto">No changes</p>
<h2 dir="auto">Diagrams</h2>

<p dir="auto">N/A</p>
<h2 dir="auto">Alternatives</h2>

<p dir="auto">None</p>