<h1 dir="auto">Checklist</h1>

<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have verified that the issue exists against the <code class="notranslate">master</code> branch of Celery.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> This has already been asked to the <a href="https://github.com/celery/celery/discussions">discussions forum</a> first.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have read the relevant section in the<br>
<a href="https://docs.celeryq.dev/en/master/contributing.html#other-bugs" rel="nofollow">contribution guide</a><br>
on reporting bugs.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/issues?q=is%3Aissue+label%3A%22Issue+Type%3A+Bug+Report%22+-label%3A%22Category%3A+Documentation%22">issues list</a><br>
for similar or identical bug reports.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have checked the <a href="https://github.com/celery/celery/pulls?q=is%3Apr+label%3A%22PR+Type%3A+Bugfix%22+-label%3A%22Category%3A+Documentation%22">pull requests list</a><br>
for existing proposed fixes.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/commits/master">commit log</a><br>
to find out if the bug was already fixed in the master branch.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included all related issues and possible duplicate issues<br>
in this issue (If there are none, check this box anyway).</li>
</ul>
<h2 dir="auto">Mandatory Debugging Information</h2>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included the output of <code class="notranslate">celery -A proj report</code> in the issue.<br>
(if you are not able to do this, then at least specify the Celery<br>
version affected).</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have verified that the issue exists against the <code class="notranslate">master</code> branch of Celery.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included the contents of <code class="notranslate">pip freeze</code> in the issue.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included all the versions of all the external dependencies required<br>
to reproduce this bug.</li>
</ul>
<h2 dir="auto">Optional Debugging Information</h2>

<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have tried reproducing the issue on more than one Python version<br>
and/or implementation.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have tried reproducing the issue on more than one message broker and/or<br>
result backend.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have tried reproducing the issue on more than one version of the message<br>
broker and/or result backend.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have tried reproducing the issue on more than one operating system.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have tried reproducing the issue on more than one workers pool.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have tried reproducing the issue with autoscaling, retries,<br>
ETA/Countdown &amp; rate limits disabled.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have tried reproducing the issue after downgrading<br>
and/or upgrading Celery and its dependencies.</li>
</ul>
<h2 dir="auto">Related Issues and Possible Duplicates</h2>

<h4 dir="auto">Related Issues</h4>
<ul dir="auto">
<li>None</li>
</ul>
<h4 dir="auto">Possible Duplicates</h4>
<ul dir="auto">
<li>None</li>
</ul>
<h2 dir="auto">Environment &amp; Settings</h2>

<p dir="auto"><strong>Celery version</strong>:</p>

<details>
<summary><b><code class="notranslate">celery report</code> Output:</b></summary>
<p dir="auto">
</p><div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content=""><pre class="notranslate"><code class="notranslate"></code></pre></div>
<p dir="auto"></p>
</details>
<h1 dir="auto">Steps to Reproduce</h1>
<p dir="auto">celery dockerfile build failed<br>
I want to build a base image for my django project,my project dependencies requirements.txt like below<br>
requirements.txt</p>
<div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="amqp==2.6.1
anyjson==0.3.3
appdirs==1.4.4
asgiref==3.2.7
billiard==3.6.3.0
celery==4.4.0
celery-with-redis==3.0
certifi==2020.6.20
cffi==1.14.0
chardet==3.0.4
click==7.1.2
click-didyoumean==0.0.3
click-repl==0.1.6 
coreapi==2.3.3
coreschema==0.0.4
cryptography==3.2
distlib==0.3.1
Django==2.2.16
django-celery-beat==1.1.0
django-celery-results==1.0.0
django-cors-headers==3.5.0
django-extensions==3.1.0
django-filter==2.4.0
django-timezone-field==4.0
django-werkzeug-debugger-runserver==0.3.1
djangorestframework==3.12.1
dnspython==1.16.0
drf-extensions==0.6.0
ephem==3.7.7.1
eventlet==0.25.2
filelock==3.0.12
flower==0.9.5
greenlet==0.4.15
humanize==3.0.1
idna==2.10
importlib-metadata==2.0.0
importlib-resources==3.0.0
install==1.3.4
itypes==1.2.0
Jinja2==2.11.2
kombu==4.6.11
Markdown==3.3.3
MarkupSafe==1.1.1
monotonic==1.5
msgpack-python==0.5.6
prometheus-client==0.8.0
prompt-toolkit==3.0.8
pycparser==2.20
PyMySQL==0.10.1
pyOpenSSL==19.1.0
python-crontab==2.4.2
python-dateutil==2.8.1
pytz==2020.1
redis==3.2.0
requests==2.24.0
six==1.15.0
sqlparse==0.4.1
tornado==6.0.4
uritemplate==3.0.1
urllib3==1.25.10
vine==1.3.0
virtualenv==20.0.35
wcwidth==0.2.5
Werkzeug==1.0.1
zipp==3.3.1
gunicorn==19.8.1
gevent==1.3.4"><pre class="notranslate">amqp==2.6.1
anyjson==0.3.3
appdirs==1.4.4
asgiref==3.2.7
billiard==3.6.3.0
celery==4.4.0
celery-with-redis==3.0
certifi==2020.6.20
cffi==1.14.0
chardet==3.0.4
click==7.1.2
click-didyoumean==0.0.3
click-repl==0.1.6 
coreapi==2.3.3
coreschema==0.0.4
cryptography==3.2
distlib==0.3.1
Django==2.2.16
django-celery-beat==1.1.0
django-celery-results==1.0.0
django-cors-headers==3.5.0
django-extensions==3.1.0
django-filter==2.4.0
django-timezone-field==4.0
django-werkzeug-debugger-runserver==0.3.1
djangorestframework==3.12.1
dnspython==1.16.0
drf-extensions==0.6.0
ephem==3.7.7.1
eventlet==0.25.2
filelock==3.0.12
flower==0.9.5
greenlet==0.4.15
humanize==3.0.1
idna==2.10
importlib-metadata==2.0.0
importlib-resources==3.0.0
install==1.3.4
itypes==1.2.0
Jinja2==2.11.2
kombu==4.6.11
Markdown==3.3.3
MarkupSafe==1.1.1
monotonic==1.5
msgpack-python==0.5.6
prometheus-client==0.8.0
prompt-toolkit==3.0.8
pycparser==2.20
PyMySQL==0.10.1
pyOpenSSL==19.1.0
python-crontab==2.4.2
python-dateutil==2.8.1
pytz==2020.1
redis==3.2.0
requests==2.24.0
six==1.15.0
sqlparse==0.4.1
tornado==6.0.4
uritemplate==3.0.1
urllib3==1.25.10
vine==1.3.0
virtualenv==20.0.35
wcwidth==0.2.5
Werkzeug==1.0.1
zipp==3.3.1
gunicorn==19.8.1
gevent==1.3.4</pre></div>
<p dir="auto">my Dockerfile</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="FROM python3.6.4
WORKDIR /data/project
COPY . /data/project

RUN pip install -r requirements.txt"><pre class="notranslate"><code class="notranslate">FROM python3.6.4
WORKDIR /data/project
COPY . /data/project

RUN pip install -r requirements.txt
</code></pre></div>
<p dir="auto">my docker version</p>
<div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="Docker version 20.10.12, build e91ed57"><pre class="notranslate">Docker version 20.10.12, build e91ed57</pre></div>
<p dir="auto">when i run</p>
<div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="docker build . -t project"><pre class="notranslate">docker build <span class="pl-c1">.</span> -t project</pre></div>
<p dir="auto">however, an error occured like below</p>
<div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="ERROR: Cannot install -r ./requirements.txt (line 51) and celery==4.4.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested celery==4.4.0
    celery-with-redis 3.0 depends on celery&lt;4.0 and &gt;=3.0

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies
The command '/bin/sh -c rm -f /etc/localtime &amp;&amp;     ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &amp;&amp;     echo 'Asia/Shanghai' &gt; /etc/timezone &amp;&amp;     chmod 777 /opt &amp;&amp;     pip3 install -U pip setuptools &amp;&amp;     pip install -r ./requirements.txt' returned a non-zero code: 1
ERROR: Job failed: exit code 1
``
## Required Dependencies
&lt;!-- Please fill the required dependencies to reproduce this issue --&gt;
* **Minimal Python Version**: N/A or Unknown
* **Minimal Celery Version**: N/A or Unknown
* **Minimal Kombu Version**: N/A or Unknown
* **Minimal Broker Version**: N/A or Unknown
* **Minimal Result Backend Version**: N/A or Unknown
* **Minimal OS and/or Kernel Version**: N/A or Unknown
* **Minimal Broker Client Version**: N/A or Unknown
* **Minimal Result Backend Client Version**: N/A or Unknown


# Expected Behavior
&lt;!-- Describe in detail what you expect to happen --&gt;
excepted to build docker image success
# Actual Behavior
&lt;!--
Describe in detail what actually happened.
Please include a backtrace and surround it with triple backticks (```).
In addition, include the Celery daemon logs, the broker logs,
the result backend logs and system logs below if they will help us debug
the issue.
--&gt;
how can i solve the celery dependency conflicts,thanks~"><pre class="notranslate">ERROR: Cannot install -r ./requirements.txt (line 51) and celery==4.4.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested celery==4.4.0
    celery-with-redis 3.0 depends on celery<span class="pl-k">&lt;</span>4.0 and <span class="pl-k">&gt;</span>=3.0

To fix this you could try to:
1. loosen the range of package versions you<span class="pl-s"><span class="pl-pds">'</span>ve specified</span>
<span class="pl-s">2. remove package versions to allow pip attempt to solve the dependency conflict</span>
<span class="pl-s"></span>
<span class="pl-s">ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies</span>
<span class="pl-s">The command <span class="pl-pds">'</span></span>/bin/sh -c rm -f /etc/localtime <span class="pl-k">&amp;&amp;</span>     ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime <span class="pl-k">&amp;&amp;</span>     <span class="pl-c1">echo</span> <span class="pl-s"><span class="pl-pds">'</span>Asia/Shanghai<span class="pl-pds">'</span></span> <span class="pl-k">&gt;</span> /etc/timezone <span class="pl-k">&amp;&amp;</span>     chmod 777 /opt <span class="pl-k">&amp;&amp;</span>     pip3 install -U pip setuptools <span class="pl-k">&amp;&amp;</span>     pip install -r ./requirements.txt<span class="pl-s"><span class="pl-pds">'</span> returned a non-zero code: 1</span>
<span class="pl-s">ERROR: Job failed: exit code 1</span>
<span class="pl-s">``</span>
<span class="pl-s">## Required Dependencies</span>
<span class="pl-s">&lt;!-- Please fill the required dependencies to reproduce this issue --&gt;</span>
<span class="pl-s">* **Minimal Python Version**: N/A or Unknown</span>
<span class="pl-s">* **Minimal Celery Version**: N/A or Unknown</span>
<span class="pl-s">* **Minimal Kombu Version**: N/A or Unknown</span>
<span class="pl-s">* **Minimal Broker Version**: N/A or Unknown</span>
<span class="pl-s">* **Minimal Result Backend Version**: N/A or Unknown</span>
<span class="pl-s">* **Minimal OS and/or Kernel Version**: N/A or Unknown</span>
<span class="pl-s">* **Minimal Broker Client Version**: N/A or Unknown</span>
<span class="pl-s">* **Minimal Result Backend Client Version**: N/A or Unknown</span>
<span class="pl-s"></span>
<span class="pl-s"></span>
<span class="pl-s"># Expected Behavior</span>
<span class="pl-s">&lt;!-- Describe in detail what you expect to happen --&gt;</span>
<span class="pl-s">excepted to build docker image success</span>
<span class="pl-s"># Actual Behavior</span>
<span class="pl-s">&lt;!--</span>
<span class="pl-s">Describe in detail what actually happened.</span>
<span class="pl-s">Please include a backtrace and surround it with triple backticks (```).</span>
<span class="pl-s">In addition, include the Celery daemon logs, the broker logs,</span>
<span class="pl-s">the result backend logs and system logs below if they will help us debug</span>
<span class="pl-s">the issue.</span>
<span class="pl-s">--&gt;</span>
<span class="pl-s">how can i solve the celery dependency conflicts,thanks~</span></pre></div>
<h1 dir="auto">Checklist</h1>

<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/issues?utf8=%E2%9C%93&amp;q=is%3Aissue+label%3A%22Category%3A+Documentation%22+">issues list</a><br>
for similar or identical bug reports.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/pulls?q=is%3Apr+label%3A%22Category%3A+Documentation%22">pull requests list</a><br>
for existing proposed fixes.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/commits/master">commit log</a><br>
to find out if the bug was already fixed in the master branch.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included all related issues and possible duplicate issues in this issue<br>
(If there are none, check this box anyway).</li>
</ul>
<h2 dir="auto">Related Issues and Possible Duplicates</h2>

<h4 dir="auto">Related Issues</h4>
<ul dir="auto">
<li>None</li>
</ul>
<h4 dir="auto">Possible Duplicates</h4>
<ul dir="auto">
<li><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="352197866" data-permission-text="Title is private" data-url="https://github.com/celery/celery/issues/4993" data-hovercard-type="issue" data-hovercard-url="/celery/celery/issues/4993/hovercard" href="https://github.com/celery/celery/issues/4993">#4993</a></li>
</ul>
<h1 dir="auto">Description</h1>

<p dir="auto">The <a href="https://docs.celeryproject.org/en/latest/userguide/application.html#abstract-tasks" rel="nofollow">Abstract Tasks</a> section of your documentation says that if you override <code class="notranslate">__call__()</code> to do something every time the task body is run, then you should call <code class="notranslate">super().__call__()</code> to run that task.  This breaks some tests in my app which I'll describe below.  calling <code class="notranslate">self.run()</code> instead of <code class="notranslate">super().__call__()</code> from within <code class="notranslate">__call__()</code> to run the body works properly.  <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="352197866" data-permission-text="Title is private" data-url="https://github.com/celery/celery/issues/4993" data-hovercard-type="issue" data-hovercard-url="/celery/celery/issues/4993/hovercard" href="https://github.com/celery/celery/issues/4993">#4993</a> specifically asked about this and the recommendation was to use <code class="notranslate">super().__call__()</code>, not <code class="notranslate">self.run()</code>.</p>
<p dir="auto">The test code that is failing effectively does the following:</p>
<ol dir="auto">
<li>enqueue task with the delay() method</li>
<li>process task, - fail and enqueue for retry</li>
<li>process task</li>
</ol>
<p dir="auto">We have mimicked the code <a href="https://github.com/celery/celery/blob/v4.1.0/celery/contrib/testing/worker.py#L122">here</a> to manually process a task by creating a WorkerController and calling <code class="notranslate">wait_for_task()</code> followed by <code class="notranslate">process_queued_task()</code>.</p>
<p dir="auto">When I use <code class="notranslate">super().__call__()</code> instead of <code class="notranslate">self.run()</code>, the second call to process the task fails with a <code class="notranslate">TimeoutError</code>.  Effectively, the initial running of the task works, but something with the retry flow is failing to re-enqueue the task.</p>
<p dir="auto">Running in the debugger, I discovered</p>
<ul dir="auto">
<li>The <code class="notranslate">celery.app.task.Task.__call__()</code> method is never called if I don't call this manually with <code class="notranslate">super().__call__()</code> from my custom Task class' <code class="notranslate">__call__()</code> method.</li>
<li>The execution of the task body is triggered from <code class="notranslate">celery.app.trace.build_tracer.trace_task()</code>, which directly calls the <code class="notranslate">run()</code> function if the custom task class has not defined <code class="notranslate">__call__()</code></li>
<li>The task_trace function calls the same surrounding management code as <code class="notranslate">Task.__call__()</code> (in addition to a bunch of other things), so it seems like it would be a bad idea to call <code class="notranslate">Task.__call__()</code> from within this.</li>
</ul>
<p dir="auto">I'm not sure if anything particular to my configuration is causing the flow to go through <code class="notranslate">trace_task()</code> rather than <code class="notranslate">Task.__call__()</code>.</p>
<p dir="auto">Here are some code comments and snippets which may be useful:<br>
<a href="https://github.com/celery/celery/blob/v4.1.0/celery/app/trace.py#L284">https://github.com/celery/celery/blob/v4.1.0/celery/app/trace.py#L284</a></p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="# If the task doesn't define a custom __call__ method
# we optimize it away by simply calling the run method directly,
# saving the extra method call and a line less in the stack trace.
fun = task if task_has_custom(task, '__call__') else task.run"><pre class="notranslate"><span class="pl-c"># If the task doesn't define a custom __call__ method</span>
<span class="pl-c"># we optimize it away by simply calling the run method directly,</span>
<span class="pl-c"># saving the extra method call and a line less in the stack trace.</span>
<span class="pl-s1">fun</span> <span class="pl-c1">=</span> <span class="pl-s1">task</span> <span class="pl-k">if</span> <span class="pl-en">task_has_custom</span>(<span class="pl-s1">task</span>, <span class="pl-s">'__call__'</span>) <span class="pl-k">else</span> <span class="pl-s1">task</span>.<span class="pl-s1">run</span></pre></div>
<p dir="auto"><a href="https://github.com/celery/celery/blob/v4.1.0/celery/app/task.py#L146">https://github.com/celery/celery/blob/v4.1.0/celery/app/task.py#L146</a></p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="    &quot;&quot;&quot;Task base class.
    Note:
        When called tasks apply the :meth:`run` method.  This method must
        be defined by all tasks (that is unless the :meth:`__call__` method
        is overridden).
    &quot;&quot;&quot;"><pre class="notranslate"><code class="notranslate">    """Task base class.
    Note:
        When called tasks apply the :meth:`run` method.  This method must
        be defined by all tasks (that is unless the :meth:`__call__` method
        is overridden).
    """
</code></pre></div>
<h1 dir="auto">Suggestions</h1>

<p dir="auto">Either help me understand what I'm doing wrong and provide documentation to explain that to future users in the Abstract Tasks section of the documentation, or change the code snippet to the following:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="from celery import Task

class DebugTask(Task):

    def __call__(self, *args, **kwargs):
        print('TASK STARTING: {0.name}[{0.request.id}]'.format(self))
        return self.run(*args, **kwargs)"><pre class="notranslate"><span class="pl-k">from</span> <span class="pl-s1">celery</span> <span class="pl-k">import</span> <span class="pl-v">Task</span>

<span class="pl-k">class</span> <span class="pl-v">DebugTask</span>(<span class="pl-v">Task</span>):

    <span class="pl-k">def</span> <span class="pl-en">__call__</span>(<span class="pl-s1">self</span>, <span class="pl-c1">*</span><span class="pl-s1">args</span>, <span class="pl-c1">**</span><span class="pl-s1">kwargs</span>):
        <span class="pl-en">print</span>(<span class="pl-s">'TASK STARTING: {0.name}[{0.request.id}]'</span>.<span class="pl-en">format</span>(<span class="pl-s1">self</span>))
        <span class="pl-k">return</span> <span class="pl-s1">self</span>.<span class="pl-en">run</span>(<span class="pl-c1">*</span><span class="pl-s1">args</span>, <span class="pl-c1">**</span><span class="pl-s1">kwargs</span>)</pre></div>