<h1 dir="auto">Checklist</h1>

<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have verified that the issue exists against the <code class="notranslate">master</code> branch of Celery.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> This has already been asked to the <a href="https://groups.google.com/forum/#!forum/celery-users" rel="nofollow">discussion group</a> first.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have read the relevant section in the<br>
<a href="http://docs.celeryproject.org/en/latest/contributing.html#other-bugs" rel="nofollow">contribution guide</a><br>
on reporting bugs.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/issues?q=is%3Aissue+label%3A%22Issue+Type%3A+Bug+Report%22+-label%3A%22Category%3A+Documentation%22">issues list</a><br>
for similar or identical bug reports.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/pulls?q=is%3Apr+label%3A%22PR+Type%3A+Bugfix%22+-label%3A%22Category%3A+Documentation%22">pull requests list</a><br>
for existing proposed fixes.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have checked the <a href="https://github.com/celery/celery/commits/master">commit log</a><br>
to find out if the bug was already fixed in the master branch.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included all related issues and possible duplicate issues<br>
in this issue (If there are none, check this box anyway).</li>
</ul>
<h2 dir="auto">Mandatory Debugging Information</h2>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included the output of <code class="notranslate">celery -A proj report</code> in the issue.<br>
(if you are not able to do this, then at least specify the Celery<br>
version affected).</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have verified that the issue exists against the <code class="notranslate">master</code> branch of Celery.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included the contents of <code class="notranslate">pip freeze</code> in the issue.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included all the versions of all the external dependencies required<br>
to reproduce this bug.</li>
</ul>
<h2 dir="auto">Optional Debugging Information</h2>

<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one Python version<br>
and/or implementation.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one message broker and/or<br>
result backend.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one version of the message<br>
broker and/or result backend.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one operating system.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one workers pool.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue with autoscaling, retries,<br>
ETA/Countdown &amp; rate limits disabled.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue after downgrading<br>
and/or upgrading Celery and its dependencies.</li>
</ul>
<h2 dir="auto">Environment &amp; Settings</h2>
<p dir="auto"><strong>Celery version</strong>:</p>

<details>
<summary><b><code class="notranslate">celery report</code> Output:</b>
</summary>
<p dir="auto">
software -&gt; celery:5.1.0 (sun-harmonics) kombu:5.1.0 py:3.9.2
            billiard:3.6.4.0 redis:3.5.3
platform -&gt; system:Linux arch:64bit, ELF
            kernel version:4.4.0-198-generic imp:CPython
loader   -&gt; celery.loaders.app.AppLoader
settings -&gt; transport:redis results:redis://localhost/1
</p><p dir="auto">broker_url: 'redis://localhost:6379/1'<br>
result_backend: 'redis://localhost/1'<br>
result_serializer: 'pickle'<br>
deprecated_settings: None<br>
task_serializer: 'pickle'<br>
accept_content: ['pickle', 'json']<br>
result_expires: 3600<br>
task_default_queue: 'zabbix'</p>
<p dir="auto"></p>
</details>
<h1 dir="auto">Steps to Reproduce</h1>
<p dir="auto">Schedule a chord with beat via a task.</p>
<p dir="auto">I created a task that takes a chain/chord as parameter and executes it, since beat does not allow for scheduling anything but a simple task directly:<br>
@app.task()<br>
def execute_chain(ch):<br>
ch.apply_async(ignore_result=True)</p>
<p dir="auto">This is added to the beat schedule via<br>
ch = group(*zbxitems) | func.s() | publish_mqtt.s()<br>
sender.add_periodic_task(interval, execute_chain.s(ch), name=name)</p>
<h2 dir="auto">Required Dependencies</h2>

<h3 dir="auto">Python Packages</h3>

<details>
<summary><b><code class="notranslate">pip freeze</code> Output:</b></summary>
<p dir="auto">
amqp==5.0.6
billiard==3.6.4.0
celery==5.1.0
certifi==2021.5.30
chardet==4.0.0
click==7.1.2
click-didyoumean==0.0.3
click-plugins==1.1.1
click-repl==0.2.0
colorama==0.4.4
idna==2.10
kombu==5.1.0
paho-mqtt==1.5.1
prompt-toolkit==3.0.18
pytz==2021.1
PyYAML==5.4.1
pyzabbix==0.8.2
redis==3.5.3
requests==2.25.1
six==1.16.0
urllib3==1.26.5
vine==5.0.0
wcwidth==0.2.5
(venv) root@vmt
</p>
</details>
<h3 dir="auto">Other Dependencies</h3>
<details>
<p dir="auto">
</p><div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content=""><pre class="notranslate"></pre></div>
<p dir="auto">version 3.9.5</p>
<p dir="auto"></p>
</details>
<h1 dir="auto">Expected Behavior</h1>
<p dir="auto">This should continue to work, executing the chain again and again.</p>
<h1 dir="auto">Actual Behavior</h1>
<p dir="auto">I works for several hours till it exits with:<br>
2021-06-11 06:25:54,253:CRITICAL:worker.start:line 207:Unrecoverable error: RecursionError('maximum recursion depth exceeded while pickling an object')<br>
Traceback (most recent call last):<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/celery/worker/worker.py", line 203, in start<br>
self.blueprint.start(self)<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/celery/bootsteps.py", line 116, in start<br>
step.start(parent)<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/celery/bootsteps.py", line 365, in start<br>
return self.obj.start()<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/celery/worker/consumer/consumer.py", line 326, in start<br>
blueprint.start(self)<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/celery/bootsteps.py", line 116, in start<br>
step.start(parent)<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/celery/worker/consumer/consumer.py", line 618, in start<br>
c.loop(*c.loop_args())<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/celery/worker/loops.py", line 81, in asynloop<br>
next(loop)<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/kombu/asynchronous/hub.py", line 361, in create_loop<br>
cb(*cbargs)<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/kombu/transport/redis.py", line 1128, in on_readable<br>
self.cycle.on_readable(fileno)<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/kombu/transport/redis.py", line 398, in on_readable<br>
chan.handlers<a href="">type</a><br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/kombu/transport/redis.py", line 789, in _brpop_read<br>
self.connection._deliver(loads(bytes_to_str(item)), dest)<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/kombu/transport/virtual/base.py", line 980, in _deliver<br>
callback(message)<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/kombu/transport/virtual/base.py", line 630, in _callback<br>
return callback(message)<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/kombu/messaging.py", line 626, in _receive_callback<br>
return on_m(message) if on_m else self.receive(decoded, message)<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/celery/worker/consumer/consumer.py", line 586, in on_task_received<br>
strategy(<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/celery/worker/strategy.py", line 201, in task_message_handler<br>
handle(req)<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/celery/worker/worker.py", line 221, in _process_task_sem<br>
return self._quick_acquire(self._process_task, req)<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/kombu/asynchronous/semaphore.py", line 57, in acquire<br>
callback(*partial_args, **partial_kwargs)<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/celery/worker/worker.py", line 226, in _process_task<br>
req.execute_using_pool(self.pool)<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/celery/worker/request.py", line 691, in execute_using_pool<br>
result = apply_async(<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/celery/concurrency/base.py", line 152, in apply_async<br>
return self.on_apply(target, args, kwargs,<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/billiard/pool.py", line 1530, in apply_async<br>
self._quick_put((TASK, (result._job, None, func, args, kwds)))<br>
File "/usr/share/nd_zabbixexport/venv/lib/python3.9/site-packages/celery/concurrency/asynpool.py", line 859, in send_job<br>
body = dumps(tup, protocol=protocol)<br>
RecursionError: maximum recursion depth exceeded while pickling an object</p>
<p dir="auto">When json dumping the task, I see this after a few iterations (mind the kwargs):<br>
2021-06-11 16:44:16,370:DEBUG:beat.execute_chain:line 90:{"task": "celery.chord", "args": [], "kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {<br>
"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {"kwargs": {}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}, "header": [{"task": "tasks.zabbix.get<br>
_zabbix_items_value", "args": ["Memory Free", "rd servers"], "kwargs": {}, "options": {}, "subtask_type": null, "immutable": true}, {"task": "tasks.zabbix.get_zabbix_items_value", "args": ["Available memory", "rd servers"], "kwargs": {}, "options": {}, "subtask_type": n<br>
ull, "immutable": true}], "body": {"task": "celery.chain", "args": [], "kwargs": {"tasks": [{"task": "tasks.aggregate.percentage_per_item", "args": [], "kwargs": {}, "options": {}, "subtask_type": null, "immutable": false}, {"task": "tasks.mqtt.publish_mqtt", "args": ["<br>
vmmqtt01", "zabbix/"], "kwargs": {}, "options": {}, "subtask_type": null, "immutable": false}]}, "options": {}, "subtask_type": "chain", "immutable": false}}, "options": {}, "subtask_type": "chord", "immutable": false}</p>
<p dir="auto">I tried also by cloning the chord first in the scheduled task and executing the clone, but same result.</p>
<p dir="auto">AsyncResult memory leak on task producers when producers retrieves task results from redis results backend. We are using redis as broker and results backend.</p>
<p dir="auto">This is issue is similar to <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="334774919" data-permission-text="Title is private" data-url="https://github.com/celery/celery/issues/4839" data-hovercard-type="pull_request" data-hovercard-url="/celery/celery/pull/4839/hovercard" href="https://github.com/celery/celery/pull/4839">#4839</a>. I have attempted to backport this fix: <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="409140221" data-permission-text="Title is private" data-url="https://github.com/celery/celery/issues/5332" data-hovercard-type="pull_request" data-hovercard-url="/celery/celery/pull/5332/hovercard" href="https://github.com/celery/celery/pull/5332">#5332</a> with vine 1.3 + celery 4.2.2. However, leak of AsyncResult objects continues to happen for task that retrieves the results from redis backend. Tasks that ignores the results do not exhibit this leak when changes are backported from <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="409140221" data-permission-text="Title is private" data-url="https://github.com/celery/celery/issues/5332" data-hovercard-type="pull_request" data-hovercard-url="/celery/celery/pull/5332/hovercard" href="https://github.com/celery/celery/pull/5332">#5332</a>. Only patch that worked for me was removing <strong>del</strong> method completely: <a href="https://github.com/celery/celery/blame/v4.2.2/celery/result.py#L382">https://github.com/celery/celery/blame/v4.2.2/celery/result.py#L382</a></p>
<p dir="auto">Celery version: 4.2.2<br>
Kombu: 4.2.2<br>
Vine: 1.3.0<br>
Python: 2.7</p>
<p dir="auto">I have attempted to reproduce the issue with celery 4.1.1 but it works on that version.</p>
<h1 dir="auto">Checklist</h1>

<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have verified that the issue exists against the <code class="notranslate">master</code> branch of Celery.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> This has already been asked to the <a href="https://groups.google.com/forum/#!forum/celery-users" rel="nofollow">discussion group</a> first.</li>
<li>[x ] I have read the relevant section in the<br>
<a href="http://docs.celeryproject.org/en/latest/contributing.html#other-bugs" rel="nofollow">contribution guide</a><br>
on reporting bugs.</li>
<li>[ x] I have checked the <a href="https://github.com/celery/celery/issues?q=is%3Aissue+label%3A%22Issue+Type%3A+Bug+Report%22+-label%3A%22Category%3A+Documentation%22">issues list</a><br>
for similar or identical bug reports.</li>
<li>[x ] I have checked the <a href="https://github.com/celery/celery/pulls?q=is%3Apr+label%3A%22PR+Type%3A+Bugfix%22+-label%3A%22Category%3A+Documentation%22">pull requests list</a><br>
for existing proposed fixes.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have checked the <a href="https://github.com/celery/celery/commits/master">commit log</a><br>
to find out if the bug was already fixed in the master branch.</li>
<li>[ x] I have included all related issues and possible duplicate issues<br>
in this issue (If there are none, check this box anyway).</li>
</ul>
<h2 dir="auto">Mandatory Debugging Information</h2>
<ul class="contains-task-list">
<li>[x ] I have included the output of <code class="notranslate">celery -A proj report</code> in the issue.<br>
(if you are not able to do this, then at least specify the Celery<br>
version affected).</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have verified that the issue exists against the <code class="notranslate">master</code> branch of Celery.</li>
<li>[ x] I have included the contents of <code class="notranslate">pip freeze</code> in the issue.</li>
<li>[x ] I have included all the versions of all the external dependencies required<br>
to reproduce this bug.</li>
</ul>
<h2 dir="auto">Optional Debugging Information</h2>

<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one Python version<br>
and/or implementation.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one message broker and/or<br>
result backend.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one version of the message<br>
broker and/or result backend.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one operating system.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one workers pool.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue with autoscaling, retries,<br>
ETA/Countdown &amp; rate limits disabled.</li>
<li>[x ] I have tried reproducing the issue after downgrading<br>
and/or upgrading Celery and its dependencies.</li>
</ul>