<h1 dir="auto">Checklist</h1>

<ul dir="auto">
<li>[ X] I have verified that the issue exists against the <code class="notranslate">master</code> branch of Celery.</li>
<li>[ X] This has already been asked to the <a href="https://groups.google.com/forum/#!forum/celery-users" rel="nofollow">discussion group</a> first.</li>
<li>[ X] I have read the relevant section in the<br>
<a href="http://docs.celeryproject.org/en/latest/contributing.html#other-bugs" rel="nofollow">contribution guide</a><br>
on reporting bugs.</li>
<li>[ X] I have checked the <a href="https://github.com/celery/celery/issues?q=is%3Aissue+label%3A%22Issue+Type%3A+Bug+Report%22+-label%3A%22Category%3A+Documentation%22">issues list</a><br>
for similar or identical bug reports.</li>
<li>[ X] I have checked the <a href="https://github.com/celery/celery/pulls?q=is%3Apr+label%3A%22PR+Type%3A+Bugfix%22+-label%3A%22Category%3A+Documentation%22">pull requests list</a><br>
for existing proposed fixes.</li>
<li>[ X] I have checked the <a href="https://github.com/celery/celery/commits/master">commit log</a><br>
to find out if the bug was already fixed in the master branch.</li>
<li>[ X] I have included all related issues and possible duplicate issues<br>
in this issue (If there are none, check this box anyway).</li>
</ul>
<h2 dir="auto">Mandatory Debugging Information</h2>
<ul dir="auto">
<li>[ X] I have included the output of <code class="notranslate">celery -A proj report</code> in the issue.<br>
(if you are not able to do this, then at least specify the Celery<br>
version affected).</li>
<li>[ X] I have verified that the issue exists against the <code class="notranslate">master</code> branch of Celery.</li>
<li>[ X] I have included the contents of <code class="notranslate">pip freeze</code> in the issue.</li>
<li>[ X] I have included all the versions of all the external dependencies required<br>
to reproduce this bug.</li>
</ul>
<h2 dir="auto">Optional Debugging Information</h2>

<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one Python version<br>
and/or implementation.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one message broker and/or<br>
result backend.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one version of the message<br>
broker and/or result backend.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one operating system.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one workers pool.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue with autoscaling, retries,<br>
ETA/Countdown &amp; rate limits disabled.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue after downgrading<br>
and/or upgrading Celery and its dependencies.</li>
</ul>
<h2 dir="auto">Related Issues and Possible Duplicates</h2>

<h4 dir="auto">Related Issues</h4>
<ul dir="auto">
<li>None</li>
</ul>
<h4 dir="auto">Possible Duplicates</h4>
<ul dir="auto">
<li>None</li>
</ul>
<h2 dir="auto">Environment &amp; Settings</h2>

<p dir="auto"><strong>Celery version</strong>:</p>

<details>
<summary><b><code class="notranslate">
software -&gt; celery:5.0.2 (singularity) kombu:5.0.2 py:3.7.4
            billiard:3.6.3.0 py-amqp:5.0.2
platform -&gt; system:Linux arch:64bit, ELF
            kernel version:5.9.16_1 imp:CPython
loader   -&gt; celery.loaders.app.AppLoader
settings -&gt; transport:amqp results:rpc:///
</code></b><p dir="auto"><b><code class="notranslate">broker_url: 'amqp://guest:********@localhost:5672//'<br>
result_backend: 'rpc:///'<br>
deprecated_settings: None<br>
</code> Output:</b></p></summary><p dir="auto"></p>
<p dir="auto">
</p><div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content=""><pre class="notranslate"><code class="notranslate"></code></pre></div>
<p dir="auto"></p>
</details>
<h1 dir="auto">Steps to Reproduce</h1>
<h2 dir="auto">Required Dependencies</h2>

<ul dir="auto">
<li><strong>Minimal Python Version</strong>: N/A or Unknown</li>
<li><strong>Minimal Celery Version</strong>: N/A or Unknown</li>
<li><strong>Minimal Kombu Version</strong>: N/A or Unknown</li>
<li><strong>Minimal Broker Version</strong>: N/A or Unknown</li>
<li><strong>Minimal Result Backend Version</strong>: N/A or Unknown</li>
<li><strong>Minimal OS and/or Kernel Version</strong>: N/A or Unknown</li>
<li><strong>Minimal Broker Client Version</strong>: N/A or Unknown</li>
<li><strong>Minimal Result Backend Client Version</strong>: N/A or Unknown</li>
</ul>
<h3 dir="auto">Python Packages</h3>

<details>
<summary><b><code class="notranslate">
asn1crypto==0.24.0
brotlipy==0.7.0
certifi==2020.6.20
cffi==1.14.0
chardet==3.0.4
conda==4.8.3
conda-package-handling==1.7.0
configparser==3.7.4
cryptography==2.9.2
idna @ file:///tmp/build/80754af9/idna_1593446292537/work
OWSLib==0.19.2
pycosat==0.6.3
pycparser @ file:///tmp/build/80754af9/pycparser_1594388511720/work
pyOpenSSL @ file:///tmp/build/80754af9/pyopenssl_1594392929924/work
pyproj==2.5.0
PySocks @ file:///tmp/build/80754af9/pysocks_1594394576006/work
python-dateutil==2.8.1
pytz==2019.3
PyYAML==5.3
requests @ file:///tmp/build/80754af9/requests_1592841827918/work
ruamel-yaml==0.15.87
six==1.15.0
tqdm @ file:///tmp/build/80754af9/tqdm_1593446365756/work
urllib3==1.25.9
</code></b><p dir="auto"><b><code class="notranslate"></code> Output:</b></p></summary><p dir="auto"></p>
<p dir="auto">
</p><div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content=""><pre class="notranslate"><code class="notranslate"></code></pre></div>
<p dir="auto"></p>
</details>
<h3 dir="auto">Other Dependencies</h3>

<details>
<p dir="auto">
# packages in environment at /opt/miniconda3/envs/[redacted]:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
aggdraw                   1.3.12           py37hdf465ad_1    conda-forge
amqp                      5.0.2              pyhd8ed1ab_0    conda-forge
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
argon2-cffi               20.1.0           py37h4abf009_2    conda-forge
asgiref                   3.3.1              pyhd8ed1ab_0    conda-forge
astroid                   2.4.2            py37hc8dfbb8_1    conda-forge
async_generator           1.10                       py_0    conda-forge
attrs                     20.3.0             pyhd3deb0d_0    conda-forge
autopep8                  1.5.4              pyh9f0ad1d_0    conda-forge
azure-core                1.9.0                    pypi_0    pypi
azure-storage-blob        12.6.0                   pypi_0    pypi
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                        py_2    conda-forge
backports.functools_lru_cache 1.6.1                      py_0    conda-forge
billiard                  3.6.3.0          py37h8f50634_2    conda-forge
black                     20.8b1                     py_1    conda-forge
bleach                    3.2.1              pyh9f0ad1d_0    conda-forge
boost-cpp                 1.72.0               h8e57a91_0    conda-forge
brotlipy                  0.7.0           py37hb5d75c8_1001    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.17.1               h36c2ea0_0    conda-forge
ca-certificates           2020.12.5            ha878542_0    conda-forge
cairocffi                 1.2.0                    pypi_0    pypi
cairosvg                  2.5.0                    pypi_0    pypi
celery                    5.0.2              pyhd8ed1ab_0    conda-forge
certifi                   2020.12.5        py37h89c1867_1    conda-forge
cffi                      1.14.4           py37h11fe52a_0    conda-forge
chardet                   4.0.0            py37h89c1867_0    conda-forge
click                     7.1.2              pyh9f0ad1d_0    conda-forge
click-didyoumean          0.0.3              pyh8c360ce_0    conda-forge
click-repl                0.1.6                      py_0    conda-forge
colorzero                 1.1                      pypi_0    pypi
cryptography              3.3.1            py37h7f0c10b_0    conda-forge
cssselect2                0.4.1                    pypi_0    pypi
cycler                    0.10.0                   pypi_0    pypi
dataclasses               0.7                pyhb2cacf7_7    conda-forge
dbus                      1.13.6               hfdff14a_1    conda-forge
decorator                 4.4.2                      py_0    conda-forge
defusedxml                0.6.0                      py_0    conda-forge
django                    3.1.5              pyhd8ed1ab_0    conda-forge
django-crispy-forms       1.10.0                   pypi_0    pypi
drawsvg                   1.7.0                    pypi_0    pypi
entrypoints               0.3             pyhd8ed1ab_1003    conda-forge
expat                     2.2.9                he1b5a44_2    conda-forge
flake8                    3.8.4                      py_0    conda-forge
fontconfig                2.13.1            h86ecdb6_1001    conda-forge
freeimageplus             3.18.0               hf484d3e_2    dlr-sc
freetype                  2.10.4               h7ca028e_0    conda-forge
future                    0.18.2           py37h89c1867_2    conda-forge
gettext                   0.19.8.1          hf34092f_1004    conda-forge
glib                      2.66.3               h58526e2_0    conda-forge
gmp                       6.2.1                h58526e2_0    conda-forge
gmsh                      4.6.0                h7d08f12_0    conda-forge
gpiozero                  1.5.1                    pypi_0    pypi
gst-plugins-base          1.14.5               h0935bb2_2    conda-forge
gstreamer                 1.14.5               h36ae1b5_2    conda-forge
guizero                   1.1.1                    pypi_0    pypi
h11                       0.12.0                   pypi_0    pypi
icu                       64.2                 he1b5a44_1    conda-forge
idna                      2.10               pyh9f0ad1d_0    conda-forge
ifcopenshell              v0.6.0           py37h4488930_3    conda-forge
imageio                   2.9.0                    pypi_0    pypi
importlib-metadata        3.3.0            py37h89c1867_2    conda-forge
importlib_metadata        3.3.0                hd8ed1ab_2    conda-forge
ipykernel                 5.4.2            py37h888b3d9_0    conda-forge
ipython                   7.19.0           py37h888b3d9_1    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
ipywidgets                7.6.2              pyhd3deb0d_0    conda-forge
isodate                   0.6.0                    pypi_0    pypi
isort                     5.7.0              pyhd8ed1ab_0    conda-forge
jedi                      0.17.2           py37h89c1867_1    conda-forge
jinja2                    2.11.2             pyh9f0ad1d_0    conda-forge
jpeg                      9d                   h36c2ea0_0    conda-forge
jsonschema                3.2.0                      py_2    conda-forge
jupyter                   1.0.0                      py_2    conda-forge
jupyter_client            6.1.7                      py_0    conda-forge
jupyter_console           6.2.0                      py_0    conda-forge
jupyter_core              4.7.0            py37h89c1867_0    conda-forge
jupyterlab_pygments       0.1.2              pyh9f0ad1d_0    conda-forge
jupyterlab_widgets        1.0.0              pyhd8ed1ab_1    conda-forge
kiwisolver                1.3.1                    pypi_0    pypi
kombu                     5.0.2            py37h89c1867_1    conda-forge
krb5                      1.17.2               h926e7f8_0    conda-forge
lazy-object-proxy         1.4.3            py37h8f50634_2    conda-forge
lcms2                     2.11                 hcbb858e_1    conda-forge
libblas                   3.9.0                6_openblas    conda-forge
libclang                  9.0.1           default_hde54327_0    conda-forge
libcurl                   7.71.1               hcdd3856_8    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libffi                    3.2.1             he1b5a44_1007    conda-forge
libgcc-ng                 9.3.0               h5dbcf3e_17    conda-forge
libgfortran-ng            9.3.0               he4bcb1c_17    conda-forge
libgfortran5              9.3.0               he4bcb1c_17    conda-forge
libglib                   2.66.3               hbe7bbb4_0    conda-forge
libglu                    9.0.0             he1b5a44_1001    conda-forge
libgomp                   9.3.0               h5dbcf3e_17    conda-forge
libiconv                  1.16                 h516909a_0    conda-forge
liblapack                 3.9.0                6_openblas    conda-forge
libllvm9                  9.0.1                hf817b99_2    conda-forge
libnghttp2                1.41.0               h8cfc5f6_2    conda-forge
libopenblas               0.3.12          pthreads_h4812303_1    conda-forge
libpng                    1.6.37               h21135ba_2    conda-forge
libsodium                 1.0.18               h36c2ea0_1    conda-forge
libssh2                   1.9.0                hab1572f_5    conda-forge
libstdcxx-ng              9.3.0               h2ae2ef3_17    conda-forge
libtiff                   4.2.0                hdc55705_0    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libwebp-base              1.1.0                h36c2ea0_3    conda-forge
libxcb                    1.13              h14c3975_1002    conda-forge
libxkbcommon              0.10.0               he1b5a44_0    conda-forge
libxml2                   2.9.10               hee79883_0    conda-forge
libxslt                   1.1.33               h31b3aaa_0    conda-forge
lz4-c                     1.9.3                h9c3ff4c_0    conda-forge
markupsafe                1.1.1            py37hb5d75c8_2    conda-forge
matplotlib                2.2.2                    pypi_0    pypi
mccabe                    0.6.1                      py_1    conda-forge
mistune                   0.8.4           py37h4abf009_1002    conda-forge
mono                      5.14.0.177        he1b5a44_1001    conda-forge
msrest                    0.6.19                   pypi_0    pypi
mu-editor                 1.0.3                    pypi_0    pypi
mypy_extensions           0.4.3            py37h89c1867_2    conda-forge
nbclient                  0.5.1                      py_0    conda-forge
nbconvert                 6.0.7            py37h89c1867_3    conda-forge
nbformat                  5.0.8                      py_0    conda-forge
ncurses                   6.2                  h58526e2_4    conda-forge
nest-asyncio              1.4.3              pyhd8ed1ab_0    conda-forge
notebook                  6.1.6            py37h89c1867_0    conda-forge
nspr                      4.29                 h9c3ff4c_1    conda-forge
nss                       3.59                 h2c00c37_0    conda-forge
nudatus                   0.0.4                    pypi_0    pypi
numpy                     1.19.4                   pypi_0    pypi
oauthlib                  3.1.0                    pypi_0    pypi
occt                      7.4.0                h9121d39_3    conda-forge
olefile                   0.46               pyh9f0ad1d_1    conda-forge
opencascade               7.4.0                h6bb024c_5    dlr-sc
openssl                   1.1.1i               h7f98852_0    conda-forge
owslib                    0.21.0             pyhd8ed1ab_0    conda-forge
packaging                 20.8               pyhd3deb0d_0    conda-forge
pandoc                    2.11.3.2             h7f98852_0    conda-forge
pandocfilters             1.4.2                      py_1    conda-forge
parso                     0.7.1              pyh9f0ad1d_0    conda-forge
pathspec                  0.8.1              pyhd3deb0d_0    conda-forge
pcre                      8.44                 he1b5a44_0    conda-forge
pexpect                   4.8.0              pyh9f0ad1d_2    conda-forge
pgzero                    1.2                      pypi_0    pypi
pickleshare               0.7.5                   py_1003    conda-forge
pigpio                    1.78                     pypi_0    pypi
pillow                    8.1.0            py37he6b4880_0    conda-forge
pip                       20.3.3             pyhd8ed1ab_0    conda-forge
pluggy                    0.13.1           py37he5f6b98_3    conda-forge
proj                      7.2.0                h8b9fe22_0    conda-forge
prometheus_client         0.9.0              pyhd3deb0d_0    conda-forge
prompt-toolkit            3.0.8              pyha770c72_0    conda-forge
prompt_toolkit            3.0.8                hd8ed1ab_0    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
punycodes                 1.0.1                    pypi_0    pypi
pycodestyle               2.4.0                    pypi_0    pypi
pycparser                 2.20               pyh9f0ad1d_2    conda-forge
pydocstyle                5.1.1                      py_0    conda-forge
pyflakes                  2.0.0                    pypi_0    pypi
pygame                    1.9.6                    pypi_0    pypi
pygments                  2.7.3              pyhd8ed1ab_0    conda-forge
pylint                    2.6.0            py37hc8dfbb8_1    conda-forge
pyls                      0.1.6                    pypi_0    pypi
pyopenssl                 20.0.1             pyhd8ed1ab_0    conda-forge
pyparsing                 2.4.7              pyh9f0ad1d_0    conda-forge
pyproj                    3.0.0.post1      py37hf874b97_0    conda-forge
pyqt5                     5.14.1                   pypi_0    pypi
pyqt5-sip                 12.8.1                   pypi_0    pypi
pyqtchart                 5.14.0                   pypi_0    pypi
pyqtwebengine             5.12.1                   pypi_0    pypi
pyrsistent                0.17.3           py37h4abf009_1    conda-forge
pyserial                  3.4                      pypi_0    pypi
pyside2                   5.13.2           py37hbee1fde_2    conda-forge
pysocks                   1.7.1            py37he5f6b98_2    conda-forge
python                    3.7.4                h265db76_1  
python-dateutil           2.8.1                      py_0    conda-forge
python-jsonrpc-server     0.4.0              pyh9f0ad1d_0    conda-forge
python-language-server    0.36.2             pyhd8ed1ab_0    conda-forge
python_abi                3.7                     1_cp37m    conda-forge
pythonnet                 2.4.0            py37haf93ef1_1    conda-forge
pythonocc-core            7.4.1            py37h2bc3f7f_0    dlr-sc
pytz                      2020.5             pyhd8ed1ab_0    conda-forge
pyyaml                    5.3.1            py37hb5d75c8_1    conda-forge
pyzmq                     20.0.0           py37h5a562af_1    conda-forge
qscintilla                2.11.4                   pypi_0    pypi
qt                        5.12.5               hd8c4c69_1    conda-forge
qtconsole                 4.3.1                    pypi_0    pypi
qtpy                      1.9.0                      py_0    conda-forge
readline                  7.0               hf8c457e_1001    conda-forge
regex                     2020.11.13       py37h5e8e339_0    conda-forge
reportlab                 3.5.59           py37h69800bb_0    conda-forge
requests                  2.25.1             pyhd3deb0d_0    conda-forge
requests-oauthlib         1.3.0                    pypi_0    pypi
rope                      0.18.0             pyh9f0ad1d_0    conda-forge
semver                    2.13.0                   pypi_0    pypi
send2trash                1.5.0                      py_0    conda-forge
setuptools                49.6.0           py37he5f6b98_2    conda-forge
six                       1.15.0             pyh9f0ad1d_0    conda-forge
snowballstemmer           2.0.0                      py_0    conda-forge
sqlalchemy                1.3.22           py37h5e8e339_1    conda-forge
sqlite                    3.33.0               h62c20be_0  
sqlparse                  0.4.1              pyh9f0ad1d_0    conda-forge
tbb                       2020.2               h4bd325d_2    conda-forge
terminado                 0.9.1            py37h89c1867_1    conda-forge
testpath                  0.4.4                      py_0    conda-forge
tinycss2                  1.1.0                    pypi_0    pypi
tk                        8.6.10               h21135ba_1    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tornado                   6.1              py37h4abf009_0    conda-forge
traitlets                 5.0.5                      py_0    conda-forge
typed-ast                 1.4.2            py37h5e8e339_0    conda-forge
typing_extensions         3.7.4.3                    py_0    conda-forge
ujson                     4.0.1            py37h3340039_1    conda-forge
urllib3                   1.26.2             pyhd8ed1ab_0    conda-forge
uvicorn                   0.13.3                   pypi_0    pypi
vine                      5.0.0              pyhd8ed1ab_1    conda-forge
wcwidth                   0.2.5              pyh9f0ad1d_2    conda-forge
webencodings              0.5.1                      py_1    conda-forge
wheel                     0.36.2             pyhd3deb0d_0    conda-forge
widgetsnbextension        3.5.1            py37h89c1867_4    conda-forge
wrapt                     1.11.2           py37h8f50634_1    conda-forge
xorg-fixesproto           5.0               h14c3975_1002    conda-forge
xorg-kbproto              1.0.7             h14c3975_1002    conda-forge
xorg-libice               1.0.10               h516909a_0    conda-forge
xorg-libsm                1.2.3             h84519dc_1000    conda-forge
xorg-libx11               1.6.12               h516909a_0    conda-forge
xorg-libxau               1.0.9                h14c3975_0    conda-forge
xorg-libxdmcp             1.1.3                h516909a_0    conda-forge
xorg-libxext              1.3.4                h516909a_0    conda-forge
xorg-libxfixes            5.0.3             h516909a_1004    conda-forge
xorg-libxmu               1.1.3                h516909a_0    conda-forge
xorg-libxrender           0.9.10            h516909a_1002    conda-forge
xorg-libxt                1.1.5             h516909a_1003    conda-forge
xorg-renderproto          0.11.1            h14c3975_1002    conda-forge
xorg-xextproto            7.3.0             h14c3975_1002    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
yaml                      0.2.5                h516909a_0    conda-forge
yapf                      0.30.0             pyh9f0ad1d_0    conda-forge
zeromq                    4.3.3                h58526e2_3    conda-forge
zipp                      3.4.0                   alias xclip="xclip -selection c"   py_0    conda-forge
zlib                      1.2.11            h516909a_1010    conda-forge
zstd                      1.4.8                ha95c52a_1    conda-forge
</p>
</details>
<h2 dir="auto">Minimally Reproducible Test Case</h2>

<details>
<p dir="auto">
</p><p dir="auto">I am working on a program that needs to use another dept.'s dotnet core dll's and it works swimmingly when executed normally.<br>
For example, considering this code:</p>
<p dir="auto">tasks.py:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import os
import sys
sys.path.append(&quot;./dll_lib&quot;)
import clr
clr.AddReference(&quot;System.IO&quot;)
import System.IO

from celery import Celery
app = Celery('Test', broker='amqp://guest:guest@localhost:5672//', backend=&quot;rpc://&quot;)

@app.task
def test(json_file):
    json_content = System.IO.File.ReadAllText(json_file)
    return 1
"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">os</span>
<span class="pl-k">import</span> <span class="pl-s1">sys</span>
<span class="pl-s1">sys</span>.<span class="pl-s1">path</span>.<span class="pl-en">append</span>(<span class="pl-s">"./dll_lib"</span>)
<span class="pl-k">import</span> <span class="pl-s1">clr</span>
<span class="pl-s1">clr</span>.<span class="pl-v">AddReference</span>(<span class="pl-s">"System.IO"</span>)
<span class="pl-k">import</span> <span class="pl-v">System</span>.<span class="pl-v">IO</span>

<span class="pl-k">from</span> <span class="pl-s1">celery</span> <span class="pl-k">import</span> <span class="pl-v">Celery</span>
<span class="pl-s1">app</span> <span class="pl-c1">=</span> <span class="pl-v">Celery</span>(<span class="pl-s">'Test'</span>, <span class="pl-s1">broker</span><span class="pl-c1">=</span><span class="pl-s">'amqp://guest:guest@localhost:5672//'</span>, <span class="pl-s1">backend</span><span class="pl-c1">=</span><span class="pl-s">"rpc://"</span>)

<span class="pl-en">@<span class="pl-s1">app</span>.<span class="pl-s1">task</span></span>
<span class="pl-k">def</span> <span class="pl-en">test</span>(<span class="pl-s1">json_file</span>):
    <span class="pl-s1">json_content</span> <span class="pl-c1">=</span> <span class="pl-v">System</span>.<span class="pl-v">IO</span>.<span class="pl-v">File</span>.<span class="pl-v">ReadAllText</span>(<span class="pl-s1">json_file</span>)
    <span class="pl-k">return</span> <span class="pl-c1">1</span></pre></div>
<p dir="auto">start.sh:</p>
<div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="#!/bin/bash

CELERY_ALWAYS_EAGER=True
CELERYD_MAX_TASKS_PER_CHILD=1
CELERYD_CONCURRENCY=1
CELERYD_PREFETCH_MULTIPLIER=1
CELERY_RESULT_BACKEND=&quot;rpc://&quot;
CELERY_RESULT_PERSISTENT=False

celery -A tasks worker --loglevel=INFO --max-tasks-per-child=1"><pre class="notranslate"><span class="pl-c"><span class="pl-c">#!</span>/bin/bash</span>

CELERY_ALWAYS_EAGER=True
CELERYD_MAX_TASKS_PER_CHILD=1
CELERYD_CONCURRENCY=1
CELERYD_PREFETCH_MULTIPLIER=1
CELERY_RESULT_BACKEND=<span class="pl-s"><span class="pl-pds">"</span>rpc://<span class="pl-pds">"</span></span>
CELERY_RESULT_PERSISTENT=False

celery -A tasks worker --loglevel=INFO --max-tasks-per-child=1</pre></div>
<p dir="auto">When I test this task it runs fine as a standalone function.<br>
But this problem is nebulous, because I can issue a task with the path to a small json (&lt;1mb) and it succeeds:</p>
<div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="$ ipython -i
Python 3.7.4 (default, Aug 13 2019, 20:35:49)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.19.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import tasks

In [2]: res = tasks.test.delay(&quot;/path/small.json&quot;)

In [3]: res.ready()
Out[3]: True"><pre class="notranslate">$ ipython -i
Python 3.7.4 (default, Aug 13 2019, 20:35:49)
Type <span class="pl-s"><span class="pl-pds">'</span>copyright<span class="pl-pds">'</span></span>, <span class="pl-s"><span class="pl-pds">'</span>credits<span class="pl-pds">'</span></span> or <span class="pl-s"><span class="pl-pds">'</span>license<span class="pl-pds">'</span></span> <span class="pl-k">for</span> more information
IPython 7.19.0 -- An enhanced Interactive Python. Type <span class="pl-s"><span class="pl-pds">'</span>?<span class="pl-pds">'</span></span> <span class="pl-k">for</span> help.

In [1]: import tasks

In [2]: res = tasks.test.delay(<span class="pl-s"><span class="pl-pds">"</span>/path/small.json<span class="pl-pds">"</span></span>)

In [3]: <span class="pl-en">res.ready</span>()
Out[3]: True</pre></div>
<p dir="auto">But when I do this with the path to a bigger json (~7mb), it stalls indefinitely:</p>
<div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="In [4]: res_big = tasks.test.delay(&quot;/path/bigger.json&quot;)

In [5]: res_big.ready()
Out[5]: False"><pre class="notranslate">In [4]: res_big = tasks.test.delay(<span class="pl-s"><span class="pl-pds">"</span>/path/bigger.json<span class="pl-pds">"</span></span>)

In [5]: <span class="pl-en">res_big.ready</span>()
Out[5]: False</pre></div>
<p dir="auto">Does anybody have an idea what can be related to this issue ? It looks connected to memory used.</p>
<p dir="auto">NOTE:<br>
This problem is not IO related, as I have tested it by reading a json as a string in the python way and using Newtonsoft.Json.dll to deserialize it. This test behaves along the similiarly with the same files, works with the small json, stalls on the bigger one.</p>
<p dir="auto"></p>
</details>
<h1 dir="auto">Expected Behavior</h1>

<p dir="auto">I expect a task to succeed independent of the size of the supplied json.</p>
<h1 dir="auto">Actual Behavior</h1>

<p dir="auto">Sizeable (&lt;1 mB)  json's succeed, but bigger (~ 7 mB) json stall; the task is received, but never succeeds.</p>
<h1 dir="auto">Checklist</h1>

<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have verified that the issue exists against the <code class="notranslate">master</code> branch of Celery.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> This has already been asked to the <a href="https://groups.google.com/forum/#!forum/celery-users" rel="nofollow">discussion group</a> first.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have read the relevant section in the<br>
<a href="http://docs.celeryproject.org/en/latest/contributing.html#other-bugs" rel="nofollow">contribution guide</a><br>
on reporting bugs.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/issues?q=is%3Aissue+label%3A%22Issue+Type%3A+Bug+Report%22+-label%3A%22Category%3A+Documentation%22">issues list</a><br>
for similar or identical bug reports.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/pulls?q=is%3Apr+label%3A%22PR+Type%3A+Bugfix%22+-label%3A%22Category%3A+Documentation%22">pull requests list</a><br>
for existing proposed fixes.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/commits/master">commit log</a><br>
to find out if the bug was already fixed in the master branch.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included all related issues and possible duplicate issues<br>
in this issue (If there are none, check this box anyway).</li>
</ul>
<h2 dir="auto">Mandatory Debugging Information</h2>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have included the output of <code class="notranslate">celery -A proj report</code> in the issue.<br>
(if you are not able to do this, then at least specify the Celery<br>
version affected).</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have verified that the issue exists against the <code class="notranslate">master</code> branch of Celery.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included the contents of <code class="notranslate">pip freeze</code> in the issue.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have included all the versions of all the external dependencies required<br>
to reproduce this bug.</li>
</ul>
<h2 dir="auto">Optional Debugging Information</h2>

<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one Python version<br>
and/or implementation.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one message broker and/or<br>
result backend.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one version of the message<br>
broker and/or result backend.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one operating system.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one workers pool.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue with autoscaling, retries,<br>
ETA/Countdown &amp; rate limits disabled.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have tried reproducing the issue after downgrading<br>
and/or upgrading Celery and its dependencies.</li>
</ul>
<h2 dir="auto">Related Issues and Possible Duplicates</h2>

<h4 dir="auto">Related Issues</h4>
<ul dir="auto">
<li>None</li>
</ul>
<h4 dir="auto">Possible Duplicates</h4>
<ul dir="auto">
<li>None</li>
</ul>
<h2 dir="auto">Environment &amp; Settings</h2>

<p dir="auto"><strong>Celery version</strong>:</p>

<details>
<summary><b><code class="notranslate">celery report</code> Output:</b></summary>
<p dir="auto">
</p><div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="celery==4.4.0
django-redis==4.10.0
redis==3.3.11"><pre class="notranslate"><code class="notranslate">celery==4.4.0
django-redis==4.10.0
redis==3.3.11
</code></pre></div>
<p dir="auto"></p>
</details>
<h1 dir="auto">Steps to Reproduce</h1>
<h2 dir="auto">Required Dependencies</h2>

<ul dir="auto">
<li><strong>Minimal Python Version</strong>: 3.6</li>
<li><strong>Minimal Celery Version</strong>: 4.4.0</li>
<li><strong>Minimal Kombu Version</strong>: 4.6.7</li>
<li><strong>Minimal Broker Version</strong>: rabbitmq</li>
<li><strong>Minimal Result Backend Version</strong>: redis</li>
<li><strong>Minimal OS and/or Kernel Version</strong>: ubuntu 18.04</li>
<li><strong>Minimal Broker Client Version</strong>: N/A or Unknown</li>
<li><strong>Minimal Result Backend Client Version</strong>: N/A or Unknown</li>
</ul>
<h3 dir="auto">Python Packages</h3>

<details>
<summary><b><code class="notranslate">pip freeze</code> Output:</b></summary>
<p dir="auto">
</p><div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="celery==4.4.0
kombu==4.6.7
redis==3.3.11"><pre class="notranslate"><code class="notranslate">celery==4.4.0
kombu==4.6.7
redis==3.3.11
</code></pre></div>
<p dir="auto"></p>
</details>
<h3 dir="auto">Other Dependencies</h3>

<details>
<p dir="auto">
N/A
</p>
</details>
<h2 dir="auto">Minimally Reproducible Test Case</h2>

<details>
<p dir="auto">
</p>
</details>
<h1 dir="auto">Actual Behavior</h1>

<p dir="auto">I have the following code that run as expected on a single worker / code base :</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="@task()
def fail(*args, **kwargs):
    raise NotImplementedError

@task()
def set_error(*args, **kwargs):
    print('im in')

fail.s().on_error(set_error.s()).delay('foo', foo='bar').get()"><pre class="notranslate"><span class="pl-en">@<span class="pl-en">task</span>()</span>
<span class="pl-k">def</span> <span class="pl-en">fail</span>(<span class="pl-c1">*</span><span class="pl-s1">args</span>, <span class="pl-c1">**</span><span class="pl-s1">kwargs</span>):
    <span class="pl-k">raise</span> <span class="pl-v">NotImplementedError</span>

<span class="pl-en">@<span class="pl-en">task</span>()</span>
<span class="pl-k">def</span> <span class="pl-en">set_error</span>(<span class="pl-c1">*</span><span class="pl-s1">args</span>, <span class="pl-c1">**</span><span class="pl-s1">kwargs</span>):
    <span class="pl-en">print</span>(<span class="pl-s">'im in'</span>)

<span class="pl-s1">fail</span>.<span class="pl-en">s</span>().<span class="pl-en">on_error</span>(<span class="pl-s1">set_error</span>.<span class="pl-en">s</span>()).<span class="pl-en">delay</span>(<span class="pl-s">'foo'</span>, <span class="pl-s1">foo</span><span class="pl-c1">=</span><span class="pl-s">'bar'</span>).<span class="pl-en">get</span>()</pre></div>
<p dir="auto">As expected, behavior is:</p>
<ol dir="auto">
<li><code class="notranslate">fail</code> task is called and raise a error</li>
<li><code class="notranslate">set_error</code> is called</li>
<li>client gets a <code class="notranslate">NotImplementedError</code> exception</li>
</ol>
<p dir="auto">Now let say <code class="notranslate">fail</code> is on another worker. code becomes:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="signature('proj.fail', queue='some-queue').on_error(set_error.s()).delay('foo', foo='bar').get()"><pre class="notranslate"><span class="pl-en">signature</span>(<span class="pl-s">'proj.fail'</span>, <span class="pl-s1">queue</span><span class="pl-c1">=</span><span class="pl-s">'some-queue'</span>).<span class="pl-en">on_error</span>(<span class="pl-s1">set_error</span>.<span class="pl-en">s</span>()).<span class="pl-en">delay</span>(<span class="pl-s">'foo'</span>, <span class="pl-s1">foo</span><span class="pl-c1">=</span><span class="pl-s">'bar'</span>).<span class="pl-en">get</span>()</pre></div>
<p dir="auto">Behavior:</p>
<ol dir="auto">
<li><code class="notranslate">fail</code> task is called and raise an error.  kombu tries to serialize the error leading to another exception (see Traceback below)</li>
</ol>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Traceback (most recent call last):
  File &quot;/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/billiard/pool.py&quot;, line 1791, in safe_apply_callback
    fun(*args, **kwargs)
  File &quot;/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/celery/worker/request.py&quot;, line 528, in on_failure
    store_result=self.store_errors,
  File &quot;/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/celery/backends/base.py&quot;, line 165, in mark_as_failure
    self._call_task_errbacks(request, exc, traceback)
  File &quot;/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/celery/backends/base.py&quot;, line 188, in _call_task_errbacks
    errback(request, exc, traceback)
  File &quot;/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/celery/canvas.py&quot;, line 613, in __call__
    return self.apply_async(args, kwargs)
  File &quot;/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/celery/canvas.py&quot;, line 642, in apply_async
    dict(self.options, **options) if options else self.options))
  File &quot;/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/celery/canvas.py&quot;, line 669, in run
    first_task.apply_async(**options)
  File &quot;/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/celery/canvas.py&quot;, line 235, in apply_async
    return _apply(args, kwargs, **options)
  File &quot;/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/celery/app/base.py&quot;, line 771, in send_task
    amqp.send_task_message(P, name, message, **options)
  File &quot;/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/celery/app/amqp.py&quot;, line 559, in send_task_message
    **properties
  File &quot;/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/kombu/messaging.py&quot;, line 169, in publish
    compression, headers)
  File &quot;/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/kombu/messaging.py&quot;, line 252, in _prepare
    body) = dumps(body, serializer=serializer)
  File &quot;/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/kombu/serialization.py&quot;, line 221, in dumps
    payload = encoder(data)
  File &quot;/.pyenv/versions/3.6.6/lib/python3.6/contextlib.py&quot;, line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File &quot;/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/kombu/serialization.py&quot;, line 54, in _reraise_errors
    reraise(wrapper, wrapper(exc), sys.exc_info()[2])
  File &quot;/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/vine/five.py&quot;, line 194, in reraise
    raise value.with_traceback(tb)
  File &quot;/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/kombu/serialization.py&quot;, line 50, in _reraise_errors
    yield
  File &quot;/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/kombu/serialization.py&quot;, line 221, in dumps
    payload = encoder(data)
  File &quot;/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/kombu/utils/json.py&quot;, line 70, in dumps
    **dict(default_kwargs, **kwargs))
  File &quot;/.pyenv/versions/3.6.6/lib/python3.6/json/__init__.py&quot;, line 238, in dumps
    **kw).encode(obj)
  File &quot;/.pyenv/versions/3.6.6/lib/python3.6/json/encoder.py&quot;, line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File &quot;/.pyenv/versions/3.6.6/lib/python3.6/json/encoder.py&quot;, line 257, in iterencode
    return _iterencode(o, 0)
  File &quot;/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/kombu/utils/json.py&quot;, line 59, in default
    return super(JSONEncoder, self).default(o)
  File &quot;/.pyenv/versions/3.6.6/lib/python3.6/json/encoder.py&quot;, line 180, in default
    o.__class__.__name__)
kombu.exceptions.EncodeError: Object of type 'Context' is not JSON serializable"><pre class="notranslate"><code class="notranslate">Traceback (most recent call last):
  File "/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/billiard/pool.py", line 1791, in safe_apply_callback
    fun(*args, **kwargs)
  File "/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/celery/worker/request.py", line 528, in on_failure
    store_result=self.store_errors,
  File "/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/celery/backends/base.py", line 165, in mark_as_failure
    self._call_task_errbacks(request, exc, traceback)
  File "/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/celery/backends/base.py", line 188, in _call_task_errbacks
    errback(request, exc, traceback)
  File "/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/celery/canvas.py", line 613, in __call__
    return self.apply_async(args, kwargs)
  File "/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/celery/canvas.py", line 642, in apply_async
    dict(self.options, **options) if options else self.options))
  File "/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/celery/canvas.py", line 669, in run
    first_task.apply_async(**options)
  File "/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/celery/canvas.py", line 235, in apply_async
    return _apply(args, kwargs, **options)
  File "/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/celery/app/base.py", line 771, in send_task
    amqp.send_task_message(P, name, message, **options)
  File "/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/celery/app/amqp.py", line 559, in send_task_message
    **properties
  File "/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/kombu/messaging.py", line 169, in publish
    compression, headers)
  File "/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/kombu/messaging.py", line 252, in _prepare
    body) = dumps(body, serializer=serializer)
  File "/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/kombu/serialization.py", line 221, in dumps
    payload = encoder(data)
  File "/.pyenv/versions/3.6.6/lib/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File "/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/kombu/serialization.py", line 54, in _reraise_errors
    reraise(wrapper, wrapper(exc), sys.exc_info()[2])
  File "/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/vine/five.py", line 194, in reraise
    raise value.with_traceback(tb)
  File "/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/kombu/serialization.py", line 50, in _reraise_errors
    yield
  File "/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/kombu/serialization.py", line 221, in dumps
    payload = encoder(data)
  File "/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/kombu/utils/json.py", line 70, in dumps
    **dict(default_kwargs, **kwargs))
  File "/.pyenv/versions/3.6.6/lib/python3.6/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/.pyenv/versions/3.6.6/lib/python3.6/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/.pyenv/versions/3.6.6/lib/python3.6/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/.pyenv/versions/3.6.6/envs/my-env/lib/python3.6/site-packages/kombu/utils/json.py", line 59, in default
    return super(JSONEncoder, self).default(o)
  File "/.pyenv/versions/3.6.6/lib/python3.6/json/encoder.py", line 180, in default
    o.__class__.__name__)
kombu.exceptions.EncodeError: Object of type 'Context' is not JSON serializable
</code></pre></div>
<ol start="2" dir="auto">
<li><code class="notranslate">set_error</code> is never called</li>
<li>client wait forever</li>
</ol>
<h1 dir="auto">Expected Behavior</h1>

<p dir="auto"><code class="notranslate">Context</code> object should be serialized and error propagated.</p>
<h1 dir="auto">Workaround</h1>
<p dir="auto">using <code class="notranslate">pickle</code> as default serializer fix the issue</p>
<p dir="auto">Best regards<br>
Nicolas</p>