<h1 dir="auto">Checklist</h1>

<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have read the relevant section in the<br>
<a href="http://docs.celeryproject.org/en/latest/contributing.html#other-bugs" rel="nofollow">contribution guide</a><br>
on reporting bugs.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/issues?q=is%3Aissue+label%3A%22Issue+Type%3A+Bug+Report%22+-label%3A%22Category%3A+Documentation%22">issues list</a><br>
for similar or identical bug reports.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/pulls?q=is%3Apr+label%3A%22PR+Type%3A+Bugfix%22+-label%3A%22Category%3A+Documentation%22">pull requests list</a><br>
for existing proposed fixes.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/commits/master">commit log</a><br>
to find out if the bug was already fixed in the master branch.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included all related issues and possible duplicate issues<br>
in this issue (If there are none, check this box anyway).</li>
</ul>
<h2 dir="auto">Mandatory Debugging Information</h2>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included the output of <code class="notranslate">celery -A proj report</code> in the issue.<br>
(if you are not able to do this, then at least specify the Celery<br>
version affected).</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have verified that the issue exists against the <code class="notranslate">master</code> branch of Celery.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have included the contents of <code class="notranslate">pip freeze</code> in the issue.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included all the versions of all the external dependencies required<br>
to reproduce this bug.</li>
</ul>
<h2 dir="auto">Optional Debugging Information</h2>

<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one Python version<br>
and/or implementation.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one message broker and/or<br>
result backend.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one version of the message<br>
broker and/or result backend.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one operating system.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one workers pool.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue with autoscaling, retries,<br>
ETA/Countdown &amp; rate limits disabled.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have tried reproducing the issue after downgrading<br>
and/or upgrading Celery and its dependencies.</li>
</ul>
<h2 dir="auto">Related Issues and Possible Duplicates</h2>

<h4 dir="auto">Related Issues</h4>
<p dir="auto"><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="312784816" data-permission-text="Title is private" data-url="https://github.com/celery/celery/issues/4651" data-hovercard-type="issue" data-hovercard-url="/celery/celery/issues/4651/hovercard" href="https://github.com/celery/celery/issues/4651">#4651</a><br>
<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="211155296" data-permission-text="Title is private" data-url="https://github.com/celery/celery/issues/3878" data-hovercard-type="issue" data-hovercard-url="/celery/celery/issues/3878/hovercard" href="https://github.com/celery/celery/issues/3878">#3878</a></p>
<h4 dir="auto">Possible Duplicates</h4>
<ul dir="auto">
<li>None</li>
</ul>
<h2 dir="auto">Environment &amp; Settings</h2>

<p dir="auto"><strong>Celery version</strong>:</p>

<details>
software -&gt; celery:3.1.20 (Cipater) kombu:3.0.37 py:2.7.15rc1
            billiard:3.3.0.23 py-amqp:1.4.9
platform -&gt; system:Linux arch:64bit imp:CPython
loader   -&gt; celery.loaders.app.AppLoader
settings -&gt; transport:pyamqp results:disabled
<p dir="auto">
</p><div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content=""><pre class="notranslate"><code class="notranslate"></code></pre></div>
<p dir="auto"></p>
</details>
<h1 dir="auto">Steps to Reproduce</h1>
<h3 dir="auto">Python Packages</h3>

<details>
<summary><b><code class="notranslate">pip freeze</code> Output:</b></summary>
<p dir="auto">
</p><div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content=""><pre class="notranslate"><code class="notranslate"></code></pre></div>
<p dir="auto"></p>
</details>
<h3 dir="auto">Other Dependencies</h3>

<details>
<p dir="auto">
pyrfc==1.9.3
</p>
</details>
<h2 dir="auto">Minimally Reproducible Test Case</h2>

<details>
<p dir="auto">
</p><p dir="auto">This is my <code class="notranslate">tasks.py</code></p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import pyrfc
import os
import datetime
from celery import Celery
from celery.utils.log import get_task_logger
import logging
import requests
from celery.exceptions import SoftTimeLimitExceeded
from argparse import ArgumentParser

#logger = get_task_logger(__name__)
logging.basicConfig(level=logging.INFO)
logger = logging

app = Celery('tasks', broker='pyamqp://guest@localhost//')

config =  # redacted

con = pyrfc.Connection(**config)

function_name = # redacted 
function_args = # redacted

@app.task
def extract():
    logger.info(&quot;PID %s&quot;, os.getpid())
    logger.info(&quot;Connecting&quot;)
    try:
        con.call(function_name, **function_args)
    except SoftTimeLimitExceeded:
        logger.info(&quot;Exceeded RFC time&quot;)
    logger.info(&quot;Disconnected&quot;)
    return 1

@app.task()
def request():
    logger.info(&quot;Connecting&quot;)
    logger.info(&quot;PID %s&quot;, os.getpid())
    try:
        r = requests.get('http://127.0.0.1:8000')
        logger.info(&quot;Disconnected&quot;)
        return r.text
    except SoftTimeLimitExceeded:
        logger.info(&quot;Exceeded time&quot;)


def main():
    parser = ArgumentParser()
    parser.add_argument('func', choices=['request', 'extract'])
    args = parser.parse_args()
    if args.func == 'request':
        request()
    elif args.func == 'extract':
        extract()
    else:
        raise


if __name__ == '__main__':"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">pyrfc</span>
<span class="pl-k">import</span> <span class="pl-s1">os</span>
<span class="pl-k">import</span> <span class="pl-s1">datetime</span>
<span class="pl-k">from</span> <span class="pl-s1">celery</span> <span class="pl-k">import</span> <span class="pl-v">Celery</span>
<span class="pl-k">from</span> <span class="pl-s1">celery</span>.<span class="pl-s1">utils</span>.<span class="pl-s1">log</span> <span class="pl-k">import</span> <span class="pl-s1">get_task_logger</span>
<span class="pl-k">import</span> <span class="pl-s1">logging</span>
<span class="pl-k">import</span> <span class="pl-s1">requests</span>
<span class="pl-k">from</span> <span class="pl-s1">celery</span>.<span class="pl-s1">exceptions</span> <span class="pl-k">import</span> <span class="pl-v">SoftTimeLimitExceeded</span>
<span class="pl-k">from</span> <span class="pl-s1">argparse</span> <span class="pl-k">import</span> <span class="pl-v">ArgumentParser</span>

<span class="pl-c">#logger = get_task_logger(__name__)</span>
<span class="pl-s1">logging</span>.<span class="pl-en">basicConfig</span>(<span class="pl-s1">level</span><span class="pl-c1">=</span><span class="pl-s1">logging</span>.<span class="pl-v">INFO</span>)
<span class="pl-s1">logger</span> <span class="pl-c1">=</span> <span class="pl-s1">logging</span>

<span class="pl-s1">app</span> <span class="pl-c1">=</span> <span class="pl-v">Celery</span>(<span class="pl-s">'tasks'</span>, <span class="pl-s1">broker</span><span class="pl-c1">=</span><span class="pl-s">'pyamqp://guest@localhost//'</span>)

<span class="pl-s1">config</span> <span class="pl-c1">=</span>  <span class="pl-c"># redacted</span>

<span class="pl-s1">con</span> <span class="pl-c1">=</span> <span class="pl-s1">pyrfc</span>.<span class="pl-v">Connection</span>(<span class="pl-c1">**</span><span class="pl-s1">config</span>)

<span class="pl-s1">function_name</span> <span class="pl-c1">=</span> <span class="pl-c"># redacted </span>
<span class="pl-s1">function_args</span> <span class="pl-c1">=</span> <span class="pl-c"># redacted</span>

@<span class="pl-s1">app</span>.<span class="pl-s1">task</span>
<span class="pl-k">def</span> <span class="pl-en">extract</span>():
    <span class="pl-s1">logger</span>.<span class="pl-en">info</span>(<span class="pl-s">"PID %s"</span>, <span class="pl-s1">os</span>.<span class="pl-en">getpid</span>())
    <span class="pl-s1">logger</span>.<span class="pl-en">info</span>(<span class="pl-s">"Connecting"</span>)
    <span class="pl-k">try</span>:
        <span class="pl-s1">con</span>.<span class="pl-en">call</span>(<span class="pl-s1">function_name</span>, <span class="pl-c1">**</span><span class="pl-s1">function_args</span>)
    <span class="pl-k">except</span> <span class="pl-v">SoftTimeLimitExceeded</span>:
        <span class="pl-s1">logger</span>.<span class="pl-en">info</span>(<span class="pl-s">"Exceeded RFC time"</span>)
    <span class="pl-s1">logger</span>.<span class="pl-en">info</span>(<span class="pl-s">"Disconnected"</span>)
    <span class="pl-k">return</span> <span class="pl-c1">1</span>

<span class="pl-en">@<span class="pl-s1">app</span>.<span class="pl-en">task</span>()</span>
<span class="pl-k">def</span> <span class="pl-en">request</span>():
    <span class="pl-s1">logger</span>.<span class="pl-en">info</span>(<span class="pl-s">"Connecting"</span>)
    <span class="pl-s1">logger</span>.<span class="pl-en">info</span>(<span class="pl-s">"PID %s"</span>, <span class="pl-s1">os</span>.<span class="pl-en">getpid</span>())
    <span class="pl-k">try</span>:
        <span class="pl-s1">r</span> <span class="pl-c1">=</span> <span class="pl-s1">requests</span>.<span class="pl-en">get</span>(<span class="pl-s">'http://127.0.0.1:8000'</span>)
        <span class="pl-s1">logger</span>.<span class="pl-en">info</span>(<span class="pl-s">"Disconnected"</span>)
        <span class="pl-k">return</span> <span class="pl-s1">r</span>.<span class="pl-s1">text</span>
    <span class="pl-k">except</span> <span class="pl-v">SoftTimeLimitExceeded</span>:
        <span class="pl-s1">logger</span>.<span class="pl-en">info</span>(<span class="pl-s">"Exceeded time"</span>)


<span class="pl-k">def</span> <span class="pl-en">main</span>():
    <span class="pl-s1">parser</span> <span class="pl-c1">=</span> <span class="pl-v">ArgumentParser</span>()
    <span class="pl-s1">parser</span>.<span class="pl-en">add_argument</span>(<span class="pl-s">'func'</span>, <span class="pl-s1">choices</span><span class="pl-c1">=</span>[<span class="pl-s">'request'</span>, <span class="pl-s">'extract'</span>])
    <span class="pl-s1">args</span> <span class="pl-c1">=</span> <span class="pl-s1">parser</span>.<span class="pl-en">parse_args</span>()
    <span class="pl-k">if</span> <span class="pl-s1">args</span>.<span class="pl-s1">func</span> <span class="pl-c1">==</span> <span class="pl-s">'request'</span>:
        <span class="pl-en">request</span>()
    <span class="pl-k">elif</span> <span class="pl-s1">args</span>.<span class="pl-s1">func</span> <span class="pl-c1">==</span> <span class="pl-s">'extract'</span>:
        <span class="pl-en">extract</span>()
    <span class="pl-k">else</span>:
        <span class="pl-k">raise</span>


<span class="pl-k">if</span> <span class="pl-s1">__name__</span> <span class="pl-c1">==</span> <span class="pl-s">'__main__'</span>:</pre></div>
<p dir="auto">This is run with <code class="notranslate">celery -A tasks worker --loglevel=info -c 1 --soft-time-limit 30</code></p>
<p dir="auto">There are two tasks</p>
<ul dir="auto">
<li><strong>Request</strong>: Performs a HTTP GET to a dummy server, which contains a delay of 60 seconds.</li>
<li><strong>Extract</strong>: Performs a RFC call to a SAP Function, which also contains a delay of 60 seconds.</li>
</ul>
<p dir="auto"></p>
</details>
<h1 dir="auto">Expected Behavior</h1>
<p dir="auto">Both tasks should handle the soft time limit equally</p>
<h1 dir="auto">Actual Behavior</h1>

<p dir="auto">The <code class="notranslate">extract</code> task doesn't handle the soft time limit exception until the server replies.</p>
<p dir="auto">Running <code class="notranslate">request.delay()</code> we obtain as expected</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="[2019-05-29 16:45:17,745: INFO/MainProcess] Received task: tasks.request[47e3fad6-42f2-4896-956d-2e54f7080fb4]
[2019-05-29 16:45:17,746: INFO/Worker-1] Connecting
[2019-05-29 16:45:17,747: INFO/Worker-1] PID 2758
[2019-05-29 16:45:47,750: WARNING/MainProcess] Soft time limit (30.0s) exceeded for tasks.request[47e3fad6-42f2-4896-956d-2e54f7080fb4]
[2019-05-29 16:45:47,751: INFO/Worker-1] Exceeded time
[2019-05-29 16:45:47,752: INFO/MainProcess] Task tasks.request[47e3fad6-42f2-4896-956d-2e54f7080fb4] succeeded in 30.005404981s: None"><pre class="notranslate"><code class="notranslate">[2019-05-29 16:45:17,745: INFO/MainProcess] Received task: tasks.request[47e3fad6-42f2-4896-956d-2e54f7080fb4]
[2019-05-29 16:45:17,746: INFO/Worker-1] Connecting
[2019-05-29 16:45:17,747: INFO/Worker-1] PID 2758
[2019-05-29 16:45:47,750: WARNING/MainProcess] Soft time limit (30.0s) exceeded for tasks.request[47e3fad6-42f2-4896-956d-2e54f7080fb4]
[2019-05-29 16:45:47,751: INFO/Worker-1] Exceeded time
[2019-05-29 16:45:47,752: INFO/MainProcess] Task tasks.request[47e3fad6-42f2-4896-956d-2e54f7080fb4] succeeded in 30.005404981s: None
</code></pre></div>
<p dir="auto">Note the inmediate logging "Exceeded time" after the Celery warning</p>
<p dir="auto">When running <code class="notranslate">extract.delay()</code> we obtain</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="[2019-05-29 16:46:03,361: INFO/MainProcess] Received task: tasks.extract[213f1eae-a872-4bc7-83af-69fd3350bce4]
[2019-05-29 16:46:03,362: INFO/Worker-1] PID 2758
[2019-05-29 16:46:03,362: INFO/Worker-1] Connecting
[2019-05-29 16:46:33,363: WARNING/MainProcess] Soft time limit (30.0s) exceeded for tasks.extract[213f1eae-a872-4bc7-83af-69fd3350bce4]
[2019-05-29 16:47:04,065: INFO/Worker-1] Exceeded RFC time
[2019-05-29 16:47:04,065: INFO/Worker-1] Disconnected
[2019-05-29 16:47:04,066: INFO/MainProcess] Task tasks.extract[213f1eae-a872-4bc7-83af-69fd3350bce4] succeeded in 60.704235596s: 1"><pre class="notranslate"><code class="notranslate">[2019-05-29 16:46:03,361: INFO/MainProcess] Received task: tasks.extract[213f1eae-a872-4bc7-83af-69fd3350bce4]
[2019-05-29 16:46:03,362: INFO/Worker-1] PID 2758
[2019-05-29 16:46:03,362: INFO/Worker-1] Connecting
[2019-05-29 16:46:33,363: WARNING/MainProcess] Soft time limit (30.0s) exceeded for tasks.extract[213f1eae-a872-4bc7-83af-69fd3350bce4]
[2019-05-29 16:47:04,065: INFO/Worker-1] Exceeded RFC time
[2019-05-29 16:47:04,065: INFO/Worker-1] Disconnected
[2019-05-29 16:47:04,066: INFO/MainProcess] Task tasks.extract[213f1eae-a872-4bc7-83af-69fd3350bce4] succeeded in 60.704235596s: 1
</code></pre></div>
<p dir="auto">Note the "Exceeded RFC time" is logged well after the Celery Warning. Remembering the 60 seconds server delay, we see the task handles the Exception only after the server reply.</p>
<p dir="auto">I tested this both with my current celery version (3.1.20) and with the newer (4.3.0) with the same results.</p>
<h1 dir="auto">Other Tests</h1>
<p dir="auto">Running the tasks outside celery and sending the <strong>SIGSUR1</strong> signal via <code class="notranslate">kill -s USR1 &lt;pid&gt;</code></p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="$ python tasks.py request  
INFO:root:Connecting
INFO:root:PID 3645
[1]    3645 user-defined signal 1  python tasks.py request"><pre class="notranslate"><code class="notranslate">$ python tasks.py request  
INFO:root:Connecting
INFO:root:PID 3645
[1]    3645 user-defined signal 1  python tasks.py request
</code></pre></div>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="$  python tasks.py extract
INFO:root:PID 3699
INFO:root:Connecting
[1]    3699 user-defined signal 1  python tasks.py extract"><pre class="notranslate"><code class="notranslate">$  python tasks.py extract
INFO:root:PID 3699
INFO:root:Connecting
[1]    3699 user-defined signal 1  python tasks.py extract
</code></pre></div>
<p dir="auto">Both functions do receive the signal (without delay) which makes me believe (perhaps incorrectly) the issue is not related to <code class="notranslate">pyrfc</code> compatibility with the signal.</p>
<p dir="auto">I do understand this may not be a Celery fault, but any hint on where to look at / test is welcomed.</p>
<p dir="auto">Regards</p>
<h1 dir="auto">Checklist</h1>

<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have verified that the issue exists against the <code class="notranslate">master</code> branch of Celery.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> This has already been asked to the <a href="https://github.com/celery/celery/discussions">discussions forum</a> first.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have read the relevant section in the<br>
<a href="http://docs.celeryproject.org/en/latest/contributing.html#other-bugs" rel="nofollow">contribution guide</a><br>
on reporting bugs.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/issues?q=is%3Aissue+label%3A%22Issue+Type%3A+Bug+Report%22+-label%3A%22Category%3A+Documentation%22">issues list</a><br>
for similar or identical bug reports.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/pulls?q=is%3Apr+label%3A%22PR+Type%3A+Bugfix%22+-label%3A%22Category%3A+Documentation%22">pull requests list</a><br>
for existing proposed fixes.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/commits/master">commit log</a><br>
to find out if the bug was already fixed in the master branch.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included all related issues and possible duplicate issues<br>
in this issue (If there are none, check this box anyway).</li>
</ul>
<h1 dir="auto">Expected Behavior</h1>

<p dir="auto">I can have the following sqs broker configuration</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="    broker_transport_options={
        &quot;predefined_queues&quot;: {
            &quot;MY_QUEUE&quot;: {
                &quot;url&quot;: &quot;MYURL.fifo&quot;
            },
    },"><pre class="notranslate"><code class="notranslate">    broker_transport_options={
        "predefined_queues": {
            "MY_QUEUE": {
                "url": "MYURL.fifo"
            },
    },
</code></pre></div>
<p dir="auto">And no error will raise when calling my tasks.</p>
<h1 dir="auto">Actual Behavior</h1>

<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="  File &quot;/usr/local/lib/python3.9/site-packages/celery/app/task.py&quot;, line 425, in delay
    return self.apply_async(args, kwargs)
  File &quot;/usr/local/lib/python3.9/site-packages/celery/app/task.py&quot;, line 575, in apply_async
    return app.send_task(
  File &quot;/usr/local/lib/python3.9/site-packages/celery/app/base.py&quot;, line 788, in send_task
    amqp.send_task_message(P, name, message, **options)
  File &quot;/usr/local/lib/python3.9/site-packages/celery/app/amqp.py&quot;, line 510, in send_task_message
    ret = producer.publish(
  File &quot;/usr/local/lib/python3.9/site-packages/kombu/messaging.py&quot;, line 177, in publish
    return _publish(
  File &quot;/usr/local/lib/python3.9/site-packages/kombu/connection.py&quot;, line 524, in _ensured
    return fun(*args, **kwargs)
  File &quot;/usr/local/lib/python3.9/site-packages/kombu/messaging.py&quot;, line 199, in _publish
    return channel.basic_publish(
  File &quot;/usr/local/lib/python3.9/site-packages/kombu/transport/virtual/base.py&quot;, line 604, in basic_publish
    return self._put(routing_key, message, **kwargs)
  File &quot;/usr/local/lib/python3.9/site-packages/kombu/transport/SQS.py&quot;, line 392, in _put
    c.send_message(**kwargs)
  File &quot;/usr/local/lib/python3.9/site-packages/botocore/client.py&quot;, line 391, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File &quot;/usr/local/lib/python3.9/site-packages/botocore/client.py&quot;, line 719, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (MissingParameter) when calling the SendMessage operation: The request must contain the parameter MessageGroupId."><pre class="notranslate"><code class="notranslate">  File "/usr/local/lib/python3.9/site-packages/celery/app/task.py", line 425, in delay
    return self.apply_async(args, kwargs)
  File "/usr/local/lib/python3.9/site-packages/celery/app/task.py", line 575, in apply_async
    return app.send_task(
  File "/usr/local/lib/python3.9/site-packages/celery/app/base.py", line 788, in send_task
    amqp.send_task_message(P, name, message, **options)
  File "/usr/local/lib/python3.9/site-packages/celery/app/amqp.py", line 510, in send_task_message
    ret = producer.publish(
  File "/usr/local/lib/python3.9/site-packages/kombu/messaging.py", line 177, in publish
    return _publish(
  File "/usr/local/lib/python3.9/site-packages/kombu/connection.py", line 524, in _ensured
    return fun(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/kombu/messaging.py", line 199, in _publish
    return channel.basic_publish(
  File "/usr/local/lib/python3.9/site-packages/kombu/transport/virtual/base.py", line 604, in basic_publish
    return self._put(routing_key, message, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/kombu/transport/SQS.py", line 392, in _put
    c.send_message(**kwargs)
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 391, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 719, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (MissingParameter) when calling the SendMessage operation: The request must contain the parameter MessageGroupId.
</code></pre></div>
<h1 dir="auto">Explanation of the bug:</h1>
<p dir="auto">In the <a href="https://docs.celeryproject.org/projects/kombu/en/stable/_modules/kombu/transport/SQS.html" rel="nofollow">konbu</a> documentation there is the following lines that manage this exception.</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="        if queue.endswith('.fifo'):
            if 'MessageGroupId' in message['properties']:
                kwargs['MessageGroupId'] = \
                    message['properties']['MessageGroupId']
            else:
                kwargs['MessageGroupId'] = 'default'"><pre class="notranslate">        <span class="pl-k">if</span> <span class="pl-s1">queue</span>.<span class="pl-en">endswith</span>(<span class="pl-s">'.fifo'</span>):
            <span class="pl-k">if</span> <span class="pl-s">'MessageGroupId'</span> <span class="pl-c1">in</span> <span class="pl-s1">message</span>[<span class="pl-s">'properties'</span>]:
                <span class="pl-s1">kwargs</span>[<span class="pl-s">'MessageGroupId'</span>] <span class="pl-c1">=</span> \
                    <span class="pl-s1">message</span>[<span class="pl-s">'properties'</span>][<span class="pl-s">'MessageGroupId'</span>]
            <span class="pl-k">else</span>:
                <span class="pl-s1">kwargs</span>[<span class="pl-s">'MessageGroupId'</span>] <span class="pl-c1">=</span> <span class="pl-s">'default'</span></pre></div>
<p dir="auto">Since <code class="notranslate">Konbu</code> checks that  <code class="notranslate">routing_key</code> ends with <code class="notranslate">.fifo</code>, and not what it has been configured inside celery as <code class="notranslate">MY_URL</code>, means that the property <code class="notranslate">MessageGroupId</code> is never set, and the <code class="notranslate">botocore</code> error is raised.</p>
<p dir="auto">I can see why this is not a celery bug, or not a bug at all, but I would say this case should be at least documented: having to force celery queues name to follow the SQS paradigm, in my opinion, can mislead people.</p>