<p dir="auto">Using the Redis backend, If you make modifications to your queues/exchanges/routing keys, you can end up with strange behaviour where queues get duplicated, and you end up with tasks being executed multiple times;</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Queue                        Exchange                     Routing Key
----------------             ----------------             ----------------
worker                       tasks                        worker.#
sender                       tasks                        sender.#
notification_send            tasks                        dynamic.notification_send.#
notification_build           tasks                        dynamic.notification_build.#
notification_send            tasks                        notification_send
default                      tasks                        task.default.#
notification_build           tasks                        notification_build
notification_build           tasks                        task.notification_build.#
default                      tasks                        default.#
notification_send            tasks                        notification_send.#
builder                      tasks                        builder.#
notification_build           tasks                        task.notification_build
default                      tasks                        task.#
notification_build           tasks                        notification_build.#"><pre class="notranslate"><code class="notranslate">Queue                        Exchange                     Routing Key
----------------             ----------------             ----------------
worker                       tasks                        worker.#
sender                       tasks                        sender.#
notification_send            tasks                        dynamic.notification_send.#
notification_build           tasks                        dynamic.notification_build.#
notification_send            tasks                        notification_send
default                      tasks                        task.default.#
notification_build           tasks                        notification_build
notification_build           tasks                        task.notification_build.#
default                      tasks                        default.#
notification_send            tasks                        notification_send.#
builder                      tasks                        builder.#
notification_build           tasks                        task.notification_build
default                      tasks                        task.#
notification_build           tasks                        notification_build.#
</code></pre></div>
<p dir="auto">To clear this out, I have to manually delete the Redis DB or find the keys via CLI and delete them - at which point things turn back to normal</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Queue                        Exchange                     Routing Key
----------------             ----------------             ----------------
notification_build           notification_build           notification_build.#
default                      tasks                        task.#
worker                       worker                       worker.#"><pre class="notranslate"><code class="notranslate">Queue                        Exchange                     Routing Key
----------------             ----------------             ----------------
notification_build           notification_build           notification_build.#
default                      tasks                        task.#
worker                       worker                       worker.#
</code></pre></div>
<p dir="auto">There should be the ability to not only purge tasks, but also purge out bindings / routing entries - because queue changes are going to happen and this is very unexpected behaviour.</p>
<p dir="auto">Cal</p>
<h1 dir="auto">Checklist</h1>
<ul dir="auto">
<li>[x ] I have read the relevant section in the<br>
<a href="http://docs.celeryproject.org/en/latest/contributing.html#other-bugs" rel="nofollow">contribution guide</a><br>
on reporting bugs.</li>
<li>[x ] I have checked the <a href="https://github.com/celery/celery/issues?q=is%3Aissue+label%3A%22Issue+Type%3A+Bug+Report%22+-label%3A%22Category%3A+Documentation%22">issues list</a><br>
for similar or identical bug reports.</li>
<li>[x ] I have checked the <a href="https://github.com/celery/celery/pulls?q=is%3Apr+label%3A%22PR+Type%3A+Bugfix%22+-label%3A%22Category%3A+Documentation%22">pull requests list</a><br>
for existing proposed fixes.</li>
<li>[x ] I have checked the <a href="https://github.com/celery/celery/commits/master">commit log</a><br>
to find out if the bug was already fixed in the master branch.</li>
<li>[x ] I have included all related issues and possible duplicate issues<br>
in this issue (If there are none, check this box anyway).</li>
</ul>
<h1 dir="auto">Steps to Reproduce</h1>
<h2 dir="auto">Minimally Reproducible Test Case</h2>
<p dir="auto">
</p><p dir="auto">Inside a task, raise an exception that does something tricky with its arguments.  For example, urllib3's <code class="notranslate">MaxRetryError</code> does this:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="class PoolError(HTTPError):
    &quot;Base exception for errors caused within a pool.&quot;
    def __init__(self, pool, message):
        self.pool = pool
        HTTPError.__init__(self, &quot;%s: %s&quot; % (pool, message))

    def __reduce__(self):
        # For pickling purposes.
        return self.__class__, (None, None)

class RequestError(PoolError):
    &quot;Base exception for PoolErrors that have associated URLs.&quot;
    def __init__(self, pool, url, message):
        self.url = url
        PoolError.__init__(self, pool, message)

    def __reduce__(self):
        # For pickling purposes.
        return self.__class__, (None, self.url, None)

class MaxRetryError(RequestError):
    &quot;&quot;&quot;Raised when the maximum number of retries is exceeded.

    :param pool: The connection pool
    :type pool: :class:`~urllib3.connectionpool.HTTPConnectionPool`
    :param string url: The requested Url
    :param exceptions.Exception reason: The underlying error

    &quot;&quot;&quot;

    def __init__(self, pool, url, reason=None):
        self.reason = reason

        message = &quot;Max retries exceeded with url: %s (Caused by %r)&quot; % (
            url, reason)

        RequestError.__init__(self, pool, url, message)
"><pre class="notranslate"><code class="notranslate">class PoolError(HTTPError):
    "Base exception for errors caused within a pool."
    def __init__(self, pool, message):
        self.pool = pool
        HTTPError.__init__(self, "%s: %s" % (pool, message))

    def __reduce__(self):
        # For pickling purposes.
        return self.__class__, (None, None)

class RequestError(PoolError):
    "Base exception for PoolErrors that have associated URLs."
    def __init__(self, pool, url, message):
        self.url = url
        PoolError.__init__(self, pool, message)

    def __reduce__(self):
        # For pickling purposes.
        return self.__class__, (None, self.url, None)

class MaxRetryError(RequestError):
    """Raised when the maximum number of retries is exceeded.

    :param pool: The connection pool
    :type pool: :class:`~urllib3.connectionpool.HTTPConnectionPool`
    :param string url: The requested Url
    :param exceptions.Exception reason: The underlying error

    """

    def __init__(self, pool, url, reason=None):
        self.reason = reason

        message = "Max retries exceeded with url: %s (Caused by %r)" % (
            url, reason)

        RequestError.__init__(self, pool, url, message)

</code></pre></div>
<p dir="auto">If you use this exception inside a celery task, like this:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="from urllib3.exceptions import MaxRetryError
raise MaxRetryError(None, 'http://fakeurl')"><pre class="notranslate"><code class="notranslate">from urllib3.exceptions import MaxRetryError
raise MaxRetryError(None, 'http://fakeurl')
</code></pre></div>
<p dir="auto">What this results in is an exception whose <code class="notranslate">args</code> contain only a single string, instead of the minimum 2 arguments that MaxRetry requires.</p>
<p dir="auto">This will make it impossible to use its AsyncResult, because when it tries to rebuild the exception it doesn't have the right number of arguments.</p>
<p dir="auto">The problem is in celery/backends/base.py in Backend.prepare_exception, which is</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="    def prepare_exception(self, exc, serializer=None):
        &quot;&quot;&quot;Prepare exception for serialization.&quot;&quot;&quot;
        serializer = self.serializer if serializer is None else serializer
        if serializer in EXCEPTION_ABLE_CODECS:
            return get_pickleable_exception(exc)
        return {'exc_type': type(exc).__name__,
                'exc_message': ensure_serializable(exc.args, self.encode),
                'exc_module': type(exc).__module__}"><pre class="notranslate"><code class="notranslate">    def prepare_exception(self, exc, serializer=None):
        """Prepare exception for serialization."""
        serializer = self.serializer if serializer is None else serializer
        if serializer in EXCEPTION_ABLE_CODECS:
            return get_pickleable_exception(exc)
        return {'exc_type': type(exc).__name__,
                'exc_message': ensure_serializable(exc.args, self.encode),
                'exc_module': type(exc).__module__}
</code></pre></div>
<p dir="auto">The issue is that <code class="notranslate">exc.args</code> can't be relied upon to reinstantiate this exception.  Instead, I think we should use the result of <code class="notranslate">exc.__reduce__()</code> which exists to preserve, more-or-less, the constructor arguments for pickling.  I think the fix is something like:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="    def prepare_exception(self, exc, serializer=None):
        &quot;&quot;&quot;Prepare exception for serialization.&quot;&quot;&quot;
        serializer = self.serializer if serializer is None else serializer
        if serializer in EXCEPTION_ABLE_CODECS:
            return get_pickleable_exception(exc)
        exc_class, exc_args = exc.__reduce__()
        return {'exc_type': exc_class.__name__,
                'exc_message': ensure_serializable(exc_args, self.encode),
                'exc_module': exc_class.__module__}"><pre class="notranslate"><code class="notranslate">    def prepare_exception(self, exc, serializer=None):
        """Prepare exception for serialization."""
        serializer = self.serializer if serializer is None else serializer
        if serializer in EXCEPTION_ABLE_CODECS:
            return get_pickleable_exception(exc)
        exc_class, exc_args = exc.__reduce__()
        return {'exc_type': exc_class.__name__,
                'exc_message': ensure_serializable(exc_args, self.encode),
                'exc_module': exc_class.__module__}
</code></pre></div>
<p dir="auto"></p>