<p dir="auto">Hi everyone,</p>
<p dir="auto">I was using Celery 3.0.11 + Redis 2.6.2 as a broker. In my development environment (Mac OS X 10.8.2, Python 2.7.3, Django 1.4.3, 1 celery worker with concurrency = 4) I created a couple of simple tasks for my project (in tasks.py):</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="@task
def invoice_delete(invoice):
    i = reload_instance(invoice)
    if not i.deleted and i.amount_paid == 0:
        i.delete()

@task
def expire_invoices():
    invoices_to_expire = core.models.Invoice.objects.filter(
        expires__lt=(now() + timedelta(minutes=1)),
        deleted=False,
    )
    for i in invoices_to_expire:
        invoice_delete.apply_async(args=[i], eta=i.expires)"><pre class="notranslate"><span class="pl-en">@<span class="pl-s1">task</span></span>
<span class="pl-k">def</span> <span class="pl-en">invoice_delete</span>(<span class="pl-s1">invoice</span>):
    <span class="pl-s1">i</span> <span class="pl-c1">=</span> <span class="pl-en">reload_instance</span>(<span class="pl-s1">invoice</span>)
    <span class="pl-k">if</span> <span class="pl-c1">not</span> <span class="pl-s1">i</span>.<span class="pl-s1">deleted</span> <span class="pl-c1">and</span> <span class="pl-s1">i</span>.<span class="pl-s1">amount_paid</span> <span class="pl-c1">==</span> <span class="pl-c1">0</span>:
        <span class="pl-s1">i</span>.<span class="pl-en">delete</span>()

<span class="pl-en">@<span class="pl-s1">task</span></span>
<span class="pl-k">def</span> <span class="pl-en">expire_invoices</span>():
    <span class="pl-s1">invoices_to_expire</span> <span class="pl-c1">=</span> <span class="pl-s1">core</span>.<span class="pl-s1">models</span>.<span class="pl-v">Invoice</span>.<span class="pl-s1">objects</span>.<span class="pl-en">filter</span>(
        <span class="pl-s1">expires__lt</span><span class="pl-c1">=</span>(<span class="pl-en">now</span>() <span class="pl-c1">+</span> <span class="pl-en">timedelta</span>(<span class="pl-s1">minutes</span><span class="pl-c1">=</span><span class="pl-c1">1</span>)),
        <span class="pl-s1">deleted</span><span class="pl-c1">=</span><span class="pl-c1">False</span>,
    )
    <span class="pl-k">for</span> <span class="pl-s1">i</span> <span class="pl-c1">in</span> <span class="pl-s1">invoices_to_expire</span>:
        <span class="pl-s1">invoice_delete</span>.<span class="pl-en">apply_async</span>(<span class="pl-s1">args</span><span class="pl-c1">=</span>[<span class="pl-s1">i</span>], <span class="pl-s1">eta</span><span class="pl-c1">=</span><span class="pl-s1">i</span>.<span class="pl-s1">expires</span>)</pre></div>
<p dir="auto">and scheduled expire_invoices tasks to run every minute (in settings.py):</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="CELERYBEAT_SCHEDULE = {
    'expire_invoices': {
        'task': 'core.tasks.expire_invoices',
        'schedule': crontab(),
        }
}"><pre class="notranslate"><span class="pl-v">CELERYBEAT_SCHEDULE</span> <span class="pl-c1">=</span> {
    <span class="pl-s">'expire_invoices'</span>: {
        <span class="pl-s">'task'</span>: <span class="pl-s">'core.tasks.expire_invoices'</span>,
        <span class="pl-s">'schedule'</span>: <span class="pl-en">crontab</span>(),
        }
}</pre></div>
<p dir="auto">it worked, and I pushed it into production - Amazon Linux, Celery 3.0.11 - 1 worker with concurency=10, Redis 2.6.2, Python 2.7.3, Django 1.4.3. Celery and Redis were running under Supervisord. In production I got a problem: my expire_invoices task was launched 3 times within every minute, instead of 1, so I got duplicated invoice.deleted events. I tried the following:</p>
<ul dir="auto">
<li>changed crontab() to timedelta(seconds=60) - did not helped, still launched 3 times within a minute. If I tried seconds=30, it was launched twice within 30 seconds interval;</li>
<li>upgraded celery to 3.0.15 - no effect;</li>
<li>purged broker storage (using FLUSHALL - very rude) and cleaning schedule file and restarting everything - no effect;</li>
<li>switched from celery beat schedule storage from a file to a database, via "-S djcelery.schedulers.DatabaseScheduler" option - no effect;</li>
<li>temporally switched broker to 'django://'. I know that it is not recommended for production, I just wanted to see what happens. And... it helped. My task was launching exactly once a minute, as needed. Finally I ended up with installation of RabbitMQ and switching broker url to it, works fine as well.</li>
</ul>
<p dir="auto">My supervisord configs in production - celery:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="[program:%(project_name)s_celery]
command=%(project_env)s/bin/python manage.py celery worker -E --loglevel=INFO --concurrency=10
directory=%(project_app)s
user=%(project_user)s
numprocs=1
stdout_logfile=%(project_logs)s/supervisor_celery_stdout.log
stderr_logfile=%(project_logs)s/supervisor_celery_stderr.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs = 600
priority=998"><pre class="notranslate"><code class="notranslate">[program:%(project_name)s_celery]
command=%(project_env)s/bin/python manage.py celery worker -E --loglevel=INFO --concurrency=10
directory=%(project_app)s
user=%(project_user)s
numprocs=1
stdout_logfile=%(project_logs)s/supervisor_celery_stdout.log
stderr_logfile=%(project_logs)s/supervisor_celery_stderr.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs = 600
priority=998
</code></pre></div>
<p dir="auto">...and celery beat:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="
[program:%(project_name)s_celery_beat]
command=%(project_env)s/bin/python manage.py celery beat --loglevel=INFO -s %(celerybeat_dir)s/schedule
directory=%(project_app)s
user=%(project_user)s
numprocs=1
stdout_logfile=%(project_logs)s/supervisor_celery_beat_stdout.log
stderr_logfile=%(project_logs)s/supervisor_celery_beat_stderr.log
autostart=true
autorestart=true
startsecs=10
priority=999"><pre class="notranslate"><code class="notranslate">
[program:%(project_name)s_celery_beat]
command=%(project_env)s/bin/python manage.py celery beat --loglevel=INFO -s %(celerybeat_dir)s/schedule
directory=%(project_app)s
user=%(project_user)s
numprocs=1
stdout_logfile=%(project_logs)s/supervisor_celery_beat_stdout.log
stderr_logfile=%(project_logs)s/supervisor_celery_beat_stderr.log
autostart=true
autorestart=true
startsecs=10
priority=999
</code></pre></div>
<h1 dir="auto">Checklist</h1>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have verified that the issue exists against the <code class="notranslate">master</code> branch of Celery.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> This has already been asked to the <a href="https://groups.google.com/forum/#!forum/celery-users" rel="nofollow">discussion group</a> first.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have read the relevant section in the<br>
<a href="http://docs.celeryproject.org/en/latest/contributing.html#other-bugs" rel="nofollow">contribution guide</a><br>
on reporting bugs.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/issues?q=is%3Aissue+label%3A%22Issue+Type%3A+Bug+Report%22+-label%3A%22Category%3A+Documentation%22">issues list</a><br>
for similar or identical bug reports.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/pulls?q=is%3Apr+label%3A%22PR+Type%3A+Bugfix%22+-label%3A%22Category%3A+Documentation%22">pull requests list</a><br>
for existing proposed fixes.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have checked the <a href="https://github.com/celery/celery/commits/master">commit log</a><br>
to find out if the bug was already fixed in the master branch.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included all related issues and possible duplicate issues<br>
in this issue (If there are none, check this box anyway).</li>
</ul>
<h2 dir="auto">Mandatory Debugging Information</h2>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have included the output of <code class="notranslate">celery -A proj report</code> in the issue.<br>
(if you are not able to do this, then at least specify the Celery<br>
version affected).</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have verified that the issue exists against the <code class="notranslate">master</code> branch of Celery.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included the contents of <code class="notranslate">pip freeze</code> in the issue.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have included all the versions of all the external dependencies required<br>
to reproduce this bug.</li>
</ul>
<h2 dir="auto">Optional Debugging Information</h2>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one Python version<br>
and/or implementation.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one message broker and/or<br>
result backend.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one version of the message<br>
broker and/or result backend.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one operating system.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue on more than one workers pool.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> I have tried reproducing the issue with autoscaling, retries,<br>
ETA/Countdown &amp; rate limits disabled.</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I have tried reproducing the issue after downgrading<br>
and/or upgrading Celery and its dependencies.</li>
</ul>
<h2 dir="auto">Related Issues and Possible Duplicates</h2>
<h4 dir="auto">Related Issues</h4>
<ul dir="auto">
<li><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="559621218" data-permission-text="Title is private" data-url="https://github.com/celery/celery/issues/5952" data-hovercard-type="pull_request" data-hovercard-url="/celery/celery/pull/5952/hovercard" href="https://github.com/celery/celery/pull/5952">#5952</a></li>
<li><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="114504575" data-permission-text="Title is private" data-url="https://github.com/celery/celery/issues/2903" data-hovercard-type="issue" data-hovercard-url="/celery/celery/issues/2903/hovercard" href="https://github.com/celery/celery/issues/2903">#2903</a></li>
</ul>
<h4 dir="auto">Possible Duplicates</h4>
<ul dir="auto">
<li>None</li>
</ul>
<h2 dir="auto">Environment &amp; Settings</h2>
<p dir="auto"><strong>Celery version</strong>: 4.4.1</p>
<h1 dir="auto">Steps to Reproduce</h1>
<h2 dir="auto">Required Dependencies</h2>
<h3 dir="auto">Python Packages</h3>
<details>
<summary><b><code class="notranslate">pip freeze</code> Output:</b></summary>
<p dir="auto">
```
redis==3.4.1
billiard==3.6.3.0
celery==4.4.1
kombu==4.6.8
```
</p>
</details>
<h3 dir="auto">Other Dependencies</h3>
<details>
<p dir="auto">
N/A
</p>
</details>
<h2 dir="auto">Minimally Reproducible Test Case</h2>
<details>
<p dir="auto">
</p><div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content=""><pre class="notranslate"></pre></div>
<p dir="auto"></p>
</details>
<h1 dir="auto">Expected Behavior</h1>
<p dir="auto">Celery just works</p>
<h1 dir="auto">Actual Behavior</h1>
<p dir="auto">After upgrading Celery from 4.4.0 to 4.4.1 I suddenly see <code class="notranslate">TypeError: __init__() got an unexpected keyword argument 'socket_keepalive'</code>.</p>
<p dir="auto">This seems to be caused by PR <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="559621218" data-permission-text="Title is private" data-url="https://github.com/celery/celery/issues/5952" data-hovercard-type="pull_request" data-hovercard-url="/celery/celery/pull/5952/hovercard" href="https://github.com/celery/celery/pull/5952">#5952</a>, which adds this kwarg for Redis connections. However, not all Redis connection constructors take the same arguments (e.g. <code class="notranslate">UnixDomainSocketConnection</code> doesn't take <code class="notranslate">socket_keepalive</code>). This seems to be happened before as noted in issue <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="114504575" data-permission-text="Title is private" data-url="https://github.com/celery/celery/issues/2903" data-hovercard-type="issue" data-hovercard-url="/celery/celery/issues/2903/hovercard" href="https://github.com/celery/celery/issues/2903">#2903</a>.</p>
<p dir="auto">Everything works fine if I downgrade to 4.4.0 again.</p>