I have long running jobs (say, 5 minutes on average - up to an hour). I originally used celery (after picloud shut down) but it just doesn&#x27;t work well with those charcteristics. Each worker reserved an extra job so it was impossible to get good cpu utilisation.<p>I switched to rq and it&#x27;s all been much easier. The behaviour is easy to understand and it&#x27;s easy to inspect redis to see what&#x27;s going on.<p>In terms of the code restart angle - I&#x27;m fairly sure you can effectively restart the workers. They run as a single process that forks to do work. Each copy you run only has a single worker, so you need to run multiple instances yourself. If you kill the parent it waits until the child has finished the job it is on before terminating.<p>I could be wrong about some of the details. I&#x27;d recommend giving it a shot. I must have run 100,000s of jobs through it now and I haven&#x27;t had a single issue.<p><a href="http:&#x2F;&#x2F;python-rq.org&#x2F;docs&#x2F;workers&#x2F;" rel="nofollow">http:&#x2F;&#x2F;python-rq.org&#x2F;docs&#x2F;workers&#x2F;</a>