Advice on Python/Django and message queues
Stompserver is a good option. It's lightweight, easy to install and easy to use from Django/python.
We have a system using stompserver in production for sending out emails and processing other jobs asynchronously.
Django saves the emails to the database, a model.post_save handler in Django sends an event to stompserver and stompserver passes the event to a consumer process which does the asynchronous task (sends the email).
It scales up quite nicely because you can add consumer processes at runtime - two consumers can send twice as many emails, and the consumers can be on seperate machines. One slight complication is that each consumer needs its own named queue so Django needs to know how many consumers are available and send events to each queue in a round-robin way. (Two consumers listening on the same queue will both get each message = duplication). If you only want one consumer process then this isn't an issue.
We previously had processes which polled the database continuously for jobs but found that it was adding a lot of load to the system, even when nothing needed to be processed.
I have an application in Django, that needs to send a large number of emails to users in various use cases. I don't want to handle this synchronously within the application for obvious reasons.
Has anyone any recommendations for a message queuing server which integrates well with Python, or they have used on a Django project? The rest of my stack is Apache, mod_python, MySQL.