Serving many on-the-fly generated images with Django
If one user is all you need to bring your webserver down then the problem is not apache or mod_wsgi.
First you should optimize your tiling routines and check if you really only deliver the data a user actually sees.
After that a faster cpu, more ram, a ssd and aggressive caching will give you more performance.
At last you may get some extra points for using another webserver, but dont expect too much from that.
Similar to a tiling server for spatial image data, I want to view many on-the-fly generated images in my Django based web application (merge images, color change, etc.). Since one client can easily request many (>100) images in a short time, it is easy to bring the web server (Apache + mod_wsgi) down.
Hence, I am looking for alternative ways. Since we already use Celery, it might be a good idea to do this image processing asynchronously and push the generated data to the client. To get started with that I switched the WSGI server to be gevent with Apache used as a proxy. However, I haven't managed to get the push thing working yet and I am not quite sure if this is the right direction anyway. Based on that I have three questions:

Do you think this (Celery, gevent, Socket.IO) is a sensible way to allow many clients to use the application without bringing the web server down? Do you see alternatives?
If I hand over the image processing to Celery and let it push the image data to the browser when it is done, the connection won't go through Apache, will it?
If some kind of pushing to the client is used, would it be better to use one connection or one for each image (and close it when done)?

Background:
The Django application I am working on allows a user to display very large images. This is done by tiling the large images before and show only the currently relevant tiles in a grid to the user. From what I understand this is the standard way to serve data in the field of mapping and spatial image data (e.g. OpenStreetMap). But unlike mapping data, we also have many slices in Z a user can scroll through (biological images).
All this works fine when the tiles are statically served. Now I added the option to generate those tiles on the fly -- different images are merged, color corrected, …. This works, but is some heavy load for the web server as one image takes about 0.1s to be generated. Currently we use Apache with mod_wsgi (WSGIRestrictedEmbedded On) and it is easy to bring the server down. Just browsing through the image stack will lead to a hanging web server. I already tried to adjust MaxClients, etc. and turned KeepAlive off. I also tried different thread/processes combinations for mod_wsgi. However, nothing helped enough to allow usage for more than one user. Therefore, I thought a Comet/WebSocket way could help here.