django-celery infrastructure over multiple servers, broker is redis
Celery actually makes this pretty simple, since you're already putting the tasks on a queue. All that changes with more workers is that each worker takes whatever's next on the queue - so multiple workers can process at once, each on their own machine.
There's three parts to this, and you've already got one of them.

Shared storage, so that all machines can access the same files 
A broker that can hand out tasks to multiple workers - redis is fine for that 
Workers on multiple machines

Here's how you set it up:

User uploads file to front-end server, which stores in your shared storage (e.g. S3, Samba, NFS, whatever), and stores the reference in the database
Front-end server kicks off a celery task to process the file e.g.

def my_view(request):
    # ... deal with storing the file
    file_in_db = store_file(request)
    my_process_file_task.delay(file_in_db.id)  # Use PK of DB record
    # do rest of view logic...


On each processing machine, run celery-worker:
python manage.py celery worker --loglevel=INFO -Q default -E

Then as you add more machines, you'll have more workers and the work will be split between them.
Key things to ensure:

You must have shared storage, or this gets much more complicated
Every worker machine must have the right Django/Celery settings to be able to find the redis broker and the shared storage (e.g. S3 bucket, keys etc)
Currently we have everything setup on single cloud server, that includes:

Database server
Apache
Celery
redis to serve as a broker for celery and for some other tasks
etc

Now we are thinking to break apart the main components to separate servers e.g. separate database server, separate storage for media files, web servers behind load balancers. The reason is to not to pay for one heavy server and use load balancers to create servers on demand to reduce cost and improve overall speed.
I am really confused about celery only, have anyone ever used celery on multiple production servers behind load balancers? Any guidance would be appreciated.
Consider one small use case which is currently how it is been done on single server (confusion is that how that can be done when we use multiple servers):

User uploads a abc.pptx file->reference is stored in database->stored on server disk
A task (convert document to pdf) is created and goes in redis (broker) queue
celery which is running on same server picks the task from queue

Read the file, convert it to pdf using software called docsplit
create a folder on server disk (which will be used as static content later on) puts pdf file and its thumbnail and plain text and the original file


Considering the above use case, how can you setup up multiple web servers which can perform the same functionality?