Creating a Persistent Data Object In Django
Consider running it in another process. You could have your Django application submit samples via a socket that the classifier process listens on, or you could run a queue and have Django submit requests to the queue.
I have a Python-based maximum entropy classifier. It's large, stored as a Pickle, and takes about a minute to unserialize. It's also not thread safe. However, it runs fast and can classify a sample (a simple Python dictionary) in a few milliseconds.
I'd like to create a basic Django web app, so users can submit samples to classify in realtime. How would I load the classifier into persistent memory once, and then regulate it so that each request can access the object without conflicting with other requests?