Stack Exchange
Sharing saved model of tensorflow among child processes
I have built a predictor (Python)class that uses tensorflow to predict classes
class IndustryPredictor:
    def __init__(self):
        self.predictor = self.load_predictor()

    def load_predictor(self):
        import tensorflow as tf
        confi_obj = ConfigParser()
        classifier_dir = confi_obj.classifier_directory
        predictor_model_dir_name = confi_obj.predictor_directory
        model_path = os.path.join(classifier_dir, predictor_model_dir_name)
        return tf.contrib.predictor.from_saved_model(model_path)

    def _create_float(v):
        return tf.train.Feature(float_list=tf.train.FloatList(value=[v]))

    def _create_str(v):
        return tf.train.Feature(bytes_list=tf.train.BytesList(value=[bytes(v, 'utf-8')]))

    def predict(description):
        doc_text = preprocess(description)

        text = _create_str(doc_text)
        dlen = _create_float(len(doc_text.split()))

        predicate = {'clean_text': text, 'len': dlen}

        example = tf.train.Example(features=tf.train.Features(feature=predicate))
        inputs = example.SerializeToString()

        preds = self.predictor({"inputs": [inputs]})
        return preds

This runs perfectly well on a single process environment. I am trying to use multiprocessing module to speed up my processing. I can create this object in the child processes and its running fine, but since my model is itself 1GB in size, I can only launch child processes up to a certain limit. 
What I was thinking is to load the saved model in the parent process and somehow pass it to child process this way I have to load the model only once. I tried doing that but the process hangs.
def main():
    workers = 8
    predictor = load_predictor()
    pool = Pool(processes=workers)
    for i in range(0, workers):
        pool.apply_async(consume, args=(predictor,), error_callback=handle_error)

    # Stay alive
    try:
        while True:
            continue
    except KeyboardInterrupt:
        logger.error(' [*] Exiting...')
        pool.terminate()
        pool.join()

Is there a way to share the tf.contrib.predictor object of tensorflow in child processes. Would writing a keras wrapper on this predictor help me solve this issue.

Were you able to solve?

Multiprocessing forks your process which is why you have a copy. Having shared memory space is possible via e.g. joblib. In essence, your object becomes a memory-mapped object on a disk. Great solution if you have a decent SSD (or better).
Manager from multiprocessing could be an option, but I'd not bet on it.
