Hi,
I need to unload model which loaded by torch::jit::load() with C++. But I couldn’t find anything about it just like ‘torch::jit::unload()’.

In my C++ code, there are two functions and provided api using pybind11:

#include <iostream>
#include <vector>
#include <string>

#include <torch/torch.h>
#include <torch/script.h>

#include <opencv2/core.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc.hpp>
...
torch::jit::script::Module initialize(const std::string &fpath) {
    torch::jit::script::Module model = torch::jit::load(fpath, "cuda");
    model.to("cuda:0");
    return model;
}


std::tuple<at::Tensor, at::Tensor> do_inference(torch::jit::scriot::Module model, ...) {
    ...  // pre-processing
    at::Tensor out = model.forward(input).toTensor();
    ...  // post processing
}

PYBIND11_MODULE(...) {
    m.def("initialize", &initialize, ...)
    ...
    m.def("do_inference",  &do_inference, ...)
}


Build it to foo.so, and import this module in Python3.6, python is a web service, just like:

import foo
from flask import Flask  # a web service
...

model = foo.initialize("./resnet50.pt")

img = open("./kitty.jpg", "rb")
foo.doInference(img)

# This service is running


The Web service is working fine, but how to unload model from device(GPU or CPU), If I want to update the model without stopping the service.
Is there a solution for that out there?
Thanks!