Thank you so much for your reply. I was able to use your fix to get my code up and running, but I realized that this method of statically quantizing my model means that all the computations will be run on the CPU. I read online that Quantization Aware Training allows for me to still train on the GPU, so I was hoping to try and use that instead.
I quantized my model using the following code:
class faster_countor_NN(Countor_NN):
    def __init__(self):
        super(faster_countor_NN, self).__init__()
        self.conv = torch.nn.Conv2d(1, 1, 1)
        self.relu = torch.nn.ReLU()

    def forward(self, images, boxes, boxes_ids, ROI_images):
        det_boxes, det_scores, det_labels, tck_boxes, tck_scores, tck_labels, tck_ids = super().forward(images, boxes, boxes_ids, ROI_images)
        return det_boxes, det_scores, det_labels, tck_boxes, tck_scores, tck_labels, tck_ids

class QuantizedRCNN(torch.nn.Module):
    def __init__(self, model_fp32):
        super(QuantizedRCNN, self).__init__()
        self.model_fp32 = model_fp32

    def forward(self, images, boxes, boxes_ids, ROI_images):
        det_boxes, det_scores, det_labels, tck_boxes, tck_scores, tck_labels, tck_ids = self.model_fp32(images, boxes, boxes_ids, ROI_images)
        return det_boxes, det_scores, det_labels, tck_boxes, tck_scores, tck_labels, tck_ids

init_net = faster_countor_NN()
init_net.load_state_dict(torch.load(paths['path_faster_RCNN_weigths'],map_location="cuda"), strict=False)
init_net.cpu()
init_net_fused = copy.deepcopy(init_net)
init_net.train()
init_net_fused.train()
init_net = torch.quantization.fuse_modules(init_net,[['conv', 'relu']])
init_net.eval()
init_net_fused.eval()
countor_net = QuantizedRCNN(model_fp32=init_net_fused)
quantization_config = torch.quantization.get_default_qconfig("fbgemm")
countor_net.qconfig = quantization_config
torch.quantization.prepare_qat(countor_net, inplace=True)
countor_net.train()
countor_net.to('cpu')
countor_net = torch.quantization.convert(countor_net, inplace=True)
countor_net.cuda()
countor_net.eval()

However, I am still getting the following error in my code.
    File "/home/maria/anaconda3/envs/engie/lib/python3.8/site-packages/torch/nn/quantized/modules/conv.py", line 331, in forward
        return ops.quantized.conv2d(

RuntimeError: Could not run 'quantized::conv2d.new' with arguments from the 'CUDA' backend. 'quantized::conv2d.new' is only available for these backends: [QuantizedCPU, BackendSelect, Named, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, Tracer, Autocast, Batched, VmapMode].

CPU: registered at /opt/conda/conda-bld/pytorch_1607370172916/work/build/aten/src/ATen/CPUType.cpp:2127 [kernel]
BackendSelect: fallthrough registered at /opt/conda/conda-bld/pytorch_1607370172916/work/aten/src/ATen/core/BackendSelectFallbackKernel.cpp:3 [backend fallback]
Named: registered at /opt/conda/conda-bld/pytorch_1607370172916/work/aten/src/ATen/core/NamedRegistrations.cpp:7 [backend fallback]
AutogradOther: registered at /opt/conda/conda-bld/pytorch_1607370172916/work/torch/csrc/autograd/generated/VariableType_2.cpp:8078 [autograd kernel]
AutogradCPU: registered at /opt/conda/conda-bld/pytorch_1607370172916/work/torch/csrc/autograd/generated/VariableType_2.cpp:8078 [autograd kernel]
AutogradCUDA: registered at /opt/conda/conda-bld/pytorch_1607370172916/work/torch/csrc/autograd/generated/VariableType_2.cpp:8078 [autograd kernel] 
AutogradXLA: registered at /opt/conda/conda-bld/pytorch_1607370172916/work/torch/csrc/autograd/generated/VariableType_2.cpp:8078 [autograd kernel]
AutogradPrivateUse1: registered at /opt/conda/conda-bld/pytorch_1607370172916/work/torch/csrc/autograd/generated/VariableType_2.cpp:8078 [autograd kernel]
AutogradPrivateUse2: registered at /opt/conda/conda-bld/pytorch_1607370172916/work/torch/csrc/autograd/generated/VariableType_2.cpp:8078 [autograd kernel]
AutogradPrivateUse3: registered at /opt/conda/conda-bld/pytorch_1607370172916/work/torch/csrc/autograd/generated/VariableType_2.cpp:8078 [autograd kernel]
Tracer: registered at /opt/conda/conda-bld/pytorch_1607370172916/work/torch/csrc/autograd/generated/TraceType_2.cpp:9654 [kernel]
Autocast: fallthrough registered at /opt/conda/conda-bld/pytorch_1607370172916/work/aten/src/ATen/autocast_mode.cpp:254 [backend fallback]
Batched: registered at /opt/conda/conda-bld/pytorch_1607370172916/work/aten/src/ATen/BatchingRegistrations.cpp:511 [backend fallback]
VmapMode: fallthrough registered at /opt/conda/conda-bld/pytorch_1607370172916/work/aten/src/ATen/VmapModeRegistrations.cpp:33 [backend fallback]

Even when I try to resolve this issue by sending my images back to the CPU (which I don’t want), I am getting a segmentation fault since my model is on the GPU. Do you know any way that I can resolve this issue to successfully run my model on the GPU? Is this possible, or am I not understanding the Quantizd Aware Training correctly?