This is the correct answer. The stubs are properly replaced when I change
model.qconfig = torch.quantization.get_default_qconfig(backend)

to
quantized_model.qconfig = torch.quantization.get_default_qconfig(backend)

Thanks!