Thanks Karthik,
One more question,
If I would like to fuse input_conv + batch_norm + activation at encoder layers, and mask_conv + batch_norm at decoder layers, how can I distinguish those layers(encoder_1, encoder_2, …, decoder_1, decoder_2,…) when iterating the loop?
For example,
for m in model.modules():
        if m.methods_for_layers_name == "encoder_1":
            torch.quantization.fuse_modules(m, ["input_conv", "batch_normalization", "activation"], inplace=True)
        elif m.methods_for_layers_name == "decoder_1":
            torch.quantization.fuse_modules(m, ["mask_conv", "batch_normalization"], inplace=True)

Any idea?