The fusing is based on the layer names. So in your case that would be encoder_1', ‘encoder_2’ etc. However, I don’t think PyTorch supports fusing those layers. And I’m not sure you can even fuse the conv2d and bn layers since they are actually part of PartialConvLayer.
So if you really wanted to fuse then you may have to write that code yourself.