I think fusing conv, conv, relu isn’t supported yet. The only layers you can currently fuse are:
    conv, bn
    conv, bn, relu
    conv, relu
    linear, relu
    bn, relu

From: Fuse Modules Recipe — PyTorch Tutorials 1.10.0+cu102 documentation 2
Maybe try fusing just the last conv and relu?