hello, crane. How do you quantize the layer_norm module.  I try to use this code to convert the encoder-decoder transformer from fp32 to int8, but it seems the layer_norm is still fp32.

from torch.quantization import QuantStub, DeQuantStub, float_qparams_weight_only_qconfig, default_qconfig
eager mode
backend = “fbgemm”
#model.qconfig = torch.quantization.get_default_qconfig(backend)
model.qconfig = default_qconfig
model.encoder.embeddings.qconfig = float_qparams_weight_only_qconfig
model.decoder.embeddings.qconfig = float_qparams_weight_only_qconfig
model_fp32_prepared = torch.quantization.prepare(model)
model_int8 = torch.quantization.convert(model_fp32_prepared)

And when I try to inference the model, it raise error about Could not run 'quantized::layer_norm' with arguments from the 'CPU' backend.