hi @naveen_raj , we currently do not have a quantized kernel for aten::mm.  There are a couple of options here:

you could leave this op in fp32 (this is going to be the fastest in terms of dev time)
if someone is up for helping write the quantized kernel for aten::mm, we would accept a PR