I just post my second question here because as you mentioned, the inference time is not only depends on number of parameters, and one of the main factor has effect on inference time is number of operation.
I did count the number of parameters in the model by the following script.
def count_parameters(model):
        return sum(p.numel() for p in model.parameters() if p.requires_grad)
    
    print("number of parameters in the model is: ", count_parameters(model))

@ptrblck I was wondering how can I count the number of operations in the model?
Is there any recommended way to do this?
Thanks