MrOCW:

Could not run ‘quantized::conv2d.new’ with arguments from the ‘QuantizedCUDA’ backend … … ‘quantized::conv2d.new’ is only available for these backends: [QuantizedCPU, …].


this error occurs when you try to run a quantized op with weight or input on cuda. For example if you were to take a correctly quantized model and then do .to(‘cuda’) and then run the model, you’d get this error.
based on the second error message, your weight is on cuda. Note: changing where x.to(‘cpu’) is located will not fix this problem if the actual op weight is on cuda.
my guess is that somewhere in your code you have model.to(‘cuda’) (likely during training) and you are not converting it back to cpu i.e. model.to(‘cpu’) before trying to do quantization.
addtionally, it looks like your self.act op is aten::silu which isn’t being converted to a quantized op (looks like it doesn’t have a quantized implementation pytorch/activation.py at master · pytorch/pytorch · GitHub 2). You can either implement it yourself or change to something along the lines of
y = sigmoid(x)
x = y * x
I would also maybe start with a less weird model and make sure the flow works for you before iterating on that. Something like: (beta) Static Quantization with Eager Mode in PyTorch — PyTorch Tutorials 1.9.1+cu102 documentation 6 could be a good starting point.