Hi @supriyar
I have done Quantizing my model, now I tried to save it with ‘jit.save(jit.script(model))’ but it pops the error as ‘‘aten::slice.t(t[] l, int start, int end=9223372036854775807, int step=1) -> (t[]): could not match type tensor to list[t] in argument ‘l’: cannot match list[t] to tensor.’’ very similar to this 2 more pops also arised. I googled about this error and in some discussions I found that it is the error regarding slicing(: , : , : ) operator says that ‘jit.script will not support scripting for directly used slicing operator’. Is it the error actually pointed to that? Because I too used slicing operator in the middle of my model and error throws at the same line.
This is one thing and now I go with another alternative to save quantized model i.e., with state_dict(). I’m able to save model with this but when I want to perform inference I have to initialize the model with the params which I have saved earlier with state_dict(). Now the params are quantized one but our model is defined for float. So error popped up as ‘exception occured : (‘copying from quantized tensor to non-quantized tensor is not allowed, please use dequantize to get a float tensor from a quantized tensor’’. This is the issue that I can’t save my model with jit and if I do so with state_dict() here I can’t initialize my model to go with inference.
Can you suggest any alternative?
Thanks @supriyar
Aravind.