LOL ok I got my question completely wrong. The Torchserve issue was completely different since I’d screwed up the ModelHandler  messing up the preprocess method
The quantization is different, but I still haven’t figured it out yet.  Here’s the model.
I’m not sure how I can quantize this, compared to this sweet simple tutorial.

  
      github.com
  
  
    VishakBharadwaj94/quant_wip/blob/master/quant.ipynb 1
{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "quant.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "code",
      "metadata": {
        "id": "Eay3SI8fa6P9"
      },


  This file has been truncated. show original