I work at Xilinx as well. We released Graffitist 8 earlier this year, a TensorFlow package for quantization-aware training. I’d like to point out that there is a critical issue in the way FakeQuant’s gradients (wrt min/max vars) are implemented in TensorFlow. We shed more light on it in our paper 10, but it has to do with the correct straight-through-estimator (STE) for training thresholds or scale-factors. It’d be nice to address this in PyTorch early on (I’m happy to be of help and can contribute as well). This is essential for quantizing traditionally difficult networks such as mobilenets, with almost no loss in accuracy (refer Table 4 in paper 10).
I agree with Paolo, in that having control over the JIT IR pass is essential for target-specific transforms on the graph. I did go over the documentation (JIT README), but some things are not clear to me yet. For instance, how to (i) pattern-match and manipulate sub-graphs, (ii) insert custom ops, (iii) invoke custom passes on the IR.
Please let us know
(i) if you can visit Xilinx (San Jose) for a JIT deep-dive, and
(ii) how we can be of help/contribute.