@supriyar added quint4x2 datatype to quantization, some tests can be found in pytorch/test_quantize_fx.py at master · pytorch/pytorch · GitHub 7, but we do have have int4 datatype support for any ops except embedding_bag.
We are working on a design doc to support custom backends currently and I think it will be useful to support this as well, please keep an eye out for design docs in github issues, I may update here if I still remember this post.