Solved by supriyar in post #7 
              
              
                int4 support is still in development, @supriyar has more context on that. 

We support torch.quint4x2 dtype, which packs two 4bit values into a byte. In order to use this dtype in operators we need kernels that understand this underlying type and can optimally operate on it. But if you wish to use …