int4 support is still in development, @supriyar has more context on that.

We support torch.quint4x2 dtype, which packs two 4bit values into a byte. In order to use this dtype in operators we need kernels that understand this underlying type and can optimally operate on it. But if you wish to use this dtype to save storage space, then it should be supported.
You can find the dtype in the nightly versions.