text: Title: Torchao `int4_weight_only` save error when passing layout
  Body: ### System Info
  
  Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.
  
  - `transformers` version: 4.49.0.dev0
  - Platform: Linux-4.18.0-425.3.1.el8.x86_64-x86_64-with-glibc2.39
  - Python version: 3.12.3
  - Huggingface_hub version: 0.27.1
  - Safetensors version: 0.4.5
  - Accelerate version: 1.4.0.dev0
  - Accelerate config:    - compute_environment: LOCAL_MACHINE
          - distributed_type: MULTI_GPU
          - mixed_precision: bf16
          - use_cpu: False
          - debug: False
          - num_processes: 2
          - machine_rank: 0
          - num_machines: 1
          - gpu_ids: 5,6
          - rdzv_backend: static
          - same_network: True
          - main_training_function: main
          - enable_cpu_affinity: False
          - downcast_bf16: no
          - tpu_use_cluster: False
          - tpu_use_sudo: False
          - tpu_env: []
  - PyTorch version (GPU?): 2.6.0+cu126 (True)
  - Tensorflow version (GPU?): not installed (NA)
  - Flax version (CPU?/GPU?/TPU?): not installed (NA)
  - Jax version: not installed
  - JaxLib version: not installed
  - Using distributed or parallel set-up in script?: <fill in>
  - Using GPU in script?: <fill in>
  - GPU type: NVIDIA A100 80GB PCIe
  
  
  ### Who can help?
  
  @SunMarc 
  
  ### Information
  
  - [ ] The official example scripts
  - [ ] My own modified scripts
  
  ### Tasks
  
  - [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
  - [ ] My own task or dataset (give details below)
  
  ### Reproduction
  
  More details refer to [torchao](https://github.com/pytorch/ao/issues/1704)
  
  
  ### Expected behavior
  
  Hi @SunMarc . Do you think we can handle this in transformers?