text: Title: [BUG] Right Enviroment for custom Qwen2-VL quantization using AutoGPTQ
  Body: Hi,
  Since last couple of weeks I am struggling to quantize my custom Qwen2-VL model using GTPQ. 
  There is a lot of confusion regarding the correct version of CUDA, PyTorch, Auto-GPTQ, transformers and tokenizers required to successfully quantize the mode. 
  If anyone can help me out for the same, that would be great. 
  For now my environment is:
  CUDA : 12.1
  Python : 3.12
  Pytorch : 2.4 
  auto_gptq : 0.5.0 (also tried 0.6.0 and 0.7.0 but not working)
  transformers : 4.46.3
  tokenizers : 0.20.3
  
  My quantization code : 
  ```
  from transformers import AutoTokenizer, TextGenerationPipeline
  from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
  import logging
  import ast
  
  logging.basicConfig(
      format="%(asctime)s %(levelname)s [%(name)s] %(message)s", level=logging.INFO, datefmt="%Y-%m-%d %H:%M:%S"
  )
  
  pretrained_model_dir = "/home/bhavya/Desktop/bhavya/llm/LLaMA-Factory/models/qwen2_vl_lora_sft"
  quantized_model_dir = "/home/bhavya/Desktop/bhavya/llm/LLaMA-Factory/models/qwen2_vl_7b_4bit_gptq"
  
  tokenizer = AutoTokenizer.from_pretrained(pretrained_model_dir, use_fast=True)
  
  # opening the file in read mode 
  my_file = open("/home/bhavya/Desktop/bhavya/llm/quantize_code/dataset_caliber.txt", "r") 
    
  # reading the file 
  data = my_file.read() 
    
  data_into_list = data.split("\n")
  datasetlist = data_into_list[:-1] 
  # printing the data 
  print(len(datasetlist))
  print(type(datasetlist[0]))
  
  dataset = []
  for x in datasetlist:
      print('x')
      print(x)
      x1 = ast.literal_eval(x)
      dataset.append(x1)
  
  print('dataset')
  print(dataset[0])
  print(type(dataset[0]))
  
  quantize_config = BaseQuantizeConfig(
      bits=4,  # quantize model to 4-bit
      group_size=128,  # it is recommended to set the value to 128
      desc_act=False,  # set to False can significantly speed up inference but the perplexity may slightly bad
  )
  
  # load un-quantized model, by default, the model will always be loaded into CPU memory
  model = AutoGPTQForCausalLM.from_pretrained(pretrained_model_dir, quantize_config)
  
  # quantize model, the examples should be list of dict whose keys can only be "input_ids" and "attention_mask"
  model.quantize(dataset)
  
  # save quantized model
  # model.save_quantized(quantized_model_dir)
  
  # save quantized model using safetensors
  model.save_quantized(quantized_model_dir, use_safetensors=True)
  
  # push quantized model to Hugging Face Hub.
  # to use use_auth_token=True, Login first via huggingface-cli login.
  # or pass explcit token with: use_auth_token="hf_xxxxxxx"
  # (uncomment the following three lines to enable this feature)
  # repo_id = f"YourUserName/{quantized_model_dir}"
  # commit_message = f"AutoGPTQ model for {pretrained_model_dir}: {quantize_config.bits}bits, gr{quantize_config.group_size}, desc_act={quantize_config.desc_act}"
  # model.push_to_hub(repo_id, commit_message=commit_message, use_auth_token=True)
  
  # alternatively you can save and push at the same time
  # (uncomment the following three lines to enable this feature)
  # repo_id = f"YourUserName/{quantized_model_dir}"
  # commit_message = f"AutoGPTQ model for {pretrained_model_dir}: {quantize_config.bits}bits, gr{quantize_config.group_size}, desc_act={quantize_config.desc_act}"
  # model.push_to_hub(repo_id, save_dir=quantized_model_dir, use_safetensors=True, commit_message=commit_message, use_auth_token=True)
  
  # load quantized model to the first GPU
  # model = AutoGPTQForCausalLM.from_quantized(quantized_model_dir, device="cuda:0")
  
  # download quantized model from Hugging Face Hub and load to the first GPU
  # model = AutoGPTQForCausalLM.from_quantized(repo_id, device="cuda:0", use_safetensors=True, use_triton=False)
  
  # inference with model.generate
  # print(tokenizer.decode(model.generate(**tokenizer("auto_gptq is", return_tensors="pt").to(model.device))[0]))
  
  # or you can also use pipeline
  # pipeline = TextGenerationPipeline(model=model, tokenizer=tokenizer)
  # print(pipeline("auto-gptq is")[0]["generated_text"])
  ```
  
  Error:
  ```
  Traceback (most recent call last):
    File "/home/bhavya/Desktop/bhavya/llm/quantize_code/qwen2_quantize_gptq.py", line 45, in <module>
      model = AutoGPTQForCausalLM.from_pretrained(pretrained_model_dir, quantize_config)
    File "/home/bhavya/anaconda3/envs/autogptq-env/lib/python3.10/site-packages/auto_gptq/modeling/auto.py", line 75, in from_pretrained
      model_type = check_and_get_model_type(pretrained_model_name_or_path, trust_remote_code)
    File "/home/bhavya/anaconda3/envs/autogptq-env/lib/python3.10/site-packages/auto_gptq/modeling/_utils.py", line 305, in check_and_get_model_type
      raise TypeError(f"{config.model_type} isn't supported yet.")
  TypeError: qwen2_vl isn't supported yet.
  ```
  
  Please help me out.