text: Title: torch_dtype is actually used now?
  Body: ### System Info
  
  different transformers versions. see description
  
  ### Who can help?
  
  @ArthurZucker 
  
  ### Information
  
  - [ ] The official example scripts
  - [ ] My own modified scripts
  
  ### Tasks
  
  - [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
  - [ ] My own task or dataset (give details below)
  
  ### Reproduction
  
  Previously (v4.46.3, didn't check all versions), `torch_dtype` in the config was ignored, meaning that model weights would get loaded in fp32 by default (correct behavior for training). On latest transformers version (v4.49.0), it seems it is now used, and so the weights get loaded with whatever is in the checkpoint. Was this change intentional? I previously recall seeing somewhere in the code that you weren't going to make the change to actually use torch_dtype until v5, and I didn't see anything in release notes at a glance, although maybe I missed it.
  
  ```
  In [1]: import transformers
  
  In [2]: llama1bcfg = transformers.AutoConfig.from_pretrained('meta-llama/Llama-3.2-1B-Instruct')
  
  In [3]: llama1b = transformers.AutoModelForCausalLM.from_config(llama1bcfg)
  
  In [4]: next(llama1b.parameters()).dtype
  Out[4]: torch.bfloat16
  ```
  
  ### Expected behavior
  
  Not actually sure, would like to confirm what you expect now.