Nicemleme, Quantization Experiment, Bitsandbytesconfig, Model Quantization, Üretken Yapay Zeka.
accelerate==0.28.0 bitsandbytes==0.43.0 transformers==4.39.0 huggingface-hub==0.21.4 python-multipart==0.0.7 torch==2.2.1 Imports: import logging import torch import os from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig Modelin yüklenme kısmı 32multi Bu yapıda gpu