Hugging Face's logo

parasail-ai
/
aya-23-8B-GPTQ-4bit

Text Generation
Transformers
Safetensors
cohere
conversational
text-generation-inference
Inference Endpoints
4-bit precision
gptq
Model card Files Community
Edit model card

Model Card for parasail-ai/aya-23-8B-GPTQ-4-bit

GPTQ quantized 4-bit version of CohereForAI/aya-23-8B, calibrated using aya_dataset. See original model card for more information.

Downloads last month
15
Safetensors
Model size
1.98B params
Tensor type
I32
·
FP16
·
Inference Examples
Text Generation
Inference API (serverless) is not available, repository is disabled.

Collection including parasail-ai/aya-23-8B-GPTQ-4bit