TheBloke
/

Mixtral-8x7B-v0.1-GPTQ

Text Generation

text-generation-inference

4-bit precision

Model card Files Files and versions Community

Resources

View closed (1)

The generative output is strange

#11 opened 11 months ago by

Speeds compared to llama_cpp_python?

#10 opened over 1 year ago by

Unable to start TGI service for TheBloke/Mixtral-8x7B-v0.1-GPTQ with num_shard as 4

#9 opened over 1 year ago by

What would be the minimal Sagemaker instance to deploy this model ?

#7 opened over 1 year ago by

ValueError: Unsupported model type mixtral

#6 opened over 1 year ago by

RuntimeError: shape '[32, 8]' is invalid for input of size 0

#5 opened over 1 year ago by

Are you going to release mixtral-8x7B-v0.1-awq

#4 opened over 1 year ago by

Running the model using "pip install auto-gptq" still results in "CUDA extension not installed"

#3 opened over 1 year ago by

TypeError: mixtral isn't supported yet.

#2 opened over 1 year ago by

Build AutoGPTQ from source

#1 opened over 1 year ago by