TheBloke
/

Mixtral-8x7B-Instruct-v0.1-AWQ

Text Generation

text-generation-inference

4-bit precision

Model card Files Files and versions Community

Resources

View closed (0)

Use ybelkada/Mixtral-8x7B-Instruct-v0.1-AWQ with VLLM instead

#10 opened about 1 year ago by

Inference taking too much time

#9 opened over 1 year ago by

Update README.md

#8 opened over 1 year ago by

RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

#7 opened over 1 year ago by

TGI - response is an empty string

#6 opened over 1 year ago by

OC is not a multiple of cta_N = 64

#5 opened over 1 year ago by

lazyDataScientist

Not supporting with TGI

#4 opened over 1 year ago by

abhishek3jangid

always getting 0 in output

#3 opened over 1 year ago by

OOM under vLLM even with 80GB GPU

#2 opened over 1 year ago by

Not supported for TGI > 1.3 ?

#1 opened over 1 year ago by