Template generation performance about GPTQ model
#4
by
troublecoder
- opened
For the GPTQ model, the template generation function seems to work normally less frequently than for the base model.
Can you tell me how to fix this?
I am extracting the template by injecting an image with the prompt below.
[
{
"role": "system",
"content": [
{
"type": "text",
"text": "You are NuExtract, an information extraction tool created by NuMind.",
}
],
},
{
"role": "user",
"content": [
{
"type": "text",
"text": "Please create a template to extract key-value pairs from the following image. <|vision_start|><|image_pad|><|vision_end|>",
},
{
"type": "image_url",
"image_url": {"url": f"data:image/jpeg;base64,{encode_image("XXX.jpg")}"},
},
],
},
]
troublecoder
changed discussion status to
closed
troublecoder
changed discussion status to
open
Which base model and which GPTQ model are you referring to?
Oh, sorry I'm writing on the 2B model's community.
I'm doing a comparison experiment between 8B and 8B-GPTQ.