How can I evaluate the quality of generation of this model?

by tian9 - opened Apr 30

Discussion

tian9

Apr 30

Could you please tell me how do you make sure that the novel has a specific plot, a coherent sentence, no logical errors, and not just a simple zodiac sign? Is there some benchmark to eval the literariness of the generation?

Aratako

Owner Apr 30

@tian9
Thank you for your comment!
Unfortunately, there is currently no benchmark available. Additionally, since the model size is small at 7B, it does not always generate perfect sentences.
As a native Japanese speaker, I check the quality of the output with my own eyes.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment