Switches from English prompt to Chinese response
Unfortunately, this model does not seem to consistently respond in the language that could be gleaned from the prompt. Example (using https://huggingface.co/TheBloke/Xwin-LM-70B-V0.1-GGUF/blob/main/xwin-lm-70b-v0.1.Q5_K_M.gguf executed via llama.cpp):
A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. USER: You are a video game narrator. Your role is to turn a boring log of game events into an interesting story. Don't lose any details--preserve all facts. However, if some events are repetitive, find ways to creatively portray the time duration. For example, if the log reads: "You walked north. You walked north. You walked north." then perhaps you could improve this by stating, "You wandered north for quite some time." Each event represents the passage of about 10 minutes of time.
You are in the city of Feldspar.
You walk north.
You walk north.
You walk north.
You open a door.
You walk north.
You talk to a shopkeeper.
You walk south.
You walk south.
ASSISTANT: 你现身于 Feldspar 城市。
勇敢地，你在这个未知的环境中迈出了第一步，朝着北方向行走。随着时间的推移，你一再地踏上前进的道路，仿佛被吸引着，不断地走向北方。
在这段旅程中，你意外地发现了一扇门。你好奇地推开了它，关注着她隐藏的秘密。
然后，你又回到了南方，脚步不定，仿佛在寻找什么。你与一位商人交流了一番，试图从他那里获取有关这个城市的信息。
最后，你又回到了北方，继续追寻着旅程的下一个目标。
I run Xwin-LM-70B-V0.1.Q3_K_S.gguf with the same prompt, got the following response:
generate: n_ctx = 1600, n_batch = 512, n_predict = -1, n_keep = 0
A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. USER: You are a video game narrator. Your role is to turn a boring log of game events into an interesting story. Don't lose any details--preserve all facts. However, if some events are repetitive, find ways to creatively portray the time duration. For example, if the log reads: "You walked north. You walked north. You walked north." then perhaps you could improve this by stating, "You wandered north for quite some time." Each event represents the passage of about 10 minutes of time. You are in the city of Feldspar. You walk north. You walk north. You walk north. You open a door. You walk north. You talk to a shopkeeper.You walk south. You walk south.
Here's how I would narrate that log:
In the bustling city of Feldspar, you embark on a journey, traversing the cobblestone streets with purpose. As you walk northward, you take in the sights and sounds of the vibrant urban landscape, the smells of various food vendors wafting through the air. You continue your trek for a while, covering quite some distance, before coming across an unassuming door. Intrigued, you open it to reveal a cozy shop brimming with goods and chatter.
You exchange greetings with the friendly shopkeeper, engaging in a lively conversation about the city's latest happenings. Afterwards, you continue your exploration, venturing southward this time, retracing your steps and observing the streets from a different perspective. The sun beats down on you as you walk, the warmth a constant reminder of the passage of time. [end of text]
Does the difference happen to be a higher value of temperture?
I got vietnamese output randomly after response. Something about land management around ho chi minh city. It is not every time.
I'm getting this happen as well, randomly. Sometimes it's English, other times it seems to provide the answer in what I presume is Vietnamese. I haven't found a workaround yet, if there is one. Other than that this model so far has proven to be the most accurate for my use case, with second place going to Synthia.
EDIT: Adding "Your answer must be in English" to the system prompt so far seems to be working for me, not thoroughly tested it though. Might currently just be random luck.
I'm getting a lot of Chinese too. Makes it unusable.
Try formatting your prompt in Alpaca style. I know the model card suggests otherwise but Alpaca seems to better constrain the model to the input language without noticeable impact to its instruction following.