System Prompt to improve responses in German language

#6
by ThiloteE - opened

image.png

Test 1:
### System:\nDu sprichst grammatikalisch korrektes Deutsch auf höchstem Muttersprachler Niveau.\n### User:\n{user}\n\n### Assistant:\n{assistant}
image.png
image.png

Test 2:
### User:\n{user}\n\n### Assistant:\n{assistant}
image.png

Test 3:
Du sprichst grammatikalisch korrektes Deutsch auf höchstem Muttersprachler Niveau.\n### User:\n{user}\n\n### Assistant:\n{assistant}
image.png

Test 4:
### System:\nDu schlüpfst in die Rolle eines extrem intelligenten Muttersprachlers der deutschen Sprache. Du sprichst grammatikalisch korrektes Deutsch auf höchstem Niveau.\n### User:\n{user}\n\n### Assistant:\n{assistant}
image.png

In conclusion:

  1. I would suggest to use the prompt from test 1: ### System:\nDu sprichst grammatikalisch korrektes Deutsch auf höchstem Muttersprachler Niveau.\n### User:\n{user}\n### Assistant:\n{assistant}
  2. Test 2 was very interesting, as it showed the model can start to respond in more or less proper German, if you just continue to give it German prompts.
  3. If you use a system prompt, there is a chance the model will refer to that in its response, thereby degrading output. Test 4 showed that by refering to "Die deutsche Sprache".
  4. Many of the responses i saw had tiny errors or weird quirks in them, so there is still room for improvement, but there is lots of potential here :-)

Thank you for the model :-)

Tagging you @DavidGF , because this might be of interest to you, since your Sauerkraut model is in there.

Owner

Hi @ThiloteE ,

Thanks for such a detailed report :)

So are you saying that I should include "System" as well in the prompt template?

I am not sure. Having and not having ###System: in the system prompt triggered the model to respond in German language (which is the most important requirement for a german language model), as long as there was a german system prompt. The output from test 3 is a lot more dumb though than output from test 1. Test 2 showed, that if there is no system prompt the model favored English over German. Unfortunately, I have not tested this model a lot yet in detail. The most important thing we can learn from these short examples are the importance of system prompts, when it comes to models that are able to converse in multiple languages, but even then, probably more testing should be done.

Very interesting results! Thanks for tagging! I haven't tried the Weyaxi model yet, but your test 2 looks really good! My experience with our model was that the model without the system prompt worked better in our test. At least it followed the instructions better.

Thank you for your tests. Its amazing to see how well the model performs!

Sign up or log in to comment