Update README.md
Browse files
README.md
CHANGED
@@ -18,8 +18,8 @@ Stop sequence == "\*\*Finished.\*\*"
|
|
18 |
|
19 |
Related Links:
|
20 |
- [Augmentoolkit](https://github.com/e-p-armstrong/augmentoolkit)
|
21 |
-
- [Augmentoolkit Factual Demo Model (the products of the quickstart)](
|
22 |
-
- [gRPo model (no thoughts)](
|
23 |
|
24 |
Notes:
|
25 |
This attempt at getting emotional responses mostly succeeded — when the model writes well, it writes *well*. However an interesting quirk emerged -- the model ended up putting most of its emotional exposition in the thought process, rather than in the final actual visible response. Possibly because it was graded higher when it explained its emotions exhaustively, possibly just because models trained to think with SFT first will pad their responses with thoughts a lot. While I think this overall came out well, it also serves as an example of what to look out for because of this. I think the next version will, in addition to the LLM-as-a-reward function approach, include a function which rewards final answers above a certain length.
|
|
|
18 |
|
19 |
Related Links:
|
20 |
- [Augmentoolkit](https://github.com/e-p-armstrong/augmentoolkit)
|
21 |
+
- [Augmentoolkit Factual Demo Model (the products of the quickstart)](https://huggingface.co/Heralax/llama-Augmentoolkit-Quickstart-Factual-Demo)
|
22 |
+
- [gRPo model (no thoughts)](https://huggingface.co/Heralax/llama-gRPo-emotions-nothoughts)
|
23 |
|
24 |
Notes:
|
25 |
This attempt at getting emotional responses mostly succeeded — when the model writes well, it writes *well*. However an interesting quirk emerged -- the model ended up putting most of its emotional exposition in the thought process, rather than in the final actual visible response. Possibly because it was graded higher when it explained its emotions exhaustively, possibly just because models trained to think with SFT first will pad their responses with thoughts a lot. While I think this overall came out well, it also serves as an example of what to look out for because of this. I think the next version will, in addition to the LLM-as-a-reward function approach, include a function which rewards final answers above a certain length.
|