Update README.md
Browse files
README.md
CHANGED
@@ -367,7 +367,7 @@ Can I ask a question?<|im_end|>
|
|
367 |
</div>
|
368 |
<div class="section">
|
369 |
<h2>Training</h2>
|
370 |
-
<p>As the the Austral/Francois tradition, I built off another great finetune
|
371 |
<div class="metrics-section">
|
372 |
<details>
|
373 |
<summary>Config(Post-KTO SFT)</summary>
|
|
|
367 |
</div>
|
368 |
<div class="section">
|
369 |
<h2>Training</h2>
|
370 |
+
<p>As the the Austral/Francois tradition, I built off another great finetune Codex-24B, I did 4 epochs ontop with roughly the same datamix as Francois-Huali/Austral 70B as a R128 Lora, then KTO alignment with a mix of Instruct/Small writing datasets and then finally another 4 epoch SFT with Rep_remover (Thanks Pocket!)</p>
|
371 |
<div class="metrics-section">
|
372 |
<details>
|
373 |
<summary>Config(Post-KTO SFT)</summary>
|