Add README

README.md

@@ -0,0 +1,43 @@
+---
+base_model: meta-llama/Meta-Llama-3.1-70B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+This LoRA adapter was extracted from
+[mlabonne/Meta-Llama-3.1-70B-Instruct-lorablated](https://huggingface.co/mlabonne/Meta-Llama-3.1-70B-Instruct-lorablated)
+and uses
+[meta-llama/Llama-3.1-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-70B-Instruct)
+as a base.
+
+## Model Details
+
+The model was extracted by running
+[mlabonne/harmful_behaviors](https://huggingface.co/datasets/mlabonne/harmful_behaviors)
+and the user prompts (but not assistant responses or system messages) from
+[Guilherme34/uncensor](https://huggingface.co/datasets/Guilherme34/uncensor)
+through the original abliterated model to generate a dataset of
+prompt/completion pairs, and was trained for 2 epochs on a 8xA100s with Axolotl
+using FSDP. Since the original abliterated model isn't perfect at avoiding
+refusals, the dataset was cleaned to remove the few refusals generated prior to
+training.
+
+### Model Description
+
+- **Developed by:** @reissbaker
+- **Funded by:** Synthetic Lab
+- **License:** Apache 2.0
+- **Finetuned from model:** Llama 3.1 70B Instruct
+
+## How to Get Started with the Model
+
+Run the model with one click on [glhf.chat](https://glhf.chat).
+
+
+#### Training Hyperparameters
+
+* BF16 mixed-precision
+* 4e-4 LR
+* Linear LR schedule
+* Fused AdamW optimizer