Upload 12 files

README.md

@@ -1,3 +1,61 @@
----
-license: apache-2.0
----
+---
+base_model: OuteAI/Lite-Mistral-150M-v2-Instruct
+tags:
+- SPPO
+- alignment-handbook
+- generated_from_trainer
+datasets:
+- UCLA-AGI/data-mistral-7b-instruct-sppo-iter1
+- christopherthompson81/sppo-synthetic-dataset-lite-mistral-150m-v2
+model-index:
+- name: Lite-Mistral-150M-v2-Instruct-SPPO-Iter3
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# Lite-Mistral-150M-v2-Instruct-SPPO-Iter3
+
+This model is iteration 3 of an SPPO process applied to [OuteAI/Lite-Mistral-150M-v2-Instruct](https://huggingface.co/OuteAI/Lite-Mistral-150M-v2-Instruct) by generating synthetic datasets from the prompts of [UCLA-AGI/data-mistral-7b-instruct-sppo-iter1](https://huggingface.co/UCLA-AGI/data-mistral-7b-instruct-sppo-iter1).
+
+One of the notable learnings I had was that the prompts used for an SPPO process should only be slightly beyond the capabilities of the base model. If they're too difficult, none of the synthetic outputs will be meaningfully "better" for the autoranker to prefer over the others. Additionally, the autoranker only needs to be good enough to evaluate the target prompts, but will need to be adjudicated for the prompts to know if it is satisfactory.
+
+## Model description
+
+I made this model to practice with the SPPO method. The model selection was based on it being small and therefore fast, as well as being coherent.
+
+## Intended uses & limitations
+
+150M models are generally meant for a data scientist or technological enthusiast to familiarize themselves with a topic. This model is likely best used as a means of gaining an understanding of the results that SPPO can deliver by contrasting outputs between the base model and prior iterations of the SPPO process. Viable output is just a bonus.
+
+## Training procedure
+
+I'm still working on putting my code together in a releasable way. The code will eventually be accessible here:
+* [GitHub - SPPO Generator](https://github.com/christopherthompson81/sppo_generator)
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-07
+- train_batch_size: 1
+- eval_batch_size: 4
+- seed: 42
+- distributed_type: multi-GPU
+- gradient_accumulation_steps: 2
+- total_train_batch_size: 2
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_ratio: 0.1
+- num_epochs: 1.0
+
+### Training results
+
+Still to come. Have not run evals yet...
+
+### Framework versions
+
+- Transformers 4.44.0
+- Pytorch 2.4.0+cu121
+- Datasets 2.20.0
+- Tokenizers 0.19.1

all_results.json

@@ -0,0 +1,9 @@
+{
+    "epoch": 1.0,
+    "total_flos": 0.0,
+    "train_loss": 116322.4441848534,
+    "train_runtime": 4176.8003,
+    "train_samples": 19766,
+    "train_samples_per_second": 4.732,
+    "train_steps_per_second": 2.366
+}

config.json

@@ -0,0 +1,27 @@
+{
+  "_name_or_path": "checkpoints/Lite-Mistral-150M-v2-Instruct-SPPO-Iter2",
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "head_dim": 48,
+  "hidden_act": "silu",
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "max_position_embeddings": 2048,
+  "model_type": "mistral",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 12,
+  "num_key_value_heads": 8,
+  "rms_norm_eps": 1e-06,
+  "rope_theta": 10000.0,
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.44.0",
+  "use_cache": true,
+  "vocab_size": 32768
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "transformers_version": "4.44.0"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df99b621e32c26f67a568c4f18f6baf76942e3f244adddbc86fa1baf2d958270
+size 313050768

special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "add_prefix_space": true,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "chat_template": "{% for message in messages %}{{bos_token + message['role'] + '\n' + message['content'] + eos_token + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ bos_token + 'assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": true,
+  "model_max_length": 2048,
+  "pad_token": "</s>",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

train_results.json

@@ -0,0 +1,9 @@
+{
+    "epoch": 1.0,
+    "total_flos": 0.0,
+    "train_loss": 116322.4441848534,
+    "train_runtime": 4176.8003,
+    "train_samples": 19766,
+    "train_samples_per_second": 4.732,
+    "train_steps_per_second": 2.366
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:555140185aeb816de2e06e385717375fdf388869d1a3a2e8678f087daaf0a1b3
+size 6584