rAIfle
/

experiment_1_8b-fp16

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

rAIfle commited on Apr 29, 2024

Commit

f75f9ff

·

verified ·

1 Parent(s): 6565638

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -24,8 +24,8 @@ trainer = SFTTrainer(
     dataset_num_proc = 2,
     packing = False, # Can make training 5x faster for short sequences.
     args = TrainingArguments(
-        per_device_train_batch_size = 2,
-        gradient_accumulation_steps = 4,
         warmup_steps = 5,
         num_train_epochs=1,
         learning_rate = 2e-4,

     dataset_num_proc = 2,
     packing = False, # Can make training 5x faster for short sequences.
     args = TrainingArguments(
+        per_device_train_batch_size = 1,
+        gradient_accumulation_steps = 8,
         warmup_steps = 5,
         num_train_epochs=1,
         learning_rate = 2e-4,