Update README.md
Browse files
README.md
CHANGED
@@ -53,17 +53,14 @@ Addressing these multifaceted issues necessitates a refined and sophisticated ap
|
|
53 |
|
54 |
2. **Fine Tuning**:
|
55 |
Our fine-tuning process primarily leveraged the Low-Rank Adaptation (LoRA) technique, incorporating specific hyperparameters and strategies to ensure optimal model performance.
|
56 |
-
Masking And Function Shuffling Technique was used during the training process;
|
57 |
-
|
58 |
-
|
59 |
-
|
60 |
-
- **
|
61 |
-
- **
|
62 |
-
- **
|
63 |
-
- **
|
64 |
-
- **Batch Size**: 4
|
65 |
-
- **Gradient Accumulation Steps**: 2
|
66 |
-
- **Hardware**: 4x A100 (80G) GPUs
|
67 |
|
68 |
|
69 |
3. **Inference**:
|
|
|
53 |
|
54 |
2. **Fine Tuning**:
|
55 |
Our fine-tuning process primarily leveraged the Low-Rank Adaptation (LoRA) technique, incorporating specific hyperparameters and strategies to ensure optimal model performance.
|
56 |
+
Masking And Function Shuffling Technique was used during the training process;The Training Setup is as follows:
|
57 |
+
- **LoRA Rank**: 32
|
58 |
+
- **Learning Rate**: 5e-5
|
59 |
+
- **Warmup Steps**: 100
|
60 |
+
- **LR Scheduler Type**: Cosine
|
61 |
+
- **Batch Size**: 4
|
62 |
+
- **Gradient Accumulation Steps**: 2
|
63 |
+
- **Hardware**: 4x A100 (80G) GPUs
|
|
|
|
|
|
|
64 |
|
65 |
|
66 |
3. **Inference**:
|