transformers pytorch llama text-generation autotrain_compatible text-generation-inference endpoints_compatible region:us
---
{}
---
## Hugging Face Model Card

**Model Name:** LLaMA-based Model with LoRA and QLoRA Quantization

**Model Description:** This model is a fine-tuned version of the LLaMA language model with optimizations including LoRA (Low-Rank Adaptation) and QLoRA (Quantized LoRA). LoRA allows adaptation to new tasks with minimal parameters, enhancing efficiency and reducing memory requirements. QLoRA further optimizes this by quantizing weights to lower precision formats, facilitating faster training and inference.

**Intended Uses:** This model is suitable for various NLP tasks such as text generation, question answering, summarization, and machine translation.

**Potential Limitations:**

- Bias: The model may reflect biases present in the training data.
- Performance Variability: Model performance might vary based on specific tasks and datasets.
- Quantization Effects: Quantization may introduce slight accuracy loss compared to full-precision models.

**Training Details:**

- **Dataset:** The model was trained on the dataset "mlabonne/guanaco-llama2-1k".
- **Base Model:** The base model used is "NousResearch/Llama-2-7b-chat-hf".
- **LoRA Configuration:**
  - LoRA attention dimension (lora_r): 64
  - Alpha parameter for LoRA scaling (lora_alpha): 16
  - Dropout probability for LoRA layers (lora_dropout): 0.1
- **QLoRA Configuration:**
  - Quantization Type: nf4
  - Compute Dtype: float16
- **Training Arguments:**
  - Number of Training Epochs: 1
  - Batch Size per GPU for Training: 4
  - Batch Size per GPU for Evaluation: 4
  - Gradient Accumulation Steps: 1
  - Gradient Checkpointing: Enabled
  - Maximum Gradient Normal: 0.3
  - Initial Learning Rate: 2e-4
  - Weight Decay: 0.001
  - Optimizer: paged_adamw_32bit
  - Learning Rate Scheduler: cosine
  - Group Sequences into Batches with Same Length: Enabled
  - Logging Steps: 25
- **SFT Parameters:**
  - Maximum Sequence Length: None
  - Packing: Disabled

**Evaluation Results:** 

- Evaluation metrics on a held-out test set specific to the intended use case are recommended but not provided.

**Ethical Considerations:**

- Bias Mitigation: Be mindful of and mitigate biases present in the training data.
- Responsible Usage: Use the model responsibly and avoid generating harmful or offensive content.

**Authors:** Walter Vijayakumar

**License:** Apache 2.0

