transformers safetensors llama text-generation autotrain_compatible text-generation-inference endpoints_compatible region:us
---
library_name: transformers
tags: []
---

# Model Card for Meta-Llama-3-8B

Meta-Llama-3-8B is an advanced language model developed by Meta, part of the Llama 3 family, optimized for text generation and natural language understanding tasks.
This model leverages transformer architecture and is available in pre-trained and instruction-tuned variants.

## Model Details

### Model Description

This is the model card for the Meta-Llama-3-8B, a part of the Llama 3 model family which includes models with 8 billion and 70 billion parameters.
The model is pre-trained on a diverse dataset of publicly available text and is designed for both research and commercial use, particularly for applications requiring natural language understanding and generation.

- **Developed by:** Meta
- **Model type:** Auto-regressive language model
- **Language(s) (NLP):** English
- **License:** Meta's custom commercial license
- **Finetuned from model:** Not applicable

### Model Sources [optional]

- **Repository:** [Meta-Llama-3-8B on Hugging Face](https://huggingface.co/meta-llama/Meta-Llama-3-8B)
- **Paper [optional]:** [Meta AI Blog](https://ai.meta.com/blog/meta-llama-3/)
- **Demo [optional]:** [Hugging Face Chat Demo](https://huggingface.co/chat/models/meta-llama/Meta-Llama-3-70B-instruct)

## Uses

### Direct Use

The model can be used directly for text generation tasks, including but not limited to chatbots, content creation, and interactive applications.

### Downstream Use [optional]

When fine-tuned, the model can be adapted for specific tasks such as sentiment analysis, question answering, and more.

### Out-of-Scope Use

The model should not be used for generating harmful content, spam, impersonation, or any other unethical activities as outlined in Meta's usage policy.

## Bias, Risks, and Limitations

### Recommendations

Users should be aware of the model's limitations and biases. It is recommended to use the model responsibly and with caution, especially in sensitive applications.

## How to Get Started with the Model

```python
from transformers import pipeline
import torch

model_id = "meta-llama/Meta-Llama-3-8B"
pipeline = pipeline(
    "text-generation", model=model_id, model_kwargs={"torch_dtype": torch.bfloat16}, device_map="auto"
)
print(pipeline("Hello, how are you today?"))
```

## Training Details

### Training Data

The model was trained on a mix of publicly available online data, totaling over 15 trillion tokens.

### Training Procedure

The model utilizes Grouped-Query Attention (GQA) for improved inference scalability and was trained using supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF).

#### Training Hyperparameters

- **Training regime:** Mixed precision (bfloat16)

## Evaluation

### Testing Data, Factors & Metrics

#### Testing Data

Testing was performed using standard natural language processing benchmarks.

#### Metrics

Metrics include standard NLP benchmarks for language models, such as perplexity, accuracy, and other task-specific measures.

### Results

Llama 3 models have demonstrated strong performance on various benchmarks, showing competitive results with other large-scale language models.

## Environmental Impact

Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute).

- **Hardware Type:** NVIDIA A100 GPUs
- **Hours used:** Not specified
- **Cloud Provider:** Not used
- **Compute Region:** Not specified
- **Carbon Emitted:** Not specified

## Technical Specifications

Llama 3 sharded model for an easier inference and fine tuning process on lower to mid end processing systems.

### Model Architecture and Objective

Llama 3 is an auto-regressive transformer-based language model designed for text generation and understanding.

### Compute Infrastructure

#### Hardware

Training utilized a cluster of NVIDIA A100 GPUs.

#### Software

The model is compatible with PyTorch and Hugging Face's transformers library.

## Citation

**BibTeX:**

```bibtex
@article{meta2024llama3,
  title={Meta Llama 3: An Open-Source Large Language Model},
  author={Meta AI},
  journal={Meta AI Blog},
  year={2024},
  url={https://ai.meta.com/blog/meta-llama-3/}
}
```

**APA:**

Meta AI. (2024). Meta Llama 3: An Open-Source Large Language Model. Meta AI Blog. Retrieved from https://ai.meta.com/blog/meta-llama-3/

## Glossary

- **Auto-regressive model:** A type of model that generates sequences by predicting the next element based on previous elements.
- **Transformer architecture:** A neural network architecture designed for handling sequential data, particularly for tasks in NLP.

## More Information

For more details about Meta, visit the [Meta Llama website](https://llama.meta.com).

## Model Card Authors

realshyfox

## Model Card Contact

For any questions or feedback, please contact the Meta AI team via their [GitHub repository](https://github.com/meta-llama/llama3).
