Hugging Face's logo

comaniac
/
Meta-Llama-3-70B-Instruct-FP8-v2

Text Generation
Transformers
Safetensors
llama
conversational
text-generation-inference
Inference Endpoints
fp8
Model card Files Community
YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co/docs/hub/model-cards#model-card-metadata)

Llama-3-70B-Instruct-FP8-v2

  • Weights and activations are per-tensor quantized to float8_e4m3.
  • Quantization with AutoFP8 with the updated activation scaling factor names.
  • Calibration dataset: Ultrachat (mgoin/ultrachat_2k)
  • Samples: 1024
  • Sequence length: 4096

Evaluation

TBA

Downloads last month
7
Safetensors
Model size
70.6B params
Tensor type
BF16
·
F8_E4M3
·
Inference Providers NEW
Text Generation
This model is not currently available via any of the supported third-party Inference Providers, and the model is not deployed on the HF Inference API.