Models from inference-optimization org containing quantization bit info
-
inference-optimization/Meta-Llama-3.1-8B-Instruct-NVFP4-FP8-Dynamic_5.0-bits
5B • Updated • 1 -
inference-optimization/Meta-Llama-3.1-8B-Instruct-NVFP4-FP8-Dynamic_5.25-bits
6B • Updated • 3 -
inference-optimization/Meta-Llama-3.1-8B-Instruct-NVFP4-FP8-Dynamic_5.5-bits
6B • Updated -
inference-optimization/Meta-Llama-3.1-8B-Instruct-NVFP4-FP8-Dynamic_5.75-bits
6B • Updated