FP8 KV Cache - a mohitsha Collection

mohitsha 's Collections

AMD ROCm AITER Kernels

AMD RyzenAI Models

FP8 KV Cache

updated Jul 4, 2024

Models with FP8 KV Cache Scales

mohitsha/Llama-2-70b-chat-hf-FP8-KV

Text Generation • Updated Jun 25, 2024 • 7
mohitsha/Llama-2-7b-chat-hf-FP8-KV

Text Generation • Updated Jun 25, 2024 • 8
mohitsha/Llama-2-7b-chat-hf-FP8-KV-AMMO

Text Generation • Updated Jun 25, 2024 • 7
mohitsha/Llama-2-70b-chat-hf-FP8-KV-AMMO

Text Generation • Updated Jun 25, 2024 • 11
mohitsha/Llama-2-7b-chat-hf-FP8-FNUZ-KV-AMMO

Text Generation • Updated Jun 25, 2024 • 6