Memory requirement for 13B 4Bit

by afoam - opened Apr 15, 2023

afoam

Apr 15, 2023

I'm new to this, how much RAM does the 13b 4 bit quantized model require? Will it run on 32 GB? Thank you

Owner Apr 15, 2023

~11GB, but some headroom is nice. A 16GB system runs the 13B model perfectly.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment