The 1-bit might be incoherent.

i1-IQ2_XXS is the bare minimum i would recommend. it run so slow (less than half a token persec 0.3 to 0.45 t/s) on Snapdragon Elite Android but it is amazingly coherent and awesome and use about 7.1GB of ram using 512 context. Haven't tried on Raspberry Pi.

Output sample using i1-IQ2_XXS on Termux android:

User: good what di you think about solar system?

Assistant: The solar system is truly fascinating! It consists of the Sun, eight
planets (in order from closest to farthest: Mercury, Venus, Earth, Mars,
Jupiter, Saturn, Uranus, and Neptune), as well as various dwarf planets
like Pluto and Ceres. The system also includes a large number of moons,
asteroids, comets, and other small objects.

Some interesting facts about the solar system:

1. It's about 4.6 billion years old, formed from a collapsing cloud of
interstellar dust and gas.

2. The Sun accounts for 99% of the mass of the solar system.

3. The planets are divided into terrestrial (rocky) planets and gas giants.

4. Many moons in the solar system show signs of activity, like geysers or
ice volcanoes.

5. The Oort Cloud, a vast collection of icy objects, is the farthest part
of the solar system.

6. We continue to make new discoveries, like the recent finding of new
dwarf planets and the exploration of Mars by rovers.

7. The Voyager 2 spacecraft has traveled further from the Sun than any
other man-made object.

8. Pluto's reclassification as a dwarf planet sparked debates about
planetary definitions.

9. The exoplanets we've discovered in other star systems have expanded our
understanding of what a "planet" can be.

10. The New Horizons mission gave us close-up views of Pluto and the Kuiper
Belt object Arrokoth.

What specific aspect of the solar system are you most interested in? I'd be
happy to share more information on any particular part of our cosmic
neighborhood!
Downloads last month
627
GGUF
Model size
23.6B params
Architecture
llama
Hardware compatibility
Log In to view the estimation

1-bit

2-bit

3-bit

4-bit

5-bit

6-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for h34v7/DXP-Zero-V1.2-24b-Small-Instruct-i1-GGUF

Quantized
(2)
this model

Collection including h34v7/DXP-Zero-V1.2-24b-Small-Instruct-i1-GGUF