|
--- |
|
license: cc-by-nc-4.0 |
|
--- |
|
|
|
# 42dot_LLM-SFT-1.3B_GGUF # |
|
|
|
* Model Creator: [42dot](https://huggingface.co/42dot) |
|
* original Model: [42dot_LLM-SFT-1.3B](https://huggingface.co/42dot/42dot_LLM-SFT-1.3B) |
|
|
|
## Description ## |
|
|
|
This repository contains the GGUF conversion and the most relevant quantizations |
|
of 42dot's |
|
[42dot_LLM-SFT-1.3B](https://huggingface.co/42dot/42dot_LLM-SFT-1.3B) model - ready |
|
to be used with [llama.cpp](https://github.com/ggerganov/llama.cpp) and similar |
|
applications. |
|
|
|
## Files ## |
|
|
|
In order to allow for fine-tuning (the model has the required LLaMA architecture) |
|
the original GGUF conversion has been made available |
|
|
|
* [42dot_LLM-SFT-1.3B.gguf](https://huggingface.co/rozek/42dot_LLM-SFT-1.3B_GGUF/blob/main/42dot_LLM-SFT-1.3B.gguf) |
|
|
|
From this file, the following quantizations were derived: |
|
|
|
* [42dot_LLM-SFT-1.3B-Q4_K_M](https://huggingface.co/rozek/42dot_LLM-SFT-1.3B_GGUF/blob/main/42dot_LLM-SFT-1.3B_Q5_K_M.gguf) |
|
* [42dot_LLM-SFT-1.3B-Q5_K_M](https://huggingface.co/rozek/42dot_LLM-SFT-1.3B_GGUF/blob/main/42dot_LLM-SFT-1.3B_Q5_K_M.gguf) |
|
* [42dot_LLM-SFT-1.3B-Q6_K](https://huggingface.co/rozek/42dot_LLM-SFT-1.3B_GGUF/blob/main/42dot_LLM-SFT-1.3B_Q6_0.gguf) |
|
* [42dot_LLM-SFT-1.3B-Q8_K](https://huggingface.co/rozek/42dot_LLM-SFT-1.3B_GGUF/blob/main/42dot_LLM-SFT-1.3B_Q8_0.gguf) |
|
|
|
(tell me if you need more) |
|
|
|
## Usage Details ## |
|
|
|
Any technical details can be found on the |
|
[original model card](https://huggingface.co/42dot/42dot_LLM-SFT-1.3B) |
|
The most important ones for using this model are |
|
|
|
* context length is 4096 |
|
* there does not seem to be a specific prompt structure - just provide the text |
|
you want to be completed |
|
|
|
### Text Completion with LLaMA.cpp ### |
|
|
|
For simple inferencing, use a command similar to |
|
|
|
``` |
|
./main -m 42dot_LLM-SFT-1.3B-Q8_K.gguf --temp 0 --top-k 4 --prompt "who was Joseph Weizenbaum?" |
|
``` |
|
|
|
### Text Tokenization with LLaMA.cpp ### |
|
|
|
To get a list of tokens, use a command similar to |
|
|
|
``` |
|
./tokenization -m 42dot_LLM-SFT-1.3B-Q8_K.gguf --prompt "who was Joseph Weizenbaum?" |
|
``` |
|
|
|
### Embeddings Calculation with LLaMA.cpp ### |
|
|
|
Text embeddings are calculated with a command similar to |
|
|
|
``` |
|
./embedding -m 42dot_LLM-SFT-1.3B-Q8_K.gguf --prompt "who was Joseph Weizenbaum?" |
|
``` |
|
|
|
## License ## |
|
|
|
The original model "_is licensed under the Creative Commons |
|
Attribution-NonCommercial 4.0 (CC BY-NC 4.0)_" - for that reason, the same |
|
license was also chosen for the conversions found in this repository. |
|
|
|
So, in order to be fair and give credits to whom they belong: |
|
|
|
* the original model was created and published by [42dot](https://huggingface.co/42dot) |
|
* besides quantization, no changes were applied to the model itself |
|
|
