I've built a small utility to split safetensors file by file.
text
I've built a small utility to split safetensors file by file.
new_line
The issue/need came up when I've tried to convert the new Deepseek V3 model  from FP8 to BF16.
text
The issue/need came up when I've tried to convert the new Deepseek V3 model  from FP8 to BF16.
new_line
The only Ada architecture GPU I have is an RTX 4080 and the 16GB vram was just wasn't enough for the conversion.
text
The only Ada architecture GPU I have is an RTX 4080 and the 16GB vram was just wasn't enough for the conversion.
new_line
new_line
BTW: I'll upload the bf16 version here:
text
BTW: I'll upload the bf16 version here:
new_line
https://huggingface.co/DevQuasar/deepseek-ai.DeepSeek-V3-Base-bf16
DevQuasar/deepseek-ai.DeepSeek-V3-Base-bf16
model
resource
https://huggingface.co/DevQuasar/deepseek-ai.DeepSeek-V3-Base-bf16
new_line
(it will take a while - days with my upload speed)
text
(it will take a while - days with my upload speed)
new_line
If anyone has access the resources to test it I'd appreciate a feedback if it's working or not.
text
If anyone has access the resources to test it I'd appreciate a feedback if it's working or not.
new_line
new_line
The tool, is available from here:
text
The tool, is available from here:
new_line
https://github.com/csabakecskemeti/ai_utils/blob/main/safetensor_splitter.py
https://github.com/csabakecskemeti/ai_utils/blob/main/safetensor_splitter.py
link
new_line
It's splitting every file to n pieces by the layers if possible, and create a new "model.safetensors.index.json" file.
text
It's splitting every file to n pieces by the layers if possible, and create a new "model.safetensors.index.json" file.
new_line
I've tested it with Llama 3.1 8B and multiple split sizes, and validated by using inference pipeline.
text
I've tested it with Llama 3.1 8B and multiple split sizes, and validated by using inference pipeline.
new_line
use 
text
use 
--help
`--help`
inline_code
 for usage
text
 for usage
new_line
Please note current version expects the model is already multiple file and have a "model.safetensors.index.json" layer-safetensor mapping file.
text
Please note current version expects the model is already multiple file and have a "model.safetensors.index.json" layer-safetensor mapping file.
I've built a small utility to split safetensors file by file.
The issue/need came up when I've tried to convert the new Deepseek V3 model  from FP8 to BF16.
The only Ada architecture GPU I have is an RTX 4080 and the 16GB vram was just wasn't enough for the conversion.

BTW: I'll upload the bf16 version here:
https://huggingface.co/DevQuasar/deepseek-ai.DeepSeek-V3-Base-bf16
(it will take a while - days with my upload speed)
If anyone has access the resources to test it I'd appreciate a feedback if it's working or not.

The tool, is available from here:
https://github.com/csabakecskemeti/ai_utils/blob/main/safetensor_splitter.py
It's splitting every file to n pieces by the layers if possible, and create a new "model.safetensors.index.json" file.
I've tested it with Llama 3.1 8B and multiple split sizes, and validated by using inference pipeline.
use `--help` for usage
Please note current version expects the model is already multiple file and have a "model.safetensors.index.json" layer-safetensor mapping file.
🔥
djuna AntleRot reonyy Tom-Neverwinter
👍
John6666 reonyy
❤️
reonyy