You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co/docs/hub/model-cards#model-card-metadata)

DenseBackwardOLMoE

自定义的OLMoE模型，使用DenseBackwardOlmoeSparseMoeBlock替换原版的MoE模块，实现dense backward功能。

用法

from transformers import AutoConfig, AutoModelForCausalLM

# 使用trust_remote_code=True加载模型
config = AutoConfig.from_pretrained("autoprogrammer/olmoe_densebackward", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("autoprogrammer/olmoe_densebackward", config=config, trust_remote_code=True)

Downloads last month: 978

Safetensors

Model size

6.92B params

Tensor type

F32

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support