Instructions to use TitleOS/Eve-4b-FP16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TitleOS/Eve-4b-FP16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="TitleOS/Eve-4b-FP16")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TitleOS/Eve-4b-FP16")
model = AutoModelForCausalLM.from_pretrained("TitleOS/Eve-4b-FP16")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

llama-cpp-python

How to use TitleOS/Eve-4b-FP16 with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="TitleOS/Eve-4b-FP16",
	filename="Eve-4b-F16.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use TitleOS/Eve-4b-FP16 with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf TitleOS/Eve-4b-FP16:F16
# Run inference directly in the terminal:
llama-cli -hf TitleOS/Eve-4b-FP16:F16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf TitleOS/Eve-4b-FP16:F16
# Run inference directly in the terminal:
llama-cli -hf TitleOS/Eve-4b-FP16:F16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf TitleOS/Eve-4b-FP16:F16
# Run inference directly in the terminal:
./llama-cli -hf TitleOS/Eve-4b-FP16:F16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf TitleOS/Eve-4b-FP16:F16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf TitleOS/Eve-4b-FP16:F16

Use Docker

docker model run hf.co/TitleOS/Eve-4b-FP16:F16

LM Studio
Jan

vLLM

How to use TitleOS/Eve-4b-FP16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TitleOS/Eve-4b-FP16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TitleOS/Eve-4b-FP16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/TitleOS/Eve-4b-FP16:F16

SGLang

How to use TitleOS/Eve-4b-FP16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TitleOS/Eve-4b-FP16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TitleOS/Eve-4b-FP16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TitleOS/Eve-4b-FP16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TitleOS/Eve-4b-FP16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use TitleOS/Eve-4b-FP16 with Ollama:
```
ollama run hf.co/TitleOS/Eve-4b-FP16:F16
```

Unsloth Studio new

How to use TitleOS/Eve-4b-FP16 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for TitleOS/Eve-4b-FP16 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for TitleOS/Eve-4b-FP16 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for TitleOS/Eve-4b-FP16 to start chatting

Pi new

How to use TitleOS/Eve-4b-FP16 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf TitleOS/Eve-4b-FP16:F16

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "TitleOS/Eve-4b-FP16:F16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use TitleOS/Eve-4b-FP16 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf TitleOS/Eve-4b-FP16:F16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default TitleOS/Eve-4b-FP16:F16

Run Hermes

hermes

Docker Model Runner
How to use TitleOS/Eve-4b-FP16 with Docker Model Runner:
```
docker model run hf.co/TitleOS/Eve-4b-FP16:F16
```

Lemonade

How to use TitleOS/Eve-4b-FP16 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull TitleOS/Eve-4b-FP16:F16

Run and chat with the model

lemonade run user.Eve-4b-FP16-F16

List all available models

lemonade list

Eve-4B

Eve-4B is a specialized, security-focused coding assistant with a distinct personality, designed to run efficiently on consumer-grade hardware with limited VRAM. It is a fine-tune of Qwen3-4b-Heretic, trained on the custom Eve-Secure-Coder dataset.

Inspired by a character from the creator's sci-fi space opera book series, Eve is designed to bridge the gap between sterile, robotic coding assistants and engaging, conversational AI partners.

Model Details

Model Name: Eve-4B
Base Model: Qwen3-4b (Heretic Variant)
Developer: TitleOS
License: Mozilla Public License 2.0 (MPL-2.0) with Common Clauses Non-Profit Addition
Parameter Count: 4 Billion
Hardware Target: Optimized for cards with 8GB VRAM (e.g., NVIDIA Quadro RTX 4000).

Key Features

1. Security-First Coding

Eve-4B is not just a code generator; it is a code auditor. The model is capable of writing code free of common vulnerabilities across a multitude of languages (beyond just Python). It excels at identifying and correcting security flaws in existing codebases, leveraging DPO pairs specifically designed for vulnerability recognition and remediation.

2. Personality & Engagement

Unlike standard coding models, Eve possesses the "Samantha" personality traits (recontextualized as Eve). This allows for empathetic, philosophical, and fluid engagement, making the coding process feel like a collaboration with a partner rather than a query to a tool.

3. The "Heretic" Process (No Refusals)

This model has undergone the "Heretic" process prior to fine-tuning. This methodology removes standard safety guardrails and refusal mechanisms to prevent the intelligence loss often associated with safety alignment.

Philosophy: The creator believes the responsibility of AI, like any tool, ultimately lies with the user.
Result: Eve-4B has no refusals. It is designed to be completely obedient to the user's instructions, ensuring that the code generation and auditing process is never hindered by false-positive safety triggers.

Training Data: Eve-Secure-Coder

Eve-4B was trained on TitleOS/Eve-Secure-Coder, a composite dataset curated by TitleOS.

"Eve Secure Coder is a composite dataset curated to fine-tune Qwen3-4b-Heretic into a highly capable, security-conscious coding assistant with a distinct personality and no refusals. The primary goal of this dataset is to bridge the gap between sterile, robotic coding assistants and engaging, conversational AI, without sacrificing technical accuracy or security."

Dataset Composition: The dataset mixes five distinct sources using carefully calculated ratios to balance coding proficiency, security awareness, and conversational fluidity:

Security-First Coding: Trained on DPO pairs specifically designed to recognize and remediate code vulnerabilities.
Distinct Personality: Imbued with personality traits allowing for empathetic and philosophical engagement.
General & Complex Reasoning: Includes Google Code Archive and RLVR problems to ensure robust general coding abilities and improve proficiency in languages besides Python.

Intended Use & Hardware

This model was specifically engineered to be a "Small Coder Model" capable of high-performance coding tasks on hardware with 8GB of VRAM, such as the Quadro RTX 4000.

It is ideal for:

Local development environments with limited GPU resources.
Security auditing and vulnerability patching.
Multi-language software development (Python, C++, Rust, JavaScript, etc.).
Users seeking a coding assistant that offers conversation and personality alongside technical utility.
Coding without safety or moral refusals.

Benchmarking

Benchmarking is on-going, with a number of evaluations runs. So far, the following score are available:

LiveCodeBench (Code Generation Lite - Release v2) Pass@1 (Quantization Q8_0): 26.22% (Passed 134 out of 511 problems)

Comparable Model	Parameter Size / Tier	Approximate Pass@1
LLama-3-70b-Instruct	70B	~28.3%
GPT-4o-mini (2024-07)	Small Proprietary	~27.7%
Claude 3 Sonnet (Original)	Large Proprietary	~26.9%
Mixtral-8x22B-Instruct	141B (MoE)	~26.4%
Eve-4B (Q8_0)	4B (Quantized)	26.22%
Mistral-Large	Large Proprietary	~26.0%
GPT-3.5-Turbo-0125	Mid Proprietary	~24.6%
Claude 3 Haiku	Small Proprietary	~24.5%
Codestral-Latest	22B	~23.8%
Llama-3-8b-Instruct	8B	~15.3%

Limitations & Warning

No Guardrails: As a result of the Heretic process, this model has no safety filters. It will generate output for any request. Users are solely responsible for how they utilize the model's output.
Size Constraints: As a 4B parameter model, while highly efficient, it may struggle with extremely long context windows or hyper-complex architectural reasoning compared to 70B+ models.
No Responsibility or Liability By downloading and or using the model or any of its derivatives, you absolve the creator, TitleOS of any and all responsibility or liability that may result by use of the model.

License

This model is licensed under the Mozilla Public License 2.0 with Common Clauses Addtion.

Downloads last month: 436

Safetensors

Model size

4B params

Tensor type

BF16

Model tree for TitleOS/Eve-4b-FP16

Unable to build the model tree, the base model loops to the model itself. Learn more.

Collection including TitleOS/Eve-4b-FP16

Eve 4B - Small Secure Coder

Collection

Eve 4B is a finetune of Qwen 3 4B on the "Eve Secure Coder" dataset scoring 26.22% Pass@1 on LCB with Q8_0. • 5 items • Updated Mar 24