Instructions to use poolside/Laguna-XS-2.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use poolside/Laguna-XS-2.1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="poolside/Laguna-XS-2.1", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("poolside/Laguna-XS-2.1", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("poolside/Laguna-XS-2.1", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use poolside/Laguna-XS-2.1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "poolside/Laguna-XS-2.1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "poolside/Laguna-XS-2.1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/poolside/Laguna-XS-2.1

SGLang

How to use poolside/Laguna-XS-2.1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "poolside/Laguna-XS-2.1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "poolside/Laguna-XS-2.1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "poolside/Laguna-XS-2.1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "poolside/Laguna-XS-2.1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use poolside/Laguna-XS-2.1 with Docker Model Runner:
```
docker model run hf.co/poolside/Laguna-XS-2.1
```

modeling_laguna.py: in-tree relative imports break trust_remote_code loading on transformers v5 (OSError: ..cache_utils.py not found)

by nikivdev - opened 7 days ago

Discussion

nikivdev

7 days ago

Loading the model with remote code fails on transformers 5.x (verified on 5.12.1):

from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("poolside/Laguna-XS-2.1", trust_remote_code=True)

OSError: poolside/Laguna-XS-2.1 does not appear to have a file named ..cache_utils.py.
Checkout 'https://huggingface.co/poolside/Laguna-XS-2.1/tree/main' for available files.

It fails while resolving the remote module, before any weights are fetched. Loading from a local snapshot directory fails the same way (there as FileNotFoundError: ..cache_utils.py).

Cause

modeling_laguna.py has two function-local, in-tree-style relative imports at the top of LagunaModel.forward (lines 636–637):

from ...cache_utils import DynamicCache
from ...masking_utils import create_causal_mask, create_sliding_window_causal_mask

They look like leftovers from the in-tree transformers/models/laguna/ layout. transformers' dynamic-module machinery (get_relative_imports in dynamic_module_utils.py) scans a remote-code file for every from .<module> import … and tries to fetch <module>.py as a sibling file from the repo — for these two lines that means ..cache_utils.py / ..masking_utils.py, which don't exist, so from_pretrained aborts.

Suggested fix

All three names are already imported absolutely at the top of the same file (lines 32, 35, 42):

from transformers.cache_utils import DynamicCache
from transformers.masking_utils import create_causal_mask
from transformers.masking_utils import create_sliding_window_causal_mask

so the two function-local import lines can simply be deleted (or replaced with those absolute forms). from .configuration_laguna import LagunaConfig is the only other relative import in the file and is fine (the sibling file exists).

Until this is fixed, every trust_remote_code=True consumer has to patch the downloaded snapshot by hand before the model will load.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment