metadata
license: gemma
pipeline_tag: text-to-image
tags:
- NovelAI
Inference
from transformers.models import AutoTokenizer, T5GemmaEncoderModel
import torch
if __name__ == '__main__':
model = T5GemmaEncoderModel.from_pretrained(t5gemma_path, torch_dtype=torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained(t5gemma_path)
inputs = tokenizer('Gemma', max_length=512, padding='max_length', truncation=True, return_tensors='pt')
output = model.forward(**inputs).last_hidden_state
SD1.5 and Gemma
from diffusers import StableDiffusionPipeline
from gemma_encoder import Encoder
if __name__ == '__main__':
pipeline = StableDiffusionPipeline.from_pretrained('NovelAI/nai-anime-v2')
pipeline.enable_model_cpu_offload()
encoder = Encoder(adapter_model, t5gemma_path, device='cpu')
load_model(adapter_model, 'adapter.safetensors')
image = pipeline(
None,
prompt_embeds=encoder.encode(pipeline, text).to('cpu'),
negative_prompt='bad quality, low quality, worst quality'
).images[0]
image.save('preview.png')
Datasets
- alfredplpl/artbench-pd-256x256
- danbooru2023-florence2-caption (verb, action clauses)
- spatial-caption
- SPRIGHT-T2I/spright_coco
- sugarquark/colormix (synthetic color, fashion dataset)
- trojblue/danbooru2025-metadata