🐾 Pokemon Generator - Mistral 7B

A fine-tuned Mistral 7B model trained to generate original Pokemon with authentic names, types, and descriptions.

Model Details

Base Model: Mistral 7B v0.1
Fine-tuning Method: LoRA (Low-Rank Adaptation)
Training Data: Pokemon from Generations 1-4
Model Size: ~13GB
Languages: English
License: Apache 2.0

Quick Start

Basic Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained("Sulfruos/poke-generator-demo")
tokenizer = AutoTokenizer.from_pretrained("Sulfruos/poke-generator-demo")

# Generate a Pokemon
prompt = "Generate a new and original Pokemon that doesn't exist yet: "
inputs = tokenizer(prompt, return_tensors="pt")

# Generate with custom parameters
outputs = model.generate(
    **inputs,
    max_new_tokens=100,
    temperature=0.7,
    do_sample=True,
    top_p=0.95,
    top_k=40,
    repetition_penalty=1.2,
    pad_token_id=tokenizer.eos_token_id
)

# Decode result
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)

Using Pipeline

from transformers import pipeline

# Create text generation pipeline
generator = pipeline(
    "text-generation", 
    model="Sulfruos/poke-generator-demo",
    tokenizer="Sulfruos/poke-generator-demo"
)

# Generate Pokemon
result = generator(
    "Generate a new and original Pokemon that doesn't exist yet: ",
    max_new_tokens=100,
    temperature=0.7,
    do_sample=True
)

print(result[0]['generated_text'])

Example Outputs

Fire-type Pokemon

Name: Blazefin
Type: Fire/Water
Description: This unique Pokemon can survive in both lava and deep ocean trenches. Its fins glow red-hot when threatened, boiling the water around it as a defense mechanism.

Psychic-type Pokemon

Name: Mindwhisper
Type: Psychic
Description: Known for its ability to communicate telepathically across vast distances. Its large, luminous eyes can see into the dreams of sleeping Pokemon.

Dual-type Pokemon

Name: Crystaleon
Type: Ice/Electric
Description: Its crystalline body stores electrical energy from thunderstorms. When it moves, tiny ice crystals spark with electricity, creating beautiful aurora-like displays.

Recommended Parameters

Conservative Generation

outputs = model.generate(
    **inputs,
    max_new_tokens=80,
    temperature=0.5,      # More consistent
    top_p=0.9,
    repetition_penalty=1.1
)

Creative Generation

outputs = model.generate(
    **inputs,
    max_new_tokens=120,
    temperature=0.9,      # More creative
    top_p=0.95,
    top_k=50,
    repetition_penalty=1.3
)

Balanced Generation (Recommended)

outputs = model.generate(
    **inputs,
    max_new_tokens=100,
    temperature=0.7,      # Good balance
    top_p=0.95,
    top_k=40,
    repetition_penalty=1.2
)

Output Format

The model generates Pokemon in this consistent format:

Name: [Pokemon Name]
Type: [Type] or [Type1/Type2]
Description: [Detailed description of the Pokemon's appearance, abilities, and characteristics]

Training Details

Dataset

Source: Pokemon data from Generations 1-4
Size: ~400 authentic Pokemon entries
Format: Structured Name/Type/Description format
Preprocessing: Name uniqueness validation, type consistency checks

Training Process

Base Model: Mistral 7B v0.1
Method: LoRA fine-tuning (r=16, α=32)
Hardware: Google Colab T4 GPU
Training Time: ~45 minutes
Loss Reduction: 2.13 → 0.95
Validation: Format consistency and uniqueness checks

Model Architecture

Parameters: ~7B (base) + LoRA adapters
Context Length: 4096 tokens
Vocabulary: 32,000 tokens
Precision: fp16 (GPU) / fp32 (CPU)

Hardware Requirements

Minimum Requirements

RAM: 16GB+
Storage: 15GB free space
GPU: Optional but recommended (GTX 1060+ or equivalent)

Recommended Setup

RAM: 32GB+
GPU: RTX 3070+ or T4+ for fast inference
Storage: SSD recommended for faster loading

Performance Expectations

CPU only: 30-120 seconds per generation
GPU (GTX 1060): 15-45 seconds per generation
GPU (RTX 3070+): 5-15 seconds per generation
GPU (T4/V100): 3-10 seconds per generation

Use Cases

Creative Writing: Generate Pokemon for stories and fan fiction
Game Development: Create original creatures for Pokemon-inspired games
Educational: Learn about fine-tuning language models
Research: Study text generation and creative AI applications

Limitations

English only: Trained on English Pokemon descriptions
Generation 1-4 style: Reflects classic Pokemon characteristics
Format dependency: Works best with the specific prompt format
Creativity vs consistency: Higher temperature = more creative but less consistent

Fine-tuning Process

This model was created using LoRA (Low-Rank Adaptation) fine-tuning:

Data Collection: Gathered authentic Pokemon data
Preprocessing: Formatted as Name/Type/Description
LoRA Training: Fine-tuned on Google Colab T4
Validation: Implemented quality and uniqueness checks
Merging: Combined LoRA weights with base model for standalone deployment

Model Card Contact

Created by: @Sulfruos
Model type: Text Generation (Pokemon-focused)
Language: English
License: Apache 2.0
Base model: Mistral 7B v0.1

Citation

If you use this model in your research or projects, please cite:

@misc{pokemon-generator-mistral,
  title={Pokemon Generator: Fine-tuned Mistral 7B for Creative Pokemon Generation},
  author={Sulfruos},
  year={2024},
  howpublished={\\url{https://huggingface.co/Sulfruos/poke-generator-demo}}
}

Ethical Considerations

This model is designed for creative and educational purposes. Generated content should be used responsibly and in accordance with fair use principles. The model generates fictional creatures inspired by the Pokemon franchise but creates original content.

Sulfruos
/

poke-generator-demo