Nyx-31M-it Model

Nyx-31M-it is a Transformer-based model trained for text generation. It was fine-tuned using GPT-2 tokenization and optimized for domain-specific tasks.

Model Details

• Architecture: Decoder-only Transformer (GPT-2 style)
• Training Data: You can find the training data used here:
• Tokenizer: Based on GPT-2 (tiktoken)
• Framework: PyTorch

Usage

To use this model with Hugging Face's transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "core-outline/nyx-31M-it"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

input_text = "What is MRR?"
inputs = tokenizer(input_text, return_tensors="pt")

output = model.generate(**inputs)
decoded_text = tokenizer.decode(output[0], skip_special_tokens=True)

print(decoded_text)

Training

Optimizer: AdamW Learning Rate: 6e-5 Batch size: 32 Training steps: 12,000 Final validation loss: 0.04