BrtGPT-124M-Base

NOTE: MODEL TRAINED ON 5M TOKENS AGAIN IN 14 JUNE 2025, IF YOU DOWNLOADED WEIGHTS BEFORE; REINSTALL!

• This model trained on 5M (About 5.000.000) Tokens, With "English" Sentences.
• Model IS NOT for QA (UNLIKE ChatGPT or LLama), this model is only Pre-Trained on Large Corpus, so this is a Base Model

Model Details

Model Description

CHECK THE COMMUNITY FOR VERY IMPORTANT UPDATES!

• Developed by: Bertug Gunel (Bertuğ Günel)
• Funded by [optional]: Nobody
• Shared by [optional]: Nobody
• Model type: Decoder-Only Transformer
• Language(s) (NLP): English
• License: CC-BY-NC-4.0
• Finetuned from model [optional]: Not Fine-Tuned

Model Sources [optional]

• Repository: Cooming Soon!
• Paper [optional]: "Attention All You Need", 1706.03762
• Demo [optional]: Model is already a demo model.

Uses

This codes, loads the model (BrtGPT-124M-Base), you can use it!

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load Model and Tokenizer
model_name = "Bertug1911/BrtGPT-124m-Base"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Input
prompt = "Math is so important because"

# Tokenize it
inputs = tokenizer(prompt, return_tensors="pt")

# Generate with model
output = model.generate(
    **inputs,
    max_new_tokens=50,
    temperature=0.01,
    top_k=1,
    do_sample=True
)

# Decode output
generated_text = tokenizer.decode(output[0], skip_special_tokens=False)
generated_text = generated_text.replace(" ", "")
generated_text = generated_text.replace("Ġ", " ")
print(generated_text)

USAGE EXAMPLES:

Input	Max New Tokens	Temperature	Output
"Today"	50	0.1	"Today, a complex and multifaceted system, is often viewed as a myriad of the intricacies of the human mind and the intricacies of the human condition. It is believed to be a powerful"
"To stop world hunger, we should"	50	0.1	"To stop world hunger, we should be able to find a more stable and healthy relationship with the body. By doing so, we can make a mealtime and easier to start and maintain a healthy and balance."
"Math is so important because"	50	0.1	"MMath is so important because it's essential to carefully consider and address any potential health concerns that may arise from the condition, as it can lead to a range of health issues. By including the bleeding and potentially causing sympt..."
"To be rich, you should,"	50	0.4	"To be rich you should be on the same time, it's essential to consider the various factors that contribute to your unique needs. For instance, it's crucial to consider that you should be taking a black room,..."

Direct Use

Direct Use

Perhaps the worst part of open source models is that using them is very laborious and requires a lot of processing power, but our model solves both problems: you can use and download them for FREE and VERY EASILY from the link below!

Web (Gradio, Spaces) UI is done! To use it FREELY and EASILY Hugging Face Spaces LINK: "https://huggingface.co/spaces/Bertug1911/BrtGPT-Web-UI"

Out-of-Scope Use

Model only generates (completes) "English" sentences with "English" tokens. (And contains some Japanese/Chinese tokens.) Don't try with another Languages!

Bias, Risks, and Limitations

Model can generates: "Political" contents, USE YOUR OWN RISK

Recommendations

No big risks or biases! You can use it freely (But only "Non-commerical")

How to Get Started with the Model

You can use model for generating English texts. Model is FREE to use.

Training Details

Training Data

NOTE: MODEL TRAINED ON 5M TOKENS AGAIN IN 13 JUNE 2025, IF YOU DOWNLOADED WEIGHTS BEFORE; REINSTALL!

Model trained on: Train.csv (5M tokens, 15000+ lines)

Data Type	Training Type	Tokens (Total)
Raw (sentences)	Pre Training	About 5M (5000K)
Instruction (Coming soon!)	Cooming soon!	Cooming soon!

Training Procedure

Model Trained on: B200 GPU (Nvidia) for 21,5 Minutes.

Training Hyperparameters

• Training regime: Training precision is: FP16, Sparsity: OFF

Evaluation

NO EVALUATION (Cooming soon!)

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: GPU
• Hours used: 0.35 (21,5 Mins)
• Cloud Provider: "Runpod" (https://www.runpod.io/)
• Compute Region: EU
• Carbon Emitted: 0.138 KG (138 Gram(s)) "This is equivalent to an average light bulb burning for 2.6 hours."

Model Card Authors [optional]

• Bertug Gunel
• Turkey/Eskisehir

Model Card Contact

[email protected]