File size: 877 Bytes
255e848 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
---
license: mit
tags:
- text-generation
- character-level
- compression
- research
datasets:
- enwik8
---
# Compressed nanoGPT (enwik8)
## Outstanding Compression Results!
- **Performance**: 1.635 → 1.637 BPC (+0.002)
- **Parameters**: 28,801,536 → 27,359,744
- **Compression**: 1.053× smaller (5.0% reduction)
- **Quality Loss**: Only 0.1% degradation!
This demonstrates **near-perfect compression** of a character-level transformer.
## Usage
```python
from transformers import AutoModel
model = AutoModel.from_pretrained(
"prompterminal/nanogpt-enwik8-compressed-working",
trust_remote_code=True
)
# Generate text
import torch
prompt = torch.randint(0, 6060, (1, 10)) # Random start
output = model.generate(prompt, max_new_tokens=100)
```
## Research Impact
First successful demonstration of high-quality compression on character-level transformers!
|