metadata

license: llama2
library_name: peft
tags:
  - generated_from_trainer
base_model: unsloth/llama-3-8b-bnb-4bit
metrics:
  - accuracy
model-index:
  - name: llama3-qwantz-coherent
    results: []
pipeline_tag: text-classification

llama3-qwantz-coherent

This model is a fine-tuned version of unsloth/llama-3-8b-bnb-4bit on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.3295
Accuracy: 0.8758

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.4482	1.0	1428	0.3295	0.8758

2-4 fake words at the end (like training set):
Can save 90% of coherent strings by discarding 82% of dp strings (cutoff is -67.26065874099731)
Can save 95% of coherent strings by discarding 72% of dp strings (cutoff is -88.40824365615845)
Can save 98% of coherent strings by discarding 62% of dp strings (cutoff is -95.06730437278748)
Can save 99% of coherent strings by discarding 54% of dp strings (cutoff is -97.79982566833496)

1 fake word at the end:
Can save 90% of coherent strings by discarding 48% of dp strings (cutoff is -77.83336043357849)
Can save 95% of coherent strings by discarding 30% of dp strings (cutoff is -92.52431392669678)
Can save 98% of coherent strings by discarding 26% of dp strings (cutoff is -95.45100927352905)
Can save 99% of coherent strings by discarding 21% of dp strings (cutoff is -97.32990860939026)

Examples (2-4 fake words):
My only problem (s) have to do with  ==>  coherent: 99.12%
My only problem (s) to cheer them personally  ==>  dp: 99.69%
(in small text) crazy utahraptor  ==>  coherent: 88.54%
(in small text) ".  ==>  coherent: 54.82%
Well, I've made up my own joke to get him today. All I need to do is "  ==>  coherent: 77.98%
Well, I've made up my own joke to get him today. All I need a father and gentlemen  ==>  dp: 99.79%
I will be immortalized by kicking an evil  ==>  dp: 72.79%
I will be immortalized by kicking other punches  ==>  dp: 99.49%
Aw shoot, I was supposed to  ==>  coherent: 99.80%
Aw shoot, I was APOCALYPSE PORN  ==>  dp: 94.10%
Get it? Because CRIME DOESN'T PAY!! Listen, my story has  ==>  coherent: 66.25%
Get it? Because CRIME DOESN'T PAY!! Listen, transcriptions of it  ==>  dp: 99.75%
Utahraptor!! DON'T LISTEN TO  ==>  coherent: 99.96%
Utahraptor! This is sort of  ==>  coherent: 95.96%
Doesn't exist in my mouth, that is!! Because it's too big  ==>  coherent: 95.38%
Doesn't exist in my mouth, that is!! Because if Superman.  ==>  dp: 99.66%
Now, HERE'S how  ==>  coherent: 98.67%
Now, guys would  ==>  dp: 95.30%
But I am a rock star  ==>  coherent: 92.34%
But I am a guy come  ==>  dp: 99.40%
But I have a solution to make them interesting again: all you need is stories where not  ==>  coherent: 94.51%
But I have a solution to make them interesting again: all you need is gonna! Diseases  ==>  dp: 99.94%
At that point, there's a sequence of six nines in a row, and his joke was that he'd like to memorize pi up to that point, so that when reciting he could end with "9,9,9,9,9,9... and so on. " Others  ==>  coherent: 70.68%
At that point, there's a sequence of six nines in a row, and his joke was that he'd like to memorize pi up to that point, so that when reciting he could end with "9,9,9,9,9,9... and so it's great he looks  ==>  dp: 99.54%
This is definitely called " T -Rex's Hilarious e joke ", okay  ==>  coherent: 79.86%
This is definitely called " T -Rex's Hilarious e joke AND IN THE  ==>  dp: 98.83%
" Your mouth is full of cockroaches:  ==>  coherent: 93.64%
" Your mouth is full of smooches.  ==>  coherent: 99.10%
Excuse me, sexual congress? Everyone else on the planet is dead, and  ==>  coherent: 89.66%
Excuse me, sexual congress? Everyone else on the planet without syntactic ambiguity!  ==>  dp: 97.02%
Sony is going to write swears on my bathroom  ==>  coherent: 99.75%
Sony is going to write their babies need to  ==>  dp: 99.68%
Beginning with the most modest: why am I  ==>  coherent: 99.62%
Beginning with the most modest: why T -  ==>  dp: 92.38%
Is there any greater meaning -to anything  ==>  coherent: 95.96%
Is there any greater meaning? When you  ==>  dp: 73.11%
I've also got steaks AND  ==>  coherent: 92.46%
I've also cold -deterministic  ==>  dp: 99.48%
I had a friend (female) who dated her roommate (also female)  ==>  coherent: 98.78%
I had a friend (female) who dated her roommate, je suis grand  ==>  dp: 97.52%
Yes... TOO BAD INDEED  ==>  dp: 93.13%
Yes... TOO MANY YEARS  ==>  coherent: 65.94%

Examples (1 fake word):
My only problem (s) have to do with you  ==>  dp: 83.54%
My only problem (s) have to do with no  ==>  dp: 53.68%
(in small text  ==>  coherent: 93.38%
(in small changes  ==>  dp: 97.64%
Well, I've made up my own joke to get him today. All I need to do is "  ==>  coherent: 77.98%
Well, I've made up my own joke to get him today. All I need to do is already  ==>  dp: 97.05%
I will be immortalized by kicking an evil kangaroo  ==>  coherent: 92.81%
I will be immortalized by kicking an evil!  ==>  dp: 99.50%
Aw shoot, I was supposed  ==>  coherent: 98.61%
Aw shoot, I was how  ==>  dp: 99.55%
Get it? Because CRIME DOESN'T PAY!! Listen, my story has both a hilarious twist ending and also  ==>  coherent: 99.64%
Get it? Because CRIME DOESN'T PAY!! Listen, my story has both a hilarious twist ending and genders  ==>  dp: 88.29%
Utahraptor!! DON'T LISTEN TO MY  ==>  coherent: 94.44%
Utahraptor!! DON'T LISTEN TO THE  ==>  coherent: 97.86%
Doesn't exist in my mouth, that is!! Because it's too  ==>  coherent: 96.91%
Doesn't exist in my mouth, that is!! Because it's Well  ==>  dp: 97.87%
Now, HERE'S how putting the things  ==>  dp: 97.33%
Now, HERE'S how putting the wall  ==>  dp: 87.15%
But I am a rock  ==>  coherent: 52.13%
But I am a time  ==>  dp: 93.86%
But I have a solution to make them interesting again: all you need is  ==>  coherent: 99.83%
But I have a solution to make them interesting again: all you need to  ==>  coherent: 99.81%
At that point, there's a sequence of six nines in a row, and his joke was that he'd like to memorize pi up to that point, so that when reciting he could end with "9,9,9  ==>  coherent: 99.95%
At that point, there's a sequence of six nines in a row, and his joke was that he'd like to memorize pi up to that point, so that when reciting he could end with "9,9, and  ==>  coherent: 95.76%
This is definitely  ==>  coherent: 89.56%
This is especially  ==>  coherent: 98.87%
" Your mouth is full of cockroaches: many of them are dead, but those that  ==>  coherent: 99.96%
" Your mouth is full of cockroaches: many of them are dead, but those taste  ==>  dp: 78.53%
Excuse me, sexual congress? Everyone  ==>  dp: 94.45%
Excuse me, sexual congress? "  ==>  dp: 91.74%
Sony is going to write swears  ==>  coherent: 74.56%
Sony is going to write that  ==>  dp: 98.15%
Beginning with the  ==>  coherent: 84.62%
Beginning with!  ==>  dp: 98.17%
Is there any greater meaning -to  ==>  dp: 61.69%
Is there any greater meaning -Rex  ==>  dp: 99.31%
I've also got steaks AND pork chops  ==>  coherent: 99.84%
I've also got steaks AND pork meat  ==>  coherent: 95.80%
I had a friend (female  ==>  coherent: 85.77%
I had a friend (on  ==>  coherent: 57.30%
Yes... TOO  ==>  dp: 77.83%
Yes... All  ==>  dp: 67.88%

Framework versions

PEFT 0.10.0
Transformers 4.40.1
Pytorch 2.2.1+cu121
Datasets 2.19.0
Tokenizers 0.19.1