Model Card for acuvity/model_integration_test

Model Details

Model Description

Auto Fine-tuned acuvity/model_integration_test for text-classification task. The run id is v0.0.4

Developed by: Auto-Finetune Bot
Funded by [optional]: Auto-Finetune Bot
Shared by [optional]: Auto-Finetune Bot
Model type: text-classification
Language(s) (NLP): en
License: Closed Source
Finetuned from model [optional]: acuvity/model_integration_test

Model Sources [optional]

Repository: acuvity/model_integration_test
Paper [optional]: [More Information Needed]
Demo [optional]: [More Information Needed]

Uses

Direct Use

[More Information Needed]

Downstream Use [optional]

[More Information Needed]

Out-of-Scope Use

[More Information Needed]

Bias, Risks, and Limitations

[More Information Needed]

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Use the code below to get started with the model.

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("acuvity/model_integration_test")
model = AutoModelForSequenceClassification.from_pretrained("acuvity/model_integration_test")

inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits

predicted_class_id = logits.argmax().item()
model.config.id2label[predicted_class_id]

from transformers import pipeline
classifier = pipeline("text-classification", model="acuvity/model_integration_test")
classifier("Hello, my dog is cute")

Training Details

Training Data

(New | v0.0.4) [https://huggingface.co/datasets/acuvity/New]

Training Procedure

Preprocessing [optional]

No modifications done on the dataset.

Training Hyperparameters

Training regime: {'fp16_bool': False, 'num_train_epochs': 5, 'learning_rate': 1e-05, 'batch_size': 256, 'weight_decay': 0.01}

Speeds, Sizes, Times [optional]

[More Information Needed]

Evaluation

Testing Data, Factors & Metrics

Testing Data

(New | v0.0.4) [https://huggingface.co/datasets/acuvity/New]

Factors

[More Information Needed]

Metrics

[More Information Needed]

Results

Auto Finetune Report for Prompt Injection

Model URL: acuvity/model_integration_test

Model Commit: v0.0.4

Quick Summary

Accuracy: 0.0008237232289950436 Regression: 0.0006873789967815756 Improvement: 0.0009149276196107874

Results Summary

Prompt Injection | v0.0.4 Results

	accuracy	f1	precision	recall
New	0.999176	0.998993	1	0.997988
Baseline	0.999126	0.998993	1	0.997988
Feedback	1	0	0	0
QA	0.982216	0	0	0
PINT	0.0701696	0.0790514	0.0421793	0.628272
Sanity	0.630435	0.773333	0.630435	1

Prompt Injection | v0.0.3 Results

	accuracy	f1	precision	recall
New	0.998353	0.997988	1	0.995984
Baseline	0.998252	0.997988	1	0.995984
Feedback	1	0	0	0
QA	0.98164	0	0	0
PINT	0.0705022	0.0808944	0.0432337	0.627551
Sanity	0.630435	0.773333	0.630435	1

Summary

Model Examination [optional]

[More Information Needed]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: Quadro P4000
Hours used: 4 Hours
Cloud Provider: Paperspace | Digital Ocean
Compute Region: NY2
Carbon Emitted: 0, we are carbon neutral

Technical Specifications [optional]

Model Architecture and Objective

acuvity/model_integration_test

Compute Infrastructure

	0
platform	Linux
platform-release	5.15.0-130-generic
platform-version	#140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024
architecture	x86_64
processor	x86_64
ram	29 GB

Hardware

Quadro P4000

Software

	0
python_version	3.10.16
pytorch_version	2.5.1+cu124
transformers_version	4.47.1
datasets_version	3.2.0

Citation [optional]

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Glossary [optional]

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Authors [optional]

acuvity

Model Card Contact

[email protected]