|
--- |
|
license: mit |
|
base_model: gpt2-medium |
|
tags: |
|
- generated_from_trainer |
|
model-index: |
|
- name: gpt2-medium-finetuned-contract-gen |
|
results: [] |
|
pipeline_tag: text-generation |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# gpt2-medium-finetuned-contract-gen |
|
|
|
## Overview |
|
`gpt2-medium-finetuned-contract-gen` is a model specialized in generating Solidity contract codes. Derived from the [gpt2-medium](https://huggingface.co/gpt2-medium) model by Hugging Face, it's been meticulously trained on an extensive set of Solidity contracts and patterns, making it apt for assisting in drafting or suggesting contract structures. |
|
|
|
## Model Description |
|
This model has been designed specifically for generating Solidity contracts. Being a derivative of the `gpt2-medium` model, it retains the broader capabilities of the parent model while demonstrating a keen proficiency in understanding and generating Solidity-centric texts. |
|
|
|
### Performance |
|
The model reported a loss of `0.3127` on the evaluation set. |
|
|
|
## Intended Uses & Limitations |
|
|
|
### Intended Uses: |
|
1. Assist developers by auto-generating contract code snippets based on prompts. |
|
2. Help in understanding and drafting complex contract structures. |
|
|
|
### Limitations: |
|
1. The generated code must be reviewed for security and functional correctness. |
|
2. The clarity of the generated code largely depends on the specificity of the prompt. |
|
|
|
## Training Details |
|
|
|
### Dataset |
|
The model was fine-tuned on an undisclosed dataset comprised of a range of Solidity contracts. |
|
|
|
### Training Hyperparameters: |
|
- Learning Rate: `5e-05` |
|
- Train Batch Size: `4` |
|
- Evaluation Batch Size: `4` |
|
- Seed: `42` |
|
- Optimizer: Adam (`betas=(0.9,0.999)`, `epsilon=1e-08`) |
|
- Learning Rate Scheduler: Cosine with restarts |
|
- Warmup Steps: `241` |
|
- Epochs: `4` |
|
|
|
### Training Results: |
|
|
|
|
|
| Training Loss | Epoch | Step | Validation Loss | |
|
|:-------------:|:-----:|:-----:|:---------------:| |
|
| 0.4744 | 0.21 | 1000 | 0.4736 | |
|
| 0.467 | 0.41 | 2000 | 0.4146 | |
|
| 0.4089 | 0.62 | 3000 | 0.3852 | |
|
| 0.4018 | 0.83 | 4000 | 0.3688 | |
|
| 0.3475 | 1.04 | 5000 | 0.3523 | |
|
| 0.2751 | 1.24 | 6000 | 0.3434 | |
|
| 0.2966 | 1.45 | 7000 | 0.3334 | |
|
| 0.292 | 1.66 | 8000 | 0.3230 | |
|
| 0.2899 | 1.87 | 9000 | 0.3200 | |
|
| 0.2508 | 2.07 | 10000 | 0.3164 | |
|
| 0.28 | 2.28 | 11000 | 0.3127 | |
|
|
|
|
|
### Dependencies: |
|
- Transformers: `4.31.0` |
|
- Pytorch: `2.0.1+cu118` |
|
- Datasets: `2.14.2` |
|
- Tokenizers: `0.13.3` |
|
|
|
## How to Use |
|
If you wish to use this model to generate Solidity contract code, follow the steps below: |
|
|
|
```python |
|
from transformers import AutoTokenizer, AutoModelForCausalLM |
|
|
|
# Load the model and tokenizer |
|
tokenizer = AutoTokenizer.from_pretrained("ckandemir/gpt2-medium-finetuned-contract-gen") |
|
model = AutoModelForCausalLM.from_pretrained("ckandemir/gpt2-medium-finetuned-contract-gen") |
|
|
|
# Input your code prompt |
|
input_text = "contract MyToken" |
|
input_ids = tokenizer.encode(input_text, return_tensors='pt') |
|
sample_output = model.generate(input_ids, do_sample=True, max_length=400, num_return_sequences=1, temperature=0.7) |
|
|
|
# Decode and print the generated text |
|
generated_text = tokenizer.decode(sample_output[0], skip_special_tokens=True) |
|
print(generated_text) |
|
|
|
|
