ckandemir commited on
Commit
6d4f724
1 Parent(s): 5916c39

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +50 -26
README.md CHANGED
@@ -14,37 +14,42 @@ should probably proofread and complete it, then remove this comment. -->
14
 
15
  # gpt2-medium-finetuned-contract-gen
16
 
17
- This model is a fine-tuned version of [gpt2-medium](https://huggingface.co/gpt2-medium) on an unknown dataset.
18
- It achieves the following results on the evaluation set:
19
- - Loss: 0.3127
20
 
21
- ## Model description
 
22
 
23
- More information needed
 
24
 
25
- ## Intended uses & limitations
26
 
27
- More information needed
 
 
28
 
29
- ## Training and evaluation data
 
 
30
 
31
- More information needed
32
 
33
- ## Training procedure
 
34
 
35
- ### Training hyperparameters
 
 
 
 
 
 
 
 
36
 
37
- The following hyperparameters were used during training:
38
- - learning_rate: 5e-05
39
- - train_batch_size: 4
40
- - eval_batch_size: 4
41
- - seed: 42
42
- - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
43
- - lr_scheduler_type: cosine_with_restarts
44
- - lr_scheduler_warmup_steps: 241
45
- - num_epochs: 4
46
 
47
- ### Training results
48
 
49
  | Training Loss | Epoch | Step | Validation Loss |
50
  |:-------------:|:-----:|:-----:|:---------------:|
@@ -61,9 +66,28 @@ The following hyperparameters were used during training:
61
  | 0.28 | 2.28 | 11000 | 0.3127 |
62
 
63
 
64
- ### Framework versions
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
65
 
66
- - Transformers 4.31.0
67
- - Pytorch 2.0.1+cu118
68
- - Datasets 2.14.2
69
- - Tokenizers 0.13.3
 
14
 
15
  # gpt2-medium-finetuned-contract-gen
16
 
17
+ ## Overview
18
+ `gpt2-medium-finetuned-contract-gen` is a model specialized in generating Solidity contract codes. Derived from the [gpt2-medium](https://huggingface.co/gpt2-medium) model by Hugging Face, it's been meticulously trained on an extensive set of Solidity contracts and patterns, making it apt for assisting in drafting or suggesting contract structures.
 
19
 
20
+ ## Model Description
21
+ This model has been designed specifically for generating Solidity contracts. Being a derivative of the `gpt2-medium` model, it retains the broader capabilities of the parent model while demonstrating a keen proficiency in understanding and generating Solidity-centric texts.
22
 
23
+ ### Performance
24
+ The model reported a loss of `0.3127` on the evaluation set.
25
 
26
+ ## Intended Uses & Limitations
27
 
28
+ ### Intended Uses:
29
+ 1. Assist developers by auto-generating contract code snippets based on prompts.
30
+ 2. Help in understanding and drafting complex contract structures.
31
 
32
+ ### Limitations:
33
+ 1. The generated code must be reviewed for security and functional correctness.
34
+ 2. The clarity of the generated code largely depends on the specificity of the prompt.
35
 
36
+ ## Training Details
37
 
38
+ ### Dataset
39
+ The model was fine-tuned on an undisclosed dataset comprised of a range of Solidity contracts.
40
 
41
+ ### Training Hyperparameters:
42
+ - Learning Rate: `5e-05`
43
+ - Train Batch Size: `4`
44
+ - Evaluation Batch Size: `4`
45
+ - Seed: `42`
46
+ - Optimizer: Adam (`betas=(0.9,0.999)`, `epsilon=1e-08`)
47
+ - Learning Rate Scheduler: Cosine with restarts
48
+ - Warmup Steps: `241`
49
+ - Epochs: `4`
50
 
51
+ ### Training Results:
 
 
 
 
 
 
 
 
52
 
 
53
 
54
  | Training Loss | Epoch | Step | Validation Loss |
55
  |:-------------:|:-----:|:-----:|:---------------:|
 
66
  | 0.28 | 2.28 | 11000 | 0.3127 |
67
 
68
 
69
+ ### Dependencies:
70
+ - Transformers: `4.31.0`
71
+ - Pytorch: `2.0.1+cu118`
72
+ - Datasets: `2.14.2`
73
+ - Tokenizers: `0.13.3`
74
+
75
+ ## How to Use
76
+ If you wish to use this model to generate Solidity contract code, follow the steps below:
77
+
78
+ ```python
79
+ from transformers import AutoTokenizer, AutoModelForCausalLM
80
+
81
+ # Load the model and tokenizer
82
+ tokenizer = AutoTokenizer.from_pretrained("ckandemir/gpt2-medium-finetuned-contract-gen")
83
+ model = AutoModelForCausalLM.from_pretrained("ckandemir/gpt2-medium-finetuned-contract-gen")
84
+
85
+ # Input your code prompt
86
+ input_text = "contract MyToken"
87
+ input_ids = tokenizer.encode(input_text, return_tensors='pt')
88
+ sample_output = model.generate(input_ids, do_sample=True, max_length=400, num_return_sequences=1, temperature=0.7)
89
+
90
+ # Decode and print the generated text
91
+ generated_text = tokenizer.decode(sample_output[0], skip_special_tokens=True)
92
+ print(generated_text)
93