yujiepan commited on
Commit
7b752f3
1 Parent(s): 78e0f55

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +50 -0
README.md ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: text-generation
3
+ inference: true
4
+ widget:
5
+ - text: 'Hello!'
6
+ example_title: Hello world
7
+ group: Python
8
+ library_name: transformers
9
+ ---
10
+
11
+ This model is randomly initialized, using the config from [databricks/dbrx-instruct](https://huggingface.co/databricks/dbrx-instruct) but with smaller size.
12
+ Note the model is in float16.
13
+
14
+ Codes:
15
+ ```python
16
+ import transformers
17
+ import torch
18
+ import os
19
+ from huggingface_hub import create_repo, upload_folder
20
+
21
+ source_model_id = 'databricks/dbrx-instruct'
22
+ save_path = '/tmp/yujiepan/dbrx-tiny-random'
23
+ repo_id = 'yujiepan/dbrx-tiny-random'
24
+
25
+ config = transformers.AutoConfig.from_pretrained(
26
+ source_model_id, trust_remote_code=True)
27
+ config.attn_config.kv_n_heads = 2
28
+ config.d_model = 4
29
+ config.ffn_config.ffn_hidden_size = 8
30
+ config.n_heads = 4
31
+ config.n_layers = 2
32
+
33
+ model = transformers.AutoModelForCausalLM.from_config(
34
+ config, trust_remote_code=True)
35
+ model = model.half()
36
+ model.save_pretrained(save_path)
37
+
38
+ tokenizer = transformers.AutoTokenizer.from_pretrained(
39
+ source_model_id, trust_remote_code=True)
40
+ tokenizer.save_pretrained(save_path)
41
+
42
+ result = transformers.pipelines.pipeline(
43
+ 'text-generation',
44
+ model=model.float(), tokenizer=tokenizer)('Hello')
45
+ print(result)
46
+
47
+ os.system(f'ls -alh {save_path}')
48
+ create_repo(repo_id, exist_ok=True)
49
+ upload_folder(repo_id=repo_id, folder_path=save_path)
50
+ ```