Spaces:

fb700
/

chatglm-fitness-RLHF

Sleeping

fb700 commited on Jul 26, 2023

Commit

74767ed

•

1 Parent(s): 70452e6

Update request_llm/bridge_chatglm.py

Files changed (1) hide show

request_llm/bridge_chatglm.py CHANGED Viewed

@@ -45,7 +45,7 @@ class GetGLMHandle(Process):
                     if device=='cpu':
                         self.chatglm_model = AutoModel.from_pretrained("fb700/chatglm-fitness-RLHF", trust_remote_code=True).float()
                     else:
-                        self.chatglm_model = AutoModel.from_pretrained("fb700/chatglm-fitness-RLHF", trust_remote_code=True).half().quantize(4).cuda()
                     self.chatglm_model = self.chatglm_model.eval()
                     break
                 else:

                     if device=='cpu':
                         self.chatglm_model = AutoModel.from_pretrained("fb700/chatglm-fitness-RLHF", trust_remote_code=True).float()
                     else:
+                        self.chatglm_model = AutoModel.from_pretrained("fb700/chatglm-fitness-RLHF", trust_remote_code=True).half().quantize(8).cuda()
                     self.chatglm_model = self.chatglm_model.eval()
                     break
                 else: