Spaces:

pcalhoun
/

Convert-Modern-English-to-Jonathan-Swift-Style

Sleeping

pcalhoun commited on Nov 13, 2024

Commit

7c0f254

verified ·

1 Parent(s): 988572d

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -26,7 +26,7 @@ model = AutoModelForCausalLM.from_pretrained(
     base_model_repo_and_name,
     torch_dtype=torch.bfloat16,
     device_map="auto",
-    load_in_4bit=True,
     token=os.environ["HF_TOKEN"],
 )

     base_model_repo_and_name,
     torch_dtype=torch.bfloat16,
     device_map="auto",
+    load_in_8bit=True,
     token=os.environ["HF_TOKEN"],
 )