Spaces:

mlabonne
/

OrpoLlama-3-8B

Running on Zero

mlabonne commited on Apr 20, 2024

Commit

cc449f6

verified ·

1 Parent(s): 7c3c6a2

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -75,7 +75,7 @@ tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
 model = AutoModelForCausalLM.from_pretrained(
     MODEL_ID,
     device_map="auto",
-    quantization_config=quantization_config,
     attn_implementation="flash_attention_2",
 )

 model = AutoModelForCausalLM.from_pretrained(
     MODEL_ID,
     device_map="auto",
+    # quantization_config=quantization_config,
     attn_implementation="flash_attention_2",
 )