neuralmagic
/

DeepSeek-R1-Distill-Qwen-1.5B-FP8-dynamic

Text Generation

text-generation-inference

Inference Endpoints

compressed-tensors

Model card Files Files and versions Community

nm-research commited on 14 days ago

Commit

134fd5e

·

verified ·

1 Parent(s): 1c0ff9d

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -113,7 +113,7 @@ OpenLLM Leaderboard V1:
 ```
 lm_eval \
   --model vllm \
-  --model_args pretrained="neuralmagic/DeepSeek-R1-Distill-Qwen-1.5B-FP8-Dynamic",dtype=auto,max_model_len=4096,tensor_parallel_size=1,enable_chunked_prefill=True \
   --tasks openllm \
   --write_out \
   --batch_size auto \
@@ -125,7 +125,7 @@ OpenLLM Leaderboard V2:
 ```
 lm_eval \
   --model vllm \
-  --model_args pretrained="neuralmagic/DeepSeek-R1-Distill-Qwen-1.5B-FP8-Dynamic",dtype=auto,max_model_len=4096,tensor_parallel_size=1,enable_chunked_prefill=True \
   --apply_chat_template \
   --fewshot_as_multiturn \
   --tasks leaderboard \

 ```
 lm_eval \
   --model vllm \
+  --model_args pretrained="neuralmagic/DeepSeek-R1-Distill-Qwen-1.5B-FP8-dynamic",dtype=auto,max_model_len=4096,tensor_parallel_size=1,enable_chunked_prefill=True \
   --tasks openllm \
   --write_out \
   --batch_size auto \
 ```
 lm_eval \
   --model vllm \
+  --model_args pretrained="neuralmagic/DeepSeek-R1-Distill-Qwen-1.5B-FP8-dynamic",dtype=auto,max_model_len=4096,tensor_parallel_size=1,enable_chunked_prefill=True \
   --apply_chat_template \
   --fewshot_as_multiturn \
   --tasks leaderboard \