KeyError: 'lm_head.weight' when using sglang to load this model
#8
by
aqweteddy
- opened
Are there any changes to the architecture of this model compared to Qwen?
full error message:
[2025-01-21 05:11:11 DP1 TP0] Scheduler hit an exception: Traceback (most recent call last):
File "/root/miniconda3/envs/eval/lib/python3.10/site-packages/sglang/srt/managers/scheduler.py", line 1747, in run_sch
eduler_process
scheduler = Scheduler(server_args, port_args, gpu_id, tp_rank, dp_rank)
File "/root/miniconda3/envs/eval/lib/python3.10/site-packages/sglang/srt/managers/scheduler.py", line 239, in __init__
self.tp_worker = TpWorkerClass(
File "/root/miniconda3/envs/eval/lib/python3.10/site-packages/sglang/srt/managers/tp_worker_overlap_thread.py", line 6
3, in __init__
self.worker = TpModelWorker(server_args, gpu_id, tp_rank, dp_rank, nccl_port)
File "/root/miniconda3/envs/eval/lib/python3.10/site-packages/sglang/srt/managers/tp_worker.py", line 68, in __init__
self.model_runner = ModelRunner(
File "/root/miniconda3/envs/eval/lib/python3.10/site-packages/sglang/srt/model_executor/model_runner.py", line 185, in
__init__
self.load_model()
File "/root/miniconda3/envs/eval/lib/python3.10/site-packages/sglang/srt/model_executor/model_runner.py", line 303, in
load_model
self.model = get_model(
File "/root/miniconda3/envs/eval/lib/python3.10/site-packages/sglang/srt/model_loader/__init__.py", line 22, in get_mo
del
return loader.load_model(
File "/root/miniconda3/envs/eval/lib/python3.10/site-packages/sglang/srt/model_loader/loader.py", line 362, in load_mo
del
model.load_weights(self._get_all_weights(model_config, model))
File "/root/miniconda3/envs/eval/lib/python3.10/site-packages/sglang/srt/models/qwen2.py", line 395, in load_weights
param = params_dict[name]
KeyError: 'lm_head.weight'
well, I just realized the lm_head.weight is also included in the safetensor file, where it does not agree with the config tie_weights=true. I just fixed this issue in the latest commit, please give it a try. Thank you for your report, that help us a lot.
aqweteddy
changed discussion status to
closed