Issue Deploying InternVL 3.5 with LMDeploy
#1
by
megladagon
- opened
I tried deploying the model with lmdeploy serve as mentioned in the README, but ran into issues when loading the model. I also checked the official LMDeploy docs, and it seems there isn’t any integration yet for InternVL 3.5.
Does anyone know if support is planned and when we might expect an LMDeploy release for InternVL 3.5?
Please deploy the model using the command below. Ensure that you install lmdeploy>=0.9.1 and include the option --backend pytorch. Our initial README omitted this option, which led to some errors, and we sincerely apologize for this oversight.
lmdeploy serve api_server OpenGVLab/InternVL3_5-8B --server-port 23333 --tp 1 --backend pytorch