🚀 The feature, motivation and pitch

When I loaded the QWEN2.5-VL model fine-tuned by LoRa using vllm version 0.8.4, I encountered the following prompt:

Regarding multimodal models, vLLM currently only supports adding LoRA to language model, visual.blocks.31.mlp.up_proj will be ignored.

I found an issue #26422 with a similar problem, but it seems the PR hasn't been merged into master. How can I enable loading visual-side LORA parameters and using VLLM to accelerate inference?

Looking forward to your reply

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.