Skip to content

feat(embedding): add WeMM-Embedding support - #5439

Merged
qinxuye merged 3 commits into
xorbitsai:mainfrom
Minamiyama:models/embedding/wemm-embedding
Aug 29, 2026
Merged

feat(embedding): add WeMM-Embedding support#5439
qinxuye merged 3 commits into
xorbitsai:mainfrom
Minamiyama:models/embedding/wemm-embedding

Conversation

@Minamiyama

Copy link
Copy Markdown
Collaborator

Support multimodal text, image, video, and interleaved inputs through sentence-transformers and vLLM with Hugging Face and ModelScope sources.

Summary

Add built-in support for Tencent's multimodal WeMM-Embedding model family:

  • WeMM-Embedding-2B
  • WeMM-Embedding-4B
  • WeMM-Embedding-9B

Models are available from:

  • Hugging Face, revision main
  • ModelScope, revision master

Supported embedding engines are intentionally limited to existing Xinference backends:

  • sentence_transformers
  • vllm

Model Metadata

Model Dimensions Max tokens
WeMM-Embedding-2B 2048 262144
WeMM-Embedding-4B 2560 262144
WeMM-Embedding-9B 4096 262144
Support multimodal text, image, video, and interleaved inputs through sentence-transformers and vLLM with Hugging Face and ModelScope sources.
@XprobeBot XprobeBot added this to the v3.x milestone Aug 29, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for the WeMM-Embedding model family (2B, 4B, and 9B) in Xinference, enabling multimodal inputs (text, image, and video) across both the sentence_transformers and vllm engines. It updates API schemas, restful clients, and core embedding logic to support structured multimodal inputs, and includes a PyAV-based video reader fallback. The review feedback correctly points out that the specified dependency versions for sentence-transformers (>=5.7.0) and transformers (>=5.2.0) do not exist on PyPI and must be updated to valid, existing versions.

Comment thread xinference/model/embedding/sentence_transformers/core.py
Comment thread xinference/model/embedding/model_spec.json Outdated

@qinxuye qinxuye left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qinxuye
qinxuye merged commit 491bd25 into xorbitsai:main Aug 29, 2026
14 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3 participants