feat: support GLM-5.3-Flash inference

[sufubao]

最终镜像

registry.ms-sc-01.maoshanwangtech.com/ms-ccr/lightllm@sha256:0260e9884e46de899f4b845aa3796d0603b7b6ee7593c1fded35b7cee4462169

H100 启动命令

IMAGE="registry.ms-sc-01.maoshanwangtech.com/ms-ccr/lightllm@sha256:0260e9884e46de899f4b845aa3796d0603b7b6ee7593c1fded35b7cee4462169"

sudo docker pull "$IMAGE"
sudo docker run -d \
  --name glm53-lightllm-16k256-kpool \
  --restart unless-stopped \
  --network host --ipc host --gpus all \
  -e LIGHTLLM_ENABLE_KPOOL_DECODE_FASTPATH=1 \
  -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
  --ulimit memlock=-1 --ulimit nofile=1048576:1048576 \
  -v /home/devsft/models/GLM-5.3-Flash:/model:ro \
  -v /home/devsft/cache-glm53-lightllm:/root/.cache \
  -v /home/devsft/cache-glm53-triton:/root/.triton \
  -v /home/devsft/cache-glm53-deep-gemm:/root/.deep_gemm \
  "$IMAGE"

1M 上下文启动命令

1M 上下文使用 CUDA 12.9 兼容专用镜像（H100 完整启动及 API 生成验证通过）：

registry.ms-sc-01.maoshanwangtech.com/ms-ccr/lightllm:glm-5.3-flash-260901-1m-cu129

不可变 Digest：sha256:668f82136bafc297a157112b775f1d7ac31665f48c461ce67f2ccb69e5bae15f

IMAGE_1M="registry.ms-sc-01.maoshanwangtech.com/ms-ccr/lightllm@sha256:668f82136bafc297a157112b775f1d7ac31665f48c461ce67f2ccb69e5bae15f"

sudo docker pull "$IMAGE_1M"
sudo docker run -d \
  --name glm53-lightllm-vl-1m-tp8-c256 \
  --restart unless-stopped \
  --network host --ipc host --gpus all \
  -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
  --ulimit memlock=-1 --ulimit nofile=1048576:1048576 \
  -v /home/devsft/models/GLM-5.3-Flash:/model:ro \
  -v /home/devsft/cache-glm53-lightllm:/root/.cache \
  -v /home/devsft/cache-glm53-triton-cu129:/root/.triton \
  -v /home/devsft/cache-glm53-deep-gemm-cu129:/root/.cache/deep_gemm \
  "$IMAGE_1M" \
  /opt/sglang/bin/python -m lightllm.server.api_server \
  --model_dir /model \
  --model_name glm-5.3-flash \
  --tp 8 \
  --host 0.0.0.0 \
  --port 8002 \
  --httpserver_workers 16 \
  --mem_fraction .90 \
  --max_total_token_num 1048612 \
  --running_max_req_size 256 \
  --max_req_total_len 1048576 \
  --batch_max_tokens 16384 \
  --chunked_prefill_size 1024 \
  --linear_att_ssm_data_type bfloat16 \
  --linear_att_cache_size 256 \
  --graph_max_batch_size 256 \
  --graph_split_batch_size 8 \
  --graph_grow_step_size 16 \
  --graph_max_len_in_batch 2048 \
  --disable_flashinfer_allreduce \
  --enable_fused_shared_experts \
  --disable_dynamic_prompt_cache \
  --disable_aggressive_schedule \
  --router_max_wait_tokens 64 \
  --max_image_pixels 6272000 \
  --max_image_token_count 8000 \
  --visual_tp 1 \
  --visual_dp 8 \
  --visual_infer_batch_size 8 \
  --cache_capacity 64 \
  --schedule_time_interval 0.001 \
  --prefill_coalesce_interval 0.5 \
  --reasoning_parser glm45 \
  --tool_call_parser glm47

16K 输入 / 256 输出

同一台 8×H100 80GB、同一 FP8 checkpoint、TP8、32K batch token、关闭 prompt cache；总吞吐单位为 tok/s。

并发	LightLLM	vLLM	LightLLM / vLLM
1	5,373.86	6,542.89	82.13%
8	16,444.11	17,635.33	93.25%
16	20,669.61	25,246.99	81.87%
64	25,614.05	31,485.12	81.35%
128	25,588.22	27,988.09	91.43%
256	25,623.93	31,206.94	82.11%

精度：GSM8K 99/100；精确 16,384-token needle PASS，找回 ZEBRA-4821。