Log inSign up
Red Hat AI
2,497 posts
Red Hat AI profile banner
@RedHat_AI

Red Hat AI

@RedHat_AI
Accelerating AI innovation with open platforms and community. The future of AI is open.
ai.redhat.com
Joined May 2018
2,091
Following
12.2K
Followers
RepliesRepliesRepostsRepostsMediaMediaArticlesArticles

Log in or sign up for X

See what’s happening and join the conversation

Continue with phone
or
Log in with username or email
Terms·Privacy·Cookies·Accessibility·Ads Info·© 2026 X Corp.
  • @RedHat_AI
    Red Hat AI
    @RedHat_AI
    17h
    Flow control is an admission layer for shared GPU inference. Before a request gets routed to a model server, it's either admitted or held. Under saturation, priority decides which class of traffic advances, fairness spreads dispatch turns across tenants so no one gets starved,
    Red Hat AI Inference Server
    llm-d flow control: Priority queuing for shared GPU inference | Red Hat Developer
    From developers.redhat.com
  • @RedHat_AI
    Red Hat AI
    @RedHat_AI
    Aug 27
    Reproducible inference benchmarking is harder than it sounds, and it's exactly what the ecosystem needs. Inference Perf, from the Kubernetes community, standardizes how inference performance is measured across model servers. Great to see it published in JOSS. Congrats to the
    @TerryTangYuan
    Yuan (Terry) Tang
    @TerryTangYuan
    Aug 27
    Excited to share that our accompanying paper for the Inference Perf project (under Kubernetes SIGs) has been published in the Journal of Open Source Software! Co-authors: Ashok Chandrasekar, Sachin Mathew Varghese, Jason Kramberger, Brendan Slabe, Chen Wang The project was
    1
  • @RedHat_AI
    Red Hat AI
    @RedHat_AI
    Aug 27
    Is your CPU:GPU ratio still built for training? In agentic workloads, 50-90% of end-to-end latency is CPU-side tool processing, not GPU math (Georgia Tech + Intel). That's pushing the CPU:GPU ratio from 1:8 in training toward 1:1, sometimes 4:1. And vLLM's CPU backend already
    redhat.com
    The CPU is back: Rethinking the CPU-GPU split for LLM inference
    Why agentic AI is driving the shift back to CPU inference.
    3
  • @RedHat_AI
    Red Hat AI
    @RedHat_AI
    Aug 25
    This is awesome. Claude Code and Codex running on open models with @vllm_project, no OpenAI API in the loop. Ask Qwen3.8 about a model released after its training cutoff and it has no idea. Give it server-side web search through the Agentic API and it finds the answer, with
    00:00
    6
  • @RedHat_AI
    Red Hat AI
    @RedHat_AI
    Aug 24
    vLLM Office Hours #56 recording is up: - What's new in vLLM 0.27 - Running Codex and Claude Code CLIs on open models - Speculators v0.6 and v0.7 update - GuideLLM v0.7.0: benchmarking reasoning, tool calling, and agentic traces Watch + see slides:
    [vLLM Office Hours #56] - GuideLLM v0.7: Benchmarking Reasoning and Agentic Traces - August 20, 2026
    From youtube.com