Pinned Loading

1. ggml-org/llama.cpp ggml-org/llama.cpp Public

   LLM inference in C/C++

   C++ 127k 22.6k

2. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 90.7k 21.5k

3. deepspeedai/DeepSpeed deepspeedai/DeepSpeed Public

   DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

   Python 43k 5k

4. sgl-project/sglang sgl-project/sglang Public

   SGLang is a high-performance serving framework for large language models and multimodal models.

   Python 33k 8.4k

5. triton-lang/triton triton-lang/triton Public

   Development repository for the Triton language and compiler

   MLIR 20.1k 3.2k

6. flashinfer-ai/flashinfer flashinfer-ai/flashinfer Public

   FlashInfer: Kernel Library for LLM Serving

   Python 6.3k 1.4k