Run GGUF models easily with a KoboldAI UI. One File. Zero Install.

High-speed Large Language Model Serving for Local Deployment

Lightweight inference library for ONNX files, written in C++. It can run Stable Diffusion XL 1.0 on a RPI Zero 2 (or in 298MB of RAM) but also Mistral 7B on desktops and servers. ARM, x86, WASM, RISC-V supported. Accelerated by XNNPACK. Python, C# and JS(WASM) bindings available.

🤘 TT-NN operator library, and TT-Metalium low level kernel programming model.

Fast Multimodal LLM on Mobile Devices

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

A highly optimized LLM inference acceleration engine for Llama and its variants.

Yet Another Language Model: LLM inference in C++/CUDA, no libraries except for I/O

A high-performance inference system for large language models, designed for production environments.

Run LLMs on AMD Ryzen™ AI NPUs in minutes. Just like Ollama - but purpose-built and deeply optimized for the AMD NPUs.

CPU inference for the DeepSeek family of large language models in C++

Kolosal AI is an OpenSource and Lightweight alternative to LM Studio to run LLMs 100% offline on your device.

An Unreal Engine plugin for LLM/GenAI models & MCP UE5 server. Supports Claude Desktop App, Windsurf & Cursor, also includes OpenAI's GPT 5, Deepseek V3.1, Claude Sonnet 4 APIs and Grok 4, with plans to add Gemini, audio & realtime APIs soon. UnrealMCP is also here!! Automatic blueprint and scene generation from AI!!

Swift library to work with llama and other large language models.

llama.cpp (GGUF LLMs) and llava.cpp (GGUF VLMs) for ROS 2

Simple chat program for LLaMa, GPT-J, and MPT models.

Modern, Header-only C++ bindings for the Ollama API.

LLaVA server (llama.cpp).

Booster - open accelerator for LLM models. Better inference and debugging for AI hackers