TensorZero is an open-source stack for industrial-grade LLM applications. It unifies an LLM gateway, observability, optimization, evaluation, and experimentation.

⚡ Python-free Rust inference server — OpenAI-API compatible. GGUF + SafeTensors, hot model swap, auto-discovery, single binary. FREE now, FREE forever.

LSP-AI is an open-source language server that serves as a backend for AI-powered functionality, designed to assist and empower software engineers, not replace them.

Instant, controllable, local pre-trained AI models in Rust

`llm-chain` is a powerful rust crate for building chains in large language models allowing you to summarise text and complete complex tasks

Believe in AI democratization. llama for nodejs backed by llama-rs, llama.cpp and rwkv.cpp, work locally on your laptop CPU. support llama/alpaca/gpt4all/vicuna/rwkv model.

LLama.cpp rust bindings

Like grep but for natural language questions. Based on Mistral 7B or Mixtral 8x7B.

A font for writing tiny stories

Production-Ready LLM Agent SDK for Every Developer

User friendly CLI tool for AI tasks. Stop thinking about LLMs and prompts, start getting results!

LLaMa 7b with CUDA acceleration implemented in rust. Minimal GPU memory needed!

bott: Your Terminal Copilot

Friendly interface to chat with an Ollama instance.

"Just hoof it!" - A spotlight like interface to Ollama

A collection of serverless apps that show how Fermyon's Serverless AI (currently in private beta) works. Reference: https://developer.fermyon.com/spin/serverless-ai-tutorial

DuckDuckGo AI to OpenAI API

Modular Rust transformer/LLM library using Candle

FastLLM - Blazingly Fast Rust based LLM Inference API 🚀

a GGUF file parser