Community benchmark database for running LLMs on Apple Silicon Macs
-
Updated
Apr 22, 2026 - Shell
#
llm-benchmark
Here are 164 public repositories matching this topic...
[ICCV 2025] AdsQA: Towards Advertisement Video Understanding Arxiv: https://arxiv.org/abs/2509.08621
benchmark video-understanding video-question-answering advertisement-dataset video-llms llm-benchmark
-
Updated
Oct 30, 2025 - Python
An Open Benchmark for AI in Cybersecurity Operations
cybersecurity threat-hunting detection-engineering soc-analyst llm-evaluation llm-agents agentic-ai reasoning-models llm-benchmark
-
Updated
Jun 16, 2026 - Python
ICRTL Benchmark: Industrial-level RTL design challenges for evaluating PPA optimization, code generation, and LLM applications in EDA.
benchmark evolution natural-language ppa eda rtl verilog system-verilog yosys ic-design vlsi-design rtl-design hardware-accelerator ai-agent verilogeval rtllm llm-benchmark spec-to-rtl
-
Updated
Jul 1, 2026 - SystemVerilog
Benchmark LLMs on real professional tasks, not academic puzzles. YAML-driven experiment pipeline + live React dashboard for GDPVal Gold Subset (220 tasks across 11 industries).
microsoft dashboard evaluation openai mlops vite github-actions huggingface yaml-pipeline azure-openai ai-evaluation code-interpreter gpt-5 anthropic benchmark-automation llm-benchmark gdpval real-world-tasks self-qa professional-tasks
-
Updated
Jul 1, 2026 - Python
The AI-native wire format for structured data. 100% comprehension on every frontier model. 50-92% fewer tokens than JSON. 43B+ lossless round-trips across 17 formats. Spec v3.2 Stable.
mcp gemini tokens openai data-serialization structured-data gcf json-alternative ai-agents wire-format agent-communication llm anthropic model-context-protocol mcp-tools token-optimization token-efficiency llm-benchmark toon-alternative graph-compact-format
-
Updated
Jul 1, 2026
Открытый бенчмарк LLM: какая нейросеть лучше пишет код 1С:Предприятие (BSL). Объективная оценка LLM по методике SMOP с реальным исполнением в 1С — Claude, GPT, Gemini, DeepSeek, YandexGPT, GigaChat.
benchmark ai evaluation code-generation code-quality onescript bsl 1c 1c-enterprise llm ai-code-generation agentic function-calling llm-evaluation smop llm-benchmark
-
Updated
Jul 1, 2026 - Python
Benchmark for evaluating AI epistemic reliability - testing how well LLMs handle uncertainty, avoid hallucinations, and acknowledge what they don't know.
python nlp benchmark machine-learning ai evaluation collaborate ai-safety llm llm-evaluation hallucination-detection ai-benchmark llm-benchmark
-
Updated
Mar 22, 2026 - Python
An agent evaluation framework with native multi-turn feedback iteration.
-
Updated
Apr 29, 2026 - Python
[ICML 2026] CapBencher toolkit: Give your LLM benchmark a built-in alarm for leakage and gaming
-
Updated
May 29, 2026 - Python
Benchmark abierto en español de 141 LLMs (89 con 13K+ runs reales y juez Phi-4 independiente). Quality, costo, velocidad, long-context y fuga de credenciales como dimensiones separadas. Alternativas a Claude, GPT y Gemini para agentes n8n/OpenClaw. Calculadora interactiva con tus propios pesos.
ai-agents startup-tools n8n ai-models emprendedores openrouter ollama llm-evaluation llm-comparison ai-benchmark llm-benchmark spanish-ai openclaw hermes-agent claude-alternatives gpt-alternatives phi4-judge benchmark-en-espanol emprendedores-ia
-
Updated
Jul 1, 2026 - Python
The open-source benchmark for LLM memory decay. Measure how Naive, RAG, Chunked RAG, Cascading, and SummaryMemory degrade over 100 conversation turns. Ebbinghaus forgetting curves, 5-provider LLM eval, multi-seed CI. No API key needed.
python nlp benchmarking machine-learning memory chatbot evaluation rag ebbinghaus streamlit ai-evaluation large-language-models llm conversation-memory rag-evaluation llm-memory llm-benchmark temporal-decay memory-decay memory-evaluation
-
Updated
Jun 8, 2026 - Python
Daily LLM value rankings - compare 300+ models by intelligence, speed and price. OpenRouter + Artificial Analysis. 大模型性价比排行榜
github-pages benchmark machine-learning static-site artificial-intelligence gemini gpt price-comparison claude model-comparison ai-tools large-language-models llm generative-ai openrouter deepseek llm-benchmark value-for-money ai-ranking token-pricing
-
Updated
Jul 1, 2026 - JavaScript
Testing how well LLMs can solve jigsaw puzzles
-
Updated
Jan 8, 2026 - Python
Community-driven behavioral reliability benchmark for LLMs. 231 probes across 19 modules, deterministic scoring, perplexity correlation, layer sensitivity mapping, quant method capture, hardware-stratified community rankings. Every test contributes to the community dataset.
benchmark mcp community-driven model-evaluation ai-evaluation llm ollama sycophancy hallucination-detection llm-testing hardware-benchmark ai-trust trust-scoring behavioral-testing llm-benchmark deterministic-scoring
-
Updated
May 4, 2026 - Python
Self-hosted LLM API benchmark, monitoring & playground. Compare latency, TTFT, throughput across OpenAI, Anthropic, Gemini & any OpenAI-compatible endpoint. Deploy with one command via Docker. | 自托管 LLM API 性能测试、监控与调试平台，一键 Docker 部署，支持多家服务商对比。
docker latency self-hosted gemini openai performance-testing api-monitoring anthropic llm-playground llm-api llm-benchmarking llm-monitoring api-benchmark llm-benchmark
-
Updated
May 15, 2026 - TypeScript
🚀 A modern, production-ready refactor of the LoCoMo long-term memory benchmark.
refactoring refactor evaluation-framework locomo long-term-memory openai-api long-context llm-benchmark
-
Updated
Feb 12, 2026 - Python
Open-source multi-agent AI debate arena: pit Claude, GPT, Gemini, Ollama & HuggingFace models against each other with frozen-context fairness, evidence-first judging, 20+ personas, code review, and PDF/Markdown reports. CLI + Web UI.
python agent open-source ai chatbot multi-agent gemini code-review debate gpt claude model-comparison persona evidence-based huggingface llm ollama ai-debate llm-benchmark ai-arena
-
Updated
Apr 10, 2026 - Python
RetardBench is an open, no-censorship benchmark that ranks large language models purely on how retarded they are.
jailbreak large-language-models llm prompt-injection red-teaming-tools ollama llm-evaluation uncensored-llm open-llm-leaderboard llm-jailbreaks prompt-injection-llm-security ai-benchmark ai-red-teaming llm-benchmark
-
Updated
Mar 2, 2026 - TypeScript
Local LLM BenchMarking
react benchmarking ai self-hosted openai-api ai-tools llm llm-agent local-llm llm-tools local-ai ollama llm-benchmarking openrouter-api llm-benchmark anthrophic-api
-
Updated
Jun 28, 2026 - TypeScript
Improve this page
Add a description, image, and links to the llm-benchmark topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the llm-benchmark topic, visit your repo's landing page and select "manage topics."