The Public Standard for Real World AI Performance

Benchmark consisting of a weighted performance across finance and coding tasks. Showing the potential impact that LLMs can have on the economy.

Benchmark consisting of a weighted performance across finance, coding, and education tasks. Showing the potential impact that LLMs can have on the economy.

Comparing native provider search against independent web-search tools on legal-research and finance-analysis tasks

Tests an agent's ability to complete legal work using documents, spreadsheets, presentations, and file-system tools.

Evaluating language models on a wide range of open source legal reasoning tasks.

Evaluating agents on legal research tasks across diverse areas of US law

A private benchmark evaluating understanding of long-context credit agreements

Evaluating agents on Excel-based financial modeling tasks

Evaluating agents on core financial analyst tasks

Evaluating reading and understanding tax certificates as images

A Vals-created set of questions and responses to tax questions

Can models support the medical billing process?

Can models support doctors with their administrative work?

Can models write math proofs that are formally verified?

Graduate-level Google-Proof Q&A benchmark evaluating models on questions that require deep reasoning.

Academic multiple-choice benchmark covering 14 subjects including STEM, humanities, and social sciences.

Multimodal Multi-task Benchmark

Student Assessment with Generative Evaluation

Can language models reimplement working programs in another language?

International Olympiad in Informatics

Our Implementation of the LiveCodeBench benchmark

Can language models rebuild programs from scratch?

How important are skills for agents?

Solving production software engineering tasks

State-of-the-art set of difficult terminal-based tasks

Can models build web applications from scratch?

Can AI agents find security bugs and patch them?

Can an AI agent build and run a space program in Kerbal Space Program?

Which model can make the most money playing poker?

Can AI help people navigate SNAP benefits?