testing-best-practices

An agent skill that enforces testing best practices when writing, reviewing, or improving tests. Built from 24 research documents covering real-world testing patterns across 16 GitHub accounts, two engineering organizations, books, and long-form testing literature; grounded in practitioner work from Kent Beck (TDD) to TigerBeetle (deterministic simulation), Jane Street (expect tests), Ward Cunningham (Fit/customer examples), and Salvatore Sanfilippo / antirez (Redis-style differential fuzzing and testability seams).

When an agent uses this skill, it produces higher-quality tests than it would on its own. Specifically:

• Property-based tests appear (Hypothesis, fast-check, proptest) where broad input spaces need more than examples
• Assertions get stronger: meaningful behavior/state/error checks replace toBeDefined(), truthy/not-empty checks, and logs
• Error-handling paths are exercised with injected downstream failures instead of only invalid-input tests
• Concurrency contracts are pinned with contention-driving tests and race-detector guidance rather than observational t.Log output
• Better oracles appear: shadow models, differential checks, statistical/recall thresholds, whole-state digest roundtrips, customer-readable examples, and pre-mask assertions when the risk boundary calls for them
• Assessments become structured with severity-prioritized findings (P0/P1/P2/P3) instead of ad-hoc lists
• Anti-patterns get detected: skipped/focused tests, logging-instead-of-asserting, mock-everything "integration" tests, weak assertions, sleeps, live-network tests, blind snapshot updates, and fault-masking assertions
• Language-appropriate patterns are used: table-driven tests in Go, #[test] modules in Rust, Vitest + fast-check in TypeScript, pytest + Hypothesis in Python

npx skills add adewale/testing-best-practices

Skills appear on skills.sh automatically after users install the repo with the skills CLI. Install counts and leaderboard rankings come from anonymous CLI telemetry; opt out with DISABLE_TELEMETRY=1. The repo page customization in skills.sh.json is picked up after the repository is seen by telemetry and the cache refreshes.

The installable skill directory is testing-best-practices. It uses the Agent Skills SKILL.md format and is configured for Codex, OpenCode, Pi, Gemini CLI, and Claude Code.

After installation, ask your coding agent to use the skill when tests are being written, reviewed, or repaired:

Use testing-best-practices to review these tests for false confidence, mock drift, flaky timing, weak assertions, and missing error paths.

The skill routes itself by task mode (write, assess, upgrade, detect), detects the repository language/framework, then loads only the relevant reference files.

The skill operates in four modes:

Language-specific guidance loads on demand based on the project's language. Advanced pattern references load only when their trigger condition matches (e.g., references/characterization-testing.md loads only when refactoring legacy code). This keeps token usage efficient.

• Red-Green-Refactor TDD, with honest red-vs-green evidence reporting
• Test quality over quantity (Kent Beck's Test Desiderata, assertion strength, coverage as a map rather than proof)
• Real behavior over mocks, with a preference hierarchy from real local objects to fakes, stubs, and only then framework mocks
• Property-based testing for broad input spaces and invariants
• Error-handling path testing via injected downstream failures, timeouts, partial responses, and I/O errors
• Concurrency contract testing under contention and race detectors/thread sanitizers
• E2E, integration, contract, documentation-code sync, and external-boundary testing at the smallest useful tier
• Characterization testing for legacy refactors
• Differential, shadow-model, statistical-oracle, pirate/conformance, golden/snapshot, VCR/recorded-fixture, exhaustive, and mutation-style testing when the risk boundary calls for them
• Test data builders, customer-readable examples, fixtures, and logging fakes that keep test intent visible
• Correctness by construction — types/schemas/contracts over repeated runtime checks; invariant-proof tests plus model-gap tests
• Validation loop before reporting done, including checks for weak assertions, skips, sleeps, live network, mock drift, fault masking, and unverified TDD claims

Language-specific (one loaded per project):

Always available:

Topic-specific (loaded only when the trigger matches):

The project now uses layered evals rather than a single public prompt table:

Run the full local non-LLM gate with:

python3 skill-development/scripts/check-all.py

Eval definitions, rubrics, schema, scorecards, and health tracking live under skill-development/evals/; historical iteration outputs live under testing-best-practices-workspace/.

Built from analysis of testing patterns across 16 GitHub accounts and two engineering organizations, backed by 24 Markdown research files:

• kentbeck -- Test Desiderata (12 properties of good tests), TCR, MoneyPython; books: TDD: By Example (red/green/refactor, test list, green-bar strategies, testing patterns), XP Explained (test-what-might-break, 100% rule, ten-minute build), Tidy First? (behavior vs. structure changes)
• npryce (GOOS co-author) -- factcheck, snodge, make-it-easy, worktorule; GOOS book: walking skeleton, double feedback loop, listen-to-your-tests, mock roles not objects, allow queries / expect commands
• graydon (Rust creator) -- exhaustigen-rs, proptest-arbitrary-interop
• karpathy -- differential testing against PyTorch/tiktoken
• kepano -- fixture-based golden file testing (defuddle)
• maryrosecook -- gitlet test suite, file tree builders
• simonw (datasette, sqlite-utils, llm) -- documentation-as-tests, VCR cassettes
• bradfitz (Go team) -- protocol-faithful fake servers
• joewalnes -- minimalist testing frameworks
• ivanmoore -- TDD katas, mock object exercises
• adewale -- property-based testing, mock fidelity, test quality audits
• chrischabot -- 5-tier test architecture, API scenario tests
• tirsen -- retry patterns
• antirez (Redis creator) -- differential fuzzing vs. a reference oracle, digest roundtrips, DEBUG as a testability surface, recall testing for approximate algorithms, agent-as-QA
• danluu -- random/coverage-guided generation thesis, "world's dumbest fuzzer" (Fuzz.jl: seed/try-catch-log/denylist harness mechanics), differential testing (csv), fault injection (fs-errors), solver-backed generation (kodkod-clj, secvisor-formal-verification)
• WardCunningham (wiki/Fit/CRC inventor) -- Fit customer-readable example tables, CHECKS validation patterns, domain DSLs in test suites (wiki-client), characterization harnesses (sudokuku), test-investment-tracks-lifetime, "preserve and protect [the test suite] as if it were code" (EPISODES)
• tigerbeetle -- deterministic simulation, state-machine invariants, fault injection, testability by design
• janestreet -- expect tests, library-level simulation, Time_source, Bonsai testing

Research documents are in research/: practitioner/account/org lesson notes plus methodology and cross-cutting literature, antipattern, decision-tree, novel-testing, and correctness-by-construction notes. The research process itself is captured in research/METHODOLOGY.md — a minimum checklist (GitHub history including actual test files and commit authorship, books, long-form writing, talks/interviews, pre-GitHub tools, post-mortems and criticism), explicitly framed as a floor rather than a ceiling.

testing-best-practices/             # The installable skill (ships to agents)
  SKILL.md                          # Core instructions (~240 lines)
  references/                       # Loaded on demand
    python.md                       # Language: Python / pytest / Hypothesis
    typescript.md                   # Language: TypeScript/JavaScript / Vitest/Jest/fast-check/Playwright
    go.md                           # Language: Go
    rust.md                         # Language: Rust
    antipatterns.md                 # Always: detection signals + fixes
    test-types.md                   # Always: decision guide and test-tier hierarchy
    characterization-testing.md     # Topic: legacy code
    correctness-by-construction.md  # Topic: types/schemas/contracts vs tests
    deterministic-time.md           # Topic: clock injection, time virtualization, async/background seams
    differential-testing.md         # Topic: reference implementations, ports, SDKs, shadow/statistical oracles
    doc-sync-testing.md             # Topic: documentation drift
    exhaustive-testing.md           # Topic: small state spaces
    golden-file-testing.md          # Topic: transformation pipelines, snapshot tests, digest roundtrips
    mathematical-properties.md      # Topic: algebraic laws
    mutation-testing.md             # Topic: test quality verification and PIE/fault propagation
    test-data-builders.md           # Topic: factories, fixtures, and customer-readable examples
    vcr-cassettes.md                # Topic: external APIs and recorded fixtures

research/                           # Source material (does not ship)
  METHODOLOGY.md                    # How we research a practitioner (a floor, not a ceiling)
  LESSONS_FROM_*.md                 # One file per practitioner/account/org
  DESIGN_FOR_TESTABILITY_LITERATURE.md
  ANTIPATTERNS.md
  CORRECTNESS_BY_CONSTRUCTION.md
  DECISION_TREE.md
  NOVEL_TESTING_TYPES.md

evals/                              # Public/shared prompt eval assets
  evals.json                        # 12 public prompt evals across 4 languages
  shared-benchmark.json             # 36 shared benchmark cases
  shared-harness.md                 # Shared harness contract
  files/                            # Fixture code for public eval prompts
  fixtures/                         # Runnable prompt fixtures
  oracles/                          # Shared fixture oracle helpers

skill-development/                  # Development-only evals and quality gates
  evals/                            # 54 rubric evals, schema, scorecard, health plan, fixture oracles
  scripts/check-all.py              # Runs all local non-LLM gates
  scripts/                          # Static audit, oracle runners, mini-repos, prompt-eval runner, scoring tools

skills.sh.json                      # skills.sh presentation metadata
skills-lock.json                    # skills CLI lockfile
testing-best-practices-workspace/   # Historical eval results by iteration

Run all local checks before proposing skill changes:

python3 skill-development/scripts/check-all.py

That command runs the static audit, eval-shape checks, fixture oracle self-tests, mutation-backed mini-repos, eval-health report, best-practices audit, and skill version scoring. Generated prompt runs and caches should stay out of the installable skill directory.

• This is an agent skill, not a standalone test runner, assertion library, or framework.
• It works generically with any language, but detailed on-demand references currently cover Python, TypeScript/JavaScript, Go, and Rust.
• The local gate is non-LLM and deterministic; prompt-output quality still needs periodic review with hidden/rotating evals as model behavior changes.

MIT