Skip to content

perf(core): fix query waterfalls — getTerm parallelism, hydrated terms reuse, menu pattern caching - #1409

Merged
ascorbic merged 2 commits into
mainfrom
perf/query-waterfalls
Jun 11, 2026
Merged

perf(core): fix query waterfalls — getTerm parallelism, hydrated terms reuse, menu pattern caching#1409
ascorbic merged 2 commits into
mainfrom
perf/query-waterfalls

Conversation

@ascorbic

@ascorbic ascorbic commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

What does this PR do?

Three query-waterfall fixes found while investigating uncached-request latency (production pages show 5–7 fully serialized queries at 15–40ms each on D1):

  1. getTerm() parallelization — the usage-count and children queries both depend only on the already-fetched term row but ran sequentially. They now run under Promise.all: one round trip saved on every tag/category archive page. Locale filtering on the children query is unchanged (content_taxonomies is keyed by translation_group and has no locale column, so the count query has no locale concern).

  2. Reuse hydrated terms in templatesgetEmDashEntry/getEmDashCollection already hydrate entry.data.terms (same join, collection and locale filtering as getEntryTerms), but 10 first-party templates/demos re-queried terms per entry anyway. They now read post.data.terms?.tag ?? [] directly — one query saved per post-detail page (two in the starter, which refetched tag and category). Supporting change: the type generator (zod-generator.ts) now emits terms?: Record<string, TaxonomyTerm[]> alongside the existing bylines field (additive), and the touched sites' checked-in emdash-env.d.ts files are regenerated. Note: hydrated terms are label-sorted, so tag display order may change where the old per-entry query used a different order.

  3. Menu collection-pattern lookup request-cachedbuildMenuTree queried _emdash_collections.url_pattern per getMenu() call; a page rendering header + footer menus paid it twice. The lookup is now wrapped in requestCached keyed by the sorted slug set.

Query-count snapshots will change (decreases only) — intentionally not regenerated here; CI auto-updates them on the PR. Please review that the snapshot diff shows only decreases.

Closes #

Type of change

  • Bug fix
  • Feature (requires maintainer-approved Discussion)
  • Refactor (no behavior change)
  • Translation
  • Documentation
  • Performance improvement
  • Tests
  • Chore (dependencies, CI, tooling)

Checklist

  • I have read CONTRIBUTING.md
  • pnpm typecheck passes (packages, templates, demos, fixtures)
  • pnpm lint passes
  • pnpm test passes (or targeted tests for my change) — unit 175 files / 2,640 tests; integration 70 files / 1,039 tests
  • pnpm format has been run
  • I have added/updated tests for my changes (if applicable)
  • User-visible strings in the admin UI are wrapped for translation (if applicable). n/a — no admin UI strings.
  • I have added a changeset (if this PR changes a published package)
  • New features link to an approved Discussion: https://github.com/emdash-cms/emdash/discussions/... n/a — performance improvement.

AI-generated code disclosure

  • This PR includes AI-generated code — model/tool: Claude Fable 5 (Claude Code)

Screenshots / test output

New tests (TDD): get-term.test.ts (dialect-agnostic via describeEachDialect; behavior lock + exact 3-query budget via a counting Kysely plugin), menu-request-cache.test.ts (two getMenu calls → one url_pattern query; failed at 2 queries before the fix), zod-generator terms emission. Known pre-existing failure unrelated to this PR: demos/preview astro check fails on duplicated astro versions in node_modules (identical on main).

🤖 Generated with Claude Code


Try this PR

Open a fresh playground →

A full working EmDash site, deployed from this branch. Each visit gets its own session-scoped sandbox: no login needed and no shared state. Try the admin, edit content, hit the public site.

Tracks perf/query-waterfalls. Updated automatically when the playground redeploys.

- getTerm() runs its usage-count and children queries under Promise.all
  (both depend only on the term row), saving a round trip on every
  tag/category archive page.
- getMenu() request-caches the _emdash_collections url_pattern lookup,
  so pages rendering multiple menus only pay for it once per request.
- Templates/demos no longer call getEntryTerms() per post detail page:
  getEmDashEntry already hydrates entry.data.terms (keyed by taxonomy
  name) via getAllTermsForEntries. The type generator now emits the
  hydrated terms field so templates can use it type-safely, and the
  checked-in emdash-env.d.ts files for the touched sites are updated
  to match.

Query-count snapshots will change in CI (fewer queries per route).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@changeset-bot

changeset-bot Bot commented Jun 11, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 1893420

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 14 packages
Name Type
emdash Patch
@emdash-cms/cloudflare Patch
@emdash-cms/sandbox-workerd Patch
@emdash-cms/fixture-perf-site Patch
@emdash-cms/perf-demo-site Patch
@emdash-cms/cache-demo-site Patch
@emdash-cms/admin Patch
@emdash-cms/auth Patch
@emdash-cms/blocks Patch
@emdash-cms/gutenberg-to-portable-text Patch
@emdash-cms/x402 Patch
create-emdash Patch
@emdash-cms/auth-atproto Patch
@emdash-cms/plugin-embeds Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 11, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
emdash-demo-cache 1893420 Jun 11 2026, 02:36 PM
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 11, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
docs 1893420 Jun 11 2026, 02:35 PM
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 11, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
emdash-playground 1893420 Jun 11 2026, 02:36 PM
@github-actions

Copy link
Copy Markdown
Contributor

Scope check

This PR changes 609 lines across 27 files. Large PRs are harder to review and more likely to be closed without review.

If this scope is intentional, no action needed. A maintainer will review it. If not, please consider splitting this into smaller PRs.

See CONTRIBUTING.md for contribution guidelines.

@pkg-pr-new

pkg-pr-new Bot commented Jun 11, 2026

Copy link
Copy Markdown

Open in StackBlitz

@emdash-cms/admin

npm i https://pkg.pr.new/@emdash-cms/admin@1409

@emdash-cms/auth

npm i https://pkg.pr.new/@emdash-cms/auth@1409

@emdash-cms/auth-atproto

npm i https://pkg.pr.new/@emdash-cms/auth-atproto@1409

@emdash-cms/blocks

npm i https://pkg.pr.new/@emdash-cms/blocks@1409

@emdash-cms/cloudflare

npm i https://pkg.pr.new/@emdash-cms/cloudflare@1409

@emdash-cms/contentful-to-portable-text

npm i https://pkg.pr.new/@emdash-cms/contentful-to-portable-text@1409

emdash

npm i https://pkg.pr.new/emdash@1409

create-emdash

npm i https://pkg.pr.new/create-emdash@1409

@emdash-cms/gutenberg-to-portable-text

npm i https://pkg.pr.new/@emdash-cms/gutenberg-to-portable-text@1409

@emdash-cms/plugin-cli

npm i https://pkg.pr.new/@emdash-cms/plugin-cli@1409

@emdash-cms/plugin-types

npm i https://pkg.pr.new/@emdash-cms/plugin-types@1409

@emdash-cms/registry-client

npm i https://pkg.pr.new/@emdash-cms/registry-client@1409

@emdash-cms/registry-lexicons

npm i https://pkg.pr.new/@emdash-cms/registry-lexicons@1409

@emdash-cms/sandbox-workerd

npm i https://pkg.pr.new/@emdash-cms/sandbox-workerd@1409

@emdash-cms/x402

npm i https://pkg.pr.new/@emdash-cms/x402@1409

@emdash-cms/plugin-ai-moderation

npm i https://pkg.pr.new/@emdash-cms/plugin-ai-moderation@1409

@emdash-cms/plugin-atproto

npm i https://pkg.pr.new/@emdash-cms/plugin-atproto@1409

@emdash-cms/plugin-audit-log

npm i https://pkg.pr.new/@emdash-cms/plugin-audit-log@1409

@emdash-cms/plugin-color

npm i https://pkg.pr.new/@emdash-cms/plugin-color@1409

@emdash-cms/plugin-embeds

npm i https://pkg.pr.new/@emdash-cms/plugin-embeds@1409

@emdash-cms/plugin-field-kit

npm i https://pkg.pr.new/@emdash-cms/plugin-field-kit@1409

@emdash-cms/plugin-forms

npm i https://pkg.pr.new/@emdash-cms/plugin-forms@1409

@emdash-cms/plugin-webhook-notifier

npm i https://pkg.pr.new/@emdash-cms/plugin-webhook-notifier@1409

commit: 1893420

@ascorbic
ascorbic marked this pull request as ready for review June 11, 2026 14:26
Copilot AI review requested due to automatic review settings June 11, 2026 14:26
@github-actions github-actions Bot added the review/needs-review No maintainer or bot review yet label Jun 11, 2026
@ascorbic ascorbic added the bot:review Trigger an emdashbot code review on this PR label Jun 11, 2026
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reduces request-time query waterfalls in packages/core by parallelizing independent taxonomy queries, reusing already-hydrated taxonomy terms in first-party templates/demos, and request-caching menu collection URL pattern lookups to avoid duplicated queries within a single render.

Changes:

  • Run getTerm() usage-count + children queries concurrently (Promise.all) to save a round trip.
  • Expose hydrated entry.data.terms in generated collection types and update templates/demos/fixtures to use it instead of calling getEntryTerms() per entry.
  • Request-cache _emdash_collections.url_pattern lookups during menu tree building so multiple menus in one request share the same query.

Reviewed changes

Copilot reviewed 17 out of 27 changed files in this pull request and generated no comments.

Show a summary per file
File Description
templates/starter/src/pages/posts/[slug].astro Stop re-querying terms; use post.data.terms hydrated by getEmDashEntry.
templates/starter/emdash-env.d.ts Regenerate types to include bylines? and terms?.
templates/starter-cloudflare/src/pages/posts/[slug].astro Stop re-querying terms; use post.data.terms.
templates/starter-cloudflare/emdash-env.d.ts Regenerate types to include bylines? and terms?.
templates/blog/src/pages/posts/[slug].astro Replace getEntryTerms() call with hydrated post.data.terms?.tag.
templates/blog/emdash-env.d.ts Regenerate types to include terms?.
templates/blog-cloudflare/src/pages/posts/[slug].astro Replace getEntryTerms() call with hydrated post.data.terms?.tag.
templates/blog-cloudflare/emdash-env.d.ts Regenerate types to include terms?.
packages/core/tests/unit/taxonomies/get-term.test.ts New unit tests asserting getTerm() behavior + fixed 3-query budget.
packages/core/tests/unit/schema/zod-generator.test.ts Assert generated TS includes hydrated bylines? and terms? fields.
packages/core/tests/unit/menus/menu-request-cache.test.ts New unit tests verifying menu collection-pattern lookup is request-cached.
packages/core/src/taxonomies/index.ts Parallelize getTerm() count + children queries via Promise.all.
packages/core/src/schema/zod-generator.ts Emit terms?: Record<string, TaxonomyTerm[]> and import TaxonomyTerm in generated types.
packages/core/src/menus/index.ts Request-cache collection url_pattern lookup used during menu URL resolution.
fixtures/perf-site/src/pages/posts/[slug].astro Replace per-entry getEntryTerms() with hydrated post.data.terms?.tag.
fixtures/perf-site/emdash-env.d.ts Regenerate types to include terms?.
demos/simple/src/pages/posts/[slug].astro Replace per-entry getEntryTerms() with hydrated post.data.terms?.tag.
demos/simple/emdash-env.d.ts Regenerate types to include terms?.
demos/preview/src/pages/posts/[slug].astro Replace per-entry getEntryTerms() with hydrated post.data.terms?.tag.
demos/preview/emdash-env.d.ts Regenerate types to include terms?.
demos/postgres/src/pages/posts/[slug].astro Replace per-entry getEntryTerms() with hydrated post.data.terms?.tag.
demos/postgres/emdash-env.d.ts Regenerate types to include terms?.
demos/playground/src/pages/posts/[slug].astro Replace per-entry getEntryTerms() with hydrated post.data.terms?.tag.
demos/playground/emdash-env.d.ts Regenerate types to include terms?.
demos/cloudflare/src/pages/posts/[slug].astro Replace per-entry getEntryTerms() with hydrated post.data.terms?.tag.
demos/cloudflare/emdash-env.d.ts Regenerate types to include terms?.
.changeset/lucky-pots-repair.md Publish notes describing the three query-waterfall fixes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@github-actions github-actions Bot added review/awaiting-author Reviewed; waiting on the author to respond cla: signed and removed review/needs-review No maintainer or bot review yet labels Jun 11, 2026
@ascorbic ascorbic added bot:review Trigger an emdashbot code review on this PR and removed bot:review Trigger an emdashbot code review on this PR labels Jun 11, 2026
@ascorbic
ascorbic merged commit 7ee9467 into main Jun 11, 2026
45 checks passed
@ascorbic
ascorbic deleted the perf/query-waterfalls branch June 11, 2026 15:38
@emdashbot emdashbot Bot mentioned this pull request Jun 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/core area/templates bot:review Trigger an emdashbot code review on this PR cla: signed review/awaiting-author Reviewed; waiting on the author to respond size/XL

2 participants