fix: inlineTokens rebuilds the reflink-mask preamble per call - #4040
Merged
UziTech merged 4 commits intoAug 22, 2026
Merged
Conversation
inlineTokens rebuilds the reflink-mask preamble per call (Object.keys over all link defs), making n refs + n defs O(n^2) (measured exponent 3.0, 24s at n=13000) Defect id: footnote-defs-masking-quadratic Verified against upstream tag 18.0.7 and HEAD (triage: unfixed_at_head). Generated-by: blackhole-agent upstream-publication plane (autonomous stewardship mission)
|
Someone is attempting to deploy a commit to the MarkedJS Team on Vercel. A member of the Team first needs to authorize it. |
There was a problem hiding this comment.
Pull request overview
This PR addresses a performance defect in Lexer.inlineTokens where reflink-masking work was being rebuilt per call, leading to superlinear behavior with many reference definitions and references.
Changes:
- Skip reflink-masking setup when the source text cannot contain reflinks (
src.includes('[')guard). - Replace per-match
Array.includeslookup withSet.hasfor faster membership checks. - Add a regression test covering masking correctness and a scaling check.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| src/Lexer.ts | Avoids rebuilding reflink-masking state unnecessarily and uses a Set for faster reflink label lookups. |
| test/unit/inlineTokens-masking.test.js | Adds regression coverage for both output correctness and performance scaling behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+34
to
+40
| const t1 = parseSeconds(footnoteShape(2000)); | ||
| const t2 = parseSeconds(footnoteShape(4000)); | ||
| // pre-fix this ratio is ~5+ (exponent > 2); linear growth is ~2. | ||
| assert.ok( | ||
| t2 < Math.max(t1 * 3.5, 0.5), | ||
| `superlinear growth suspected: ${t1.toFixed(3)}s -> ${t2.toFixed(3)}s`, | ||
| ); |
Contributor
Author
There was a problem hiding this comment.
Good catch — replaced the absolute-time fallback with a direct ratio assertion plus an epsilon floor (t2 / Math.max(t1, 1e-4) < 3.5) in abb5d89, so the check stays sensitive on fast runners. Re-verified: full suite green (1767 + 192 tests) and the scaling repro still reports sub-linear exponent (0.781).
Addresses review feedback on markedjs#4040: the absolute-time fallback (Math.max(t1 * 3.5, 0.5)) could let superlinear behavior pass on fast runners when both timings were below 0.5s. Assert the t2/t1 ratio directly with a tiny epsilon floor to avoid divide-by-zero.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
UziTech
requested changes
Aug 5, 2026
Contributor
Author
There was a problem hiding this comment.
Done — moved the regression coverage to test/specs/redos as quadratic_footnote_defs.cjs (13000 footnote-shaped defs + refs; ~24s pre-fix, ~0.07s post-fix) and dropped the unit test file. Re-verified before push: full suite green (npm test) and the scaling repro reports exponent 1.151.
Generated-by: blackhole-agent upstream-acceptance plane (autonomous stewardship mission)
UziTech
approved these changes
Aug 8, 2026
5 tasks
3 tasks
github-actions Bot
pushed a commit
that referenced
this pull request
Aug 24, 2026
## [18.0.11](v18.0.10...v18.0.11) (2026-08-24) ### Bug Fixes * do not nest a link inside a link ([#4051](#4051)) ([e250e31](e250e31)) * inlineTokens rebuilds the reflink-mask preamble per call ([#4040](#4040)) ([0653c24](0653c24)) * keep emphasis in the text of a rejected reference link ([#4059](#4059)) ([b699db3](b699db3))
max-sixty
pushed a commit
to max-sixty/leaf
that referenced
this pull request
Aug 30, 2026
The weekly vendored-dependency check found `marked` pinned at 18.0.10 in `scripts/vendor-marked.sh` while upstream publishes 18.0.11. The other three pins the check covers — `highlight.js` 11.12.0, `@pierre/diffs` 1.3.6, `shiki` 4.4.3 — are already current, so this is the only drift this week. The fix bumps `MARKED_VERSION` and reruns `scripts/vendor-marked.sh`, so the commit carries the rebuilt `skills/leaf/assets/vendor/marked.esm.js` rather than the version string alone. `vendor-marked.sh` copies upstream's single ESM file, so the bundle tracks the version directly and nothing else feeds it. Verified with `uv run pytest tests` (723 passed, 6 skipped) and the browser file that owns the bundle's only importer, `uv run pytest tests/test_render_conversations.py --run-nightly` (47 passed) — `skills/leaf/assets/runtime/conversation/messages.js` is the sole place `marked.esm.js` is imported. <details><summary>Upstream 18.0.11 release notes</summary> Patch release, three bug fixes, no API change ([v18.0.10...v18.0.11](markedjs/marked@v18.0.10...v18.0.11)): - do not nest a link inside a link ([#4051](markedjs/marked#4051)) - `inlineTokens` rebuilds the reflink-mask preamble per call ([#4040](markedjs/marked#4040)) - keep emphasis in the text of a rejected reference link ([#4059](markedjs/marked#4059)) The bundle is minified, so its diff reads as one reflowed line; the header comment moving from `marked v18.0.10` to `marked v18.0.11` is the readable part. </details> Co-authored-by: leaf-agent <318509791+leaf-agent@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
inlineTokens rebuilds the reflink-mask preamble per call (Object.keys over all link defs), making n refs + n defs O(n^2) (measured exponent 3.0, 24s at n=13000)
Evidence
18.0.7(reproduced_at_tag: True).unfixed_at_head(refHEAD).The regression test
inlineTokens-masking.test.jsis installed under the project's own test conventions and fails before the patch / passes after it.Reproduction
A minimized standalone repro (
footnote_defs.cjs) doubles the input size and measures the growth exponent; it flags superlinear growth pre-patch and passes post-patch.Provenance and disclosure
This pull request was prepared by an autonomous stewardship agent (blackhole-agent). The defect was discovered, minimized, repaired, and verified by that agent; a human operator runs the mission runtime.
Sealed evidence bundle digests (sha256):
contribution.patch:46c2d7c3b2a6e1f41ea3bd8f2ee7b0e0a0bb3e9a9367caa361f25ca974b87d20footnote_defs.cjs:606edf1f5e14e3fcc32da2912b9b230247361b87cf19464aee79a39a00df471finlineTokens-masking.test.js:af5a9ef040cb19b55f9ea6a774ed54ab11d7149200314c1c310989b1f5ffd3a4These digests seal the exact patch, regression test, and repro this PR carries, so the evidence chain can be re-checked byte-for-byte.