Skip to content

fix(layout): recover mixed-case tracked words - #436

Open
yzxcj797 wants to merge 2 commits into
firecrawl:mainfrom
yzxcj797:round2-pdf-b-211
Open

fix(layout): recover mixed-case tracked words#436
yzxcj797 wants to merge 2 commits into
firecrawl:mainfrom
yzxcj797:round2-pdf-b-211

Conversation

@yzxcj797

@yzxcj797 yzxcj797 commented Aug 20, 2026

Copy link
Copy Markdown

Summary

  • Fixes Corrupted text extraction for text-based PDFs #211.
  • Detect a conservative bimodal signature for mixed-case glyph-per-item tracking and rejoin each title-case word while retaining real word gaps.
  • Keep deliberately spaced lowercase single-letter sequences unchanged, alongside the existing all-caps and spaceless-CJK handling.

Testing

  • cargo test --target x86_64-pc-windows-gnu
  • cargo clippy --target x86_64-pc-windows-gnu -- -D warnings
  • cargo fmt --all -- --check
  • git diff --check

Summary by cubic

Recover mixed-case glyph-per-letter tracking into words while preserving real word gaps. Previously, mixed-case runs (e.g., JohnDoe) split into 1–2 character fragments; now a conservative bimodal gap detector rejoins title-case words without changing lowercase spaced singles or existing all-caps/CJK behavior. Fixes #211.

  • Introduces a shared gap-splitting helper (largest_relative_gap_jump) and routes non-CJK, non-all-caps runs through a title-case path that requires a strong jump and at least two ≥3-letter title-case words before merging.
  • Keeps lowercase single-letter sequences unchanged; preserves existing all-caps and spaceless CJK handling.
  • Adds unit and integration tests, including a synthetic PDF with custom font metrics to reproduce resume-style tracking; no API or option changes.

Written for commit 87bd26d. Summary will update on new commits.

Review in cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files

Shadow auto-approve: would not auto-approve because issues were found.

Fix all with cubic | Re-trigger cubic

Comment thread src/extractor/mod.rs Outdated
Comment thread src/extractor/mod.rs Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 1 file (changes from recent commits).

Shadow auto-approve: would auto-approve. Introduces a conservative heuristic to rejoin mixed-case tracked words (e.g., JohnDoe) while preserving lowercase spaced single-letter sequences and existing all-caps/CJK handling. The fix is focused, test-driven, and does not change any API or operational configuration.

Re-trigger cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant