fix(markdown): recognize extractor bullet glyphs (▪ ‣ ◆) as list items

[shahidbeig-a11y]

Fixes #475

---

Summary by cubic

Recognizes additional bullet glyphs (▪ ‣ ◆ and related) as list items in markdown extraction, so PDF lists using them are no longer emitted as plain paragraphs or misclassified as headings. Fixes #475.

• Shares BULLET_GLYPHS across list detection, formatting, standalone bullet merging, and the layout marker guard.
• Keeps - and * as separate checks and excludes middle dot, en dash, and em dash.
• Expands tests covering extended glyphs, negative cases, heading rejection, and a square-bullet merge stream-order regression.

^{Written for commit 87d8478. Summary will update on new commits.}

[cubic-dev-ai]

2 issues found and verified against the latest diff

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/markdown/classify.rs">

<violation number="1" location="src/markdown/classify.rs:4">
P2: When a PDF places `‣` or `⁃` in standalone marker items, column detection can mistake the marker gutter for a real column because its marker guard does not know these newly supported glyphs. Add the glyphs to the layout marker set or share `BULLET_GLYPHS` so these list items reach markdown intact.</violation>

<violation number="2" location="src/markdown/classify.rs:85">
P2: Extending starts_with_bullet_marker() to the decorative square/diamond glyphs (■ □ ◆ ◇) means section/chapter headings that begin with these glyphs (common in styled documents, e.g. "■ Overview", "◆ Resources") are now demoted from heading detection in title_like()/heuristic_heading() and, via the matching is_list_item(), rendered as `-` list items. The round and small-square bullets (• ● ○ ◦ ▪ ▫ ‣ ⁃) are unambiguous list markers; the large square/diamond forms are more ambiguous and this change turns a heading into a list item when a document uses them decoratively.</violation>
</file>

<sub>Shadow auto-approve: would not auto-approve because issues were found.

Fix all with cubic | Re-trigger cubic</sub>

[cubic-dev-ai]

P2: When a PDF places ‣ or ⁃ in standalone marker items, column detection can mistake the marker gutter for a real column because its marker guard does not know these newly supported glyphs. Add the glyphs to the layout marker set or share BULLET_GLYPHS so these list items reach markdown intact.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/markdown/classify.rs, line 4:

<comment>When a PDF places `‣` or `⁃` in standalone marker items, column detection can mistake the marker gutter for a real column because its marker guard does not know these newly supported glyphs. Add the glyphs to the layout marker set or share `BULLET_GLYPHS` so these list items reach markdown intact.</comment>

<file context>
@@ -1,5 +1,15 @@
 //! Line classification: captions, lists, code detection.
 
+pub(crate) const BULLET_GLYPHS: &[char] =
+    &['•', '●', '○', '◦', '▪', '▫', '◆', '◇', '■', '□', '‣', '⁃'];
+
+fn starts_with_bullet_glyph_and_space(text: &str) -> bool {
</file context>

[cubic-dev-ai]

P2: Extending starts_with_bullet_marker() to the decorative square/diamond glyphs (■ □ ◆ ◇) means section/chapter headings that begin with these glyphs (common in styled documents, e.g. "■ Overview", "◆ Resources") are now demoted from heading detection in title_like()/heuristic_heading() and, via the matching is_list_item(), rendered as - list items. The round and small-square bullets (• ● ○ ◦ ▪ ▫ ‣ ⁃) are unambiguous list markers; the large square/diamond forms are more ambiguous and this change turns a heading into a list item when a document uses them decoratively.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/markdown/classify.rs, line 85:

<comment>Extending starts_with_bullet_marker() to the decorative square/diamond glyphs (■ □ ◆ ◇) means section/chapter headings that begin with these glyphs (common in styled documents, e.g. "■ Overview", "◆ Resources") are now demoted from heading detection in title_like()/heuristic_heading() and, via the matching is_list_item(), rendered as `-` list items. The round and small-square bullets (• ● ○ ◦ ▪ ▫ ‣ ⁃) are unambiguous list markers; the large square/diamond forms are more ambiguous and this change turns a heading into a list item when a document uses them decoratively.</comment>

<file context>
@@ -64,18 +74,15 @@ pub(crate) fn is_caption_line(text: &str) -> bool {
-        || trimmed.starts_with("● ")
-        || trimmed.starts_with("○ ")
-        || trimmed.starts_with("◦ ")
+    starts_with_bullet_glyph_and_space(trimmed)
         || trimmed.starts_with("- ")
         || trimmed.starts_with("* ")
</file context>

[cubic-dev-ai]

0 issues found across 2 files (changes from recent commits).

<sub>Shadow auto-approve: would not auto-approve. Auto-approval blocked by 2 unresolved issues from previous reviews.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 2 files (changes from recent commits).

<sub>Shadow auto-approve: would not auto-approve because issues were found.
Tip: Review your code locally with the cubic CLI to iterate faster.

Fix all with cubic | Re-trigger cubic</sub>

[cubic-dev-ai]

0 issues found across 3 files (changes from recent commits).

<sub>Shadow auto-approve: would not auto-approve. Auto-approval blocked by 2 unresolved issues from previous reviews.

Re-trigger cubic</sub>