Skip to content

fix: add FLAG num support for agglutinative languages - #1090

Merged
jdkato merged 1 commit into
vale-cli:v3from
Denomas:fix/flag-num-support
May 1, 2026
Merged

fix: add FLAG num support for agglutinative languages#1090
jdkato merged 1 commit into
vale-cli:v3from
Denomas:fix/flag-num-support

Conversation

@tolgakaratas

Copy link
Copy Markdown
Contributor

Summary

  • The spell checker's affix parser used rune (single character) as the map key for affix rules, which broke dictionaries using FLAG num format
  • Numeric flags like 14308,10482,4720 were parsed by reading only the first digit, making most affix rules unreachable
  • This affected all agglutinative languages (Turkish, Hungarian, Finnish, etc.) whose Hunspell dictionaries use FLAG num with thousands of suffix groups

Changes

  • Change AffixMap key from rune to string to support multi-character flags
  • Change compoundMap key from rune to string
  • Add parseFlags() method that correctly handles all Hunspell flag formats: ASCII, num, long, and UTF-8
  • Update expand() to use parsed flag slices instead of rune iteration
  • Update compound rule parsing in gospell.go

Impact

For the Turkish dictionary (tr_TR), this fix enables correct recognition of ~59,000 suffix groups and ~15.8 million inflected word forms that were previously unreachable. The tr_TR.aff file uses FLAG num with comma-separated numeric IDs.

Before: hunspell -d tr_TR -l recognizes "belediyeye", "adaletli", "ancak" — but Vale flags them as unknown.
After: Vale correctly recognizes these words using the same dictionary files.

Test plan

  • New unit tests for parseFlags() covering ASCII, num, long, and UTF-8 formats
  • New unit test for FLAG num affix parsing and expansion
  • New integration test: newGoSpellReader with FLAG num dictionary
  • Backward compatibility test: ASCII flag dictionaries still work correctly
  • All 8 new tests pass
=== RUN   TestParseFlagsASCII
--- PASS: TestParseFlagsASCII (0.00s)
=== RUN   TestParseFlagsNum
--- PASS: TestParseFlagsNum (0.00s)
=== RUN   TestParseFlagsLong
--- PASS: TestParseFlagsLong (0.00s)
=== RUN   TestParseFlagsUTF8
--- PASS: TestParseFlagsUTF8 (0.00s)
=== RUN   TestFlagNumAffixParsing
--- PASS: TestFlagNumAffixParsing (0.00s)
=== RUN   TestFlagNumExpand
--- PASS: TestFlagNumExpand (0.00s)
=== RUN   TestFlagNumGoSpellReader
--- PASS: TestFlagNumGoSpellReader (0.00s)
=== RUN   TestASCIFlagBackwardCompatibility
--- PASS: TestASCIFlagBackwardCompatibility (0.00s)
PASS
The spell checker's affix parser used `rune` (single character) as the
map key for affix rules. This broke dictionaries that use `FLAG num`
format, where flags are comma-separated numbers (e.g., "14308,10482").

Only the first digit of each numeric flag was read, causing most affix
rules to be unreachable. This affected all agglutinative languages
(Turkish, Hungarian, Finnish, etc.) whose Hunspell dictionaries use
`FLAG num` with tens of thousands of suffix groups.

Changes:
- Change AffixMap key from `rune` to `string`
- Change compoundMap key from `rune` to `string`
- Add parseFlags() method that handles ASCII, num, long, and UTF-8 formats
- Update expand() to use parsed flag slices instead of rune iteration
- Update compound rule parsing in gospell.go

For the Turkish dictionary (tr_TR), this enables correct recognition of
~59,000 suffix groups and ~15.8M inflected word forms that were
previously unreachable.
@jdkato

jdkato commented May 1, 2026

Copy link
Copy Markdown
Member

Thanks!

@jdkato
jdkato merged commit e251662 into vale-cli:v3 May 1, 2026
tolgakaratas added a commit to Denomas/gherkin that referenced this pull request May 4, 2026
Add natural Turkish alternatives across all keyword categories.
Every keyword reviewed by native Turkish speaker.
No existing keywords removed — 100% backward compatible.

This is part of a broader effort to improve Turkish language support
in open-source tools, including:
- vale-cli/vale#1090 (FLAG num for agglutinative languages)
- https://github.com/Denomas/Turkce-yazim-denetimi (Turkish prose linting)
- https://github.com/Denomas/hunspell-tr (Hunspell tr_TR dictionary expansion)
tmeijn pushed a commit to tmeijn/dotfiles that referenced this pull request May 18, 2026
This MR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [errata-ai/vale](https://github.com/errata-ai/vale) | patch | `v3.14.1` → `v3.14.2` |

MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot).

**Proposed changes to behavior should be submitted there as MRs.**

---

### Release Notes

<details>
<summary>errata-ai/vale (errata-ai/vale)</summary>

### [`v3.14.2`](https://github.com/vale-cli/vale/releases/tag/v3.14.2)

[Compare Source](vale-cli/vale@v3.14.1...v3.14.2)

#### Changelog

- [`810868b`](vale-cli/vale@810868b2) chore: drop yaml v2
- [`20fcb4d`](vale-cli/vale@20fcb4df) fix: move to gopkg.in/yaml.v3
- [`cdf6f9e`](vale-cli/vale@cdf6f9e5) Bugfix/path expansion ([#&#8203;1096](vale-cli/vale#1096))
- [`3b9c8bf`](vale-cli/vale@3b9c8bf9) chore(deps): upgrade github.com/olekukonko/tablewriter from v0.0.5 to v1.1.4 ([#&#8203;1092](vale-cli/vale#1092))
- [`e251662`](vale-cli/vale@e251662e) fix: add FLAG num support for agglutinative languages ([#&#8203;1090](vale-cli/vale#1090))
- [`831ded9`](vale-cli/vale@831ded96) chore: update clone path
- [`806283c`](vale-cli/vale@806283cf) chore(deps): bump github.com/tomwright/dasel/v3 from 3.3.1 to 3.3.2 ([#&#8203;1088](vale-cli/vale#1088))

</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - At any time (no schedule defined)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever MR is behind base branch, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this MR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xODAuMCIsInVwZGF0ZWRJblZlciI6IjQzLjE4MC4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiLCJhdXRvbWF0aW9uOmJvdC1hdXRob3JlZCIsImRlcGVuZGVuY3ktdHlwZTo6cGF0Y2giXX0=-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants