fix: add FLAG num support for agglutinative languages - #1090
Merged
Conversation
The spell checker's affix parser used `rune` (single character) as the map key for affix rules. This broke dictionaries that use `FLAG num` format, where flags are comma-separated numbers (e.g., "14308,10482"). Only the first digit of each numeric flag was read, causing most affix rules to be unreachable. This affected all agglutinative languages (Turkish, Hungarian, Finnish, etc.) whose Hunspell dictionaries use `FLAG num` with tens of thousands of suffix groups. Changes: - Change AffixMap key from `rune` to `string` - Change compoundMap key from `rune` to `string` - Add parseFlags() method that handles ASCII, num, long, and UTF-8 formats - Update expand() to use parsed flag slices instead of rune iteration - Update compound rule parsing in gospell.go For the Turkish dictionary (tr_TR), this enables correct recognition of ~59,000 suffix groups and ~15.8M inflected word forms that were previously unreachable.
Member
|
Thanks! |
tolgakaratas
added a commit
to Denomas/gherkin
that referenced
this pull request
May 4, 2026
Add natural Turkish alternatives across all keyword categories. Every keyword reviewed by native Turkish speaker. No existing keywords removed — 100% backward compatible. This is part of a broader effort to improve Turkish language support in open-source tools, including: - vale-cli/vale#1090 (FLAG num for agglutinative languages) - https://github.com/Denomas/Turkce-yazim-denetimi (Turkish prose linting) - https://github.com/Denomas/hunspell-tr (Hunspell tr_TR dictionary expansion)
4 tasks
tmeijn
pushed a commit
to tmeijn/dotfiles
that referenced
this pull request
May 18, 2026
This MR contains the following updates: | Package | Update | Change | |---|---|---| | [errata-ai/vale](https://github.com/errata-ai/vale) | patch | `v3.14.1` → `v3.14.2` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>errata-ai/vale (errata-ai/vale)</summary> ### [`v3.14.2`](https://github.com/vale-cli/vale/releases/tag/v3.14.2) [Compare Source](vale-cli/vale@v3.14.1...v3.14.2) #### Changelog - [`810868b`](vale-cli/vale@810868b2) chore: drop yaml v2 - [`20fcb4d`](vale-cli/vale@20fcb4df) fix: move to gopkg.in/yaml.v3 - [`cdf6f9e`](vale-cli/vale@cdf6f9e5) Bugfix/path expansion ([#​1096](vale-cli/vale#1096)) - [`3b9c8bf`](vale-cli/vale@3b9c8bf9) chore(deps): upgrade github.com/olekukonko/tablewriter from v0.0.5 to v1.1.4 ([#​1092](vale-cli/vale#1092)) - [`e251662`](vale-cli/vale@e251662e) fix: add FLAG num support for agglutinative languages ([#​1090](vale-cli/vale#1090)) - [`831ded9`](vale-cli/vale@831ded96) chore: update clone path - [`806283c`](vale-cli/vale@806283cf) chore(deps): bump github.com/tomwright/dasel/v3 from 3.3.1 to 3.3.2 ([#​1088](vale-cli/vale#1088)) </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever MR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box --- This MR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xODAuMCIsInVwZGF0ZWRJblZlciI6IjQzLjE4MC4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiLCJhdXRvbWF0aW9uOmJvdC1hdXRob3JlZCIsImRlcGVuZGVuY3ktdHlwZTo6cGF0Y2giXX0=-->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
rune(single character) as the map key for affix rules, which broke dictionaries usingFLAG numformat14308,10482,4720were parsed by reading only the first digit, making most affix rules unreachableFLAG numwith thousands of suffix groupsChanges
AffixMapkey fromrunetostringto support multi-character flagscompoundMapkey fromrunetostringparseFlags()method that correctly handles all Hunspell flag formats:ASCII,num,long, andUTF-8expand()to use parsed flag slices instead of rune iterationgospell.goImpact
For the Turkish dictionary (
tr_TR), this fix enables correct recognition of ~59,000 suffix groups and ~15.8 million inflected word forms that were previously unreachable. Thetr_TR.afffile usesFLAG numwith comma-separated numeric IDs.Before:
hunspell -d tr_TR -lrecognizes "belediyeye", "adaletli", "ancak" — but Vale flags them as unknown.After: Vale correctly recognizes these words using the same dictionary files.
Test plan
parseFlags()covering ASCII, num, long, and UTF-8 formatsFLAG numaffix parsing and expansionnewGoSpellReaderwithFLAG numdictionary