fix(tools): glob '**/' now matches zero or more directories (root & direct children)

[azanux]

Problem

Fixes #1751.

Glob matching used Java NIO PathMatcher (glob:…), where a / after ** requires at least one directory, unlike ripgrep/git/bash globstar, which treat **/ as zero or more. As a result:

• src/**/*.sol silently skipped files directly in src/ (only returned nested ones)
• **/*.sol missed root-level files
• FileReadTools.findFiles also returned directories as if they were files

The failure was silent (no error), hit the documented recommended pattern **/*.kt, and caused two concrete harms:

1. Agents (FileReadTools.findFiles, an @LlmTool) acted on incomplete file listings.
2. RAG (DirectoryTextSearch.fileGlob) silently never indexed top-level files, causing context loss / hallucinations.

Separately, DirectoryTextSearch.extractChunksAroundMatches located match positions on a lowercased copy of the file content (content.lowercase()) and then used those offsets to slice the original content via substring. String.lowercase() can change the string length for some Unicode characters (e.g. İ U+0130 → i̇, 2 chars). When such a character appears before a match, the offsets computed on the lowercased copy are shifted relative to the original, so the resulting chunk is sliced from the wrong region and no longer contains the matched term.

Fix

Glob semantics. Translate the glob to a single regex in the shared globMatcher helper (FileReadTools.kt), mirroring the JDK translator (sun.nio.fs.Globs) with one change: a **/ segment becomes (?:[^/]+/)* (zero or more directory segments) instead of forcing a literal separator.

Each **/ now matches independently in a single pass, so any number of them in one pattern works: a/**/b/**/c matches all 0/1/n combinations, not just the all-zero or all-≥1 extremes. Every other glob feature is preserved: * / ? stay within a segment, braces {a,b}, char classes [a-z] / [!a], and regex: passthrough. findFiles also gained an isRegularFile filter so directories are never returned.

DirectoryTextSearch (RAG) previously built its own inline PathMatcher; it now switches to the shared globMatcher helper (a one-line swap) and inherits the same semantics. globMatcher's public signature is unchanged.

Unicode offsets. Search the original content directly, case-insensitively, via content.indexOf(term, index, ignoreCase = true). Offsets are then valid for substring, and the redundant content.lowercase() in this method is removed.

Behavior change

findFiles now applies an isRegularFile filter, so directories matching the glob are no longer returned, only files.

Why a hand-rolled glob to regex translator?

Glob patterns here are written by the LLM, so the matcher must support the full glob vocabulary it will reach for, otherwise a pattern silently matches nothing and the agent acts on a partial/empty file list.

Tests

• FileReadToolsGlobTest: full glob spec: zero/one/many dirs, leading & prefixed **/, single-segment wildcards, braces/classes/ranges, literal-dot and backslash escaping, regex: passthrough, files-only, over-match guards (a look-alike filename and a sibling directory must not leak in), multiple/adjacent globstars, and findHighest pruning.
• DirectoryTextSearchGlobTest: same glob semantics for the RAG file filter.
• DirectoryTextSearchTest.EncodingTests: failing-first test: a file with length-changing lowercase chars before the match now yields a chunk that actually contains the searched term.
• One @Disabled test documents a deliberate semantic divergence: a non-isolated ** (src/**Foo.sol) falls back to .* and crosses /, where minimatch/bash treat it as a plain *. Product decision, not a bug.

mvn test on embabel-agent-api and embabel-agent-rag passes green.

[igordayen]

@azanux - than you, few comments to get better readability.

[alexheifetz]

Overall: LGTM, good test coverage. Two non-blocking questions:

1. Case-sensitivity: NIO's getPathMatcher("glob:") was case-insensitive on Windows; the new Pattern.compile isn't. So findFiles("**/*.KT") on Windows now returns nothing. Intended? If so, worth noting in "Behavior change."

2. Char classes match /: unlike JDK Globs, [!x] → [^x] can cross a dir boundary (foo[!x]bar matches foo/bar). Edge case — worth excluding / from generated classes.

[azanux]

Thanks @alexheifetz for the thorough review!

• Good catch on the char classes - fixed. Negated classes now intersect with [^/] , so foo[!x]bar no longer matches foo/bar. Added a test.
• The case-sensitivity change is intentional: matching now depends only on the pattern, not the host OS, which keeps results deterministic across platforms. Documented it in the KDoc as a behavior change.
• While in there, I also fixed a related bug: a literal [ inside a class (e.g. [[]) threw a PatternSyntaxException instead of matching a [ file - NIO accepts it, now we do too.