Fast codePointCount implementation for BytesRef#140388
Merged
parkertimmins merged 12 commits intoelastic:mainfrom Jan 12, 2026
Merged
Fast codePointCount implementation for BytesRef#140388parkertimmins merged 12 commits intoelastic:mainfrom
parkertimmins merged 12 commits intoelastic:mainfrom
Conversation
parkertimmins
requested review from
martijnvg and
romseygeek
and removed request for
martijnvg
Collaborator
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
Collaborator
|
Hi @parkertimmins, I've created a changelog YAML for you. |
martijnvg
approved these changes
Jan 9, 2026
Member
martijnvg
left a comment
martijnvg
left a comment
There was a problem hiding this comment.
One minor comment, LGTM otherwise.
...g/elasticsearch/index/mapper/blockloader/docvalues/fn/Utf8CodePointsFromOrdsBlockLoader.java
Outdated
Show resolved
Hide resolved
Contributor
Author
|
The failing tests are:
Since this is unrelated to this PR, I'll go ahead with the merge. |
Contributor
For reference, PR to fix the unrelated test failures - #140557 |
Member
|
Neat! Did you get any performance numbers on this one? I imagine this could be plugged into a bunch of other places too. |
Contributor
Author
|
@nik9000 |
eranweiss-elastic
pushed a commit
to eranweiss-elastic/elasticsearch
that referenced
this pull request
Jan 15, 2026
Lucene's UnicodeUtil.codePointCount is used to count the number of code points in a unicode string. It processes a single byte at a time. We can improve upon this by loading 8 bytes into a long and processing them at once.
spinscale
pushed a commit
to spinscale/elasticsearch
that referenced
this pull request
Jan 21, 2026
Lucene's UnicodeUtil.codePointCount is used to count the number of code points in a unicode string. It processes a single byte at a time. We can improve upon this by loading 8 bytes into a long and processing them at once.
parkertimmins
added a commit
that referenced
this pull request
Jan 21, 2026
Add Panama SIMD implementation of codePointCount. Keep SWAR version from #140388 as fallback if SIMD not available. This results in a very large speedup on long strings, for example those over 100 bytes. Lucene's UnicodeUtil.codePointCount remains faster for small strings, so continue to use this version if byte length is below a threshold.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lucene's UnicodeUtil.codePointCount is used to count the number of code points in a unicode string. It processes a single byte at a time. We can improve upon this by loading 8 bytes into a long and processing them at once.