Skip to content

Fixes memory leak in BytesRefLongBlockHash#137050

Merged
ncordon merged 5 commits intoelastic:mainfrom
ncordon:BlockHashRandomizedTests-fix
Oct 24, 2025
Merged

Fixes memory leak in BytesRefLongBlockHash#137050
ncordon merged 5 commits intoelastic:mainfrom
ncordon:BlockHashRandomizedTests-fix

Conversation

@ncordon
Copy link
Contributor

@ncordon ncordon commented Oct 23, 2025

Closes #137021.

Repro:

./gradlew ":x-pack:plugin:esql:compute:test" --tests "org.elasticsearch.compute.aggregation.blockhash.BlockHashRandomizedTests.testWithCranky {forcePackedHash=false groups=2 maxValuesPerPosition=1 dups=0 allowedTypes=[Basic[elementType=LONG], Basic[elementType=BYTES_REF]]}" -Dtests.seed=F336120F4E4CDBE -Dtests.locale=en-DG -Dtests.timezone=America/Sitka -Druntime.java=25

The test BlockHashRandomizedTests#testWithCranky was failing in the after phase when we were checking all of the allocated arrays had been released. That test only failed once every 20 times (the frequency CrankyCircuitBreakerService kicks in).

    public void testWithCranky() {
        CircuitBreakerService service = new CrankyCircuitBreakerService();
        CircuitBreaker breaker = service.getBreaker(CircuitBreaker.REQUEST);
        BigArrays bigArrays = new MockBigArrays(PageCacheRecycler.NON_RECYCLING_INSTANCE, service);
        try {
            test(new MockBlockFactory(breaker, bigArrays));
            logger.info("cranky let us finish!");
        } catch (CircuitBreakingException e) {
            logger.info("cranky", e);
            assertThat(e.getMessage(), equalTo(CrankyCircuitBreakerService.ERROR_MESSAGE));
        }
    }
@elasticsearchmachine
Copy link
Collaborator

Hi @ncordon, I've created a changelog YAML for you.

var dict = blockFactory.newBytesRefArrayVector(bytes, Math.toIntExact(bytes.size()));
dict = blockFactory.newBytesRefArrayVector(bytes, Math.toIntExact(bytes.size()));
bytes = null; // transfer ownership to dict
k1 = new OrdinalBytesRefBlock(ordinals.build(), dict);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A circuit breaker could kick in here, in the ordinals.build() call and we would not release dict

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our normal way to do this is to set dict to null after you give it to k1 and then:

Releasables.closeExpectNoException(bytes, dict);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha!

@ncordon ncordon marked this pull request as ready for review October 23, 2025 16:38
@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Oct 23, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

Copy link
Member

@dnhatn dnhatn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks Nacho!

@ncordon ncordon added the auto-backport Automatically create backport pull requests when merged label Oct 23, 2025
@ncordon ncordon merged commit 88d4704 into elastic:main Oct 24, 2025
34 checks passed
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
9.2 Commit could not be cherrypicked due to conflicts
8.19 Commit could not be cherrypicked due to conflicts
9.1 Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 137050

@ncordon ncordon deleted the BlockHashRandomizedTests-fix branch October 24, 2025 07:25
@ncordon ncordon removed backport pending auto-backport Automatically create backport pull requests when merged v9.2.1 v8.19.7 v9.1.7 labels Oct 24, 2025
@ncordon
Copy link
Contributor Author

ncordon commented Oct 24, 2025

No need to backport this, it seems this bug was introduced only for 9.3

fzowl pushed a commit to voyage-ai/elasticsearch that referenced this pull request Nov 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Analytics/Compute Engine Analytics in ES|QL >bug Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v9.3.0

4 participants