workaround premature recycling (compression related)

[jackshirazi]

What does this PR do?

lose the compression rather than retry looking for strategy, so that the application proceeds with minimum delay - branch to see if this works for the customer as a temporary workaround

Checklist

• This is an enhancement of existing features, or a new feature in existing plugins
  • I have updated CHANGELOG.asciidoc
  • I have added tests that prove my fix is effective or that my feature works
  • Added an API method or config option? Document in which version this will be introduced
  • I have made corresponding changes to the documentation
• This is a bugfix
  • I have updated CHANGELOG.asciidoc
  • I have added tests that would fail without this fix
• This is a new plugin
  • I have updated CHANGELOG.asciidoc
  • My code follows the style guidelines of this project
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have updated supported-technologies.asciidoc
  • Added an API method or config option? Document in which version this will be introduced
  • Added an instrumentation plugin? Describe how you made sure that old, non-supported versions are not instrumented by accident.
• This is something else
  • I have updated CHANGELOG.asciidoc

[ghost]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-03-31T13:31:48.499+0000

• Duration: 48 min 27 sec

Test stats 🧪

Test	Results
Failed	0
Passed	2840
Skipped	20
Total	2860

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• run benchmark tests : Run the benchmark tests.

• run jdk compatibility tests : Run the JDK Compatibility tests.

• run integration tests : Run the Agent Integration tests.

• run end-to-end tests : Run the APM-ITs.

• run windows tests : Build & tests on windows.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[efdknittlfrank]

Thank you for working on this! I'd be happy to test the workaround.

I was wondering if the retry could have a kind of fail-safe, aborting after a fixed number of retries. Something similar to:

Suggested change

	String compressionStrategy = composite.getCompressionStrategy();
	if (compressionStrategy == null) {
	//lose the compression rather than retry, so that the application proceeds with minimum delay
	return false;
	}
	String compressionStrategy;
	for (int i = 0; i < 100 && compressionStrategy == null; ++i) {
	compressionStrategy = composite.getCompressionStrategy();
	}
	if (compressionStrategy == null) {
	// lose the compression rather than retry, so that the application proceeds with minimum delay
	return false;
	}

But I assume the workaround will be removed soon, once the root cause is found?

[jackshirazi]

Let's see how it looks when you try it. We might prefer to leave the fast fail so that there is minimum impact to the application. When we've eliminated premature recycling, this concurrent race should happen rarely. At most I think I'd retry only once