[filebeat][ABS] - Segregated max_workers from batch/page size of the response

[ShourieG]

Type of change

• Enhancement
• Docs

Proposed commit message

Decoupled max_workers from the actual page size configurations while
keeping backwards compatibility. A minimum/default value validation was
not implemented intentionally to keep backwards compatibility in
scenarios the users would upgrade the stack but not change  any
relevant configurations.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

None

Author's Checklist

• [ ]

How to test this PR locally

Related issues

• Closes [filebeat][ABS] - Segregate batch size from worker count #44491
• Relates [Meta][ABS] - Improvements and addition of new features to the Azure Blob Storage input #43312

Use cases

Screenshots

Logs

[github-actions]

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @ShourieG? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[colleenmcginnis]

Some minor suggestions below.

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b abs/batch_size upstream/abs/batch_size
git merge upstream/main
git push upstream abs/batch_size

[efd6]

Please address Colleen's comment before merge.

[colleenmcginnis]

✨