Add leader election variable in kube state metrics integration

[MichaelKatsoulis]

What does this PR do?

What does this PR do?

It adds an extra setting leaderelection: true to the kubernetes package in data_streams that collect cluster wise metrics (kube_state*, apiserver, events) so as to indicate that that these specific data_streams should be enabled only when leadership lock is obtained.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• If I'm introducing a new feature, I have modified the Kibana version constraint in my package's manifest.yml file to point to the latest Elastic stack release (e.g. ^7.13.0).

How to test this PR locally

1. Build the kubernetes package elastic-package build inside integrations/packages/kubernetes folder
2. Bring up elastic stack elastic-package stack up --version=7.14.0-SNAPSHOT -v -d
3. Set up multi-node k8s cluster with kind (https://kind.sigs.k8s.io/docs/user/quick-start/#multinode-clusters)
4. Connect kind nodes to elastic stack network docker-connect.sh.zip
5. In Kibana ui create a new agent policy with kubernetes integration. Leave Leader Election activated. Copy the enrolment token of the new policy.
6. Update elastic-agent-managed daemonset variables (FLEET_URL: "http://fleet-server:8220", FLEET_ENROLLMENT_TOKEN: token from step 5 and apply the manifests kubectl apply -f .
7. After agents are successfully enrolled to fleet and assigned the policy check in discovery that kube_state metrics (for example metricset=state_pod) come from only one agent.name which is the leader

Related issues

• Closes [Kubernetes] Tune package's data_streams to leverage leaderelection by default #1311

Screenshots

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2021-07-13T08:04:54.379+0000

• Duration: 27 min 26 sec

• Commit: ecd9eb9

Test stats 🧪

Test	Results
Failed	0
Passed	109
Skipped	0
Total	109

Trends 🧪

[elasticmachine]

Pinging @elastic/integrations (Team:Integrations)

[ChrsMark]

Great enhancement, feel free to ship it!

Some partially related comments:

1. Testing notes are really good (+ scripts etc), maybe we can document them somewhere for the future. @MichaelKatsoulis could you take care of this?
2. With this condition added the only missing part from standlone version is the conditions so as to run control-plane's module only on master using autodiscovery. See https://github.com/elastic/beats/blob/master/deploy/kubernetes/elastic-agent-standalone-kubernetes.yaml#L347. @MichaelKatsoulis it would be nice to have a follow-up PR/issue for this.

In general this managed version of k8s integration which more or less stands equally next to the standalone version sounds like a good demo candidate.

[MichaelKatsoulis]

Thanks @ChrsMark !

1. We have plans on working on issue Use upstream's manifest to deploy agent on k8s for tests elastic-package#328 which will make some of the manual steps obsolete. Meanwhile I can document steps needed to test leader election like functionalities in multi node clusters.
2. I will take care of creating the follow up issue.