Skip to content

Modify configuration to be aligned with new structure - #649

Merged
uri-weisman merged 15 commits into
elastic:mainfrom
uri-weisman:cis_aws_e2e
Jan 9, 2023
Merged

Modify configuration to be aligned with new structure#649
uri-weisman merged 15 commits into
elastic:mainfrom
uri-weisman:cis_aws_e2e

Conversation

@uri-weisman

Copy link
Copy Markdown
Collaborator

What does this PR do?

  1. Update policy version to latest - v1.2.9
  2. Modify config serialization to fit new fleet configuration.

Related Issues

Checklist

  • I have added tests that prove my fix is effective or that my feature works
  • I have added the necessary README/documentation (if appropriate)
@mergify

mergify Bot commented Jan 5, 2023

Copy link
Copy Markdown
Contributor

This pull request does not have a backport label. Could you fix it @uri-weisman? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v./d./d./d is the label to automatically backport to the 8./d branch. /d is the digit
    NOTE: backport-skip has been added to this pull request.
@mergify mergify Bot added the backport-skip label Jan 5, 2023
@uri-weisman uri-weisman linked an issue Jan 5, 2023 that may be closed by this pull request
@github-actions

github-actions Bot commented Jan 5, 2023

Copy link
Copy Markdown
@uri-weisman uri-weisman changed the title Modify configuration serialization to be aligned with new structure Jan 5, 2023
@uri-weisman
uri-weisman marked this pull request as ready for review January 5, 2023 09:53
@uri-weisman
uri-weisman requested review from a team as code owners January 5, 2023 09:53
@uri-weisman
uri-weisman requested a review from olegsu January 5, 2023 14:28
@oren-zohar
oren-zohar requested review from jeniawhite and removed request for olegsu January 9, 2023 08:50
Comment thread config/config.go Outdated

type BenchmarkConfig struct {
ID string `config:"benchmark"`
AWSConfig aws.ConfigAWS `config:"aws.credentials"`

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see why AWSConfig should be a part of BenchmarkConfig.
For example, vulnerability management is not a benchmark but we will want to reuse the AWS authentication logic there.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see.
We can keep this structure and rename BenchmarkConfig to something more general to include vulnerability management.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Besides the integration already sending the AWS credentials as part of the benchmark config, is there a real reason to enforce that coupling? In the future we will have also GCPConfig and AzureConfig are they all going to be part of an object that also contains the benchmark details?
I think we should have a CloudConfig object, that for now holds only aws credentials, while benchmark config should be a separate object.

Comment thread deploy/aws/cloudbeat-aws.yml Outdated
benchmark: cis_aws
aws:
credentials:
access_key_id: ${AWS_ACCESS_KEY_ID:no_access_key_was_set}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we have these values instead of nil or empty values that will omitempty the keys in unmarshal flow?
no_access_key_was_set
no_secret_was_set

I think this creates a more complex flow where we populate fields with mock values just to fail later on.
I would expect the behavior to be different where we do not populate the fields and just fail on validation that the fields were non-existent or continue the flow without keys in case of anonymous access (which this change blocks since it mocks something invalid).

Obviously, this would require changes to the config.go and potentially changes to the validator in case we do not want to rely on config.go, but as of today all of our validations happen in the config.go layer.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No real reason just wanted to be aligned with the EKS config file.
I'll revert it to an empty string.

@uri-weisman
uri-weisman requested a review from jeniawhite January 9, 2023 10:50
@uri-weisman
uri-weisman requested a review from amirbenun January 9, 2023 12:00

@gurevichdmitry gurevichdmitry left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@uri-weisman, deploy dir files approved. Please also update cloudbeat-eks-pytest.yaml in tests dir.

Comment on lines +5 to +8
aws:
credentials:
access_key_id: ${AWS_ACCESS_KEY_ID:no_access_key_was_set}
secret_access_key: ${AWS_SECRET_ACCESS_KEY:no_secret_was_set}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@uri-weisman, the file tests/deploy/cloudbeat-eks-pytest.yaml should be also updated with the code above.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gurevichdmitry - thanks updated.

@uri-weisman
uri-weisman requested a review from a team as a code owner January 9, 2023 15:21

@gurevichdmitry gurevichdmitry left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@uri-weisman, deploy files approved.

@uri-weisman
uri-weisman merged commit 8f22eaa into elastic:main Jan 9, 2023
@uri-weisman
uri-weisman deleted the cis_aws_e2e branch January 9, 2023 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

4 participants