Skip to content

feat: Add os_release_label arg to aws_emr_cluster #43018

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

acwwat
Copy link
Contributor

@acwwat acwwat commented Jun 14, 2025

Rollback Plan

If a change needs to be reverted, we will publish an updated version of the library.

Changes to Security Controls

n/a

Description

This PR is to add the os_release_label argument to the aws_emr_cluster resource.

Note that there are three existing acceptance test case failures described below, but they are unrelated to this PR.

Relations

Closes #42977

References

Referred to RunJobFlow for specs and wordings.

Output from Acceptance Testing

Note that there are three unrelated test case that failed:

  • TestAccEMRCluster_ebs and TestAccEMRCluster_CustomAMI_id failed because the core and primary instances kept failing bootstrap with a timeout. I suspect this is related to the release and application being included in the configuration. I am not familiar with EMR enough to continue troubleshooting.
  • TestAccEMRCluster_visibleToAllUsers failed because the visible_to_all_users arg did not persist on an update. Looking at CloudTrail, it is as if Terraform didn't detect the change so it didn't make the API call to update the flag.

I will open two separate issues for these acceptance test failures. Meanwhile I don't think they are related to this enhancement, so I'd like a maintainer to review and merge this PR to add the feature first.

$ make testacc TESTS=TestAccEMRCluster_ PKG=emr
make: Verifying source code with gofmt...
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go1.23.10 test ./internal/service/emr/... -v -count 1 -parallel 8 -run='TestAccEMRCluster_'  -timeout 360m -vet=off
2025/06/14 13:50:10 Initializing Terraform AWS Provider...
=== RUN   TestAccEMRCluster_basic
=== PAUSE TestAccEMRCluster_basic
=== RUN   TestAccEMRCluster_autoTerminationPolicy
=== PAUSE TestAccEMRCluster_autoTerminationPolicy
=== RUN   TestAccEMRCluster_additionalInfo
=== PAUSE TestAccEMRCluster_additionalInfo
=== RUN   TestAccEMRCluster_disappears
=== PAUSE TestAccEMRCluster_disappears
=== RUN   TestAccEMRCluster_sJSON
=== PAUSE TestAccEMRCluster_sJSON
=== RUN   TestAccEMRCluster_CoreInstanceGroup_autoScalingPolicy
=== PAUSE TestAccEMRCluster_CoreInstanceGroup_autoScalingPolicy
=== RUN   TestAccEMRCluster_CoreInstanceGroup_bidPrice
=== PAUSE TestAccEMRCluster_CoreInstanceGroup_bidPrice
=== RUN   TestAccEMRCluster_CoreInstanceGroup_instanceCount
=== PAUSE TestAccEMRCluster_CoreInstanceGroup_instanceCount
=== RUN   TestAccEMRCluster_CoreInstanceGroup_instanceType
=== PAUSE TestAccEMRCluster_CoreInstanceGroup_instanceType
=== RUN   TestAccEMRCluster_CoreInstanceGroup_name
=== PAUSE TestAccEMRCluster_CoreInstanceGroup_name
=== RUN   TestAccEMRCluster_EC2Attributes_defaultManagedSecurityGroups
=== PAUSE TestAccEMRCluster_EC2Attributes_defaultManagedSecurityGroups
=== RUN   TestAccEMRCluster_Kerberos_clusterDedicatedKdc
=== PAUSE TestAccEMRCluster_Kerberos_clusterDedicatedKdc
=== RUN   TestAccEMRCluster_MasterInstanceGroup_bidPrice
=== PAUSE TestAccEMRCluster_MasterInstanceGroup_bidPrice
=== RUN   TestAccEMRCluster_MasterInstanceGroup_instanceCount
=== PAUSE TestAccEMRCluster_MasterInstanceGroup_instanceCount
=== RUN   TestAccEMRCluster_MasterInstanceGroup_instanceType
=== PAUSE TestAccEMRCluster_MasterInstanceGroup_instanceType
=== RUN   TestAccEMRCluster_MasterInstanceGroup_name
=== PAUSE TestAccEMRCluster_MasterInstanceGroup_name
=== RUN   TestAccEMRCluster_security
=== PAUSE TestAccEMRCluster_security
=== RUN   TestAccEMRCluster_Step_basic
=== PAUSE TestAccEMRCluster_Step_basic
=== RUN   TestAccEMRCluster_Step_mode
=== PAUSE TestAccEMRCluster_Step_mode
=== RUN   TestAccEMRCluster_Step_multiple
=== PAUSE TestAccEMRCluster_Step_multiple
=== RUN   TestAccEMRCluster_Step_multiple_listStates
=== PAUSE TestAccEMRCluster_Step_multiple_listStates
=== RUN   TestAccEMRCluster_Bootstrap_ordering
=== PAUSE TestAccEMRCluster_Bootstrap_ordering
=== RUN   TestAccEMRCluster_PlacementGroupConfigs
=== PAUSE TestAccEMRCluster_PlacementGroupConfigs
=== RUN   TestAccEMRCluster_terminationProtected
=== PAUSE TestAccEMRCluster_terminationProtected
=== RUN   TestAccEMRCluster_keepJob
=== PAUSE TestAccEMRCluster_keepJob
=== RUN   TestAccEMRCluster_visibleToAllUsers
=== PAUSE TestAccEMRCluster_visibleToAllUsers
=== RUN   TestAccEMRCluster_s3Logging
=== PAUSE TestAccEMRCluster_s3Logging
=== RUN   TestAccEMRCluster_s3LogEncryption
=== PAUSE TestAccEMRCluster_s3LogEncryption
=== RUN   TestAccEMRCluster_tags
=== PAUSE TestAccEMRCluster_tags
=== RUN   TestAccEMRCluster_RootVolume_size
=== PAUSE TestAccEMRCluster_RootVolume_size
=== RUN   TestAccEMRCluster_StepConcurrency_level
=== PAUSE TestAccEMRCluster_StepConcurrency_level
=== RUN   TestAccEMRCluster_ebs
=== PAUSE TestAccEMRCluster_ebs
=== RUN   TestAccEMRCluster_CustomAMI_id
=== PAUSE TestAccEMRCluster_CustomAMI_id
=== RUN   TestAccEMRCluster_InstanceFleet_basic
=== PAUSE TestAccEMRCluster_InstanceFleet_basic
=== RUN   TestAccEMRCluster_InstanceFleetMaster_only
=== PAUSE TestAccEMRCluster_InstanceFleetMaster_only
=== RUN   TestAccEMRCluster_unhealthyNodeReplacement
=== PAUSE TestAccEMRCluster_unhealthyNodeReplacement
=== RUN   TestAccEMRCluster_osReleaseLabel
=== PAUSE TestAccEMRCluster_osReleaseLabel
=== CONT  TestAccEMRCluster_basic
=== CONT  TestAccEMRCluster_Step_multiple
=== CONT  TestAccEMRCluster_tags
=== CONT  TestAccEMRCluster_EC2Attributes_defaultManagedSecurityGroups
=== CONT  TestAccEMRCluster_InstanceFleet_basic
=== CONT  TestAccEMRCluster_s3LogEncryption
=== CONT  TestAccEMRCluster_CoreInstanceGroup_autoScalingPolicy
=== CONT  TestAccEMRCluster_keepJob
=== CONT  TestAccEMRCluster_PlacementGroupConfigs
--- PASS: TestAccEMRCluster_Step_multiple (343.73s)
--- PASS: TestAccEMRCluster_basic (374.19s)
=== CONT  TestAccEMRCluster_terminationProtected
--- PASS: TestAccEMRCluster_keepJob (403.30s)
=== CONT  TestAccEMRCluster_s3Logging
--- PASS: TestAccEMRCluster_CoreInstanceGroup_autoScalingPolicy (453.21s)
=== CONT  TestAccEMRCluster_CoreInstanceGroup_instanceType
--- PASS: TestAccEMRCluster_tags (587.51s)
=== CONT  TestAccEMRCluster_CoreInstanceGroup_name
--- PASS: TestAccEMRCluster_terminationProtected (375.59s)
=== CONT  TestAccEMRCluster_Bootstrap_ordering
--- PASS: TestAccEMRCluster_s3LogEncryption (777.71s)
=== CONT  TestAccEMRCluster_Step_multiple_listStates
--- PASS: TestAccEMRCluster_s3Logging (514.03s)
=== CONT  TestAccEMRCluster_ebs
--- PASS: TestAccEMRCluster_PlacementGroupConfigs (580.96s)
=== CONT  TestAccEMRCluster_CustomAMI_id
--- PASS: TestAccEMRCluster_Step_multiple_listStates (421.25s)
=== CONT  TestAccEMRCluster_unhealthyNodeReplacement
--- PASS: TestAccEMRCluster_CoreInstanceGroup_instanceType (789.70s)
=== CONT  TestAccEMRCluster_osReleaseLabel
--- PASS: TestAccEMRCluster_InstanceFleet_basic (1380.85s)
=== CONT  TestAccEMRCluster_InstanceFleetMaster_only
--- PASS: TestAccEMRCluster_CoreInstanceGroup_name (842.24s)
=== CONT  TestAccEMRCluster_MasterInstanceGroup_name
--- PASS: TestAccEMRCluster_Bootstrap_ordering (1086.27s)
=== CONT  TestAccEMRCluster_Step_mode
--- PASS: TestAccEMRCluster_EC2Attributes_defaultManagedSecurityGroups (1844.03s)
=== CONT  TestAccEMRCluster_Step_basic
--- PASS: TestAccEMRCluster_unhealthyNodeReplacement (691.53s)
=== CONT  TestAccEMRCluster_security
--- PASS: TestAccEMRCluster_osReleaseLabel (721.68s)
=== CONT  TestAccEMRCluster_StepConcurrency_level
=== CONT  TestAccEMRCluster_MasterInstanceGroup_instanceCount
--- PASS: TestAccEMRCluster_InstanceFleetMaster_only (650.75s)
=== CONT  TestAccEMRCluster_MasterInstanceGroup_instanceType
--- PASS: TestAccEMRCluster_MasterInstanceGroup_name (696.10s)
=== CONT  TestAccEMRCluster_RootVolume_size
--- PASS: TestAccEMRCluster_Step_basic (443.22s)
=== CONT  TestAccEMRCluster_visibleToAllUsers
--- PASS: TestAccEMRCluster_security (451.73s)
=== CONT  TestAccEMRCluster_CoreInstanceGroup_instanceCount
--- PASS: TestAccEMRCluster_StepConcurrency_level (512.24s)
=== CONT  TestAccEMRCluster_CoreInstanceGroup_bidPrice
--- PASS: TestAccEMRCluster_CoreInstanceGroup_instanceCount (1046.40s)
=== CONT  TestAccEMRCluster_Kerberos_clusterDedicatedKdc
--- PASS: TestAccEMRCluster_Step_mode (794.22s)
=== CONT  TestAccEMRCluster_MasterInstanceGroup_bidPrice
--- PASS: TestAccEMRCluster_MasterInstanceGroup_instanceType (655.46s)
=== CONT  TestAccEMRCluster_disappears
=== NAME  TestAccEMRCluster_visibleToAllUsers
    cluster_test.go:1297: Step 3/4 error: Check failed: Check 2/2 error: aws_emr_cluster.test: Attribute 'visible_to_all_users' expected "false", got "true"
--- PASS: TestAccEMRCluster_RootVolume_size (697.15s)
=== CONT  TestAccEMRCluster_sJSON
--- FAIL: TestAccEMRCluster_visibleToAllUsers (637.61s)
=== CONT  TestAccEMRCluster_additionalInfo
--- PASS: TestAccEMRCluster_disappears (345.48s)
=== CONT  TestAccEMRCluster_autoTerminationPolicy
--- PASS: TestAccEMRCluster_Kerberos_clusterDedicatedKdc (439.97s)
--- PASS: TestAccEMRCluster_MasterInstanceGroup_instanceCount (1091.32s)
--- PASS: TestAccEMRCluster_sJSON (308.75s)
--- PASS: TestAccEMRCluster_additionalInfo (379.02s)
--- PASS: TestAccEMRCluster_MasterInstanceGroup_bidPrice (673.73s)
--- PASS: TestAccEMRCluster_CoreInstanceGroup_bidPrice (892.45s)
--- PASS: TestAccEMRCluster_autoTerminationPolicy (708.26s)
=== NAME  TestAccEMRCluster_ebs
    cluster_test.go:1540: Step 1/2 error: Error running apply: exit status 1

        Error: waiting for EMR Cluster (j-1VHVE90R7M3RG) create: unexpected state 'TERMINATING', wanted target 'RUNNING, WAITING'. last error: INTERNAL_ERROR: Failed to start the job flow due to an internal error

          with aws_emr_cluster.test,
          on terraform_plugin_test.tf line 193, in resource "aws_emr_cluster" "test":
         193: resource "aws_emr_cluster" "test" {

--- FAIL: TestAccEMRCluster_ebs (2933.15s)
=== NAME  TestAccEMRCluster_CustomAMI_id
    cluster_test.go:1574: Step 1/2 error: Error running apply: exit status 1

        Error: waiting for EMR Cluster (j-5ZQSNUYR6LYN) create: unexpected state 'TERMINATING', wanted target 'RUNNING, WAITING'. last error: INTERNAL_ERROR: Failed to start the job flow due to an internal error

          with aws_emr_cluster.test,
          on terraform_plugin_test.tf line 305, in resource "aws_emr_cluster" "test":
         305: resource "aws_emr_cluster" "test" {

--- FAIL: TestAccEMRCluster_CustomAMI_id (2997.24s)
FAIL
FAIL    github.com/hashicorp/terraform-provider-aws/internal/service/emr        3922.175s
FAIL
make: *** [GNUmakefile:635: testacc] Error 1

$
...
@acwwat acwwat requested a review from a team as a code owner June 14, 2025 22:34
Copy link

Community Guidelines

This comment is added to every new Pull Request to provide quick reference to how the Terraform AWS Provider is maintained. Please review the information below, and thank you for contributing to the community that keeps the provider thriving! 🚀

Voting for Prioritization

  • Please vote on this Pull Request by adding a 👍 reaction to the original post to help the community and maintainers prioritize it.
  • Please see our prioritization guide for additional information on how the maintainers handle prioritization.
  • Please do not leave +1 or other comments that do not add relevant new information or questions; they generate extra noise for others following the Pull Request and do not help prioritize the request.

Pull Request Authors

  • Review the contribution guide relating to the type of change you are making to ensure all of the necessary steps have been taken.
  • Whether or not the branch has been rebased will not impact prioritization, but doing so is always a welcome surprise.
Copy link

github-actions bot commented Jun 14, 2025

✅ Thank you for correcting the previously detected issues! The maintainers appreciate your efforts to make the review process as smooth as possible.

@github-actions github-actions bot added needs-triage Waiting for first response or review from a maintainer. documentation Introduces or discusses updates to documentation. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. service/emr Issues and PRs that pertain to the emr service. size/M Managed by automation to categorize the size of a PR. external-maintainer Contribution from a trusted external contributor. labels Jun 14, 2025
@ewbankkit ewbankkit self-assigned this Jun 16, 2025
@github-actions github-actions bot added size/L Managed by automation to categorize the size of a PR. prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. labels Jun 16, 2025
Copy link
Contributor

@ewbankkit ewbankkit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀.

% make testacc TESTARGS='-run=TestAccEMRCluster_osReleaseLabel\|TestAccEMRCluster_basic\|TestAccEMRCluster_upgradeV5_100_0' PKG=emr
make: Verifying source code with gofmt...
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go1.23.10 test ./internal/service/emr/... -v -count 1 -parallel 20  -run=TestAccEMRCluster_osReleaseLabel\|TestAccEMRCluster_basic\|TestAccEMRCluster_upgradeV5_100_0 -timeout 360m -vet=off
2025/06/16 11:12:12 Initializing Terraform AWS Provider...
=== RUN   TestAccEMRCluster_basic
=== PAUSE TestAccEMRCluster_basic
=== RUN   TestAccEMRCluster_osReleaseLabel
=== PAUSE TestAccEMRCluster_osReleaseLabel
=== RUN   TestAccEMRCluster_upgradeV5_100_0
=== PAUSE TestAccEMRCluster_upgradeV5_100_0
=== CONT  TestAccEMRCluster_basic
=== CONT  TestAccEMRCluster_upgradeV5_100_0
=== CONT  TestAccEMRCluster_osReleaseLabel
--- PASS: TestAccEMRCluster_basic (292.57s)
--- PASS: TestAccEMRCluster_upgradeV5_100_0 (422.84s)
--- PASS: TestAccEMRCluster_osReleaseLabel (708.88s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/emr	713.775s
Copy link
Member

@jar-b jar-b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

@ewbankkit
Copy link
Contributor

@acwwat Thanks for the contribution 🎉 👏.

@ewbankkit ewbankkit merged commit a22f3f8 into hashicorp:main Jun 24, 2025
48 checks passed
Copy link

Warning

This Issue has been closed, meaning that any additional comments are much easier for the maintainers to miss. Please assume that the maintainers will not see them.

Ongoing conversations amongst community members are welcome, however, the issue will be locked after 30 days. Moving conversations to another venue, such as the AWS Provider forum, is recommended. If you have additional concerns, please open a new issue, referencing this one where needed.

@github-actions github-actions bot added this to the v6.1.0 milestone Jun 24, 2025
@acwwat acwwat deleted the f-aws_emr_cluster-add_os_release_label_arg branch June 26, 2025 00:16
@github-actions github-actions bot removed the prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. label Jun 26, 2025
Copy link

This functionality has been released in v6.1.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Introduces or discusses updates to documentation. enhancement Requests to existing resources that expand the functionality or scope. external-maintainer Contribution from a trusted external contributor. service/emr Issues and PRs that pertain to the emr service. size/L Managed by automation to categorize the size of a PR. size/M Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
3 participants