Add steps to copy release images to Docker Hub in Buildkite pipeline by teresaromero · Pull Request #1692 · elastic/package-registry

teresaromero · 2026-04-28T12:29:30Z

Summary

Adds Buildkite pipeline steps to copy released Docker images from docker.elastic.co to DockerHub (hub.docker.com/u/elastic) as part of the release process.

Changes

`package-registry/package-registry` (empty image)

On every semver release tag (v*.*.*), two new trigger steps copy the standard and UBI images to DockerHub after the publish step completes:

package-registry/package-registry:<tag>
package-registry/package-registry:<tag>-ubi

`package-registry/distribution` (distribution image)

The existing release distribution pipeline now also copies all four image variants to DockerHub after the release steps complete:

package-registry/distribution:<tag> (production)
package-registry/distribution:<tag>-ubi (production UBI)
package-registry/distribution:lite-<tag> (lite)
package-registry/distribution:lite-<tag>-ubi (lite UBI)

Notes

The original issue scoped only the empty package-registry/package-registry image for DockerHub publishing, due to concerns about the large size of distribution images. This PR also includes the distribution image variants to cover all images released by these pipelines — however, whether package-registry/distribution (and in particular the lite variants) should be published to DockerHub was not explicitly confirmed. This is flagged for reviewer awareness.

Related https://github.com/elastic/platform-engineering-productivity/issues/954#issuecomment-4305256985

- Introduced new steps in `.buildkite/pipeline.yml` to trigger the copying of release images and UBI images to Docker Hub based on version tags. - Updated `.buildkite/release-package-registry-distribution.yml` to include commands for copying release images, UBI images, and lite images to Docker Hub, with appropriate tagging.

prodsecmachine · 2026-04-28T12:29:44Z

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues
✅	Licenses	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

mrodm · 2026-04-28T17:28:10Z

      - elastic/vault-docker-login#v0.6.0:
          secret_path: 'kv/ci-shared/platform-ingest/elastic_docker_registry'

+  - trigger: unified-release-copy-elastic-images-to-dockerhub


Does this pipeline offer a dry run option ? It would be interesting being able to test these pipelines before a new tag is pushed.

Hey @mrodm unified-release-copy-elastic-images-to-dockerhub does not currently support a dry-run mode. When the job runs, it performs an actual crane copy from docker.elastic.co to Docker Hub ( index.docker.io).

The pipeline is driven by IMAGES_NAMES (comma-separated namespace/name) and IMAGES_TAG.
To test before using a production tag, you can try triggering the pipeline with a non-production tag that already exists upstream and verify the results on Docker Hub.

…dockerhub

…opying step - Updated the command syntax in `.buildkite/release-package-registry-distribution.yml` to use a single-line format for improved readability and consistency.

…ution - Updated the notification step to clarify pipeline execution conditions. - Improved error messaging in the tag validation script for better user guidance. - Added concurrency and retry settings to the release distribution steps for improved reliability. - Changed async behavior to false for Docker image copy steps to ensure sequential execution.

teresaromero · 2026-05-08T09:19:05Z

Added some changes with the feedback from the official buildkite skill to review pipelines

for https://github.com/elastic/package-registry/pull/1692/changes#diff-c7cfaf86811e41aa2b847f18fdce17cb8285ab9fd1e337faa1431e0457bfa08f

  • Make pipeline intent visible in the UI: Swapped a log-only echo step for a buildkite-agent annotate message so the “who
    should run this and how” contract is visible on the build page without digging into logs.

  • Avoid conditional dependency traps: The validate step depended on an input step that is skipped for non-UI triggers.
    Adding allow_dependency_failure: true prevents builds from getting stuck or behaving inconsistently when a dependency is
     skipped.

  • Improve resilience to transient infra failures: Added retry.automatic for common transient exit statuses (-1, 143, 255)
    so agent loss/spot termination/connection hiccups don’t require manual reruns.

  • Prevent concurrent release races: Added concurrency + concurrency_group so two releases can’t run at once and fight over
     the same Docker tags (a common release-pipeline hazard).

  • Make downstream trigger results observable and retries safer: Switched Docker Hub copy triggers to async: false so the
    build reflects their success/failure, and added stable key: values to make retried uploads/steps more deterministic and
    easier to reason about.

for https://github.com/elastic/package-registry/pull/1692/changes#diff-a3991ebf1475eb82acab13946ffc1eca02b43917f6d5bec3898f45c8b0b9bd53

  • Added timeouts to prevent hung jobs consuming agents indefinitely:
    • lint/build: 15 min
    • unit tests + smoke test + publish: 30 min
  • Added automatic retries for transient infra issues (exit_status: -1, 143, 255) on
    lint/build/smoke/tests/publish.
  • Added per-branch concurrency guard on publish to prevent tag/branch image races:
    • concurrency_group: "package-registry/publish-${BUILDKITE_BRANCH}", ordered
  • Saved CI compute on failures: cancel_on_build_failing: true on each OS unit-test step.
  • Made Docker Hub copy triggers observable: switched the two dockerhub copy triggers to async: false so the
    build reflects copy success/failure.
  • Documented release-tag gating: added a comment explaining the tag regex is release-only and excludes
    prereleases by design.

- Added timeout and retry configurations for lint, build, smoke test, and test steps to improve reliability. - Set cancel_on_build_failing to true for test steps to halt execution on failure. - Updated Docker image publishing step to include concurrency settings and adjusted async behavior to false for sequential execution.

mrodm · 2026-05-08T09:57:11Z

        key: test-linux
        command:
          - ".buildkite/scripts/run-tests.sh"
+        cancel_on_build_failing: true


I think I will keep this with the default value (false).

If these steps are run, we get more information about all the errors in test even if the first steps fail.

mrodm · 2026-05-08T09:57:14Z

+            - exit_status: 143
+              limit: 2
+            - exit_status: 255
+              limit: 2


I think I would keep these steps as before.
Not sure about the exit codes that the scripts will raise to allow retrying.

mrodm · 2026-05-08T12:59:47Z

    depends_on:
      - step: "input"
        allow_failure: false
+    allow_dependency_failure: true


Adding this allow_dependency_failure setting, would it be the same as the current depends_on field?

nop, this was a wrong recommendation, as adding this will continue with the release regardless of any dependency failing...

…dockerhub

Co-authored-by: Mario Rodriguez Molins <marrodmo@gmail.com>

elasticmachine · 2026-05-15T09:10:58Z

💚 Build Succeeded

Buildkite Build
Commit: 2e048e1

History

💚 Build #1987 succeeded fbd84e3
💚 Build #1983 succeeded cb8358a
💚 Build #1974 succeeded c511822
💚 Build #1973 succeeded c6f571e
💚 Build #1949 succeeded 2fe9a19

mrodm

LGTM

It will require to pay attention to these pipeline when new tags are pushed:

New tag pushed to package-registry via GitHub releases that triggers the pipeline https://buildkite.com/elastic/package-registry
New tag pushed to fleet-server that triggers the pipeline https://buildkite.com/elastic/package-registry-release-package-registry-distribution

teresaromero requested a review from a team as a code owner April 28, 2026 12:29

mrodm reviewed Apr 28, 2026

View reviewed changes

teresaromero added 5 commits May 8, 2026 10:46

Merge branch 'main' of github.com:elastic/package-registry into push-…

1fcf404

…dockerhub

Refactor command syntax in Buildkite configuration for Docker image c…

e66b6a6

…opying step - Updated the command syntax in `.buildkite/release-package-registry-distribution.yml` to use a single-line format for improved readability and consistency.

Add dummy test step to Buildkite pipeline configuration

d3e8432

Remove dummy test step from Buildkite pipeline configuration

c6f571e

mrodm reviewed May 8, 2026

View reviewed changes

teresaromero added 4 commits May 11, 2026 14:04

Revert timeouts and options

f1fed89

Revert timeouts and label

bb67b04

Revert validate-tag

2e3c9a5

Merge branch 'main' of github.com:elastic/package-registry into push-…

cb8358a

…dockerhub

mrodm reviewed May 12, 2026

View reviewed changes

Comment thread .buildkite/release-package-registry-distribution.yml

Comment thread .buildkite/release-package-registry-distribution.yml Outdated

Update .buildkite/release-package-registry-distribution.yml

fbd84e3

Co-authored-by: Mario Rodriguez Molins <marrodmo@gmail.com>

mrodm reviewed May 13, 2026

View reviewed changes

Comment thread .buildkite/release-package-registry-distribution.yml

Update .buildkite/release-package-registry-distribution.yml

2e048e1

Co-authored-by: Mario Rodriguez Molins <marrodmo@gmail.com>

mrodm approved these changes May 15, 2026

View reviewed changes

teresaromero merged commit daf7987 into elastic:main May 18, 2026
6 checks passed

teresaromero mentioned this pull request Jul 3, 2026

Publish distribution image to DockerHub as package-registry-distribution #1808

Merged

Navigation Menu

Uh oh!

Add steps to copy release images to Docker Hub in Buildkite pipeline - #1692

Add steps to copy release images to Docker Hub in Buildkite pipeline#1692
teresaromero merged 13 commits into
elastic:mainfrom
teresaromero:push-dockerhub

teresaromero commented Apr 28, 2026 •

edited

Loading

prodsecmachine commented Apr 28, 2026 •

edited

Loading

Uh oh!

Uh oh!

mrodm Apr 28, 2026

navyau09 May 6, 2026

teresaromero commented May 8, 2026 •

edited

Loading

mrodm May 8, 2026

mrodm May 8, 2026

mrodm May 8, 2026

teresaromero May 13, 2026

Uh oh!

Uh oh!

Uh oh!

elasticmachine commented May 15, 2026

mrodm left a comment

Uh oh!

Labels

5 participants

Uh oh!

Conversation

teresaromero commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

package-registry/package-registry (empty image)

package-registry/distribution (distribution image)

Notes

prodsecmachine commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Snyk checks have passed. No issues have been found so far.

Uh oh!

Uh oh!

mrodm Apr 28, 2026

Choose a reason for hiding this comment

navyau09 May 6, 2026

Choose a reason for hiding this comment

teresaromero commented May 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

mrodm May 8, 2026

Choose a reason for hiding this comment

mrodm May 8, 2026

Choose a reason for hiding this comment

mrodm May 8, 2026

Choose a reason for hiding this comment

teresaromero May 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

elasticmachine commented May 15, 2026

💚 Build Succeeded

History

mrodm left a comment

Choose a reason for hiding this comment

Uh oh!

Labels

5 participants

teresaromero commented Apr 28, 2026 •

edited

Loading

`package-registry/package-registry` (empty image)

`package-registry/distribution` (distribution image)

prodsecmachine commented Apr 28, 2026 •

edited

Loading

teresaromero commented May 8, 2026 •

edited

Loading