Tags: ai-dynamo/nixl
Tags
CI: key release wheel folders by build number instead of nixl sha (#2259 ) Release wheels published to `release/<ver>/<nixl-sha8>/` gave every rebuild of a commit the same folder, so a re-spin overwrote its predecessor in place and a half-finished upload was indistinguishable from a complete one. This publishes to `release/<ver>/<build-number>/` instead, with the nixl sha recorded as an Artifactory property rather than in the path. - `cuda_major` becomes a matrix axis instead of a job parameter, so one run covers both CUDA majors (20 cells) and a build-number folder still holds a complete cu12+cu13 set. `build-llm-container` keeps finding both variants under a single `WHEEL_BUILD_ID`; the nightly cron drops to one entry and `CUDA_MAJOR` is removed. - `pipeline_stop` now owns the release folder: on SUCCESS it stamps a folder-level `NIXL_SHA` (with `recursive=0`, so per-wheel values survive) and points `currentBuild.description` at the folder in the Artifactory UI; on anything else it deletes the folder, since a partial set has to be rebuilt anyway. Safe only because the folder is keyed by build number and owned by one build. - Publish attaches `NIXL_SHA`, `NIXL_VERSION`, `JOB_NAME` and `BUILD_NUMBER` to every wheel unconditionally — outside the `build_options.env` guard, so a built ref predating `--build-options-file` still records its sha. - The poller's published check becomes one AQL property query per release branch instead of one folder GET per commit, and triggers once per commit rather than per CUDA variant. Because only a green build sets the marker, a partial upload no longer reads as published — a gap the old check had. - The job description links the release tree in the Artifactory UI. - `verification/` is untouched: those folders are keyed by sha pair and shared between builds, so they are never deleted. Release folders now accumulate one per re-spin and `cleanup-spec.json` has no `release/**` retention rule. Left alone deliberately — purging published release artifacts is a policy call rather than a CI one. Two things need confirming against the live Artifactory on the first run: the UI tree URL shape (`/ui/repos/tree/General/<repo>/<path>`) and that AQL returns folder-level properties for `type: folder`. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Nightly releases now build CUDA 12 and CUDA 13 wheel variants together. - Release artifacts are organized by version and build number for easier identification. - Wheels include build, source-commit, and resolved build-option metadata. - **Bug Fixes** - Incomplete release folders are automatically removed after failed builds. - Release monitoring now detects published wheel sets more reliably and retries missing or failed releases. - Partial uploads are excluded from completion checks. - Duplicate builds are prevented while a release is in progress. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: NirWolfer <nwolfer@nvidia.com>
test: wait-for-wheels + publish-crate on gitlab_ci_runners, PR #2076
test: wait-for-wheels DAG (nixl-compliance.yml), PR #2076
chore: update attributions for 1.4.1 (#2129) Bump the nixl, nixl-cu12 and nixl-cu13 self-version entries from 1.4.0 to 1.4.1. No third-party dependency changes between 1.4.0 and 1.4.1: Rust crates verified 68/68 against Cargo.lock, all Meson wrap versions unchanged, and the wheel's torch/numpy deps are unchanged. ## What? _Describe what this PR is doing._ ## Why? _Justification for the PR. If there is an existing issue/bug, please reference it. For bug fixes, the 'Why?' and 'What?' can be merged into a single item._ ## How? _It is optional, but for complex PRs, please provide information about the design, architecture, approach, etc._ Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
nixl 1.4.0-rc3 — re-test compliance after nspect fix (MR !34)
nixl 1.4.0-rc2 — test release-automation compliance trigger
chore: 1.4.0 attributions update (#2088) Full re-verification of the C++ dependency surface against what the 1.4.0 build actually produces (contrib/Dockerfile.manylinux wheel builder on manylinux_2_28/AlmaLinux 8, contrib/Dockerfile container on Ubuntu 24.04), cross-checked against the shared objects bundled in a published nixl-cu13 wheel. Version corrections (attributed version did not match what is built/shipped): - grpc 1.17.0 -> 1.73.0 (GRPC_TAG; wheel ships libgrpc++.so.1.73.0) - protobuf 3.12.4 -> 31.0 (gRPC submodule; wheel ships libprotobuf.so.31.0.0) and fix License URL, which pointed at Boost's LICENSE_1_0.txt - c-ares 1.18.1 -> 1.19.1 (gRPC v1.73.0 submodule) - RE2 2023-03-01 -> 2022-04-01 (gRPC v1.73.0 submodule) - LibFabric 2.3.0 -> 1.21.0 (LIBFABRIC_VERSION in all three Dockerfiles) - libcurl 8.5.0 -> 8.19.0 (built from source for OpenSSL 3.x linkage) - libxml2 2.9.14 -> 2.15 (built from source; wheel ships libxml2.so.16.1.3) - cuda 12.8 -> 13.0, cudart 12.8 -> 13.0.88, cufile 1.13.1.3 -> 1.15.1.6 - NVIDIA DOCA 3.1.0 -> 3.3.0 (DOCA_v3.3.0 host package) - glibc 2.35 -> 2.39, libgcc-s1/libstdc++ 12.1.0 -> 14.2.0 (Ubuntu 24.04) - libunwind 1.3.2 -> 1.3.1, Google test 1.17.0 -> 1.14.0 Missing dependencies added: - hwloc 2.12.2 (BSD-3-Clause) - a required Meson dependency of the libfabric plugin, built from source in the wheel builder - openssl 3.0.16 (Apache-2.0) - built from source; gRPC, libcurl, the AWS SDK and the Azure SDK all link it - openldap 2.4.46 (OpenLDAP Public License v2.8) - pulled in through libcurl Mislabelled entry fixed: - libucm 48.0-1 -> libibumad 48.0-1; the entry's URLs and license already pointed at rdma-core, and libibumad was the only rdma-core component installed by both Dockerfiles that was not yet attributed Python: add the nixl-cu12 self-entry, which was missing while nixl and nixl-cu13 were present, even though the cu12 wheel is built and published. Verified unchanged: all Rust entries (59/59 Cargo.lock registry crates), all Meson subproject wraps, and the resolved Python dependency tree. ## What? _Describe what this PR is doing._ ## Why? _Justification for the PR. If there is an existing issue/bug, please reference it. For bug fixes, the 'Why?' and 'What?' can be merged into a single item._ ## How? _It is optional, but for complex PRs, please provide information about the design, architecture, approach, etc._
[release/1.3.2] libfabric: flush FI_MORE per rail (#1989) ## Summary Backports #1966 onto `release/1.3.2`. This cherry-picks `78ece0c9897f6751052514c7cb37fde6ab7228ec` as `9dce5cb6615e45cfd71a45fcb6f42cfa5b28dfa9`. ## Details - Flushes FI_MORE WRITE batches per physical rail instead of by descriptor position. - Keeps batching opt-in via an explicit `apply_fi_more` decision passed into `prepareAndSubmitTransfer`. - Preserves this release branch's inline `postXfer` flow while adapting the per-rail flush precompute from the original PR. ## Validation - `git diff --check HEAD~1 HEAD` Full Meson build was not run locally because `meson` / Python `mesonbuild` is not installed in this environment. Signed-off-by: Arun Karthik <akkart@amazon.com> Co-authored-by: Arun Karthik <akkart@amazon.com>
chore: update NIXL attributions to 1.3.1 (#1901) Bump the nixl and nixl-cu13 self-version headers in ATTRIBUTIONS-Python.md from 1.3.0 to 1.3.1 to match the released package version (pyproject is already at 1.3.1). No third-party dependency attributions changed for the 1.3.1 patch release. ## What? _Describe what this PR is doing._ ## Why? _Justification for the PR. If there is an existing issue/bug, please reference it. For bug fixes, the 'Why?' and 'What?' can be merged into a single item._ ## How? _It is optional, but for complex PRs, please provide information about the design, architecture, approach, etc._ <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Updated license attribution entries for two packages to reflect the latest versions. * Kept the same Apache 2.0 license details while refreshing the recorded package versions. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PreviousNext