Skip to content

ci: skip builds when a change touches no build inputs - #3763

Open
KuSh wants to merge 1 commit into
rtk-ai:developfrom
KuSh:ci/path-filtering-189
Open

ci: skip builds when a change touches no build inputs#3763
KuSh wants to merge 1 commit into
rtk-ai:developfrom
KuSh:ci/path-filtering-189

Conversation

@KuSh

@KuSh KuSh commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator

What

Adds path-based change filtering to .github/, which had none. Every push to
develop ran pre-release plus 8 build jobs (5 platforms + DEB + RPM + Create
Release) and published a dev-*-rc.N prerelease; every PR ran 10 job instances.
Replaying the last 100 merges into develop, 8 were pure markdown and paid for
all of it.

.github/inert-paths.conf holds git pathspecs for paths that cannot change
the binary or CI. changed-scope.sh subtracts them from a diff range and
answers through its exit status. Git does the matching, so there is no glob
engine of ours to get wrong.

About #189 — read before merging

This does not implement what #189 proposed, and that is deliberate. The
issue asked to gate the binary build on the release path. Two problems, both
verified:

  1. The proposed diff can never return false. It checks
    git diff PREV_TAG..HEAD -- '*.rs' 'Cargo.toml' 'Cargo.lock' 'build.rs', but
    release-please's own release commit always bumps the version in Cargo.toml
    and Cargo.lock. code_changed would be true on every release.
  2. Skipping the build on master breaks installs. install.sh resolves the
    version by following the redirect on /releases/latest, then downloads that
    tag's tarballs by name. A stable release published without assets becomes
    "Latest" and every fresh curl | sh fails.

The savings #189 was after are taken on the develop and PR side instead, where
they are roughly 8x larger (369 RCs vs ~46 stable tags) and carry no such risk.
The master path is moved to its own workflow and left ungated.

Closing #189 on merge for that reason — if you would rather keep it open to
track the release-path idea, drop the Fixes line before merging.

Design

Every failure direction leads to building rather than to shipping a stale
artifact:

  • The list is inverted, so an unrecognised path builds.
  • The skip verdict is exit status 3, not 1, because bash hands out the low
    statuses itself (1 for set -u, 2 for syntax, 127 for missing). An accident
    cannot be mistaken for a skip.
  • Gated jobs skip only on an explicit build=false, and the gate's own guards
    run before it writes that, so a red gate runs CI rather than waving a PR
    through green.
  • --no-renames throughout: rename detection reports only a rename's
    destination, so moving an embedded file into docs/ would otherwise look
    inert on a commit that no longer compiles.

The ,glob trap

hooks/** is deliberately not excluded. src/hooks/init.rs include_str!s
ten files out of hooks/, six of them .md. A bare :(exclude)*.md would
swallow them, because without ,glob a pathspec * also matches / — and a
docs-only commit would then ship a stale binary.

check-embed-scope.sh enforces this mechanically: it resolves every
include_str!/include_bytes! target under all tracked *.rs (collapsing
newlines first, since rustfmt wraps long paths, then splitting each macro onto
its own line so grep -o's non-overlapping windows cannot let one embed hide
the next) and fails if any is excluded, missing, unresolvable, or written in a
form it cannot verify. It runs ungated, so it still fires on the docs-only PRs
that skip everything else.

Other changes

  • stable-release.yml — the master path moves out of cd.yml so a bug in
    the develop gate cannot reach a release. It is not path-gated.
  • verify-assets in release.yml — asserts all 8 assets uploaded non-empty
    and that checksums.txt names nothing that failed to upload. On failure it
    demotes the release to prerelease, taking it out of /releases/latest so
    installs fall back to the last good tag. Discord and the Homebrew tap wait on
    it. It detects and demotes; it cannot prevent — see Stable releases are 'Latest' for ~6-7 minutes before their assets exist, breaking install.sh #3759.
  • .claude/hooks/ is excluded from the build gate but routed to the
    security job via .github/security-paths.conf. Those shell scripts run on
    contributors' machines and semgrep is Rust-only; that job's pattern scan was
    Rust-only too, so it gains a Shell script scan step.
  • cd.yml diffs from the last RC rather than github.event.before: develop
    runs are cancel-in-progress, and a docs push landing on a cancelled source
    build would otherwise skip and strand that source change with no RC.
  • Security scan diff range. Three existing steps diffed
    origin/master...HEAD, which on a PR into develop spans every merge since
    the last release rather than the PR. On this PR that was 88 files against 10,
    and it reported Cargo.toml, tracking.rs and init.rs as "critical files
    modified" by a PR touching none of them. All four scans now share one
    BASE_SHA resolved once, so they report the PR's own changes. Resolving it
    once also lets each scan separate its git call from the grep that needs
    || true for a no-match — a failing git now fails the step instead of
    passing as a clean verdict.
  • pr-target-check.yml gains branches: [master], matching what its job
    already tested; every PR into develop was starting a run that only skipped
    (all 20 most recent runs did nothing), and it holds a pull-requests:write
    app token on pull_request_target. It also stops flagging release-please:
    its only exemption was the developmaster promotion, so release-please's
    own version-bump PR has been labelled wrong-base and told to target
    develop on every release since 0.37.1. Both exemptions are keyed by branch
    (the account behind them is not stable — the promotion has been opened by a
    human and by two different apps) and require the branch to live in this repo,
    since a fork's head branch name is the contributor's to choose.

Verification

  • changed-scope.sh --self-test — 40 assertions, wired into CI.
  • check-embed-scope.sh — 49 embed sites, verified non-vacuous by injecting a
    docs/ embed (flagged DRIFT) and by removing ,glob (flagged the six
    embedded hooks/*.md).
  • Historical replay over the last 100 merges into develop: 92 build /
    8 skip, and every file in all 8 skipped merges is markdown — no .rs, no
    hooks/, no Cargo.*.
  • verify-assets logic run against the real v0.46.0 release: clean on the
    good release, catches a dropped tarball in both checks, and aborts without
    demoting when gh is unavailable.
  • pr-target-check.yml replayed over the last 100 PRs into master: 17
    verdicts flip, all of them release-please's, 0 unintended exemptions, 65
    contributor PRs still flagged. The same-repo clause changes no verdict.
  • Security scans replayed against the real refs/pull/3763/merge: the summary
    now lists exactly the files this PR touches, and with BASE_SHA unset every
    scan fails hard rather than reporting clean.

Known limitations

Follow-ups opened while working on this

🤖 Generated with Claude Code

@amandeavor amandeavor left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the path classifier, embedded-file guard, CI dependency graph, prerelease range selection, and stable-release asset verification. The classifier passes all 40 self-test cases, the PR checks pass across Linux, Windows, and macOS, and the gate consistently fails open when it cannot prove a change is inert. I also checked the release demotion and downstream notification/Homebrew dependencies. I did not find a correctness blocker.

@pszymkowiak

Copy link
Copy Markdown
Collaborator

Solid PR overall — the design write-up and the ,glob trap section in particular are great, and the self-test/replay verification approach is exactly right.

One correctness bug worth fixing before merge, on the notify-discord/homebrew gate:

notify-discord:
  needs: [verify-assets]
  if: ${{ !inputs.prerelease }}

needs: changed from [release] to [verify-assets], but if: didn't change with it. GitHub Actions only applies an implicit success() check on needs when a job has no custom if: — the moment you add your own condition, that implicit gate is gone and you have to write the success check yourself:

"The success() function ... this status check function will cause a job to be skipped unless a previous job's status is success. It is recommended to use this expression whenever you use a custom condition alongside needs."
GitHub Docs: Using conditions to control job execution

So as written, both jobs run as soon as verify-assets completes — success or failure — as long as !inputs.prerelease holds. Since verify-assets demotes a broken release via gh release edit --prerelease and then exits 1 (rather than flipping inputs.prerelease, which is just the original workflow input), the demotion doesn't stop either downstream job: Discord announces the release and Homebrew's formula gets updated pointing at assets that were just flagged incomplete — the exact scenario "Discord and the Homebrew tap wait on it" in the PR description says this prevents.

Fix:

if: ${{ !inputs.prerelease && needs.verify-assets.result == 'success' }}

on both notify-discord and homebrew.

(Rest of the review — duplicated merge-ref guard, hardcoded asset list in verify-assets, inconsistent diff ranges between the two security-job steps, changed-scope.sh reimplementing what dorny/paths-filter already does — are cleanup/simplification notes, not blockers.)

@KuSh
KuSh force-pushed the ci/path-filtering-189 branch 5 times, most recently from e18a05e to c26851c Compare August 31, 2026 23:06
There was no path filtering anywhere in .github/. Every push to develop ran
pre-release plus 8 build jobs (5 platforms + DEB + RPM + Create Release) and
published a dev-*-rc.N prerelease, and every PR ran 10 job instances. Replaying
the last 100 merges into develop, 8 were pure markdown and paid for all of it.

.github/inert-paths.conf holds git pathspecs for the paths that cannot change
the binary or CI. changed-scope.sh subtracts them from a diff range and answers
through its exit status. Git does the matching, so there is no glob engine of
ours to get wrong.

Every failure direction leads to building rather than to shipping a stale
artifact:

- The list is inverted, so an unrecognised path builds.
- The skip verdict is status 3, not 1, because bash hands out the low statuses
  itself -- 1 for a set -u violation or a failed command, 2 for a syntax error,
  127 for a missing one. An accident cannot be mistaken for a skip.
- Gated jobs skip only on an explicit build=false, and the gate's own guards
  run before it writes that, so a red gate runs CI rather than waving the PR
  through green.
- --no-renames throughout: rename detection reports only a rename's
  destination, so moving an embedded file into docs/ would otherwise look inert
  on a commit that no longer compiles.

hooks/** is deliberately not excluded: src/hooks/init.rs include_str!s ten
files out of hooks/, six of them .md. A bare :(exclude)*.md would swallow them,
because without `,glob` a pathspec `*` also matches `/`. check-embed-scope.sh
fails the build if that ever happens, and runs ungated so it still fires on the
docs-only PRs that skip everything else.

.claude/hooks/ is excluded from the build gate but routed to the security job
through .github/security-paths.conf: those shell scripts run on contributors'
machines, and semgrep is Rust-only. That job's pattern scan was Rust-only too,
so it gains a shell scan.

check-embed-scope.sh splits each macro onto its own line before matching.
grep -o does not match overlapping windows, so the 80 characters trailing a
literal embed swallowed whichever embed followed it, and that one was dropped
as literal and never reported -- the exact concat!(env!(..)) form the guard
exists to catch was invisible whenever a plain include_str! preceded it.

pr-target-check.yml stops flagging release-please. Its only exemption was the
develop->master promotion, so release-please's own version-bump PR was labelled
wrong-base and told to target develop on every release since 0.37.1. Both
maintainer routes are now keyed by branch, because the account behind them is
not stable: the promotion has been opened by a human and by two different apps.
Replayed over the last 100 PRs into master, that flips 17 verdicts, all of them
release-please's, and exempts nothing else. Both exemptions are conditioned on
the branch living in this repo, because a fork's head branch name is the
contributor's to choose; every exempt route already satisfies that, so the
clause changes no verdict.

pr-target-check.yml also gains the matching base-branch filter. Its job only acts
on a PR into master, so every PR into develop was starting a run that skipped;
the last 20 runs did nothing. It also holds a pull-requests:write app token on
pull_request_target, so a run it cannot act on is exposure with no upside.

That job's three existing scans diffed origin/master...HEAD, which on a PR into
develop spans every merge since the last release rather than the PR. They now
share one resolved base with the new shell scan, so they report the PR's own
changes instead of naming files the author never touched. Resolving the base
once also means each scan separates its git call from the grep that needs
`|| true` for a no-match, so a git failure fails the step rather than passing
as a clean verdict.

cd.yml diffs from the last RC rather than from github.event.before: develop
runs are cancel-in-progress, and a docs push landing on a cancelled source
build would otherwise skip and strand that source change with no RC.

The master path moves to its own workflow, stable-release.yml, and is not
gated. install.sh follows the redirect on /releases/latest and downloads that
tag's tarballs, so a stable release without assets breaks every fresh install;
a bug in the develop gate must not be able to cause that. release.yml gains a
verify-assets job that asserts all 8 assets uploaded non-empty and that
checksums.txt names nothing that failed to upload, demoting the release out of
/releases/latest if not, with Discord and the Homebrew tap waiting on it.

Fixes rtk-ai#189

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@KuSh
KuSh force-pushed the ci/path-filtering-189 branch from c26851c to ecada25 Compare August 31, 2026 23:12
@KuSh

KuSh commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks — two of the cleanup notes turned out to be worth more than the blocker, and both are now fixed in the PR. On the blocker itself I think the rule is the other way round, and I'd rather not add the condition.

The implicit success() is not removed by a custom if:

The quote is real, but it says the opposite of the reading. From Expressions → status check functions:

A default status check of success() is applied unless you include one of these functions.

"These functions" is the status-check set — success(), always(), cancelled(), failure(). What drops the default is including one of those, not writing a custom condition. jobs.<job_id>.needs says the same from the other side:

If a job fails or is skipped, all jobs that need it are skipped unless the jobs use a conditional expression that causes the job to continue.

and its worked example of such an expression is always().

if: ${{ !inputs.prerelease }} contains no status check function, so success() still applies and both jobs skip when verify-assets fails.

The strongest evidence is inside the PR. verify-assets is the job that does have to run on failure, and the only reason it can is the !cancelled():

verify-assets:
  needs: [release]
  if: ${{ !cancelled() && !inputs.prerelease }}

If a custom if: already removed the implicit success(), that !cancelled() would be dead weight — !inputs.prerelease alone would run it on a failed release. It isn't dead weight: without it the job is skipped exactly when a release is incomplete, which was a HIGH finding in an earlier round here. The two facts are one rule seen from both ends — !cancelled() on verify-assets is load-bearing because the plain if: downstream is sufficient.

So needs.verify-assets.result == 'success' would be a no-op: both checks would have to pass and the implicit one already does the work. I'd rather leave it out, because a redundant success check sitting beside a deliberate !cancelled() reads as if the implicit gate were unreliable — which is what would invite someone to "fix" verify-assets the same way and quietly break the demotion path.

Happy to be shown wrong by a run: if you can point at one where a job with needs and a status-function-free if: ran after its dependency failed, I'll take the change.

Diff ranges — you were right, and it was worse than inconsistent

Fixed in the PR. Measured on this PR's own merge ref: origin/master...HEAD resolved to 88 files against the PR's 10, because on a PR into develop it spans every merge since the last release. The critical-files scan was reporting

.github/workflows/stale.yml
Cargo.toml
src/core/tracking.rs
src/hooks/init.rs

as modified by this PR, which touches none of them, and then demanding "Manual security review by 2 maintainers". Misattribution rather than noise — and a scan that cries wolf every time is one people learn to click past.

All four scans now share one BASE_SHA resolved once. It is also target-aware, which the old range was not: on a PR into develop it is the PR alone, and on a PR into master it is the whole release delta. (On the last release PR the old range even pulled in the previous release's CHANGELOG.md and .release-please-manifest.json, since merge-base(master, develop) predates release-please's own commits on master.)

That surfaced a second thing: grep returns 1 on no-match, so each step needed || true, which also swallowed a failing git diff and turned a broken scan into "No dangerous patterns detected". Each scan now separates its git call from the grep, so git failure fails the step. Verified by running them with BASE_SHA unset — all four exit 128 instead of reporting clean.

Duplicated merge-ref guard — your note is what made it worth doing

I had this as not worth factoring: one line of plumbing across two jobs, with opposite responses. Fixing the diff ranges changed the arithmetic — four steps needed the same base — so it now lives once, in a Resolve the PR diff base step placed before the toolchain install so a broken assumption costs seconds rather than minutes. The shell scan lost its private copy.

Hardcoded asset list — deliberate, and now commented

Deriving want.txt from the build matrix would drop a vanished matrix entry from both sides of the comm, so the check would agree nothing was missing. The independence is the point. Adding a target only grows have, which is ignored, so it cannot cause a false demotion.

dorny/paths-filter

Considered and rejected in the design, for a reason specific to this use: a file matching no filter yields all filters false, so an unclassified new path reads as "nothing to build" and skips. The fail-safe direction is inverted from what this gate needs. changed-scope.sh subtracts an inert list instead, so anything unlisted builds — which is also why the classification lives in one file both workflows read rather than in either workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants