Split CI: depot for unit tests, Xcode Cloud for UI tests#659
Split CI: depot for unit tests, Xcode Cloud for UI tests#659lawrencecchen wants to merge 3 commits intomainfrom
Conversation
Depot runners can't run XCUITests (no GUI session). Move UI tests to Xcode Cloud (ci_post_clone.sh already handles xcframework download). Depot now only runs unit tests with a pre-built xcframework download. Also trigger nightly builds on every merge to main with top-level concurrency so rapid merges cancel in-progress runs. Switch nightly from self-hosted to depot-macos-latest.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review infoConfiguration used: defaults Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughCI workflows were changed to run on Changes
Sequence Diagram(s)mermaid Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Greptile SummarySuccessfully splits CI responsibilities: depot handles fast unit tests (~2-3 min) while Xcode Cloud handles UI tests. The migration from self-hosted to depot-macos-latest removes infrastructure dependencies and improves CI reliability. Key changes include:
Confidence Score: 5/5
Important Files Changed
Last reviewed commit: 36facd2 |
![chatgpt-codex-connector[bot]](https://cdn.statically.io/img/avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 36facd2846
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| concurrency: | ||
| group: nightly | ||
| cancel-in-progress: true |
There was a problem hiding this comment.
With schedule still enabled hourly and workflow-level concurrency.cancel-in-progress: true, any nightly run that lasts past the next :30 tick will be canceled by the new scheduled invocation before it can finish tagging/releasing. This creates a starvation loop in slow/notarization-heavy runs where nightlies can keep restarting and never complete, so the cancellation policy should be scoped (for example, only for push runs) rather than applied globally.
Useful? React with 👍 / 👎.
![coderabbitai[bot]](https://cdn.statically.io/img/avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (2)
.github/workflows/ci.yml (1)
92-94: Consider verifying archive integrity.The download and extraction work correctly, but for a build dependency, you might want to verify the archive's integrity with a checksum before extraction—especially since this artifact directly affects the build output.
♻️ Optional: Add checksum verification
tar xzf GhosttyKit.xcframework.tar.gz rm GhosttyKit.xcframework.tar.gz test -d GhosttyKit.xcframework + # Optional: verify expected contents exist + test -d GhosttyKit.xcframework/macos-arm64_x86_64🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/ci.yml around lines 92 - 94, Add a checksum verification step before extracting GhosttyKit.xcframework.tar.gz: compute or obtain the expected SHA256 (e.g. from an env var or accompanying .sha256 file) and validate the archive (instead of directly running "tar xzf GhosttyKit.xcframework.tar.gz"), failing the job if the checksum does not match; keep the subsequent "tar xzf GhosttyKit.xcframework.tar.gz", "rm GhosttyKit.xcframework.tar.gz", and "test -d GhosttyKit.xcframework" lines, but precede them with the checksum check so the workflow aborts on integrity failure..github/workflows/nightly.yml (1)
90-90: Depot runner label is valid for third-party CI.The actionlint warning about
depot-macos-latestbeing an unknown label is a false positive. Depot is a third-party CI provider that offers custom macOS runners. To silence this warning locally, you can add the label to anactionlint.yamlconfig file:self-hosted-runner: labels: - depot-macos-latest🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/nightly.yml at line 90, The actionlint warning is a false positive for the custom runner label used in the workflow; add an actionlint config to whitelist that label by creating an actionlint.yaml containing a self-hosted-runner > labels entry that includes the string "depot-macos-latest" so actionlint recognizes the runs-on: depot-macos-latest value in the nightly.yml workflow.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In @.github/workflows/ci.yml:
- Around line 92-94: Add a checksum verification step before extracting
GhosttyKit.xcframework.tar.gz: compute or obtain the expected SHA256 (e.g. from
an env var or accompanying .sha256 file) and validate the archive (instead of
directly running "tar xzf GhosttyKit.xcframework.tar.gz"), failing the job if
the checksum does not match; keep the subsequent "tar xzf
GhosttyKit.xcframework.tar.gz", "rm GhosttyKit.xcframework.tar.gz", and "test -d
GhosttyKit.xcframework" lines, but precede them with the checksum check so the
workflow aborts on integrity failure.
In @.github/workflows/nightly.yml:
- Line 90: The actionlint warning is a false positive for the custom runner
label used in the workflow; add an actionlint config to whitelist that label by
creating an actionlint.yaml containing a self-hosted-runner > labels entry that
includes the string "depot-macos-latest" so actionlint recognizes the runs-on:
depot-macos-latest value in the nightly.yml workflow.
The cmux scheme had buildConfiguration="Release" for TestAction, which strips all #if DEBUG test instrumentation code. The app launches but never writes setup data files because the UI test support code is compiled out. Switch to Debug so Xcode Cloud UI tests work.
1 participant
Summary
runs-onfromself-hostedtodepot-macos-latest. Download pre-built GhosttyKit.xcframework instead of building from source (same curl+retry pattern asci_post_clone.sh). Unit tests only (~2-3 min).push: [main]trigger so every merge builds a nightly immediately. Add top-levelconcurrency: { group: nightly, cancel-in-progress: true }so rapid merges cancel stale runs. Switch fromself-hostedtodepot-macos-latest.depot-macosinstead ofself-hosted.UI tests move to Xcode Cloud (manual setup in Xcode,
ci_post_clone.shhook already merged in PR #447).Test plan
testsjob runs unit tests only and passes quicklySummary by CodeRabbit
Chores
Tests