Fix discussion bot: invalid Copilot model + silent drop on bare repo links - #45
Conversation
…links - actions/ai-inference@v3 requests model gpt-5.5, which isn't a valid Copilot CLI model (docs list gpt-4.1 as default, claude-sonnet-4.5 as the other example). Every real comment hit "Copilot CLI exited with code 1" in the parse step, confirmed in runs 31776838711 and 31778709534. Switch to gpt-4.1. - When Copilot parsing is unavailable, the URL-fallback path only kept candidates that already included a release tag, so a bare repo link (e.g. "https://github.com/org/repo") produced action=ignore and no reply at all -- the bot silently dropped genuine submission attempts. Now a bare GitHub link with no candidates falls back to action=question, which already has a reply asking for the full release URL.
Untracked __pycache__/*.pyc showed up from local script testing; exclude it going forward instead of committing build artifacts.
The ai-inference action masks the CLI's stderr unless ACTIONS_STEP_DEBUG is set at the org/repo level. Model swap (gpt-5.5 -> gpt-4.1) made no difference to the "exited with code 1" failure, so invoke the CLI directly here to see the real error and find the actual root cause. To be reverted once diagnosed.
Confirmed via a raw, unmasked Copilot CLI invocation (run 31781411798):
Error: Authentication token found but could not be validated.
Failed to fetch PAT user login (401): GitHub returned: Bad credentials
COPILOT_PAT is expired/revoked -- auth fails before any model or prompt
is considered. That's why swapping gpt-5.5 for gpt-4.1 changed nothing,
and why PR #44's "only run if COPILOT_PAT is set" approach wouldn't have
helped either: the secret is set, it's just no longer a valid token.
This requires regenerating the PAT and updating the repo secret; no
workflow or script change can fix it. Debug step served its purpose,
removing it.
![chatgpt-codex-connector[bot]](https://cdn.statically.io/img/avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e5bedb2920
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| def has_github_link(comment: str) -> bool: | ||
| return re.search(r"https?://github\.com/[^\s<>)\]\"']+", comment or "") is not None |
There was a problem hiding this comment.
When Copilot parsing is unavailable, this predicate also matches unrelated GitHub URLs such as issue, pull-request, profile, and documentation links. Because those URLs produce no candidate, a comment like Can someone explain https://github.com/acme/widget/issues/42? is misclassified as a submission clarification and receives the canned request for a release URL. Parse the URL as an actual owner/repository root before activating this fallback.
Useful? React with 👍 / 👎.
| system-prompt-file: .github/prompts/parse-app-submission-system.md | ||
| prompt-file: runtime-parse-prompt.txt | ||
| model: gpt-5.5 | ||
| model: gpt-4.1 |
There was a problem hiding this comment.
Once parsing succeeds with this model, a bare-repository submission can produce the schema-valid candidate {"repo_url":"https://github.com/acme/paper","tag":""}. candidates_from_parsed() retains that as ("acme/paper", ""), so it bypasses the new no-candidate clarification path and calls verify() with an empty tag, replying with an HTTP verification failure instead of requesting the release URL. Filter tagless parsed candidates or route them to the clarification response before restoring this parser path.
Useful? React with 👍 / 👎.
| system-prompt-file: .github/prompts/parse-app-submission-system.md | ||
| prompt-file: runtime-parse-prompt.txt | ||
| model: gpt-5.5 | ||
| model: gpt-4.1 |
There was a problem hiding this comment.
In the inspected discussion-bot workflow, changing this input does not restore parsing because the raw Copilot CLI diagnostic recorded in run 31781411798 fails earlier while validating COPILOT_PAT with 401 Bad credentials; the follow-up diagnosis in repository history explicitly reports that swapping gpt-5.5 for gpt-4.1 made no difference. Until the secret is regenerated and updated, every comment still takes the limited URL fallback, so questions and submissions without a recognizable URL remain unhandled despite this commit claiming to repair the bot.
Useful? React with 👍 / 👎.
Still getting "Copilot CLI exited with code 1" after the PAT rotation. Re-checking the raw CLI error to see whether it's still a bad-credentials issue or something new (e.g. no Copilot entitlement on the token's account). To be reverted once diagnosed.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 21f31f8d1f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_PAT }} | ||
| run: | | ||
| set +e | ||
| copilot -p "say hi" -s --no-ask-user --model gpt-4.1 |
There was a problem hiding this comment.
In the inspected discussion_comment path, every non-bot comment now executes this diagnostic prompt before the real actions/ai-inference parse. When COPILOT_PAT is valid, copilot -p "say hi" performs an additional unrelated model invocation for every comment, consuming quota and adding latency without affecting the bot result; when authentication is broken, it still adds a redundant failing step. Remove this temporary diagnostic or restrict it to an explicit manual debug input.
Useful? React with 👍 / 👎.
Auth is now fully working (COPILOT_PAT fine-grained token with Copilot Requests permission). New error: "Model \"gpt-4.1\" from --model flag is not available" -- the ai-inference action's hardcoded default model isn't entitled on this account/plan. Rather than guess another model name, omit --model entirely (empty model input) so the CLI uses whatever model it's actually entitled to.
Root cause chain, fully diagnosed and fixed: 1. COPILOT_PAT was expired/revoked (401 Bad credentials). 2. Rotated PAT was a classic token; Copilot CLI requires fine-grained. 3. Fine-grained PAT was missing the "Copilot Requests" permission. 4. Once authenticated, the hardcoded default model "gpt-4.1" wasn't entitled on this account/plan. Fixed by rotating to a properly-scoped fine-grained PAT and letting the Copilot CLI pick its own default model (empty model input) instead of hardcoding one, so this doesn't silently break again if the entitled model changes. Verified end-to-end via workflow_dispatch (run 31786772162): Copilot correctly parsed a natural-language submission, extracted the release candidate, verification passed, and the bot produced the expected should_update/should_reply/added_count result -- no more URL-only fallback.
2 participants
Copilot CLI model (docs list gpt-4.1 as default, claude-sonnet-4.5 as
the other example). Every real comment hit "Copilot CLI exited with
code 1" in the parse step, confirmed in runs 31776838711 and
31778709534. Switch to gpt-4.1.
candidates that already included a release tag, so a bare repo link
(e.g. "https://github.com/org/repo") produced action=ignore and no
reply at all -- the bot silently dropped genuine submission attempts.
Now a bare GitHub link with no candidates falls back to action=question,
which already has a reply asking for the full release URL.