Skip to content

add workflow to automate running llm-sync-models script#9724

Merged
Light2Dark merged 4 commits into
mainfrom
sham/add-workflow-llm-info
May 30, 2026
Merged

add workflow to automate running llm-sync-models script#9724
Light2Dark merged 4 commits into
mainfrom
sham/add-workflow-llm-info

Conversation

@Light2Dark

Copy link
Copy Markdown
Member

📝 Summary

  • Add a weekly GitHub Actions workflow to sync the latest models.dev entries into packages/llm-info/data/models.yml.
  • Limit the sync to the 5 newest models per provider and open a draft PR when changes are found.

📋 Pre-Review Checklist

  • For large changes, or changes that affect the public API: this change was discussed or approved through an issue, on Discord, or the community discussions (Please provide a link if applicable).
  • Any AI generated code has been reviewed line-by-line by the human PR author, who stands by it.
  • Video or media evidence is provided for any visual changes (optional).

✅ Merge Checklist

  • I have read the contributor guidelines.
  • Documentation has been updated where applicable, including docstrings for API changes.
  • Tests have been added for the changes made.
Copilot AI review requested due to automatic review settings May 29, 2026 10:46
@vercel

vercel Bot commented May 29, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
marimo-docs Ready Ready Preview, Comment May 29, 2026 2:05pm

Request Review

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a GitHub Actions workflow to automate weekly synchronization of the LLM model catalog from models.dev into packages/llm-info/data/models.yml.

Changes:

  • Adds scheduled and manual workflow triggers for syncing LLM model metadata.
  • Runs sync-models, code generation, and llm-info tests.
  • Creates a draft PR when catalog changes are detected and no existing update PR is open.
Comment thread .github/workflows/sync-llm-info.yml
Comment thread .github/workflows/sync-llm-info.yml Outdated
Comment thread .github/workflows/sync-llm-info.yml Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 issues found across 1 file

Architecture diagram
sequenceDiagram
    participant Scheduler as Weekly Cron (GitHub)
    participant Workflow as GitHub Actions Workflow
    participant Repo as Git Repository
    participant ModelsDev as models.dev (External API)
    participant LLMInfo as llm-info package
    participant PRBot as PR Creation (gh CLI)

    Note over Scheduler,PRBot: Sync LLM Model Catalog Workflow

    Scheduler->>Workflow: Trigger (Monday 08:30 UTC or manual dispatch)
    Workflow->>Repo: Checkout code (fetch-depth: 0)
    Workflow->>LLMInfo: Install repo dependencies
    Workflow->>LLMInfo: Run sync-models -n 5
    LLMInfo->>ModelsDev: Fetch 5 newest models per provider
    ModelsDev-->>LLMInfo: Return model data
    LLMInfo->>LLMInfo: Update packages/llm-info/data/models.yml

    Workflow->>LLMInfo: Run codegen
    LLMInfo->>LLMInfo: Generate derived data
    Workflow->>LLMInfo: Run tests
    LLMInfo-->>Workflow: Test results

    Workflow->>Repo: Check for changes in models.yml
    alt Changes detected
        Workflow->>Workflow: Set has_changes=true
        Workflow->>Repo: Check for existing open PR with title "Update LLM model catalog"
        alt No existing PR
            Workflow->>Repo: Create new branch (automation/sync-llm-info-models-*)
            Workflow->>Repo: Commit updated models.yml
            Workflow->>Repo: Push branch
            Workflow->>PRBot: gh pr create --draft
            PRBot-->>Workflow: PR created
        else Existing PR found
            Workflow->>Workflow: Skip PR creation
        end
    else No changes
        Workflow->>Workflow: Skip PR creation
    end
Loading

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread .github/workflows/sync-llm-info.yml
Comment thread .github/workflows/sync-llm-info.yml Outdated
Comment thread .github/workflows/sync-llm-info.yml Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

@Light2Dark Light2Dark requested a review from kirangadhave May 29, 2026 14:11

@kirangadhave kirangadhave left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@Light2Dark Light2Dark merged commit 0e837b1 into main May 30, 2026
31 checks passed
@Light2Dark Light2Dark deleted the sham/add-workflow-llm-info branch May 30, 2026 02:06
Light2Dark added a commit that referenced this pull request Jun 26, 2026
**This pull request was authored by a coding agent.**

## 📝 Summary

The weekly [Sync LLM
models](https://github.com/marimo-team/marimo/actions/workflows/sync-llm-info.yml)
workflow has failed on every run since it was added (#9724). Sync,
codegen, tests, commit, and push all succeed, but `gh pr create` fails
with:

```
Resource not accessible by integration (createPullRequest)
```

The release GitHub App token has `contents: write` (enough to push
`automation/sync-llm-info-models`) but not `pull-requests: write`. I
switched the PR lookup and creation steps to `GITHUB_TOKEN`, which the
workflow already grants `pull-requests: write`. The app token is still
used for checkout and push.

## 📋 Pre-Review Checklist

- [ ] For large changes, or changes that affect the public API: this
change was discussed or approved through an issue, on
[Discord](https://marimo.io/discord?ref=pr), or the community
[discussions](https://github.com/marimo-team/marimo/discussions) (Please
provide a link if applicable).
- [x] Any AI generated code has been reviewed line-by-line by the human
PR author, who stands by it.
- [ ] Video or media evidence is provided for any visual changes
(optional).

## ✅ Merge Checklist

- [x] I have read the [contributor
guidelines](https://github.com/marimo-team/marimo/blob/main/CONTRIBUTING.md).
- [ ] Documentation has been updated where applicable, including
docstrings for API changes.
- [ ] Tests have been added for the changes made.

Made with [Cursor](https://cursor.com)

<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/marimo-team/marimo/pull/9998?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

3 participants