QA: run_qa v1.6 form + ExplicitImports (root + sublibs)#984
Open
ChrisRackauckas-Claude wants to merge 5 commits into
Open
QA: run_qa v1.6 form + ExplicitImports (root + sublibs)#984ChrisRackauckas-Claude wants to merge 5 commits into
ChrisRackauckas-Claude wants to merge 5 commits into
Conversation
Convert every QA env (root test/qa and all 10 lib/<Sub>/test/qa) from the hand-rolled Aqua + separate explicit_imports.jl bodies to a single SciMLTesting.run_qa(...) call with explicit_imports = true, so all six ExplicitImports checks now run (the pre-1.6 files ran only three). Per env: - qa.jl -> `using SciMLTesting, <Module>, Test` (+ `using JET` for SciPy) and a single run_qa(<Module>; ...). Aqua sub-check tweaks (ambiguities recursive=false, piracies treat_as_own, stale_deps/deps_compat ignore-lists, persistent_tasks off on the umbrella) move into aqua_kwargs. ExplicitImports ignore/skip-lists move into ei_kwargs; genuinely-large findings use ei_broken. - test/qa/explicit_imports.jl removed (folded into run_qa); the QA group body in each runtests.jl simplified to `body = joinpath(@__DIR__, "qa", "qa.jl")`. - test/qa/Project.toml: + SciMLTesting (compat "1.6"); ExplicitImports dropped (transitive via SciMLTesting); Aqua kept (ambiguities runs) and JET kept (SciPy). [sources]/develop wiring preserved verbatim. ExplicitImports findings (FIX>IGNORE>BROKEN): the two new public-API checks (*_are_public) and all_explicit_imports_via_owners are satisfied with per-check name ignore-lists for non-public / re-exported names owned by NonlinearSolveBase, SciMLBase, SciMLLogging, ForwardDiff, CommonSolve, ArrayInterface, etc. Two large no_implicit_imports findings are marked ei_broken (Homotopy: ~20 names via heavy `using`; SciPy and the FirstOrder/QuasiNewton single-module cases use skip-lists). NonlinearSolveBase keeps its @verbosity_specifier macro-name no_stale ignore-list. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ent_tasks broken Verified each QA env against released SciMLTesting 1.6.0 (activate qa env, develop [sources], instantiate, run qa.jl). Expanded the ExplicitImports per-check ignore lists to the full set of non-public / re-exported names that surface once every extension in the env is loaded (the analyzable submodule set grows with the loaded trigger deps), for NonlinearSolveBase, SimpleNonlinearSolve, BracketingNonlinearSolve and the root umbrella. Switched the single-name no_implicit_imports skips (QuasiNewton SciMLOperators, FirstOrder CommonSolve.init, SciPy module names) to `ignore` (skip-by-module did not catch a module-name implicit import). Marked persistent_tasks `aqua_broken` for NonlinearSolveHomotopyContinuation and NonlinearSolveSciPy: their qa envs develop NonlinearSolveBase but not SciMLJacobianOperators, so on Julia >=1.11 Aqua's develop-by-path persistent-tasks check honors the leaked `[sources]` in the registered NonlinearSolveBase and errors on the missing sibling path. This is the upstream Pkg [sources]-leak bug (JuliaLang/Pkg.jl#4705 + JuliaTesting/Aqua.jl#387); it flakes the same way on master, is not a real persistent task, and run_qa's clean_sources mitigation cannot reach a broken entry that is not in the active env. Recorded Broken (not Fail) pending the fix. Sample verified green-or-broken (0 Fail) vs SciMLTesting 1.6.0: SciMLJacobianOperators 17/17, SCCNonlinearSolve 17/17, NonlinearSolveQuasiNewton 17/17, NonlinearSolveFirstOrder 17/17, NonlinearSolveSpectralMethods 17/17, BracketingNonlinearSolve 17/17, NonlinearSolveBase 17/17, SimpleNonlinearSolve 17/17, root 11/11; Homotopy 15 pass + 2 broken; SciPy 17 pass + 1 broken. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…round SciMLTesting 1.7.0 now runs the two public-API ExplicitImports checks (check_all_qualified_accesses_are_public / check_all_explicit_imports_are_public) only on Julia >= 1.11 (skipped on the 1.10 LTS), and SciMLBase 3.24.0, CommonSolve 0.2.9, and ArrayInterface 7.26.0 made many previously-internal names public. This sweeps the per-repo public-API ignore-lists in every qa.jl (root + all sublibs) down to only what is still genuinely non-public. Removed 75 now-public entries owned by SciMLBase / CommonSolve / ArrayInterface (e.g. build_solution, has_jac/jvp/vjp, NLStats, AbstractNonlinearProblem, AbstractDEAlgorithm, AutoSpecialize, init/solve/solve!/step!, can_setindex, fast_scalar_indexing, ismutable, isstructured, aos_to_soa, undefmatrix). Kept entries that are still non-public in their owning packages: NonlinearSolveBase's own internal API (the bulk), remaining SciMLBase internals (__init, __solve, NonlinearAliasSpecifier, NoSpecialize, AbstractNonlinearAlgorithm/Function, build_linear_solution, strip_solution, allowsbounds, Void, the *Originator types, the override/init/concrete-solve helpers, ...), and non-SciML deps (ForwardDiff, SciMLOperators, StaticArraysCore, Tracker, ReverseDiff, TaylorDiff, LinearSolve, SIAMFANLEquations, SpeedMapping, NLSolversBase, LinearAlgebra, Base, Core, FunctionWrappers). The no_implicit_imports / no_stale_explicit_imports / *_via_owners ignores and aqua_broken/ei_broken markers are untouched. Verified against the registered releases on Julia 1.12 (public checks run): all 11 QA groups green (Homotopy/SciPy retain their pre-existing aqua_broken/ei_broken). Verified on Julia 1.10 LTS (public checks skipped by SciMLTesting 1.7): green, 0 hard failures. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…MLTesting 1.7 The base-lib make-public round shipped SciMLBase 3.27.0 (60 newly-public names) and SciMLTesting 1.7.0 (gates the two public-API ExplicitImports checks to Julia >=1.11). Bump each qa env's SciMLTesting compat to "1.7" and re-derive the all_qualified_accesses_are_public / all_explicit_imports_are_public ignore lists against the registered releases on Julia 1.12, dropping every name that is now public and keeping only the genuine survivors. Names dropped (now public in SciMLBase 3.27.0): NoInit, OverrideInit, get_initial_values, allows_late_binding_tstops, allowsbounds, NonlinearAliasSpecifier, AbstractNonlinearAlgorithm. Survivors kept (still non-public in their owners): ForwardDiff (Dual/Partials/ Tag/value/...), Base/Core (add_sum/Compiler/return_type/Cartesian/setindex), LinearAlgebra (inv!/AdjOrTransAbsVec), FunctionWrappers, SciMLOperators (AbstractSciMLOperator), StaticArraysCore (StaticArray), SparseArrays, LeastSquaresOptim (Cholesky/LSMR/QR), TaylorDiff, Tracker/ReverseDiff, NLsolve re-export (NonDifferentiable), NonlinearSolveFirstOrder (RUS/Bastin), SciMLBase internals (AbstractSteadyStateProblem/__init/__solve/...), and NonlinearSolveBase's own internal API. Owner attributions corrected (Bastin/RUS -> NonlinearSolveFirstOrder, LSMR/QR/Cholesky -> LeastSquaresOptim). Verified on Julia 1.12 (public checks run) against the registered releases: all 11 QA groups green with 0 hard failures (root, NonlinearSolveBase, SimpleNonlinearSolve, Bracketing, FirstOrder, QuasiNewton, SpectralMethods, SCC, SciMLJacobianOperators 17/17; SciPy 17+1 persistent_tasks broken; Homotopy 15+2 broken). Verified on Julia 1.10 (public checks skipped per SciMLTesting 1.7): root 9/9, NonlinearSolveBase 15/15 green. via_owners / no_implicit_imports / no_stale_explicit_imports ignores and aqua_broken/ei_broken markers untouched. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reduce the run_qa ExplicitImports ignore/broken footprint across the monorepo by fixing the root cause rather than ignoring, verified green against released SciMLBase 3.30.0 / SciMLLogging / SciMLTesting 1.7.0 on Julia 1.12: - Drop now-public SciMLBase names from every ignore list (__init, __solve, has_initialization_data, AbstractODEIntegrator, AbstractSteadyStateProblem are public in SciMLBase 3.30.0). - Migrate SciMLLogging-owned re-exports (@SciMLMessage, AbstractVerbosityPreset, None) off NonlinearSolveBase to their public owner SciMLLogging in Bracketing / FirstOrder / QuasiNewton / SpectralMethods / SimpleNonlinearSolve (add SciMLLogging to [deps]/[compat]); this clears the all_explicit_imports_via_owners and all_explicit_imports_are_public flags. - Migrate ImmutableNonlinearProblem to its owner SciMLBase in the NonlinearSolveBase ForwardDiff ext and the SimpleNonlinearSolve Tracker/ReverseDiff/ChainRulesCore exts (drops via_owners flags). - Make Homotopy's ~20 implicit imports explicit (drop ei_broken no_implicit_imports entirely); make SciPy's CommonSolve/SciMLBase/ NonlinearSolveBase module names + FirstOrder's CommonSolve.init explicit (drop the no_implicit_imports ignores). Remaining exceptions are genuinely-irreducible non-public external API (NonlinearSolveBase internal API by design, ForwardDiff/Tracker/ReverseDiff/ TaylorDiff/LeastSquaresOptim internals, Base/Core/LinearAlgebra internals, NLSolversBase NonDifferentiable) plus the documented persistent_tasks aqua_broken (upstream Pkg [sources]-leak JuliaLang/Pkg.jl#4705). Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ignore until reviewed by @ChrisRackauckas.
Converts every QA env in the monorepo — the root
test/qaand all 10lib/<Sub>/test/qa— from the hand-rolledAqua.test_all+ separateexplicit_imports.jlbodies to a singleSciMLTesting.run_qa(...; explicit_imports = true)call (SciMLTesting 1.6.0 form). All six ExplicitImports checks now run (the pre-1.6 files ran only three); the standaloneexplicit_imports.jlfiles are removed and eachruntests.jlQA group body collapses tobody = joinpath(@__DIR__, "qa", "qa.jl").Per env: Aqua sub-check tweaks (ambiguities
recursive=false, piraciestreat_as_own,stale_deps/deps_compatignore-lists,persistent_tasks=falseon the umbrella) move intoaqua_kwargs; ExplicitImports ignore/skip-lists move intoei_kwargs.test/qa/Project.tomlgainsSciMLTesting(compat"1.6"), dropsExplicitImports(transitive via SciMLTesting), keepsAqua(ambiguities runs) andJET(SciPy). The[sources]/develop wiring is preserved verbatim.EI policy (FIX > IGNORE > BROKEN, goal 0 hard FAIL): the two new public-API checks and
all_explicit_imports_via_ownersare satisfied with per-check name ignore-lists for non-public / re-exported names owned by NonlinearSolveBase, SciMLBase, SciMLLogging, ForwardDiff, CommonSolve, ArrayInterface, etc. (grouped by source package in comments). Largeno_implicit_importsfindings useei_broken(Homotopy). NonlinearSolveBase keeps its@verbosity_specifiermacro-nameno_staleignore-list.Two
persistent_taskschecks (Homotopy, SciPy) are markedaqua_broken: their qa envs develop NonlinearSolveBase but not SciMLJacobianOperators, so on Julia ≥1.11 Aqua's develop-by-path check honors the registered NonlinearSolveBase's leaked[sources]and errors on the missing sibling path — the upstream Pkg[sources]-leak bug (JuliaLang/Pkg.jl#4705 + JuliaTesting/Aqua.jl#387). It flakes the same way on master, is not a real persistent task, andclean_sourcescannot reach an entry that is not in the active env. Recorded Broken (not Fail) pending the upstream fix.Verified locally vs released SciMLTesting 1.6.0 (activate qa env → develop
[sources]→ instantiate → run qa.jl), green-or-broken with 0 Fail:Sublibs converted: 10 (+ root). CI covers the full matrix.
🤖 Generated with Claude Code