feat(bundle): bundle the bit cli with esbuild - #10590
Draft
GiladShoham wants to merge 418 commits into
Draft
Conversation
…undle require() of the ESM-only mdx options package may be hijacked by a stale @babel/register pirates hook (left after the mocha tester runs in-process), which compiles the ESM source as CJS and crashes on the export token. load it with a native import() and pass it to createRspackConfig.
…dx aspect is not loaded the mdx aspect used to be a core aspect, so its import detector for md/mdx files was always registered. now that it's a regular env, docs imports (e.g. a component's .docs.mdx importing another component) silently dropped from the model when no workspace component loads the mdx aspect, breaking preview bundling in capsules. register a regex-based fallback detector in dependency-resolver; the mdx aspect's compile-based detector takes precedence whenever it is loaded.
…in md/mdx import detection imports shown inside fenced code blocks are documentation examples, not real ESM imports - detecting them produced self-import and missing-package issues for docs that demonstrate usage. also anchor imports to line starts, as MDX only treats block-level statements as ESM.
…e .js loader a tester running in-process (e.g. mocha) may leave a @babel/register pirates hook that replaces the .js extension loader and compiles ESM sources as CJS scripts, crashing on the export token (seen when the published react env lazily requires the ESM-only mdx options package during GeneratePreview). capture the pristine loader at bootstrap and retry a failed ESM require with it - node >= 22.12 handles require() of ESM natively.
…nvs-from-workspace # Conflicts: # .bitmap # components/legacy/e2e-helper/excluded-fixtures/extensions/dev-files-env/dev-files-env.extension.ts # components/legacy/e2e-helper/excluded-fixtures/extensions/env-add-dependencies/add-deps-env.extension.ts # components/legacy/e2e-helper/excluded-fixtures/extensions/node-env-dev-dep/node-env.extension.ts # pnpm-lock.yaml # scopes/react/ui/docs/apply-providers/apply-providers.tsx # scripts/e2e-test-timings.json
…nvs-from-workspace # Conflicts: # .bitmap
…d by getEnvId slot match
…om-manifest # Conflicts: # .bitmap # pnpm-lock.yaml # scopes/mdx/mdx/mdx.detector.spec.ts # scopes/mdx/mdx/mdx.detector.ts
…v source removed)
…y not-yet-compiled reloads
…om-manifest # Conflicts: # .bitmap
…om-manifest # Conflicts: # .bitmap # pnpm-lock.yaml
…, not only legacy
…om-manifest # Conflicts: # .bitmap # e2e/harmony/env.e2e.ts # pnpm-lock.yaml # scopes/compilation/compiler/compiler.main.runtime.ts # scopes/workspace/workspace/workspace.ts
…om-manifest # Conflicts: # .bitmap # e2e/harmony/deps-in-capsules.e2e.ts # e2e/harmony/dev-files.e2e.ts # e2e/harmony/pkg-manager-config.e2e.ts # e2e/harmony/root-components.e2e.ts # pnpm-lock.yaml
…after merging master
…om-manifest # Conflicts: # .bitmap # e2e/harmony/dependency-resolver.e2e.ts # pnpm-lock.yaml # scripts/generate-e2e-timings.js
…ortedArchitectures
…n in the grouped pipeline
…om-manifest # Conflicts: # .bitmap # pnpm-lock.yaml
…into bit-bundle3 # Conflicts: # .circleci/config.yml # pnpm-lock.yaml
…e medium+ The job now runs on every branch, so doc-only pushes (e.g. bundle-plan/*.md) still pay for the full 40-way e2e sweep. skip_when_doc_only already exists for exactly this and no-ops on master. Also drop the medium+ override, which predates the resource savings from bundling and no longer seems needed.
Adds bundle_push_build + a second bundle_publish_to_gcloud into build_and_test, gated to bit-bundle* branches: it reuses setup_esbuild_bundle/build_ui_prebundle's persisted workspace output instead of rebuilding from scratch, so no manual run_bundle_deploy trigger is needed to get an installable bvm dev build. resolve_bundle_version, pack_bvm_tars, and inject_ui_prebundle are extracted as shared commands so the manual bundle_deploy workflow and this one can't drift on how a tar is actually built. Also adds bundle-plan/25-pre-merge-cleanup.md, a running checklist of hygiene items to resolve before this branch merges into master.
Core aspects (aspect-loader, compiler, workspace, scope, typescript, preview, webpack, ...) satisfy isAspect() since they're themselves built with the aspect/env env, so resolveWorkspaceAspects' installed- legacy-env classification wrongly swept them into installedLegacyEnvs. That made them BFS roots for the later runtime-dependency closure, which are exempt from stop-boundaries by design, so the traversal walked into each core aspect's own internal @teambit/*.modules.* packages and tried resolving them as independently-installed npm packages - failing only in the esbuild-bundled CLI's sparse node_modules (e2e_test_esbuild_bundle, e2e_test_ui_prebundle), never in a full monorepo install. Excludes core aspect ids before they can enter installedLegacyEnvs, and adds a matching stop-boundary at core-aspect nodes in the legacyRuntimeClosure traversal itself as defense in depth.
…om-manifest # Conflicts: # .bitmap # e2e/harmony/custom-env.e2e.ts
…into b3merge # Conflicts: # pnpm-lock.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bundles the CLI into a single 60 MB CJS file plus 10 externals that cannot be inlined, with a generated shim package per core aspect re-exporting its slice of the bundle.
npm run bundlebuilds it;--seaalso builds a node single executable.1.2 GB / 141k files → 160 MB / ~2.8k files.
bit --help~0.53s warm.The shims emit the same
dist/*.aspect.jsanddist/*.main.runtime.jsfilenames the aspect loader already discovers, so the runtime needed no changes. The one source fix is real and independent of bundling:hook-requirepatchedmodule.constructor.prototype.require, which under any bundler installs an enumerablerequireonObject.prototype.bit startnow works too: it serves the pre-built UI/preview bundles instead of running a bundler at all (shouldServeBundleUi/writePreviewEntryhash-match and serve from the shippedartifacts/, nopublic/written). Verified end to end from a freshbit init+bit createworkspace: UI shell, workspace/scope roots, and the component's own preview all served from the pre-bundle.@rspack/dev-serverand, as of this PR,@rspack/coreitself (42 MB — was the single biggest external) are both fully excluded from the default build'spackage.json/node_modules, not just externalized — neither is reachable oncebit startserves the pre-bundle instead of rebuilding.Current size breakdown (measured on the real published-package shape,
inPlace: true):bit.app.js(the bundle)artifacts/, shipped inside the shims)node_modules)typescript23 MB,@pnpm22 MBThe UI pre-bundle shrink (82.7 MB → 16.7 MB) comes from upstream #10628 (SSR fix + minifier, 58→24 MB) and #10629 (single rspack compilation shared by both UI roots instead of two separate builds, 24→16 MB), both now merged into this branch, plus #10631's
bit startsanity e2e.esbuild's own
metafile.json(8.9 MB of build-analysis JSON, never read at runtime) is no longer written into the published package — still produced for localnpm run bundleiteration and CI's diagnostic capture.Producing the UI/preview pre-bundle locally needs a real
bit build --tasks BundleUI,PreBundlePreview; it's now cached under a gitignored.bundle-cache/(with a commit-hash + datemeta.json) sonode_moduleswipes don't force re-deriving it every time.Verified from an isolated dir: 40+ commands including
create,status,tag,export,import,watch,server,start, andbuild --unmodified(all 9 tasks, rspack included).npm run e2e-test:bundle/:searun the suite against the artifact; CircleCI builds it once insetup_esbuild_bundleand shares it across the e2e nodes (gated to^bit-bundle.*branches).Full architecture, measurements, externals breakdown, script-vs-SEA analysis, the publishable package layout and open questions are in
bundle-plan.md.Draft: based on
remove-core-envs-from-manifest, so the diff includes that branch. Opened to get CircleCI running.