Skip to content

feat(cli): configurable insert_mode (fail-fast / fail-at-end) for strict compile-failure handling - #241

Open
sebastianbraun25 wants to merge 1 commit into
VectifyAI:mainfrom
sebastianbraun25:feat/issue-239-insert-mode
Open

feat(cli): configurable insert_mode (fail-fast / fail-at-end) for strict compile-failure handling#241
sebastianbraun25 wants to merge 1 commit into
VectifyAI:mainfrom
sebastianbraun25:feat/issue-239-insert-mode

Conversation

@sebastianbraun25

@sebastianbraun25 sebastianbraun25 commented Aug 31, 2026

Copy link
Copy Markdown

Note

This PR was created in collaboration between a human and AI: implementation, tests, and
PR text were created by an AI assistant under the guidance and review of the human author.

Problem

openkb add currently reports a whole-file "added" outcome even when one or more individual
concepts/entities failed to generate during compilation. _compile_concepts in
openkb/agent/compiler.py collects exceptions from the per-concept/per-entity generation tasks
via asyncio.gather(..., return_exceptions=True), logs a [WARN] ... planned but only N written
line, and continues — but the file's hash is still registered and the source is still eligible for
auto_delete_added_files, so a partially-compiled document looks identical to a fully successful
one from the CLI/API caller's point of view.

Solution / Changes

  • New insert_mode config key (.openkb/config.yaml) with three values:
    • "normal" (default): unchanged behavior — a concept/entity generation failure during compile
      is logged as a warning and the file is still reported "added".
    • "fail-fast": the first concept/entity generation failure cancels every other still-pending
      generation in the batch and immediately raises ConceptCompilationError — nothing from the
      batch is written.
    • "fail-at-end": every planned concept/entity generation is attempted (so every failure for the
      document is logged in one pass) before ConceptCompilationError is raised if anything failed.
  • Both strict modes rely entirely on the existing mutation-snapshot rollback
    (openkb.add_coordinator/openkb.mutation) to discard the add and report it "failed" — no new
    rollback path needed. The existing "keep raw/ on failed" and "keep the debug log on a
    non-'added' outcome" behaviors already cover raw-file and log-preservation for strict mode.
  • _compile_concepts's three early-return paths (unparseable plan, scalar plan,
    all-items-filtered-as-malformed) now also raise under a strict insert_mode, not just individual
    concept/entity generation failures — a genuinely empty plan (nothing was ever planned) still
    counts as complete success in every mode.
  • compile_short_doc/compile_long_doc resolve insert_mode from the already-loaded KB config, so
    no CLI-level plumbing is needed.
  • Backward compatible: default "normal" behavior is unchanged.

Issues

…ict compile-failure handling

- Add `insert_mode` config key (`.openkb/config.yaml`) with three values:
  - "normal" (default): unchanged behavior — a concept/entity generation
    failure during compile is logged as a warning and the file is still
    reported "added".
  - "fail-fast": the first concept/entity generation failure cancels every
    other still-pending generation in the batch and immediately raises
    `ConceptCompilationError` — nothing from the batch is written.
  - "fail-at-end": every planned concept/entity generation is attempted (so
    every failure for the document is logged in one pass) before
    `ConceptCompilationError` is raised if anything failed.
- Both strict modes rely entirely on the existing mutation-snapshot rollback
  (`openkb.add_coordinator`/`openkb.mutation`) to discard the add and report
  it "failed" — no new rollback path needed. The existing "keep raw/ on
  failed" and "keep the debug log on a non-'added' outcome" behaviors already
  cover the raw-file and log-preservation requirements for strict mode.
- `_compile_concepts`'s three early-return paths (unparseable plan, scalar
  plan, all-items-filtered-as-malformed) now also raise under a strict
  insert_mode, not just individual concept/entity generation failures — a
  genuinely empty plan (nothing was ever planned) still counts as complete
  success in every mode.
- `compile_short_doc`/`compile_long_doc` resolve `insert_mode` from the
  already-loaded KB config, so no CLI-level plumbing is needed.

Resolves VectifyAI#239

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant