Skip to content

docs: parallel.md advertises unsupported "tests" mode; ExecutionValidator rejects it with a misleading error #35

Description

@shossain786

Summary

docs-site/docs/guides/parallel.md documents parallel: tests as a supported TestNG mode, but ExecutionValidator (called unconditionally at suite bootstrap) rejects any value other than none, methods, classes. Any user who follows the guide's own table and sets parallel: tests gets a suite that fails to start, with a misleading error message.

Reproduction

execution:
  mode: local
  parallel: tests
  threadCount: 4

browser:
  name: chrome

timeouts:
  explicit: 10
  pageLoad: 30

Run any TestNG suite with this config.

Expected

Either the suite starts and runs with TestNG's tests-level parallelism (per the docs), or — if tests genuinely isn't supported — the docs shouldn't advertise it.

Actual

FrameworkBootstrap.initialize() calls ExecutionValidator.validate(config.getExecution()) unconditionally on every suite start (src/main/java/com/seleniumboot/lifecycle/FrameworkBootstrap.java:36), before any test runs. ExecutionValidator only accepts none/methods/classes:

// src/main/java/com/seleniumboot/execution/ExecutionValidator.java:14-16
if (!"none".equalsIgnoreCase(parallel)
        && !"methods".equalsIgnoreCase(parallel)
        && !"classes".equalsIgnoreCase(parallel)) {
    throw new IllegalStateException("Parallel execution configuration missing");   // line 17
}

parallel: tests throws IllegalStateException: Parallel execution configuration missing at bootstrap, before a single test method runs.

Docs claiming otherwise

docs-site/docs/guides/parallel.md:88:

TestNG supports several parallel modes. Selenium Boot works with all of them:

followed by a table (lines 90–94) that includes:

Mode Description Recommended
tests Each <test> tag in suite XML runs in a thread For suite-level isolation

The inline config sample at line 20 also lists it: parallel: methods # none (default) | methods | classes | tests.

This is consistent with the repo's own CLAUDE.md Configuration Reference, which lists only parallel: none | methods | classes — the docs-site page is the outlier, not the code.

Bug 2 — misleading error message

The thrown message, "Parallel execution configuration missing", is wrong for this case: the value is present, just unsupported. A user setting parallel: tests is told their config is missing, which sends them looking in the wrong place (e.g. re-checking YAML indentation) instead of realizing the value itself is rejected. The message should name the offending value and list the accepted ones, e.g. Unsupported parallel mode 'tests' — expected one of: none, methods, classes. This should be fixed regardless of which option below is chosen for bug 1.

Bug 3 — smaller doc inaccuracy, same file

docs-site/docs/guides/parallel.md:22:

maxActiveSessions: 4    # semaphore cap — cannot exceed threadCount

Nothing in the code enforces this relationship. DriverManager.getOrInitSemaphore() builds a Semaphore sized purely from execution.getMaxActiveSessions() (src/main/java/com/seleniumboot/driver/DriverManager.java:44-51), with no comparison to threadCount anywhere — I grepped SeleniumBootDefaults, ExecutionValidator, and DriverManager and found no such check. The comment reads as an enforced constraint but is only advice; worth a one-word wording fix (e.g. "should not exceed") separate from bugs 1–2.

Two fix options for bug 1 (maintainer decision, not actioned here)

  • (a) Add "tests" to the accepted values in ExecutionValidator. tests is a legitimate TestNG parallel mode and nothing else in the framework appears to care which mode is used — this looks like the more correct fix, but it's a code change.
  • (b) Delete the tests row from the table and the "works with all of them" claim, and change the inline sample comment to none (default) | methods | classes.

Either way, bug 2 (error message) should be fixed regardless of which option is chosen.

This is a framework code decision, so per the current roadmap it's deferred to the 2026-07-29 review gate — filing this to record the defect, not to propose action now.

Context

This surfaced while writing a public blog post that links to this guide: https://seleniumboot.com/blog/parallel-selenium-tests-java-without-thread-safety-bugs — the post deliberately documents modes as none | methods | classes to avoid repeating the docs page's false claim, which is why the two pages are now inconsistent with each other.

Note: the YAML config keys in this same guide (execution: nesting, missing timeouts: block) were already fixed on 2026-07-23 in commit 400e3f3. This tests-mode issue is separate and still open — not a regression of that fix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdocumentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions