Summary

Add two new built-in checks that verify whether the output contains any or all items from a provided list of strings. This is a common need for topic coverage validation (e.g., "does the response mention at least one of these required topics?").

Motivation

Currently, users must write custom FnCheck lambdas for list-based string matching. Competitors like DeepEval and promptfoo ship this as a built-in. Having a registered, serializable check improves usability and enables YAML/JSON-based test definitions.

Implementation Guide

Pattern to follow

Reference: libs/giskard-checks/src/giskard/checks/builtin/text_matching.py (StringMatching)

Steps

1. Create checks in libs/giskard-checks/src/giskard/checks/builtin/text_matching.py (or a new file)
2. Subclass Check and register with @Check.register("contains_any") and @Check.register("contains_all")
3. Implement async run(self, trace: Trace) -> CheckResult
4. Support:
   • values: list[str] — the list of strings to check against
   • case_sensitive: bool = False — case sensitivity toggle
   • JSONPath extraction via key: JSONPathStr for the value to check
   • Unicode normalization (reuse normalize_string from utils/normalization.py)
5. Add to __init__.py exports
6. Add tests in tests/builtin/

Example usage

from giskard.checks import ContainsAny, ContainsAll, Scenario

scenario = (
    Scenario(name="topic_coverage")
    .interact(inputs="Explain machine learning", outputs="ML is a subset of AI...")
    .check(ContainsAny(values=["machine learning", "ML", "artificial intelligence"]))
    .check(ContainsAll(values=["definition", "example"]))
)

Acceptance Criteria

• ContainsAny passes if output contains at least one item from the list
• ContainsAll passes if output contains every item from the list
• Case sensitivity is configurable
• JSONPath extraction works for nested trace values
• Unicode normalization is supported
• Checks are registered and serializable
• Tests cover: basic matching, case sensitivity, empty list, no match, JSONPath extraction