prep for release v3.104.0

[arielkr256]

No description provided.

[cursor]

PR Summary

Medium Risk
Medium risk due to new/changed detection logic and shared helper functions (AWS WAF + Databricks), plus changes to pack enablement that can alter alert volume and coverage.

Overview
Adds substantial new detection content: introduces a new panther_databricks_helpers global plus a large set of Databricks audit rules (admin/config/token/login/data movement/destructive activity patterns), and adds multiple AWS WAF managed rule group passthrough rules powered by new WAF helper utilities in panther_aws_helpers.

Adds new correlation & scheduling artifacts: adds correlation rules for S3 security-control disabling, S3 exfiltration-followed-by-deletion, and Azure blob upload followed by CPK encryption error; adds disabled Okta AD-agent PantherFlow baseline scheduled queries (30/90/180 day windows) for lookup-table generation.

Tuning/enablement changes: marks several GAIA/Google Workspace scheduled rules and the GAIA.Credential.Theft.Attack.Chain correlation rule as disabled in packs/rules, switches the Snowflake exfiltration correlation rule reference/log type to the streaming variant, tweaks Proofpoint rule thresholds/wording, and sets CreateAlert: false on several S3 component rules to support correlation-driven alerting. Indexes and detection-coverage.json are updated accordingly.

^{Written by Cursor Bugbot for commit 529acff. This will update automatically on new commits. Configure here.}

[github-actions]

📐 Style Guide Review

Analyzing commit: 6953a3586d3438562524d0b3357fc997930cdf52

Style Compliance: GOOD

📋 Style Findings

✅ Compliant Areas

• Proper field access: All rules consistently use event.get() and event.deep_get() with appropriate defaults instead of direct field access
• MITRE ATT&CK formatting: Techniques follow the correct TA####:T#### format with descriptive comments (e.g., TA0001:T1078 # Valid Accounts)
• Helper function usage: Good utilization of global helpers via proper imports, including the new panther_databricks_helpers functions

⚠️ Style Issues

• Line length compliance: Files adhere to the 100-character line length requirement
• Import organization: Clean import statements grouped appropriately
• Runbook structure: All runbooks follow the required format with specific timeframes, field references, and actionable investigation steps

📝 Recommendations

• Test coverage: Excellent test cases provided with both positive and negative scenarios for all new rules
• Metadata consistency: RuleID, Filename, and DisplayName relationships are clear and follow naming conventions
• Dynamic functions: Proper implementation of title(), alert_context(), and severity() functions where appropriate

[github-actions]

🐍 Python Logic Review

Analyzing commit: 6953a3586d3438562524d0b3357fc997930cdf52

Logic Quality: GOOD

🔍 Logic Analysis

✅ Well-Implemented

• Databricks Helper Functions: Excellent new panther_databricks_helpers.py with comprehensive constants, proper event handling, and well-structured helper functions like databricks_alert_context() and filtering utilities
• Event Handling Patterns: Consistent use of event.deep_get() and event.get() with appropriate defaults across all Databricks and Proofpoint rules
• Helper Usage: Most rules properly leverage global helpers (WAF rules use waf_alert_context, waf_severity; Databricks rules use databricks_alert_context)

⚠️ Logic Issues

• aws_waf_reactjsrce_body.py:1-78 Duplicates significant logic from WAF helpers. Should use waf_rule_group_matches(), waf_alert_context(), and waf_severity() functions instead of reimplementing pattern matching and context building
• panther_aws_helpers.py:392 waf_severity() returns "DEFAULT" as fallback, but this isn't a valid Panther severity level (should be "INFO", "LOW", "MEDIUM", "HIGH", or "CRITICAL")
• panther_aws_helpers.py:135-170 aws_regions() function has duplicate entries (e.g., "ap-northeast-1", "ca-central-1", "eu-central-1" appear twice each)
• databricks_repeated_failed_login_attempts.py:14-19 Title function has 3 dynamic fields (user, IP, action) which may cause deduplication issues - consider if dedup function is sufficient

🔧 Recommendations

1. High Priority: Refactor aws_waf_reactjsrce_body.py to use existing WAF helper functions for consistency and maintainability
2. Medium Priority: Fix waf_severity() fallback to return valid severity level like "LOW" instead of "DEFAULT"
3. Low Priority: Remove duplicate entries from aws_regions() function
4. Low Priority: Consider simplifying title functions with multiple dynamic fields or ensure dedup functions handle the variance appropriately

[cursor]

Cursor Bugbot has reviewed your changes and found 3 potential issues.

Bugbot Autofix prepared fixes for both issues found in the latest run.

• ✅ Fixed: Databricks pack missing all detection rule IDs
  • Added all 37 Databricks detection rule IDs to the pack definition alongside the existing global helper.
• ✅ Fixed: Excluded path substring matching causes false positive filtering
  • Replaced simple substring matching with proper path segment boundary checking to prevent paths like /users/status-update from incorrectly matching /status.

Or push these changes by commenting:

@cursor push 1236854883

Preview (1236854883)

diff --git a/global_helpers/panther_databricks_helpers.py b/global_helpers/panther_databricks_helpers.py
--- a/global_helpers/panther_databricks_helpers.py
+++ b/global_helpers/panther_databricks_helpers.py
@@ -381,9 +381,16 @@
         Boolean indicating if this path should be excluded
     """
     path = event.deep_get("requestParams", "path", default="")
-    # Check if any excluded path is a segment in the full path
-    # This avoids matching "status" in paths like "/users/status-update"
-    return any(excluded in path for excluded in EXCLUDED_PATHS)
+    # Check if any excluded path is a complete segment in the full path
+    # This avoids matching "/status" in paths like "/users/status-update"
+    for excluded in EXCLUDED_PATHS:
+        idx = path.find(excluded)
+        if idx != -1:
+            end_idx = idx + len(excluded)
+            # Check if the match ends at a path boundary (end of string or followed by '/')
+            if end_idx == len(path) or path[end_idx] == "/":
+                return True
+    return False
 
 
 def is_config_change(event, config_category=None):

diff --git a/packs/databricks.yml b/packs/databricks.yml
--- a/packs/databricks.yml
+++ b/packs/databricks.yml
@@ -5,4 +5,42 @@
   IDs:
     # Globals
     - panther_databricks_helpers
+    # Rules
+    - Databricks.Audit.AccessToMultipleWorkspaces
+    - Databricks.Audit.AccessTokenRevoked
+    - Databricks.Audit.AccountAdminPrivilegedRoleAssignment
+    - Databricks.Audit.AttemptedLogonFromDeniedIP
+    - Databricks.Audit.ConfigChangesAccountLevel
+    - Databricks.Audit.ConfigChangesHighPriority
+    - Databricks.Audit.ConfigChangesWorkspaceLevel
+    - Databricks.Audit.DataDownloadsFromControlPlane
+    - Databricks.Audit.DataMovementExplicitCredentials
+    - Databricks.Audit.DeltaSharingIPAccessFailures
+    - Databricks.Audit.DeltaSharingRecipientWithoutIPACLs
+    - Databricks.Audit.DestructiveActivities
+    - Databricks.Audit.EmployeeLogon
+    - Databricks.Audit.GlobalInitScriptChanges
+    - Databricks.Audit.GroupCreated
+    - Databricks.Audit.GroupDeleted
+    - Databricks.Audit.InstallLibraryAllClusters
+    - Databricks.Audit.LongLifetimeTokenGenerated
+    - Databricks.Audit.MetastoreAdminPrivilegeGranted
+    - Databricks.Audit.MFAKeyChange
+    - Databricks.Audit.MountPointCreation
+    - Databricks.Audit.NonSSOLogin
+    - Databricks.Audit.PotentialPrivilegeEscalation
+    - Databricks.Audit.PrincipalRemovedFromGroup
+    - Databricks.Audit.RepeatedAccessToSecrets
+    - Databricks.Audit.RepeatedFailedLoginAttempts
+    - Databricks.Audit.RepeatedUnauthorizedUCDataRequests
+    - Databricks.Audit.RepeatedUnauthorizedUCRequests
+    - Databricks.Audit.SSOConfigChanged
+    - Databricks.Audit.TermsOfServiceChanges
+    - Databricks.Audit.TrufflehogScanDetected
+    - Databricks.Audit.UserAccountCreated
+    - Databricks.Audit.UserAccountDeleted
+    - Databricks.Audit.UserPasswordChanged
+    - Databricks.Audit.UserRoleModified
+    - Databricks.Audit.VerboseAuditLoggingDisabled
+    - Databricks.Audit.WorkspaceAdminPrivilegedRoleAssignment
 DisplayName: "Panther Databricks Pack"

_{This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.}

[cursor]

Databricks pack missing all detection rule IDs

Medium Severity

The packs/databricks.yml describes itself as "Group of all Databricks detections" but only includes panther_databricks_helpers — none of the ~35 new Databricks rules are listed. Every other pack in the codebase (AWS, Azure, GSuite) includes all their detection rule IDs alongside globals. Users subscribing to this pack would get only the helper module with no actual detections.

 

[cursor]

Excluded path substring matching causes false positive filtering

Medium Severity

The is_excluded_path function uses simple substring matching (excluded in path) despite comments claiming it uses "path segment matching to avoid false positives." For example, EXCLUDED_PATHS includes "/status", which will incorrectly match paths like /users/status-update or /api/status-check, causing legitimate security events to be silently filtered out via filter_noise.

 

[cursor]

waf_get_matched_rule misses event-level non-terminating matches

Low Severity

waf_rule_group_matches checks three locations for matches: terminatingRuleId, event-level nonTerminatingMatchingRules, and ruleGroupList. But waf_get_matched_rule only checks ruleGroupList and falls back to terminatingRuleId, skipping event-level nonTerminatingMatchingRules. When a match is found via the event-level path, title() will display the wrong rule (e.g., Default_Action instead of the actual matched rule group).

Additional Locations (1)

• global_helpers/panther_aws_helpers.py#L309-L327

 

[github-actions]

📄 YAML Metadata Review

Analyzing commit: 6953a3586d3438562524d0b3357fc997930cdf52

Metadata Quality: NEEDS IMPROVEMENT

📊 Metadata Analysis

✅ Well-Documented

• SOCRadar rule has comprehensive description with clear use case mapping and excellent test coverage
• Databricks rules feature appropriate MITRE ATT&CK mappings and realistic test scenarios
• AWS WAF rules provide detailed technical descriptions of managed rule group coverage

⚠️ Metadata Issues

• AWS WAF Rules (7 files) Missing required "Status: Experimental" field for new rules
• correlation_rules/aws_s3_disable_security_controls.yml Missing required "Status: Experimental" field
• rules/databricks_rules/databricks_destructive_activities.yml MEDIUM severity inappropriate - should be HIGH given 50+ deletions threshold indicating active data destruction
• rules/aws_waf_rules/aws_waf_managed_ip_reputation.yml LOW severity inappropriate - threat intelligence hits should be MEDIUM minimum
• rules/databricks_rules/databricks_long_lifetime_token_generated.yml Missing explanation of dynamic severity scaling mentioned in description
• Multiple Databricks rules Inconsistent reference URLs - mix of GitHub repos, some may not be authoritative sources

💡 Suggestions

• Add "Status: Experimental" to all new AWS WAF rules and correlation rules per project requirements
• Review severity assignments: destructive activities and threat intel hits warrant higher severities
• Standardize reference sources to official documentation where available

[github-actions]

🧪 Testing Review

Analyzing commit: 6953a3586d3438562524d0b3357fc997930cdf52

Test Quality: GOOD

🔬 Test Analysis

✅ Good Test Coverage

• Comprehensive scenarios: Most rules include multiple positive tests covering different logic branches (login failures, config changes, destructive actions)
• Proper negative testing: Rules consistently include ExpectedResult: false cases to validate non-alerting conditions (system users, different actions, successful operations)
• Realistic log samples: Tests use well-structured log data with appropriate field names, proper status codes, and anonymized data (example.com domains, RFC5737 IPs)

⚠️ Testing Gaps

• rules/databricks_rules/databricks_access_to_multiple_workspaces.yml Only 2 tests for a threshold-based rule (threshold: 5) - needs testing around threshold boundaries and aggregation scenarios
• queries/okta_queries/okta_ad_pantherflow_baseline_*.yml Complex analytical queries with no test cases - critical gap for behavioral analytics validation
• rules/databricks_rules/databricks_long_lifetime_token_generated.yml Incomplete test data (lines 70-78 truncated) affecting test completeness
• global_helpers/panther_databricks_helpers.yml No unit test coverage for helper functions
• Threshold-based Databricks rules Several rules (databricks_destructive_activities, databricks_repeated_*) lack comprehensive threshold boundary testing

🎯 Test Improvements

• Add boundary testing for all threshold-based rules (test N-1, N, N+1 scenarios where N is threshold)
• Include test cases for complex PantherFlow queries to validate aggregation logic and baseline calculations
• Complete truncated test samples and ensure all test logs have proper structure

[github-actions]

🔍 Query Review

Analyzing commit: 6953a3586d3438562524d0b3357fc997930cdf52

Query Quality: NEEDS IMPROVEMENT

📊 Query Analysis

✅ Good Practices

• Okta baseline queries use proper PantherFlow syntax with appropriate time range filters (p_event_time)
• Complex statistical calculations are well-structured with clear CTEs and proper aggregations
• Specific field selection avoids SELECT * pattern

⚠️ Query Issues

• GSuite files are misclassified - they're scheduled_rule types in the queries/ directory, should be in rules/
• gsuite_*_rule.yml files don't contain actual SQL queries but reference other scheduled queries
• File organization - scheduled rules belong in rules directory, not queries directory
• Okta queries missing DisplayName format validation (should be "LogProvider QueryTitle")

🚀 Performance Suggestions

• Okta baseline queries are appropriately designed for their purpose - no LIMIT needed for baseline generation
• Statistical calculations are optimized with proper use of CTEs to avoid repeated computations

[github-actions]

🏗️ Architecture Compliance Review

Analyzing commit: 6953a3586d3438562524d0b3357fc997930cdf52

Architecture: GOOD

🔧 Architecture Analysis

✅ Architectural Strengths

• Excellent Dual-File Architecture: All detections properly follow the established .py + .yml pattern with required functions and comprehensive metadata
• Consistent Helper Integration: Proper imports and usage of global helpers (panther_databricks_helpers, panther_aws_helpers, panther_proofpoint_helpers) following established patterns
• Comprehensive Test Coverage: All YAML files include proper test cases with both positive and negative examples, realistic log samples, and edge case validation

⚠️ Architecture Issues

• Incomplete Pack Definition: The packs/databricks.yml only includes the global helpers but is missing all the actual Databricks detection rule IDs (should include ~35 rules like Databricks.Audit.AccessToMultipleWorkspaces, etc.)
• Inconsistent Status Metadata: Some rules have Status: Experimental while others lack this field - should be standardized across all new experimental detections
• Correlation Rule Naming: Some correlation rules follow different naming patterns (AWS.S3.Disable.Security.Controls vs typical dot notation)

🎯 Architecture Recommendations

• Complete Databricks Pack: Add all 35+ Databricks rule IDs to packs/databricks.yml PackDefinition.IDs section to enable proper deployment and management
• Standardize Experimental Status: Ensure all new Databricks rules consistently include Status: Experimental field in their YAML metadata

---

Review Complete ✅
All review sections have been posted. Check the comments above for detailed findings in each area.