Convert Scheduled Rules to Real-Time Streaming Rules Using unique()

[zaynahsmith-dasilva]

Background

Changes

Migrates 7 scheduled SQL-based rules to real-time streaming rules by leveraging Panther's unique() threshold feature, replacing periodic data lake queries with per-event detection.

Rules converted:

• AWS.CloudTrail.PasswordSpraying → rules/aws_cloudtrail_rules/
• AWS.VPC.PortScanning → rules/aws_vpc_flow_rules/
• Google.Workspace.Rapid.Multi.IP.Authentication
• Kubernetes.SecretEnumeration → rules/kubernetes_rules/
• GSuite.Drive.Many.Documents.Deleted → rules/gsuite_activityevent_rules/
• Dropbox.Many.Downloads → rules/dropbox_rules/
• Dropbox.Many.Deletes → rules/dropbox_rules/

Changes per Rule:

• Replaced rule(_): return True + SQL aggregation with proper rule(), unique(), dedup(), title(), severity(), and alert_context() functions
• Added tiered severity() logic based on per-event context (e.g. root user, internal IP, external sharing, non-standard login type)
• Removed corresponding scheduled_query YAML files

Testing

THREAT-663

[cursor]

PR Summary

Medium Risk
Introduces new real-time streaming detections (with unique()/threshold semantics) while deprecating several existing scheduled-rule definitions, which may change alert volume and tuning expectations across AWS/Kubernetes/Dropbox/GSuite.

Overview
Migrates multiple detections from scheduled, query-driven rules to real-time streaming rules by adding new Python/YAML rule implementations that use dedup()/unique() with thresholds (notably AWS.CloudTrail.PasswordSpraying, AWS.VPC.PortScanning, Dropbox bulk downloads/deletes, GSuite bulk Drive deletions, and Kubernetes secret enumeration).

Deprecates the corresponding scheduled-rule YAMLs (marking them Status: Deprecated, disabling where applicable) and updates pack and index metadata (indexes/*.md, indexes/detection-coverage.json, packs/*.yml) to reference the new rules and remove the old scheduled/query entries.

^{Reviewed by Cursor Bugbot for commit 451cb42. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Autofix Details

Bugbot Autofix prepared fixes for both issues found in the latest run.

• ✅ Fixed: AWS pack IDs no longer resolve
  • I restored the migrated AWS rule IDs to the original pack-referenced IDs so packs/aws.yml entries resolve correctly again.
• ✅ Fixed: Password spray lost failure volume guardrail
  • I reintroduced the original dual-condition logic by using the scheduled-query output fields and requiring both distinctUserNames > 5 and failures > 10 before alerting.

Or push these changes by commenting:

@cursor push 50e593604a

Preview (50e593604a)

diff --git a/queries/aws_queries/anomalous_vpc_port_activity_query.yml b/queries/aws_queries/anomalous_vpc_port_activity_query.yml
new file mode 100644
--- /dev/null
+++ b/queries/aws_queries/anomalous_vpc_port_activity_query.yml
@@ -1,0 +1,24 @@
+AnalysisType: scheduled_query
+Description: Instances of a srcAddr communicating with multiple ports on a dstAddr could indicate port scanning activity.
+Enabled: false
+Query: |
+  SELECT
+    srcAddr,
+    dstAddr,
+    COUNT(DISTINCT dstPort) as distinctDstPorts,
+    vpcId,
+    region,
+    subNetId
+  FROM panther_logs.public.aws_vpcflow
+  WHERE p_occurs_since('1 hour')
+    and srcAddr != 'null'
+    and srcPort not in (443, 80, 2049, 123, 445, 53, 853)
+    and dstPort not in (443, 80, 2049, 123, 445, 53, 853)
+    and flowDirection = 'egress'
+  GROUP BY srcAddr, dstAddr, vpcId, region, subNetId
+  HAVING COUNT(DISTINCT dstPort) >=  10
+  ORDER BY COUNT(DISTINCT dstPort) DESC
+QueryName: "VPC Flow Port Scanning"
+Schedule:
+  RateMinutes: 60
+  TimeoutMinutes: 5

diff --git a/queries/aws_queries/cloudtrail_password_spraying_query.yml b/queries/aws_queries/cloudtrail_password_spraying_query.yml
new file mode 100644
--- /dev/null
+++ b/queries/aws_queries/cloudtrail_password_spraying_query.yml
@@ -1,0 +1,31 @@
+AnalysisType: scheduled_query
+QueryName: "Query.CloudTrail.Password.Spraying"
+Enabled: false
+Description: >
+  Detect password spraying in cloudtrail logs
+Query: |
+  SELECT
+    -- this information will be in the alert events
+    awsRegion as region,
+    recipientAccountId as accountid,
+    COUNT(DISTINCT useridentity:userName) as distinctUserNames,
+    COUNT(1) as failures,
+    MIN(p_event_time) as first_attempt,
+    MAX(p_event_time) as last_attempt
+  FROM
+    panther_logs.public.aws_cloudtrail
+  WHERE
+    p_occurs_since(3600)
+    AND
+    eventtype = 'AwsConsoleSignIn'
+    AND
+    responseElements:ConsoleLogin = 'Failure'
+  GROUP BY
+    region, accountid
+  HAVING
+    distinctUserNames > 5
+    AND
+    failures > 10
+Schedule:
+  RateMinutes: 60
+  TimeoutMinutes: 1

diff --git a/rules/aws_cloudtrail_rules/aws_cloudtrail_password_spraying.py b/rules/aws_cloudtrail_rules/aws_cloudtrail_password_spraying.py
--- a/rules/aws_cloudtrail_rules/aws_cloudtrail_password_spraying.py
+++ b/rules/aws_cloudtrail_rules/aws_cloudtrail_password_spraying.py
@@ -1,33 +1,19 @@
-from panther_aws_helpers import aws_rule_context
-
-
 def rule(event):
-    if event.get("eventType") != "AwsConsoleSignIn":
-        return False
-    return event.deep_get("responseElements", "ConsoleLogin", default="") == "Failure"
+    return event.get("distinctUserNames", 0) > 5 and event.get("failures", 0) > 10
 
 
 def title(event):
-    account = event.get("recipientAccountId", "Unknown Account")
-    region = event.get("awsRegion", "Unknown Region")
+    account = event.get("accountid", "Unknown Account")
+    region = event.get("region", "Unknown Region")
     return f"Password Spraying Detected in AWS Account [{account}] Region [{region}]"
 
 
-def dedup(event):
-    account = event.get("recipientAccountId", "")
-    region = event.get("awsRegion", "")
-    return f"{account}:{region}"
-
-
-def unique(event):
-    return event.deep_get("userIdentity", "userName", default="")
-
-
-def severity(event):
-    if event.deep_get("userIdentity", "type", default="") == "Root":
-        return "HIGH"
-    return "DEFAULT"
-
-
 def alert_context(event):
-    return aws_rule_context(event)
+    return {
+        "region": event.get("region"),
+        "accountid": event.get("accountid"),
+        "distinctUserNames": event.get("distinctUserNames"),
+        "failures": event.get("failures"),
+        "first_attempt": event.get("first_attempt"),
+        "last_attempt": event.get("last_attempt"),
+    }

diff --git a/rules/aws_cloudtrail_rules/aws_cloudtrail_password_spraying.yml b/rules/aws_cloudtrail_rules/aws_cloudtrail_password_spraying.yml
--- a/rules/aws_cloudtrail_rules/aws_cloudtrail_password_spraying.yml
+++ b/rules/aws_cloudtrail_rules/aws_cloudtrail_password_spraying.yml
@@ -1,16 +1,14 @@
-AnalysisType: rule
+AnalysisType: scheduled_rule
 Filename: aws_cloudtrail_password_spraying.py
-RuleID: "AWS.CloudTrail.PasswordSpraying"
+RuleID: "CloudTrail.Password.Spraying"
 DisplayName: "AWS CloudTrail Password Spraying"
 Enabled: false
 Severity: Medium
-DedupPeriodMinutes: 60
-Threshold: 6
-LogTypes:
-  - AWS.CloudTrail
 Description: >
   Detects password spraying attacks by alerting when more than 5 distinct usernames
-  fail to authenticate to the AWS console from the same account and region within 60 minutes.
+  and more than 10 failed AWS console logins occur in the same account and region within 60 minutes.
+ScheduledQueries:
+  - Query.CloudTrail.Password.Spraying
 Reports:
   MITRE ATT&CK:
     - TA0001:T1078
@@ -21,54 +19,21 @@
   2. Check if any of the targeted usernames subsequently had a successful ConsoleLogin from any sourceIPAddress in the 6 hours after the alert
   3. Find other alerts for this recipientAccountId or any of the targeted usernames in the past 7 days to determine if this is part of a broader campaign
 Tests:
-  - Name: Failed Console Login
+  - Name: Password Spraying Threshold Met
     ExpectedResult: true
     Log:
-      awsRegion: us-east-1
-      eventName: ConsoleLogin
-      eventSource: signin.amazonaws.com
-      eventType: AwsConsoleSignIn
-      recipientAccountId: "111122223333"
-      responseElements:
-        ConsoleLogin: Failure
-      userIdentity:
-        type: IAMUser
-        userName: alice
-  - Name: Successful Console Login
+      region: us-east-1
+      accountid: "111122223333"
+      distinctUserNames: 6
+      failures: 11
+      first_attempt: "2026-03-31T10:00:00Z"
+      last_attempt: "2026-03-31T10:59:59Z"
+  - Name: Distinct Usernames Met but Failures Too Low
     ExpectedResult: false
     Log:
-      awsRegion: us-east-1
-      eventName: ConsoleLogin
-      eventSource: signin.amazonaws.com
-      eventType: AwsConsoleSignIn
-      recipientAccountId: "111122223333"
-      responseElements:
-        ConsoleLogin: Success
-      userIdentity:
-        type: IAMUser
-        userName: alice
-  - Name: Non-Console Event
-    ExpectedResult: false
-    Log:
-      awsRegion: us-east-1
-      eventName: DescribeInstances
-      eventSource: ec2.amazonaws.com
-      eventType: AwsApiCall
-      recipientAccountId: "111122223333"
-      responseElements: null
-      userIdentity:
-        type: IAMUser
-        userName: alice
-  - Name: Failed Console Login - Root User
-    ExpectedResult: true
-    Log:
-      awsRegion: us-east-1
-      eventName: ConsoleLogin
-      eventSource: signin.amazonaws.com
-      eventType: AwsConsoleSignIn
-      recipientAccountId: "111122223333"
-      responseElements:
-        ConsoleLogin: Failure
-      userIdentity:
-        type: Root
-        userName: root
+      region: us-east-1
+      accountid: "111122223333"
+      distinctUserNames: 6
+      failures: 10
+      first_attempt: "2026-03-31T10:00:00Z"
+      last_attempt: "2026-03-31T10:59:59Z"

diff --git a/rules/aws_vpc_flow_rules/aws_vpc_port_scanning.yml b/rules/aws_vpc_flow_rules/aws_vpc_port_scanning.yml
--- a/rules/aws_vpc_flow_rules/aws_vpc_port_scanning.yml
+++ b/rules/aws_vpc_flow_rules/aws_vpc_port_scanning.yml
@@ -1,6 +1,6 @@
 AnalysisType: rule
 Filename: aws_vpc_port_scanning.py
-RuleID: "AWS.VPC.PortScanning"
+RuleID: "VPCFlow.Port.Scanning"
 DisplayName: "VPC Flow Port Scanning"
 Enabled: false
 Severity: Medium

_{This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: List verb ineffective in unique threshold counting
  • I updated unique() to generate a stable non-null key for list events (namespace/requestURI fallback) so list-based secret enumeration contributes to threshold counting.

Or push these changes by commenting:

@cursor push e09aefa300

Preview (e09aefa300)

diff --git a/rules/kubernetes_rules/k8s_secret_enumeration.py b/rules/kubernetes_rules/k8s_secret_enumeration.py
--- a/rules/kubernetes_rules/k8s_secret_enumeration.py
+++ b/rules/kubernetes_rules/k8s_secret_enumeration.py
@@ -25,9 +25,22 @@
 
 
 def unique(event):
-    return event.udm("name") or None
+    secret_name = event.udm("name")
+    if secret_name:
+        return secret_name
 
+    # List requests target secret collections and often omit objectRef.name.
+    if event.udm("verb") == "list":
+        namespace = event.udm("namespace")
+        if namespace:
+            return f"list:{namespace}"
+        request_uri = event.udm("requestURI")
+        if request_uri:
+            return f"list:{request_uri}"
+        return "list"
+    return None
 
+
 def severity(event):
     if not is_failed_request(event.udm("responseStatus")):
         return "HIGH"

_{This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.}

[zaynahsmith-dasilva]

@cursor push e09aefa

[CLAassistant]

All committers have signed the CLA.

[alessandrarizzo]

should these be tagged as experimental?

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 09c4e1e. Configure here.}