Skip to content

Error in Microsoft Defender for Endpoint Pipeline #9320

Description

@janniten

Hello,
I'm using the Microsoft Defender for Endpoint Integration and I an ingestion error in pipeline logs-microsoft_defender_endpoint.log-2.24.0, caused by the evidence field being []

Event.original

{"aadTenantId":"xxxxxxx-xxx-xxxx-xxxx-xxxxxxxx","alertCreationTime":"2024-03-07T11:14:02.0866667Z","assignedTo":"user@domain","category":"SuspiciousActivity","classification":"InformationalExpectedActivity","comments":[],"computerDnsName":"computer1","description":"user2@domain initiated an Automated investigation on computer1.\nThe investigation automatically identifies and reviews threat artifacts for possible remediation.\n\nDetails: AntivirusDetecton event","detectionSource":"AutomatedInvestigation","detectorId":"xxxxx-xxxxx-xxxx-xxxxx-xxxxxxx","determination":"ConfirmedActivity","domains":[],"evidence":[],"firstEventTime":"2024-03-07T11:14:01.9880757Z","id":"yyyyy_zzzzz","incidentId":32247,"investigationId":2409,"investigationState":"PartiallyInvestigated","lastEventTime":"2024-03-07T11:14:01.9880757Z","lastUpdateTime":"2024-03-08T10:45:56.5966667Z","loggedOnUsers":[{"accountName":"user2","domainName":"domain"}],"machineId":"xxxxxxxxxxxxxxxxxxxxx","mitreTechniques":[],"rbacGroupName":null,"relatedUser":null,"resolvedTime":"2024-03-08T10:45:56.54Z","severity":"Informational","status":"Resolved","threatFamilyName":null,"threatName":null,"title":"Automated investigation started manually"}

error.message

image
(and also happens for url and IpAddress)

Adding this condition !ctx.json?.evidence.isEmpty() to these processors, solves the problem

  {
    "append": {
      "field": "event.category",
      "value": "process",
      "if": "!ctx.json?.evidence.isEmpty() && ctx?.json?.evidence?.entityType == 'Process'"
    }
  }

  {
    "append": {
      "field": "event.type",
      "value": "access",
      "if": "!ctx.json?.evidence.isEmpty() && ctx.json?.evidence?.entityType != null && ctx.json?.evidence?.entityType == 'User'"
    }
  }

  {
    "rename": {
      "field": "json.evidence.url",
      "target_field": "url.full",
      "ignore_missing": true,
      "if": "!ctx.json?.evidence.isEmpty() && ctx.json?.evidence?.url != null"
    }
  }
  
   {
    "append": {
      "field": "related.ip",
      "value": "{{json.evidence.ipAddress}}",
      "if": "!ctx.json?.evidence.isEmpty() && ctx.json?.evidence?.ipAddress != null"
    }
  }

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions