Skip to content

[crowdstrike,m365_defender,microsoft_defender_endpoint,trend_micro_vision_one] map source-specific fields to process.* ECS fields - #13984

Merged
efd6 merged 4 commits into
elastic:mainfrom
mohitjha-elastic:add_ecs_for_data_analyzer
May 27, 2025
Merged

[crowdstrike,m365_defender,microsoft_defender_endpoint,trend_micro_vision_one] map source-specific fields to process.* ECS fields#13984
efd6 merged 4 commits into
elastic:mainfrom
mohitjha-elastic:add_ecs_for_data_analyzer

Conversation

@mohitjha-elastic

@mohitjha-elastic mohitjha-elastic commented May 23, 2025

Copy link
Copy Markdown
Contributor

Proposed Commit Message

crowdstrike, m365_defender, microsoft_defender_endpoint, trend_micro_vision_one: map
source-specific fields to `process.*` ECS fields.

* crowdstrike: add process.entity_id and process.parent.entity_id ECS
  mappings in alert and falcon data streams.
* m365_defender: add process.entity_id and process.parent.entity_id ECS
  mappings in all data streams.
* microsoft_defender_endpoint: add process.entity_id and
  process.parent.entity_id ECS mappings in log data stream.
* trend_micro_vision_one: add process.entity_id ECS mapping in detection
  data stream.

These mappings align the data streams with ECS standards and ensure
consistency across ingested data, supporting reliable analyzer
development.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

How to test this PR locally

To test integration

Clone integrations repo.
Install the elastic package locally.
Start the elastic stack using the elastic package.
Move to integrations/packages/<integration_name> directory.
Run the following command to run tests.
elastic-package test -v

NOTE: To test individual integrations, replace <integration_name> with one of the following: crowdstrike, m365_defender, microsoft_defender_endpoint, or trend_micro_vision_one.

Related issues

@mohitjha-elastic mohitjha-elastic self-assigned this May 23, 2025
@mohitjha-elastic
mohitjha-elastic requested a review from a team as a code owner May 23, 2025 11:39
@mohitjha-elastic mohitjha-elastic added enhancement New feature or request Integration:crowdstrike CrowdStrike Integration:microsoft_defender_endpoint Microsoft Defender for Endpoint Integration:m365_defender Microsoft Defender XDR Integration:trend_micro_vision_one TrendAI Vision One Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] Team:SDE-Crest Crest developers on the Security Integrations team [elastic/sit-crest-contractors] labels May 23, 2025
@elasticmachine

Copy link
Copy Markdown

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

…add_ecs_for_data_analyzer

Conflicts:
	packages/crowdstrike/changelog.yml
	packages/crowdstrike/data_stream/alert/sample_event.json
	packages/crowdstrike/data_stream/falcon/sample_event.json
	packages/crowdstrike/docs/README.md
	packages/m365_defender/changelog.yml
	packages/m365_defender/data_stream/alert/sample_event.json
	packages/m365_defender/data_stream/incident/sample_event.json
	packages/m365_defender/docs/README.md
	packages/microsoft_defender_endpoint/changelog.yml
	packages/microsoft_defender_endpoint/data_stream/log/sample_event.json
	packages/microsoft_defender_endpoint/docs/README.md
@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@efd6 efd6 changed the title [crowdstrike/m365_defender/microsoft_defender_endpoint/trend_micro_vision_one] Map Process.* ECS mappings to Ensure Consistency for Data Analyzer May 25, 2025
Comment on lines +298 to +309
- convert:
field: process.pid
tag: convert_process_pid_to_string
target_field: process.entity_id
type: string
ignore_missing: true
- convert:
field: process.parent.pid
tag: convert_process_parent_pid_to_string
target_field: process.parent.entity_id
type: string
ignore_missing: true

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave as numbers and note numeric keyword in the test definition.

Comment on lines +110 to +121
- convert:
field: process.pid
tag: convert_process_pid_to_string
target_field: process.entity_id
type: string
ignore_missing: true
- convert:
field: process.parent.pid
tag: convert_process_parent_pid_to_string
target_field: process.parent.entity_id
type: string
ignore_missing: true

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave as numbers.

if (!processParentPid.isEmpty()) {
ctx.process.parent.pid = convertToOrderedArray(processParentPid);
}
- foreach:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave as numbers.

Comment on lines +237 to +248
- convert:
field: process.pid
tag: convert_process_pid_to_string
target_field: process.entity_id
type: string
ignore_missing: true
- convert:
field: process.parent.pid
tag: convert_process_parent_pid_to_string
target_field: process.parent.entity_id
type: string
ignore_missing: true

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave as numbers.

field: process.pid
copy_from: trend_micro_vision_one.detection.process.pid
ignore_failure: true
- convert:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave as a number.

@mohitjha-elastic
mohitjha-elastic requested a review from efd6 May 26, 2025 12:44
@elasticmachine

Copy link
Copy Markdown

💚 Build Succeeded

History

cc @mohitjha-elastic

@efd6 efd6 changed the title [crowdstrike,m365_defender,microsoft_defender_endpoint,trend_micro_vision_one] Map Process.* ECS mappings to Ensure Consistency for Data Analyzer May 26, 2025

@efd6 efd6 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest commit message body:

* crowdstrike: add process.entity_id and process.parent.entity_id ECS
  mappings in alert and falcon data streams.
* m365_defender: add process.entity_id and process.parent.entity_id ECS
  mappings in all data streams.
* microsoft_defender_endpoint: add process.entity_id and
  process.parent.entity_id ECS mappings in log data stream.
* trend_micro_vision_one: add process.entity_id ECS mapping in detection
  data stream.

These mappings align the data streams with ECS standards and ensure
consistency across ingested data, supporting reliable analyzer
development.
@efd6
efd6 merged commit 4ecefbb into elastic:main May 27, 2025
@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package crowdstrike - 1.71.0 containing this change is available at https://epr.elastic.co/package/crowdstrike/1.71.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package m365_defender - 3.6.0 containing this change is available at https://epr.elastic.co/package/m365_defender/3.6.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package microsoft_defender_endpoint - 2.36.0 containing this change is available at https://epr.elastic.co/package/microsoft_defender_endpoint/2.36.0/

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package trend_micro_vision_one - 2.1.0 containing this change is available at https://epr.elastic.co/package/trend_micro_vision_one/2.1.0/

anupratharamachandran pushed a commit to anupratharamachandran/integrations that referenced this pull request Jun 2, 2025
…sion_one] map source-specific fields to process.* ECS fields (elastic#13984)

* crowdstrike: add process.entity_id and process.parent.entity_id ECS
  mappings in alert and falcon data streams.
* m365_defender: add process.entity_id and process.parent.entity_id ECS
  mappings in all data streams.
* microsoft_defender_endpoint: add process.entity_id and
  process.parent.entity_id ECS mappings in log data stream.
* trend_micro_vision_one: add process.entity_id ECS mapping in detection
  data stream.

These mappings align the data streams with ECS standards and ensure
consistency across ingested data, supporting reliable analyzer
development.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Integration:crowdstrike CrowdStrike Integration:m365_defender Microsoft Defender XDR Integration:microsoft_defender_endpoint Microsoft Defender for Endpoint Integration:trend_micro_vision_one TrendAI Vision One Team:SDE-Crest Crest developers on the Security Integrations team [elastic/sit-crest-contractors] Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

3 participants