[checkpoint_email] Initial release of the Check Point Harmony Email & Collaboration by brijesh-elastic · Pull Request #11472 · elastic/integrations

brijesh-elastic · 2024-10-18T13:32:06Z

Proposed commit message

Create New integration package checkpoint_email.

Added event data stream.
Added data collection logic for the data stream.
Added the ingest pipeline for the data stream.
Mapped fields according to the ECS schema and added Fields metadata in the appropriate yml files.
Added dashboard and visualizations.
Added pipeline test for the data stream.
Added system test cases for the data stream.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

How to test this PR locally

Clone integrations repo.
Install elastic package locally.
Start elastic stack using elastic-package.
Move to integrations/packages/checkpoint_email directory.
Run the following command to run tests.

elastic-package test

--- Test results for package: checkpoint_email - START ---
╭──────────────────┬─────────────┬───────────┬───────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE          │ DATA STREAM │ TEST TYPE │ TEST NAME                                 │ RESULT │ TIME ELAPSED │
├──────────────────┼─────────────┼───────────┼───────────────────────────────────────────┼────────┼──────────────┤
│ checkpoint_email │ event       │ pipeline  │ (ingest pipeline warnings test-event.log) │ PASS   │   309.6712ms │
│ checkpoint_email │ event       │ pipeline  │ test-event.log                            │ PASS   │ 247.700215ms │
╰──────────────────┴─────────────┴───────────┴───────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: checkpoint_email - END   ---
Done
--- Test results for package: checkpoint_email - START ---
╭──────────────────┬─────────────┬───────────┬──────��────┬────────┬───────────────╮
│ PACKAGE          │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │  TIME ELAPSED │
├──────────────────┼─────────────┼───────────┼───────────┼────────┼───────────────┤
│ checkpoint_email │ event       │ system    │ default   │ PASS   │ 41.094371033s │
╰──────────────────┴─────────────┴───────────┴───────────┴────────┴───────────────╯
--- Test results for package: checkpoint_email - END   ---
Done
--- Test results for package: checkpoint_email - START ---
╭──────────────────┬─────────────┬───────────┬──────────────────────────┬────────┬──────────────╮
│ PACKAGE          │ DATA STREAM │ TEST TYPE │ TEST NAME                │ RESULT │ TIME ELAPSED │
├──────────────────┼─────────────┼───────────┼──────────────────────────┼────────┼──────────────┤
│ checkpoint_email │ event       │ static    │ Verify sample_event.json │ PASS   │ 139.139436ms │
╰──────────────────┴─────────────┴───────────┴──────────────────────────┴────────┴──────────────╯
--- Test results for package: checkpoint_email - END   ---
Done
--- Test results for package: checkpoint_email - START ---
╭──────────────────┬─────────────┬───────────┬───────────────────────────────────────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE          │ DATA STREAM │ TEST TYPE │ TEST NAME                                                                 │ RESULT │ TIME ELAPSED │
├──────────────────┼─────────────┼───────────┼───────────────────────────────────────────────────────────────────────────┼────────┼──────────────┤
│ checkpoint_email │             │ asset     │ dashboard checkpoint_email-3f1885e2-633d-4aa9-bf77-56a98d484692 is loaded │ PASS   │      1.189µs │
│ checkpoint_email │             │ asset     │ search checkpoint_email-83662506-f7ee-40ba-9f16-65da29ec7ef4 is loaded    │ PASS   │        300ns │
│ checkpoint_email │ event       │ asset     │ index_template logs-checkpoint_email.event is loaded                      │ PASS   │        273ns │
│ checkpoint_email │ event       │ asset     │ ingest_pipeline logs-checkpoint_email.event-0.1.0 is loaded               │ PASS   │        180ns │
╰──────────────────┴─────────────┴───────────┴───────────────────────────────────────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: checkpoint_email - END   ---
Done

Related issues

Closes [New Integration] Check Point Harmony Email & Collaboration #11178

Screenshots

elastic-vault-github-plugin-prod · 2024-10-18T13:54:59Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

elasticmachine · 2024-10-21T12:40:15Z

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

jamiehynds · 2024-10-21T12:42:32Z

@brijesh-elastic minor nit, but Checkpoint should be Check Point to ensure alignment with their branding. Can you make sure we're using Check Point in integration title, labels, documentation, dashboards, etc?

efd6 · 2024-10-22T05:18:01Z

+        "end_date": now.format(time_layout.RFC3339),
+      })
+  ).as(state,
+    has(state.events) && type(state.events) == type({}) && has(state.events.error) ? state : // Exit early due to POST failure.


ISTM that we don't need to check for the existence of the error field; the only time that we will have a state.event that is an object is when there was an error. But we can simplify further…

Suggested change

has(state.events) && type(state.events) == type({}) && has(state.events.error) ? state : // Exit early due to POST failure.

type(state.?events.orValue(null)) == type({}) ? state : // Exit early due to POST failure.

I'm not a big fan of putting everything in state here. I think the preamble should be into different label so that there is some demarcation about what expected scopes are.

Still not wildly happy with having everything in state.

efd6 · 2024-10-22T05:25:52Z

+      field: event.category
+      tag: append_malware_into_event_category
+      value: malware
+      if: ctx.json?.type != null && ['malware', 'suspicious malware'].contains(ctx.json.type.toLowerCase())


Suggested change

if: ctx.json?.type != null && ['malware', 'suspicious malware'].contains(ctx.json.type.toLowerCase())

if: >-

ctx.json?.type != null && (

ctx.json.type.equalsIgnoreCase('malware') ||

ctx.json.type.equalsIgnoreCase('suspicious malware')

)

(similar below)

efd6 · 2024-10-22T05:28:44Z

+on_failure:
+  - append:
+      field: error.message
+      value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}'


Suggested change

value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}'

value: >-

Processor '{{{ _ingest.on_failure_processor_type }}}'

{{{#_ingest.on_failure_processor_tag}}}with tag '{{{ _ingest.on_failure_processor_tag }}}'

{{{/_ingest.on_failure_processor_tag}}}failed with message '{{{ _ingest.on_failure_message }}}'

efd6

Nits only.

efd6 · 2024-10-28T05:24:23Z

+      field: event.type
+      tag: append_indicator_into_event_type
+      value: indicator
+      if: ctx.json?.type != null && ['anomaly', 'shadow_it'].contains(ctx.json.type.toLowerCase())


elastic-sonarqube · 2024-10-28T11:15:37Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
97.4% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

elasticmachine · 2024-10-28T11:15:42Z

💚 Build Succeeded

Buildkite Build
Commit: 68fc3f5

History

💚 Build #17615 succeeded 19b8f7a
💚 Build #17481 succeeded 4a398c9
💚 Build #17348 succeeded c09a235

willemdh · 2024-10-28T18:55:08Z

Is this based on parsing the json from the Harmony Email and Collaboration SIEM integration or for the Check Point portal logs?

efd6 · 2024-10-28T20:41:12Z

@jamiehynds Are you good with this now?

brijesh-elastic · 2024-10-29T10:08:58Z

Is this based on parsing the json from the Harmony Email and Collaboration SIEM integration or for the Check Point portal logs?

It is collecting security events data from Harmony Email & Collaboration Smart API.

jamiehynds · 2024-11-06T09:16:33Z

LGTM

elastic-vault-github-plugin-prod · 2024-11-08T13:07:52Z

Package checkpoint_email - 0.1.0 containing this change is available at https://epr.elastic.co/search?package=checkpoint_email

… Collaboration (elastic#11472) Create New integration package checkpoint_email. Added event data stream. Added data collection logic for the data stream. Added the ingest pipeline for the data stream. Mapped fields according to the ECS schema and added Fields metadata in the appropriate yml files. Added dashboard and visualizations. Added pipeline test for the data stream. Added system test cases for the data stream.

brijesh-elastic added 2 commits October 18, 2024 18:51

Initial release

d68f8a7

Update changelog entry

c09a235

andrewkroh added Crest Contributions from Crest developement team. New Integration Issue or pull request for creating a new integration package. labels Oct 18, 2024

jamiehynds added the Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] label Oct 21, 2024

efd6 reviewed Oct 22, 2024

View reviewed changes

Resolve comments

4a398c9

brijesh-elastic changed the title ~~[checkpoint_email] Initial release of the Checkpoint Harmony Email & Collaboration~~ Oct 23, 2024

brijesh-elastic requested a review from efd6 October 23, 2024 06:34

efd6 reviewed Oct 28, 2024

View reviewed changes

brijesh-elastic and others added 2 commits October 28, 2024 16:08

Merge branch 'main' into package-checkpoint_email

19b8f7a

Resolve comments

68fc3f5

brijesh-elastic requested a review from efd6 October 28, 2024 10:59

efd6 approved these changes Oct 28, 2024

View reviewed changes

kcreddy merged commit 9b5a6b4 into elastic:main Nov 8, 2024

andrewkroh added the Integration:checkpoint_email Check Point Harmony Email & Collaboration label Nov 18, 2024

brijesh-elastic mentioned this pull request Jan 2, 2025

[cyberark_epm] Initial release of the CyberArk EPM #12198

Merged

4 tasks

Navigation Menu

Uh oh!

[checkpoint_email] Initial release of the Check Point Harmony Email & Collaboration - #11472

[checkpoint_email] Initial release of the Check Point Harmony Email & Collaboration#11472
kcreddy merged 5 commits into
elastic:mainfrom
brijesh-elastic:package-checkpoint_email

brijesh-elastic commented Oct 18, 2024 •

edited

Loading

elastic-vault-github-plugin-prod Bot commented Oct 18, 2024

elasticmachine commented Oct 21, 2024

jamiehynds commented Oct 21, 2024

Uh oh!

Uh oh!

Uh oh!

efd6 Oct 22, 2024

efd6 Oct 28, 2024

efd6 Oct 22, 2024

efd6 Oct 22, 2024

efd6 left a comment

efd6 Oct 28, 2024

Uh oh!

elastic-sonarqube Bot commented Oct 28, 2024

elasticmachine commented Oct 28, 2024

willemdh commented Oct 28, 2024

efd6 commented Oct 28, 2024

brijesh-elastic commented Oct 29, 2024

jamiehynds commented Nov 6, 2024

elastic-vault-github-plugin-prod Bot commented Nov 8, 2024

Labels

7 participants

	has(state.events) && type(state.events) == type({}) && has(state.events.error) ? state : // Exit early due to POST failure.
	type(state.?events.orValue(null)) == type({}) ? state : // Exit early due to POST failure.

-      if: ctx.json?.type != null && ['malware', 'suspicious malware'].contains(ctx.json.type.toLowerCase())
+      if: >-
+        ctx.json?.type != null && (
+          ctx.json.type.equalsIgnoreCase('malware') ||
+          ctx.json.type.equalsIgnoreCase('suspicious malware')
+        )

-      value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}'
+      value: >-
+        Processor '{{{ _ingest.on_failure_processor_type }}}'
+        {{{#_ingest.on_failure_processor_tag}}}with tag '{{{ _ingest.on_failure_processor_tag }}}'
+        {{{/_ingest.on_failure_processor_tag}}}failed with message '{{{ _ingest.on_failure_message }}}'

Uh oh!

Conversation

brijesh-elastic commented Oct 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

How to test this PR locally

Related issues

Screenshots

elastic-vault-github-plugin-prod Bot commented Oct 18, 2024

🚀 Benchmarks report

elasticmachine commented Oct 21, 2024

jamiehynds commented Oct 21, 2024

Uh oh!

Uh oh!

Uh oh!

efd6 Oct 22, 2024

Choose a reason for hiding this comment

efd6 Oct 28, 2024

Choose a reason for hiding this comment

efd6 Oct 22, 2024

Choose a reason for hiding this comment

efd6 Oct 22, 2024

Choose a reason for hiding this comment

efd6 left a comment

Choose a reason for hiding this comment

efd6 Oct 28, 2024

Choose a reason for hiding this comment

Uh oh!

elastic-sonarqube Bot commented Oct 28, 2024

Quality Gate passed

elasticmachine commented Oct 28, 2024

💚 Build Succeeded

History

willemdh commented Oct 28, 2024

efd6 commented Oct 28, 2024

brijesh-elastic commented Oct 29, 2024

jamiehynds commented Nov 6, 2024

elastic-vault-github-plugin-prod Bot commented Nov 8, 2024

Labels

7 participants

brijesh-elastic commented Oct 18, 2024 •

edited

Loading