Skip to content

[servicenow] Initial release of ServiceNow - #11521

Merged
efd6 merged 8 commits into
elastic:mainfrom
muskan-agarwal26:servicenow_cmdb-0.1.0
Nov 5, 2024
Merged

[servicenow] Initial release of ServiceNow#11521
efd6 merged 8 commits into
elastic:mainfrom
muskan-agarwal26:servicenow_cmdb-0.1.0

Conversation

@muskan-agarwal26

@muskan-agarwal26 muskan-agarwal26 commented Oct 25, 2024

Copy link
Copy Markdown
Contributor

Proposed commit message

Create New integration package servicenow.

  • Added event data stream.
  • Added data collection logic for the data stream.
  • Added the ingest pipeline for the data stream.
  • Mapped fields according to the ECS schema and added Fields metadata in the appropriate yml files.
  • Added dashboard and visualizations.
  • Added pipeline test for the data stream.
  • Added system test cases for the data stream.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

  • Clone integrations repo.
  • Install elastic package locally.
  • Start elastic stack using elastic-package.
  • Move to integrations/packages/servicenow directory.
  • Run the following command to run tests.

elastic-package test

elastic-package test -v
2024/11/01 21:27:41 DEBUG Enable verbose logging
2024/11/01 21:27:41 DEBUG latest version (cached): v0.106.0. Download from: https://github.com/elastic/elastic-package/releases/tag/v0.106.0 (Timestamp 2024-11-01 21:04:21.916977025 +0530 IST)
2024/11/01 21:27:41  INFO New version is available - v0.106.0. Download from: https://github.com/elastic/elastic-package/releases/tag/v0.106.0
Run asset tests for the package
2024/11/01 21:27:41 DEBUG output command: /usr/bin/docker ps -a --filter label=com.docker.compose.project=elastic-package-stack --format {{.ID}}
2024/11/01 21:27:42 DEBUG output command: /usr/bin/docker inspect f38469bc2ef6 1453beff71a5 118d99dcd6d1 721a9d54799b e8dd47b972de 5bcd903df9ee e6ec02c8d7a3 99c362a6ca9c a2909d944efb 9a8543acdeec
2024/11/01 21:27:42 DEBUG Connecting with Kibana host from current profile (profile: default, host: "https://127.0.0.1:5601")
2024/11/01 21:27:42 DEBUG GET https://127.0.0.1:5601/api/status
2024/11/01 21:27:42 DEBUG Running tests sequentially
2024/11/01 21:27:42 DEBUG installing package...
2024/11/01 21:27:42 DEBUG GET https://127.0.0.1:5601/api/fleet/epm/packages/servicenow
2024/11/01 21:27:42 DEBUG Build directory: /root/github/integrations/build/packages/servicenow/0.1.0
2024/11/01 21:27:42 DEBUG Clear target directory (path: /root/github/integrations/build/packages/servicenow/0.1.0)
2024/11/01 21:27:42 DEBUG Copy package content (source: /root/github/integrations/packages/servicenow)
2024/11/01 21:27:42 DEBUG Copy license file if needed
2024/11/01 21:27:42  INFO License text found in "/root/github/integrations/LICENSE.txt" will be included in package
2024/11/01 21:27:42 DEBUG Encode dashboards
2024/11/01 21:27:42 DEBUG Resolve external fields
2024/11/01 21:27:42 DEBUG Package has external dependencies defined
2024/11/01 21:27:43 DEBUG data_stream/event/fields/base-fields.yml: source file hasn't been changed
2024/11/01 21:27:43 DEBUG data_stream/event/fields/beats.yml: source file hasn't been changed
2024/11/01 21:27:43 DEBUG data_stream/event/fields/fields.yml: source file hasn't been changed
2024/11/01 21:27:43 DEBUG Package doesn't have to import ECS mappings
2024/11/01 21:27:43 DEBUG Build zipped package
2024/11/01 21:27:43 DEBUG Compress using archiver.Zip (destination: /root/github/integrations/build/packages/servicenow-0.1.0.zip)
2024/11/01 21:27:43 DEBUG Create work directory for archiving: /tmp/elastic-package-1922099671/servicenow-0.1.0
2024/11/01 21:27:43 DEBUG Skip validation of the built .zip package
2024/11/01 21:27:43 DEBUG POST https://127.0.0.1:5601/api/fleet/epm/packages
2024/11/01 21:27:45 DEBUG GET https://127.0.0.1:5601/api/fleet/epm/packages/servicenow
2024/11/01 21:27:46 DEBUG removing package...
2024/11/01 21:27:46 DEBUG GET https://127.0.0.1:5601/api/fleet/epm/packages/servicenow
2024/11/01 21:27:46 DEBUG DELETE https://127.0.0.1:5601/api/fleet/epm/packages/servicenow/0.1.0
--- Test results for package: servicenow - START ---
╭────────────┬─────────────┬───────────┬─────────────────────────────────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE    │ DATA STREAM │ TEST TYPE │ TEST NAME                                                           │ RESULT │ TIME ELAPSED │
├────────────┼─────────────┼───────────┼─────────────────────────────────────────────────────────────────────┼────────┼──────────────┤
│ servicenow │             │ asset     │ dashboard servicenow-014c13b4-59bc-4bfa-bcb2-fdc1828020bf is loaded │ PASS   │      1.507µs │
│ servicenow │             │ asset     │ dashboard servicenow-0305f883-12a8-4021-af73-fbe05e797c3f is loaded │ PASS   │        231ns │
│ servicenow │             │ asset     │ dashboard servicenow-22d413b7-ffb5-450f-b5f8-7dcd54fab22f is loaded │ PASS   │        216ns │
│ servicenow │             │ asset     │ dashboard servicenow-297414cc-3b65-40b1-b15b-e33cf676b3bf is loaded │ PASS   │        251ns │
│ servicenow │             │ asset     │ dashboard servicenow-488772bc-4ce5-48c6-9e8a-d948439dbf39 is loaded │ PASS   │        247ns │
│ servicenow │             │ asset     │ dashboard servicenow-648308f9-ebdf-46fe-b6ce-b90dfcf2d1dd is loaded │ PASS   │        265ns │
│ servicenow │             │ asset     │ dashboard servicenow-b182c6a6-3b4e-413f-899d-dd255b1a3623 is loaded │ PASS   │        274ns │
│ servicenow │             │ asset     │ dashboard servicenow-b4795d3e-8c8b-48b2-900e-edc89db4d336 is loaded │ PASS   │        300ns │
│ servicenow │             │ asset     │ dashboard servicenow-d2eb3f0c-4ddb-4683-afab-2820d2616ca2 is loaded │ PASS   │        342ns │
│ servicenow │             │ asset     │ dashboard servicenow-d8977587-d9b4-4487-b321-a8402da07d48 is loaded │ PASS   │        323ns │
│ servicenow │             │ asset     │ dashboard servicenow-d929c3a1-7af8-428e-8357-8b2e8d3e3050 is loaded │ PASS   │        369ns │
│ servicenow │             │ asset     │ dashboard servicenow-e6093b0d-38a6-4b05-8354-4bbaef28b667 is loaded │ PASS   │        357ns │
│ servicenow │             │ asset     │ search servicenow-00004bce-eac2-46d4-bb4e-ce6849c50a36 is loaded    │ PASS   │        202ns │
│ servicenow │             │ asset     │ search servicenow-01396f16-041b-4ec5-b104-d29fe68f09ad is loaded    │ PASS   │        288ns │
│ servicenow │             │ asset     │ search servicenow-046f04ce-7f31-4929-81fa-34561fb369f0 is loaded    │ PASS   │        249ns │
│ servicenow │             │ asset     │ search servicenow-1c323098-190c-4592-aca5-dd9e64de540b is loaded    │ PASS   │        252ns │
│ servicenow │             │ asset     │ search servicenow-26f2b1af-7d7d-4578-b325-322ac4c783fa is loaded    │ PASS   │        299ns │
│ servicenow │             │ asset     │ search servicenow-44db573d-f3e6-4f59-b1a4-328040220991 is loaded    │ PASS   │        283ns │
│ servicenow │             │ asset     │ search servicenow-722e7751-818d-41e2-b7ac-fcc90bb8241b is loaded    │ PASS   │        307ns │
│ servicenow │             │ asset     │ search servicenow-a5b0060e-4ace-4c9f-96e1-2d815bc9dd28 is loaded    │ PASS   │        326ns │
│ servicenow │             │ asset     │ search servicenow-afa470f8-4e19-4fee-8b17-5649a7b0ae51 is loaded    │ PASS   │        330ns │
│ servicenow │             │ asset     │ search servicenow-b9cc24b4-0cfe-458c-8645-23838623ca09 is loaded    │ PASS   │        358ns │
│ servicenow │             │ asset     │ search servicenow-e28e5e66-d00d-4e2c-af2e-a09d0d261b05 is loaded    │ PASS   │        433ns │
│ servicenow │ event       │ asset     │ index_template logs-servicenow.event is loaded                      │ PASS   │        302ns │
│ servicenow │ event       │ asset     │ ingest_pipeline logs-servicenow.event-0.1.0 is loaded               │ PASS   │        342ns │
╰────────────┴─────────────┴───────────┴─────────────────────────────────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: servicenow - END   ---
Done
Run pipeline tests for the package
2024/11/01 21:27:48 DEBUG output command: /usr/bin/docker ps -a --filter label=com.docker.compose.project=elastic-package-stack --format {{.ID}}
2024/11/01 21:27:48 DEBUG output command: /usr/bin/docker inspect f38469bc2ef6 1453beff71a5 118d99dcd6d1 721a9d54799b e8dd47b972de 5bcd903df9ee e6ec02c8d7a3 99c362a6ca9c a2909d944efb 9a8543acdeec
2024/11/01 21:27:48 DEBUG Connecting with Elasticsearch host from current profile (profile: default, host: "https://127.0.0.1:9200")
2024/11/01 21:27:48 DEBUG Running tests sequentially
2024/11/01 21:27:51 DEBUG Imported ECS fields definition from external schema for validation (embedded in package: false, stack uses ecs@mappings template: true)
2024/11/01 21:27:54 DEBUG Dump Elastic stack data
2024/11/01 21:27:54 DEBUG Dump stack logs (location: )
2024/11/01 21:27:54 DEBUG output command: /usr/bin/docker ps -a --filter label=com.docker.compose.project=elastic-package-stack --format {{.ID}}
2024/11/01 21:27:54 DEBUG output command: /usr/bin/docker inspect f38469bc2ef6 1453beff71a5 118d99dcd6d1 721a9d54799b e8dd47b972de 5bcd903df9ee e6ec02c8d7a3 99c362a6ca9c a2909d944efb 9a8543acdeec
2024/11/01 21:27:54 DEBUG Dump stack logs for elasticsearch
2024/11/01 21:27:54 DEBUG running command: /usr/bin/docker compose version --short
2024/11/01 21:27:54 DEBUG Determined Docker Compose version: 2.27.0
2024/11/01 21:27:54 DEBUG running command: /usr/bin/docker compose -f /root/.elastic-package/profiles/default/stack/docker-compose.yml -p elastic-package-stack logs --since 2024-11-01T15:57:48Z elasticsearch
--- Test results for package: servicenow - START ---
╭────────────┬─────────────┬───────────┬───────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE    │ DATA STREAM │ TEST TYPE │ TEST NAME                                 │ RESULT │ TIME ELAPSED │
├────────────┼─────────────┼───────────┼───────────────────────────────────────────┼────────┼──────────────┤
│ servicenow │ event       │ pipeline  │ (ingest pipeline warnings test-event.log) │ PASS   │ 698.255017ms │
│ servicenow │ event       │ pipeline  │ test-event.log                            │ PASS   │ 2.871486917s │
╰────────────┴─────────────┴───────────┴───────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: servicenow - END   ---
Done
Run policy tests for the package
2024/11/01 21:27:57 DEBUG output command: /usr/bin/docker ps -a --filter label=com.docker.compose.project=elastic-package-stack --format {{.ID}}
2024/11/01 21:27:57 DEBUG output command: /usr/bin/docker inspect f38469bc2ef6 1453beff71a5 118d99dcd6d1 721a9d54799b e8dd47b972de 5bcd903df9ee e6ec02c8d7a3 99c362a6ca9c a2909d944efb 9a8543acdeec
2024/11/01 21:27:57 DEBUG Connecting with Kibana host from current profile (profile: default, host: "https://127.0.0.1:5601")
2024/11/01 21:27:57 DEBUG GET https://127.0.0.1:5601/api/status
--- Test results for package: servicenow - START ---
No test results
--- Test results for package: servicenow - END   ---
Done
Run static tests for the package
2024/11/01 21:27:57 DEBUG Running tests sequentially
2024/11/01 21:27:58 DEBUG Imported ECS fields definition from external schema for validation (embedded in package: false, stack uses ecs@mappings template: true)
--- Test results for package: servicenow - START ---
╭────────────┬─────────────┬───────────┬──────────────────────────┬────────┬──────────────╮
│ PACKAGE    │ DATA STREAM │ TEST TYPE │ TEST NAME                │ RESULT │ TIME ELAPSED │
├────────────┼─────────────┼───────────┼──────────────────────────┼────────┼──────────────┤
│ servicenow │ event       │ static    │ Verify sample_event.json │ PASS   │ 255.703561ms │
╰────────────┴─────────────┴───────────┴──────────────────────────┴────────┴──────────────╯
--- Test results for package: servicenow - END   ---
Done
Run system tests for the package
2024/11/01 21:27:58 DEBUG output command: /usr/bin/docker ps -a --filter label=com.docker.compose.project=elastic-package-stack --format {{.ID}}
2024/11/01 21:27:58 DEBUG output command: /usr/bin/docker inspect f38469bc2ef6 1453beff71a5 118d99dcd6d1 721a9d54799b e8dd47b972de 5bcd903df9ee e6ec02c8d7a3 99c362a6ca9c a2909d944efb 9a8543acdeec
2024/11/01 21:27:58 DEBUG Connecting with Kibana host from current profile (profile: default, host: "https://127.0.0.1:5601")
2024/11/01 21:27:58 DEBUG GET https://127.0.0.1:5601/api/status
2024/11/01 21:27:58 DEBUG output command: /usr/bin/docker ps -a --filter label=com.docker.compose.project=elastic-package-stack --format {{.ID}}
2024/11/01 21:27:58 DEBUG output command: /usr/bin/docker inspect f38469bc2ef6 1453beff71a5 118d99dcd6d1 721a9d54799b e8dd47b972de 5bcd903df9ee e6ec02c8d7a3 99c362a6ca9c a2909d944efb 9a8543acdeec
2024/11/01 21:27:58 DEBUG Connecting with Elasticsearch host from current profile (profile: default, host: "https://127.0.0.1:9200")
2024/11/01 21:27:58 DEBUG Running suite...
2024/11/01 21:27:58 DEBUG Running system tests for data stream "event"
2024/11/01 21:27:58 DEBUG System runner: data stream "event" config file "test-default-config.yml" variant ""
2024/11/01 21:27:58 DEBUG Installing package...
2024/11/01 21:27:58 DEBUG GET https://127.0.0.1:5601/api/fleet/epm/packages/servicenow
2024/11/01 21:27:59 DEBUG Build directory: /root/github/integrations/build/packages/servicenow/0.1.0
2024/11/01 21:27:59 DEBUG Clear target directory (path: /root/github/integrations/build/packages/servicenow/0.1.0)
2024/11/01 21:27:59 DEBUG Copy package content (source: /root/github/integrations/packages/servicenow)
2024/11/01 21:27:59 DEBUG Copy license file if needed
2024/11/01 21:27:59  INFO License text found in "/root/github/integrations/LICENSE.txt" will be included in package
2024/11/01 21:27:59 DEBUG Encode dashboards
2024/11/01 21:27:59 DEBUG Resolve external fields
2024/11/01 21:27:59 DEBUG Package has external dependencies defined
2024/11/01 21:27:59 DEBUG data_stream/event/fields/base-fields.yml: source file hasn't been changed
2024/11/01 21:27:59 DEBUG data_stream/event/fields/beats.yml: source file hasn't been changed
2024/11/01 21:27:59 DEBUG data_stream/event/fields/fields.yml: source file hasn't been changed
2024/11/01 21:27:59 DEBUG Package doesn't have to import ECS mappings
2024/11/01 21:27:59 DEBUG Build zipped package
2024/11/01 21:27:59 DEBUG Compress using archiver.Zip (destination: /root/github/integrations/build/packages/servicenow-0.1.0.zip)
2024/11/01 21:27:59 DEBUG Create work directory for archiving: /tmp/elastic-package-1296602193/servicenow-0.1.0
2024/11/01 21:27:59 DEBUG Skip validation of the built .zip package
2024/11/01 21:27:59 DEBUG POST https://127.0.0.1:5601/api/fleet/epm/packages
2024/11/01 21:28:02 DEBUG Running tests sequentially
2024/11/01 21:28:02 DEBUG Using config: "default"
2024/11/01 21:28:02 DEBUG running test with configuration 'default'
2024/11/01 21:28:02 DEBUG creating enroll policy...
2024/11/01 21:28:02 DEBUG POST https://127.0.0.1:5601/api/fleet/agent_policies
2024/11/01 21:28:06 DEBUG creating test policy...
2024/11/01 21:28:06 DEBUG POST https://127.0.0.1:5601/api/fleet/agent_policies
2024/11/01 21:28:09 DEBUG setting up service...
2024/11/01 21:28:09 DEBUG setting up service using Docker Compose service deployer
2024/11/01 21:28:09 DEBUG running command: /usr/bin/docker compose version --short
2024/11/01 21:28:09 DEBUG Determined Docker Compose version: 2.27.0
2024/11/01 21:28:09 DEBUG output command: /usr/bin/docker network inspect elastic-package-stack_default
2024/11/01 21:28:09 DEBUG running command: /usr/bin/docker compose -f /root/github/integrations/packages/servicenow/_dev/deploy/docker/docker-compose.yml -p elastic-package-service-47870 up --build -d
WARN[0000] /root/github/integrations/packages/servicenow/_dev/deploy/docker/docker-compose.yml: `version` is obsolete 
[+] Running 2/2
 ✔ Network elastic-package-service-47870_default         Created                                                                                              0.4s 
 ✔ Container elastic-package-service-47870-servicenow-1  Started                                                                                              3.3s 
2024/11/01 21:28:13 DEBUG running command: /usr/bin/docker compose -f /root/github/integrations/packages/servicenow/_dev/deploy/docker/docker-compose.yml -p elastic-package-service-47870 ps -a -q
WARN[0000] /root/github/integrations/packages/servicenow/_dev/deploy/docker/docker-compose.yml: `version` is obsolete 
2024/11/01 21:28:13 DEBUG Wait for healthy containers: 58c5e029fe62f16e772f8f9be8aaede6cbbaa4c3818732a0d90cf1d0ee52a16d
2024/11/01 21:28:13 DEBUG output command: /usr/bin/docker inspect 58c5e029fe62f16e772f8f9be8aaede6cbbaa4c3818732a0d90cf1d0ee52a16d
2024/11/01 21:28:13 DEBUG Container 58c5e029fe62f16e772f8f9be8aaede6cbbaa4c3818732a0d90cf1d0ee52a16d status: running (no health status)
2024/11/01 21:28:13 DEBUG run command: /usr/bin/docker network connect elastic-package-stack_default elastic-package-service-47870-servicenow-1 --alias svc-servicenow
2024/11/01 21:28:13 DEBUG adding service container elastic-package-service-47870-servicenow-1 internal ports to context
2024/11/01 21:28:13 DEBUG running command: /usr/bin/docker compose -f /root/github/integrations/packages/servicenow/_dev/deploy/docker/docker-compose.yml -p elastic-package-service-47870 config
WARN[0000] /root/github/integrations/packages/servicenow/_dev/deploy/docker/docker-compose.yml: `version` is obsolete 
2024/11/01 21:28:13 DEBUG adding package data stream to test policy...
2024/11/01 21:28:13 DEBUG POST https://127.0.0.1:5601/api/fleet/package_policies
2024/11/01 21:28:17 DEBUG deleting old data in data stream...
2024/11/01 21:28:17 DEBUG GET https://127.0.0.1:5601/api/fleet/agents
2024/11/01 21:28:17 DEBUG filter agents using criteria: NamePrefix=docker-fleet-agent
2024/11/01 21:28:17 DEBUG found 1 enrolled agent(s)
2024/11/01 21:28:17 DEBUG Selected enrolled agent "243ebe11-4a3c-4c33-8b0b-2df4c8097581"
2024/11/01 21:28:17 DEBUG Set Debug log level to agent
2024/11/01 21:28:17 DEBUG POST https://127.0.0.1:5601/api/fleet/agents/243ebe11-4a3c-4c33-8b0b-2df4c8097581/actions
2024/11/01 21:28:18 DEBUG GET https://127.0.0.1:5601/api/fleet/agent_policies/4ade0c07-c560-410e-9a96-b8076f377689
2024/11/01 21:28:18 DEBUG assigning package data stream to agent...
2024/11/01 21:28:18 DEBUG PUT https://127.0.0.1:5601/api/fleet/agents/243ebe11-4a3c-4c33-8b0b-2df4c8097581/reassign
2024/11/01 21:28:19 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/243ebe11-4a3c-4c33-8b0b-2df4c8097581
2024/11/01 21:28:19 DEBUG Agent 243ebe11-4a3c-4c33-8b0b-2df4c8097581 (Host: docker-fleet-agent): Policy ID 4ade0c07-c560-410e-9a96-b8076f377689 LogLevel: info Status: updating
2024/11/01 21:28:19 DEBUG Wait until the policy (ID: 4ade0c07-c560-410e-9a96-b8076f377689, revision: 2) is assigned to the agent (ID: 243ebe11-4a3c-4c33-8b0b-2df4c8097581)...
2024/11/01 21:28:21 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/243ebe11-4a3c-4c33-8b0b-2df4c8097581
2024/11/01 21:28:21 DEBUG Agent 243ebe11-4a3c-4c33-8b0b-2df4c8097581 (Host: docker-fleet-agent): Policy ID 4ade0c07-c560-410e-9a96-b8076f377689 LogLevel: info Status: updating
2024/11/01 21:28:21 DEBUG Wait until the policy (ID: 4ade0c07-c560-410e-9a96-b8076f377689, revision: 2) is assigned to the agent (ID: 243ebe11-4a3c-4c33-8b0b-2df4c8097581)...
2024/11/01 21:28:23 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/243ebe11-4a3c-4c33-8b0b-2df4c8097581
2024/11/01 21:28:23 DEBUG Agent 243ebe11-4a3c-4c33-8b0b-2df4c8097581 (Host: docker-fleet-agent): Policy ID 4ade0c07-c560-410e-9a96-b8076f377689 LogLevel: debug Status: updating
2024/11/01 21:28:23 DEBUG Wait until the policy (ID: 4ade0c07-c560-410e-9a96-b8076f377689, revision: 2) is assigned to the agent (ID: 243ebe11-4a3c-4c33-8b0b-2df4c8097581)...
2024/11/01 21:28:25 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/243ebe11-4a3c-4c33-8b0b-2df4c8097581
2024/11/01 21:28:25 DEBUG Agent 243ebe11-4a3c-4c33-8b0b-2df4c8097581 (Host: docker-fleet-agent): Policy ID 4ade0c07-c560-410e-9a96-b8076f377689 LogLevel: debug Status: online
2024/11/01 21:28:25 DEBUG Policy revision assigned to the agent (ID: 243ebe11-4a3c-4c33-8b0b-2df4c8097581)...
2024/11/01 21:28:25 DEBUG checking for expected data in data stream (10m0s)...
2024/11/01 21:28:25 DEBUG found 0 hits in logs-servicenow.event-66166 data stream
2024/11/01 21:28:26 DEBUG found 0 hits in logs-servicenow.event-66166 data stream
2024/11/01 21:28:27 DEBUG found 0 hits in logs-servicenow.event-66166 data stream
2024/11/01 21:28:28 DEBUG found 0 hits in logs-servicenow.event-66166 data stream
2024/11/01 21:28:29 DEBUG found 0 hits in logs-servicenow.event-66166 data stream
2024/11/01 21:28:30 DEBUG found 0 hits in logs-servicenow.event-66166 data stream
2024/11/01 21:28:31 DEBUG found 0 hits in logs-servicenow.event-66166 data stream
2024/11/01 21:28:32 DEBUG found 0 hits in logs-servicenow.event-66166 data stream
2024/11/01 21:28:33 DEBUG found 0 hits in logs-servicenow.event-66166 data stream
2024/11/01 21:28:34 DEBUG found 0 hits in logs-servicenow.event-66166 data stream
2024/11/01 21:28:35 DEBUG found 0 hits in logs-servicenow.event-66166 data stream
2024/11/01 21:28:36 DEBUG found 0 hits in logs-servicenow.event-66166 data stream
2024/11/01 21:28:37 DEBUG found 0 hits in logs-servicenow.event-66166 data stream
2024/11/01 21:28:38 DEBUG found 0 hits in logs-servicenow.event-66166 data stream
2024/11/01 21:28:39 DEBUG found 0 hits in logs-servicenow.event-66166 data stream
2024/11/01 21:28:40 DEBUG found 3 hits in logs-servicenow.event-66166 data stream
2024/11/01 21:28:44 DEBUG found 3 hits in logs-servicenow.event-66166 data stream
2024/11/01 21:28:44 DEBUG running command: /usr/bin/docker compose version --short
2024/11/01 21:28:44 DEBUG Determined Docker Compose version: 2.27.0
2024/11/01 21:28:44 DEBUG running command: /usr/bin/docker compose -f /root/github/integrations/packages/servicenow/_dev/deploy/docker/docker-compose.yml -p elastic-package-service-47870 ps -a -q servicenow
WARN[0000] /root/github/integrations/packages/servicenow/_dev/deploy/docker/docker-compose.yml: `version` is obsolete 
2024/11/01 21:28:44 DEBUG output command: /usr/bin/docker inspect 58c5e029fe62f16e772f8f9be8aaede6cbbaa4c3818732a0d90cf1d0ee52a16d
2024/11/01 21:28:44 DEBUG check whether or not synthetics is enabled (data stream logs-servicenow.event-66166)...
2024/11/01 21:28:44 DEBUG data stream logs-servicenow.event-66166 has synthetics enabled: false
2024/11/01 21:28:44 DEBUG Imported ECS fields definition from external schema for validation (embedded in package: false, stack uses ecs@mappings template: true)
2024/11/01 21:28:45 DEBUG assert hit count expected 3, observed 3
2024/11/01 21:28:45 DEBUG reassigning original policy back to agent...
2024/11/01 21:28:45 DEBUG PUT https://127.0.0.1:5601/api/fleet/agents/243ebe11-4a3c-4c33-8b0b-2df4c8097581/reassign
2024/11/01 21:28:46 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/243ebe11-4a3c-4c33-8b0b-2df4c8097581
2024/11/01 21:28:46 DEBUG Agent 243ebe11-4a3c-4c33-8b0b-2df4c8097581 (Host: docker-fleet-agent): Policy ID elastic-agent-managed-ep LogLevel: info Status: updating
2024/11/01 21:28:46 DEBUG Wait until the policy (ID: elastic-agent-managed-ep, revision: 14) is assigned to the agent (ID: 243ebe11-4a3c-4c33-8b0b-2df4c8097581)...
2024/11/01 21:28:48 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/243ebe11-4a3c-4c33-8b0b-2df4c8097581
2024/11/01 21:28:48 DEBUG Agent 243ebe11-4a3c-4c33-8b0b-2df4c8097581 (Host: docker-fleet-agent): Policy ID elastic-agent-managed-ep LogLevel: info Status: updating
2024/11/01 21:28:48 DEBUG Wait until the policy (ID: elastic-agent-managed-ep, revision: 14) is assigned to the agent (ID: 243ebe11-4a3c-4c33-8b0b-2df4c8097581)...
2024/11/01 21:28:50 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/243ebe11-4a3c-4c33-8b0b-2df4c8097581
2024/11/01 21:28:50 DEBUG Agent 243ebe11-4a3c-4c33-8b0b-2df4c8097581 (Host: docker-fleet-agent): Policy ID elastic-agent-managed-ep LogLevel: info Status: updating
2024/11/01 21:28:50 DEBUG Wait until the policy (ID: elastic-agent-managed-ep, revision: 14) is assigned to the agent (ID: 243ebe11-4a3c-4c33-8b0b-2df4c8097581)...
2024/11/01 21:28:52 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/243ebe11-4a3c-4c33-8b0b-2df4c8097581
2024/11/01 21:28:52 DEBUG Agent 243ebe11-4a3c-4c33-8b0b-2df4c8097581 (Host: docker-fleet-agent): Policy ID elastic-agent-managed-ep LogLevel: info Status: online
2024/11/01 21:28:52 DEBUG Policy revision assigned to the agent (ID: 243ebe11-4a3c-4c33-8b0b-2df4c8097581)...
2024/11/01 21:28:52 DEBUG tearing down service...
2024/11/01 21:28:52 DEBUG tearing down service using Docker Compose runner
2024/11/01 21:28:52 DEBUG running command: /usr/bin/docker compose version --short
2024/11/01 21:28:52 DEBUG Determined Docker Compose version: 2.27.0
2024/11/01 21:28:52 DEBUG running command: /usr/bin/docker compose -f /root/github/integrations/packages/servicenow/_dev/deploy/docker/docker-compose.yml -p elastic-package-service-47870 stop
WARN[0000] /root/github/integrations/packages/servicenow/_dev/deploy/docker/docker-compose.yml: `version` is obsolete 
[+] Stopping 1/1
 ✔ Container elastic-package-service-47870-servicenow-1  Stopped                                                                                              1.1s 
2024/11/01 21:28:54 DEBUG running command: /usr/bin/docker compose -f /root/github/integrations/packages/servicenow/_dev/deploy/docker/docker-compose.yml -p elastic-package-service-47870 logs
WARN[0000] /root/github/integrations/packages/servicenow/_dev/deploy/docker/docker-compose.yml: `version` is obsolete 
2024/11/01 21:28:54  INFO Write container logs to file: /root/github/integrations/build/container-logs/servicenow-1730476734165175812.log
2024/11/01 21:28:54 DEBUG running command: /usr/bin/docker compose -f /root/github/integrations/packages/servicenow/_dev/deploy/docker/docker-compose.yml -p elastic-package-service-47870 down --volumes
WARN[0000] /root/github/integrations/packages/servicenow/_dev/deploy/docker/docker-compose.yml: `version` is obsolete 
[+] Running 2/2
 ✔ Container elastic-package-service-47870-servicenow-1  Removed                                                                                              0.0s 
 ✔ Network elastic-package-service-47870_default         Removed                                                                                              0.2s 
2024/11/01 21:28:54 DEBUG Deleting data stream for testing logs-servicenow.event-66166
2024/11/01 21:28:54 DEBUG reassigning original log level "info" back to agent...
2024/11/01 21:28:54 DEBUG POST https://127.0.0.1:5601/api/fleet/agents/243ebe11-4a3c-4c33-8b0b-2df4c8097581/actions
2024/11/01 21:28:55 DEBUG deleting test policies...
2024/11/01 21:28:55 DEBUG POST https://127.0.0.1:5601/api/fleet/agent_policies/delete
2024/11/01 21:28:58 DEBUG POST https://127.0.0.1:5601/api/fleet/agent_policies/delete
2024/11/01 21:29:00 DEBUG Dump Elastic stack data
2024/11/01 21:29:00 DEBUG Dump stack logs (location: /tmp/test-system-3252866976)
2024/11/01 21:29:00 DEBUG output command: /usr/bin/docker ps -a --filter label=com.docker.compose.project=elastic-package-stack --format {{.ID}}
2024/11/01 21:29:00 DEBUG output command: /usr/bin/docker inspect f38469bc2ef6 1453beff71a5 118d99dcd6d1 721a9d54799b e8dd47b972de 5bcd903df9ee e6ec02c8d7a3 99c362a6ca9c a2909d944efb 9a8543acdeec
2024/11/01 21:29:00 DEBUG Dump stack logs for package-registry
2024/11/01 21:29:00 DEBUG running command: /usr/bin/docker compose version --short
2024/11/01 21:29:00 DEBUG Determined Docker Compose version: 2.27.0
2024/11/01 21:29:00 DEBUG running command: /usr/bin/docker compose -f /root/.elastic-package/profiles/default/stack/docker-compose.yml -p elastic-package-stack logs package-registry
2024/11/01 21:29:00 DEBUG Dump stack logs for elastic-agent
2024/11/01 21:29:00 DEBUG running command: /usr/bin/docker compose version --short
2024/11/01 21:29:00 DEBUG Determined Docker Compose version: 2.27.0
2024/11/01 21:29:00 DEBUG running command: /usr/bin/docker compose -f /root/.elastic-package/profiles/default/stack/docker-compose.yml -p elastic-package-stack logs elastic-agent
2024/11/01 21:29:01 DEBUG running command: /usr/bin/docker compose version --short
2024/11/01 21:29:01 DEBUG Determined Docker Compose version: 2.27.0
2024/11/01 21:29:01 DEBUG run command: /usr/bin/docker cp elastic-package-stack-elastic-agent-1:/usr/share/elastic-agent/state/data/logs/ /tmp/test-system-3252866976/logs/elastic-agent-internal
2024/11/01 21:29:01 DEBUG Dump stack logs for fleet-server
2024/11/01 21:29:01 DEBUG running command: /usr/bin/docker compose version --short
2024/11/01 21:29:02 DEBUG Determined Docker Compose version: 2.27.0
2024/11/01 21:29:02 DEBUG running command: /usr/bin/docker compose -f /root/.elastic-package/profiles/default/stack/docker-compose.yml -p elastic-package-stack logs fleet-server
2024/11/01 21:29:17 DEBUG running command: /usr/bin/docker compose version --short
2024/11/01 21:29:17 DEBUG Determined Docker Compose version: 2.27.0
2024/11/01 21:29:17 DEBUG run command: /usr/bin/docker cp elastic-package-stack-fleet-server-1:/usr/share/elastic-agent/state/data/logs/ /tmp/test-system-3252866976/logs/fleet-server-internal
2024/11/01 21:29:21 DEBUG Dump stack logs for kibana
2024/11/01 21:29:22 DEBUG running command: /usr/bin/docker compose version --short
2024/11/01 21:29:22 DEBUG Determined Docker Compose version: 2.27.0
2024/11/01 21:29:22 DEBUG running command: /usr/bin/docker compose -f /root/.elastic-package/profiles/default/stack/docker-compose.yml -p elastic-package-stack logs kibana
2024/11/01 21:29:23 DEBUG Dump stack logs for elasticsearch
2024/11/01 21:29:24 DEBUG running command: /usr/bin/docker compose version --short
2024/11/01 21:29:24 DEBUG Determined Docker Compose version: 2.27.0
2024/11/01 21:29:24 DEBUG running command: /usr/bin/docker compose -f /root/.elastic-package/profiles/default/stack/docker-compose.yml -p elastic-package-stack logs elasticsearch
2024/11/01 21:29:28 DEBUG Uninstalling package...
2024/11/01 21:29:28 DEBUG GET https://127.0.0.1:5601/api/fleet/epm/packages/servicenow
2024/11/01 21:29:29 DEBUG DELETE https://127.0.0.1:5601/api/fleet/epm/packages/servicenow/0.1.0
--- Test results for package: servicenow - START ---
╭────────────┬─────────────┬───────────┬───────────┬────────┬───────────────╮
│ PACKAGE    │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │  TIME ELAPSED │
├────────────┼─────────────┼───────────┼───────────┼────────┼───────────────┤
│ servicenow │ event       │ system    │ default   │ PASS   │ 42.598154037s │
╰────────────┴─────────────┴───────────┴───────────┴────────┴───────────────╯
--- Test results for package: servicenow - END   ---
Done

Screenshots

snow_ss_1
snow_ss_2

@kcreddy

kcreddy commented Oct 28, 2024

Copy link
Copy Markdown
Contributor

/test

@kcreddy kcreddy added Crest Contributions from Crest developement team. Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels Oct 28, 2024
@elasticmachine

Copy link
Copy Markdown

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@kcreddy kcreddy added New Integration Issue or pull request for creating a new integration package. Integration:servicenow_cmdb labels Oct 28, 2024
Comment thread packages/servicenow_cmdb/data_stream/event/agent/stream/cel.yml.hbs Outdated
Comment thread packages/servicenow_cmdb/data_stream/event/agent/stream/cel.yml.hbs Outdated
Comment thread packages/servicenow_cmdb/data_stream/event/agent/stream/cel.yml.hbs
Comment thread packages/servicenow_cmdb/data_stream/event/agent/stream/cel.yml.hbs Outdated
1. Implemented changes.
2. Added package inside codeowners.
@muskan-agarwal26
muskan-agarwal26 requested a review from efd6 October 28, 2024 09:19

@efd6 efd6 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is too big to be safely reviewed in one sitting. I've done the best that I can, but I will take another look later.

Comment thread packages/servicenow_cmdb/_dev/build/docs/README.md Outdated
Comment thread packages/servicenow_cmdb/_dev/build/docs/README.md Outdated
Comment thread packages/servicenow_cmdb/_dev/build/docs/README.md Outdated
Comment thread packages/servicenow_cmdb/_dev/build/docs/README.md Outdated
Comment thread packages/servicenow_cmdb/_dev/build/docs/README.md Outdated
Comment thread packages/servicenow_cmdb/data_stream/event/manifest.yml Outdated
Comment thread packages/servicenow_cmdb/data_stream/event/manifest.yml Outdated
Comment thread packages/servicenow_cmdb/data_stream/event/manifest.yml Outdated
Comment thread packages/servicenow_cmdb/data_stream/event/manifest.yml
1. Readme suggestions.
2. Changed the contains stmt in pipeline.
3. Added rule in config as suggested.
4. Removed the link of this page from dashboard.
5. Updated description for table_name in manifest.
@muskan-agarwal26
muskan-agarwal26 requested a review from efd6 October 30, 2024 16:58
@efd6

efd6 commented Oct 30, 2024

Copy link
Copy Markdown
Contributor

/test

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

1. Rename package to servicenow.
2. Add next_page to the cursor and update data collection.
3. implement readme suggestions.
4. Remove (This Page) from dashboard links.
@muskan-agarwal26 muskan-agarwal26 changed the title [servicenow_cmdb] Initial release of ServiceNow CMDB Nov 1, 2024
@muskan-agarwal26
muskan-agarwal26 requested a review from efd6 November 1, 2024 16:19
@efd6

efd6 commented Nov 3, 2024

Copy link
Copy Markdown
Contributor

/test

Comment thread packages/servicenow/data_stream/event/agent/stream/aws-s3.yml.hbs
Comment thread packages/servicenow/data_stream/event/agent/stream/cel.yml.hbs
1. Replace the integrations link.
@muskan-agarwal26
muskan-agarwal26 requested a review from efd6 November 4, 2024 15:04
@efd6

efd6 commented Nov 4, 2024

Copy link
Copy Markdown
Contributor

/test

@elasticmachine

Copy link
Copy Markdown

💚 Build Succeeded

History

@efd6
efd6 merged commit 45c013f into elastic:main Nov 5, 2024
@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package servicenow - 0.1.0 containing this change is available at https://epr.elastic.co/search?package=servicenow

harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 4, 2025
Create New integration package servicenow.

* add event data stream.
* add data collection logic for the data stream.
* add the ingest pipeline for the data stream.
* map fields according to the ECS schema and add fields metadata in the
  appropriate yml files.
* add dashboard and visualizations.
* add pipeline test for the data stream.
* add system test cases for the data stream.
harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 5, 2025
Create New integration package servicenow.

* add event data stream.
* add data collection logic for the data stream.
* add the ingest pipeline for the data stream.
* map fields according to the ECS schema and add fields metadata in the
  appropriate yml files.
* add dashboard and visualizations.
* add pipeline test for the data stream.
* add system test cases for the data stream.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Crest Contributions from Crest developement team. Integration:servicenow ServiceNow New Integration Issue or pull request for creating a new integration package. Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

5 participants