Skip to content

[ti_eset] Add puaadware data stream - #19132

Open
polakovicp wants to merge 2 commits into
elastic:ti_set_polakovicp_enhancementsfrom
polakovicp:ti_eset_puaadware
Open

[ti_eset] Add puaadware data stream#19132
polakovicp wants to merge 2 commits into
elastic:ti_set_polakovicp_enhancementsfrom
polakovicp:ti_eset_puaadware

Conversation

@polakovicp

@polakovicp polakovicp commented May 21, 2026

Copy link
Copy Markdown
Contributor

[ti_eset] Add puaadware data stream

PR adds new PUA adware data stream to ti_eset integration. Documentation for this data stream is submitted as a separate PR

Related issues

Relates to PR #18709

$ elastic-package test                                                                                                                  

--- Test results for package: ti_eset - START ---
╭─────────┬─────────────┬───────────┬──────────────────────────────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME                                                        │ RESULT │ TIME ELAPSED │
├─────────┼─────────────┼───────────┼──────────────────────────────────────────────────────────────────┼────────┼──────────────┤
│ ti_eset │             │ asset     │ dashboard ti_eset-402bf6b0-8194-11ee-a704-a3364df0a052 is loaded │ PASS   │           0s │
│ ti_eset │             │ asset     │ dashboard ti_eset-4fc91930-819a-11ee-a704-a3364df0a052 is loaded │ PASS   │           0s │
│ ti_eset │             │ asset     │ dashboard ti_eset-c27d91e0-819a-11ee-a704-a3364df0a052 is loaded │ PASS   │           0s │
│ ti_eset │             │ asset     │ dashboard ti_eset-f608e350-8192-11ee-a704-a3364df0a052 is loaded │ PASS   │           0s │
│ ti_eset │ apt         │ asset     │ index_template logs-ti_eset.apt is loaded                        │ PASS   │           0s │
│ ti_eset │ apt         │ asset     │ ingest_pipeline logs-ti_eset.apt-1.10.0 is loaded                │ PASS   │           0s │
│ ti_eset │ botnet      │ asset     │ index_template logs-ti_eset.botnet is loaded                     │ PASS   │           0s │
│ ti_eset │ botnet      │ asset     │ ingest_pipeline logs-ti_eset.botnet-1.10.0 is loaded             │ PASS   │           0s │
│ ti_eset │ cc          │ asset     │ index_template logs-ti_eset.cc is loaded                         │ PASS   │           0s │
│ ti_eset │ cc          │ asset     │ ingest_pipeline logs-ti_eset.cc-1.10.0 is loaded                 │ PASS   │           0s │
│ ti_eset │ domains     │ asset     │ index_template logs-ti_eset.domains is loaded                    │ PASS   │           0s │
│ ti_eset │ domains     │ asset     │ ingest_pipeline logs-ti_eset.domains-1.10.0 is loaded            │ PASS   │           0s │
│ ti_eset │ files       │ asset     │ index_template logs-ti_eset.files is loaded                      │ PASS   │           0s │
│ ti_eset │ files       │ asset     │ ingest_pipeline logs-ti_eset.files-1.10.0 is loaded              │ PASS   │           0s │
│ ti_eset │ ip          │ asset     │ index_template logs-ti_eset.ip is loaded                         │ PASS   │           0s │
│ ti_eset │ ip          │ asset     │ ingest_pipeline logs-ti_eset.ip-1.10.0 is loaded                 │ PASS   │           0s │
│ ti_eset │ puaadware   │ asset     │ index_template logs-ti_eset.puaadware is loaded                  │ PASS   │           0s │
│ ti_eset │ puaadware   │ asset     │ ingest_pipeline logs-ti_eset.puaadware-1.10.0 is loaded          │ PASS   │           0s │
│ ti_eset │ url         │ asset     │ index_template logs-ti_eset.url is loaded                        │ PASS   │           0s │
│ ti_eset │ url         │ asset     │ ingest_pipeline logs-ti_eset.url-1.10.0 is loaded                │ PASS   │           0s │
╰─────────┴─────────────┴───────────┴──────────────────────────────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: ti_eset - END   ---
Done
Run pipeline tests for the package
--- Test results for package: ti_eset - START ---
╭─────────┬─────────────┬───────────┬─────────────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME                                       │ RESULT │ TIME ELAPSED │
├─────────┼─────────────┼───────────┼─────────────────────────────────────────────────┼────────┼──────────────┤
│ ti_eset │ apt         │ pipeline  │ (ingest pipeline warnings test-eset-ndjson.log) │ PASS   │   865.4469ms │
│ ti_eset │ apt         │ pipeline  │ test-eset-ndjson.log                            │ PASS   │   447.3099ms │
│ ti_eset │ botnet      │ pipeline  │ (ingest pipeline warnings test-eset-ndjson.log) │ PASS   │   910.9299ms │
│ ti_eset │ botnet      │ pipeline  │ test-eset-ndjson.log                            │ PASS   │   163.0392ms │
│ ti_eset │ cc          │ pipeline  │ (ingest pipeline warnings test-eset-ndjson.log) │ PASS   │   880.8849ms │
│ ti_eset │ cc          │ pipeline  │ test-eset-ndjson.log                            │ PASS   │   148.9523ms │
│ ti_eset │ domains     │ pipeline  │ (ingest pipeline warnings test-eset-ndjson.log) │ PASS   │   873.0505ms │
│ ti_eset │ domains     │ pipeline  │ test-eset-ndjson.log                            │ PASS   │    88.8606ms │
│ ti_eset │ files       │ pipeline  │ (ingest pipeline warnings test-eset-ndjson.log) │ PASS   │    890.614ms │
│ ti_eset │ files       │ pipeline  │ test-eset-ndjson.log                            │ PASS   │   195.9159ms │
│ ti_eset │ ip          │ pipeline  │ (ingest pipeline warnings test-eset-ndjson.log) │ PASS   │   861.7389ms │
│ ti_eset │ ip          │ pipeline  │ test-eset-ndjson.log                            │ PASS   │   135.4918ms │
│ ti_eset │ puaadware   │ pipeline  │ (ingest pipeline warnings test-eset-ndjson.log) │ PASS   │   941.9493ms │
│ ti_eset │ puaadware   │ pipeline  │ test-eset-ndjson.log                            │ PASS   │    68.5607ms │
│ ti_eset │ url         │ pipeline  │ (ingest pipeline warnings test-eset-ndjson.log) │ PASS   │   923.7343ms │
│ ti_eset │ url         │ pipeline  │ test-eset-ndjson.log                            │ PASS   │    178.014ms │
╰─────────┴─────────────┴───────────┴─────────────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: ti_eset - END   ---
Done
Run static tests for the package
2026/08/07 21:43:14  INFO elastic-package v0.125.1 version-hash undefined (build time: unknown)
--- Test results for package: ti_eset - START ---
╭─────────┬─────────────┬───────────┬──────────────────────────┬────────┬──────────────╮
│ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME                │ RESULT │ TIME ELAPSED │
├─────────┼─────────────┼───────────┼──────────────────────────┼────────┼──────────────┤
│ ti_eset │ apt         │ static    │ Verify sample_event.json │ PASS   │    87.0833ms │
│ ti_eset │ botnet      │ static    │ Verify sample_event.json │ PASS   │    73.0061ms │
│ ti_eset │ cc          │ static    │ Verify sample_event.json │ PASS   │    65.3181ms │
│ ti_eset │ domains     │ static    │ Verify sample_event.json │ PASS   │     63.584ms │
│ ti_eset │ files       │ static    │ Verify sample_event.json │ PASS   │    73.1127ms │
│ ti_eset │ ip          │ static    │ Verify sample_event.json │ PASS   │    63.2014ms │
│ ti_eset │ puaadware   │ static    │ Verify sample_event.json │ PASS   │    74.0053ms │
│ ti_eset │ url         │ static    │ Verify sample_event.json │ PASS   │     72.387ms │
╰─────────┴─────────────┴───────────┴──────────────────────────┴────────┴──────────────╯
--- Test results for package: ti_eset - END   ---
Done
Run system tests for the package
--- Test results for package: ti_eset - START ---
╭─────────┬─────────────┬───────────┬───────────┬────────┬──────────────╮
│ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │ TIME ELAPSED │
├─────────┼─────────────┼───────────┼───────────┼────────┼──────────────┤
│ ti_eset │ apt         │ system    │ httpjson  │ PASS   │   56.052363s │
│ ti_eset │ botnet      │ system    │ httpjson  │ PASS   │ 1m1.0934608s │
│ ti_eset │ cc          │ system    │ httpjson  │ PASS   │  50.0136356s │
│ ti_eset │ domains     │ system    │ httpjson  │ PASS   │  44.8374914s │
│ ti_eset │ files       │ system    │ httpjson  │ PASS   │  50.0249208s │
│ ti_eset │ ip          │ system    │ httpjson  │ PASS   │  49.4685785s │
│ ti_eset │ puaadware   │ system    │ cel       │ PASS   │  45.9652368s │
│ ti_eset │ url         │ system    │ httpjson  │ PASS   │   52.535885s │
╰─────────┴─────────────┴───────────┴───────────┴────────┴──────────────╯
--- Test results for package: ti_eset - END   ---
Done
@polakovicp
polakovicp requested a review from a team as a code owner May 21, 2026 14:47
@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Reviewers

Buildkite won't run for external contributors automatically; you need to add a comment:

  • /test : will kick off a build in Buildkite.

NOTE: https://github.com/elastic/integrations/blob/main/.buildkite/pull-requests.json contains all those details.

@andrewkroh andrewkroh added Integration:ti_eset ESET Threat Intelligence (Partner supported) Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels May 21, 2026
@infra-vault-gh-plugin-prod

Copy link
Copy Markdown

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@botelastic

botelastic Bot commented Jun 25, 2026

Copy link
Copy Markdown

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

@botelastic botelastic Bot added the Stalled label Jun 25, 2026
@mergify

mergify Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request
@vera-review-bot

Copy link
Copy Markdown

👀 I have started reviewing the PR


state:
url: {{url}}
username: {{username}}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 HIGH data_stream/puaadware/.../cel.yml.hbs:20

CEL stream consumes username/password vars that are never declared

The new cel template builds its Basic auth header from state.username / state.password (lines 20-21, 50), but username and password are not declared as vars anywhere this stream can reach. The data stream manifest (packages/ti_eset/data_stream/puaadware/manifest.yml) declares no username/password vars, and the only place they exist in the package is under the root policy template's httpjson input (packages/ti_eset/manifest.yml lines 42-52). The root manifest has a single policy template eset whose only input is type httpjson; there is no cel input. Because Fleet associates a data stream stream to a policy-template input by matching input type, this input: cel stream is not backed by any policy-template input, so it is not offered for configuration and the credential vars are out of scope. As committed, the Authorization header resolves to Basic + base64(":") and authentication to the ETI TAXII endpoint fails. The system test passes only because the test harness injects top-level username/password vars, which does not reflect a real Fleet policy.

Recommendation:

Add a cel input to the eset policy template (or a dedicated policy template) so the new data stream is wired up and the shared ETI credentials are in scope for it:

policy_templates:
  - name: eset
    title: ETI feeds (TAXII version 2)
    description: Collect data from ETI feeds (TAXII version 2)
    inputs:
      - type: httpjson
        title: ETI feeds (TAXII version 2)
        description: Collect data from ETI feeds (TAXII version 2)
        vars:
          - name: username
            type: text
            title: ETI Username
            required: true
            show_user: true
          - name: password
            type: password
            title: ETI Password
            required: true
            secret: true
            show_user: true
      - type: cel
        title: ETI feeds (TAXII version 2.1, CEL)
        description: Collect data from ETI feeds using CEL
        vars:
          - name: username
            type: text
            title: ETI Username
            required: true
            show_user: true
          - name: password
            type: password
            title: ETI Password
            required: true
            secret: true
            show_user: true

(The root manifest is outside this PR's changed files, so this change must be added there as part of wiring the new cel stream.)


🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

delay: 120s
field: event.ingested
settings:
num_failure_retries: -1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 MEDIUM elasticsearch/transform/puaadware_latest_ioc/transform.yml:28

Latest transform omits unattended: true

The new latest transform sets settings.num_failure_retries: -1 but does not set unattended: true. Every other ti_eset latest transform (apt, botnet, cc, domains, files, ip, url) uses settings.unattended: true. Without unattended mode the transform does not auto-recover (it stops and requires manual restart on certain failure conditions), which is exactly what unattended is required to prevent for a continuously-running TI latest transform. This is an inconsistency with the established package pattern and removes the auto-recovery guarantee.

Recommendation:

Set unattended: true to match the other ti_eset transforms (keep the retry override if desired):

settings:
  unattended: true
  num_failure_retries: -1

🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

@botelastic botelastic Bot removed the Stalled label Jul 17, 2026
# newer versions go on top
- version: "1.10.0"
changes:
- description: PLACEHOLDER - specified once all related PRs will be merged to branch ti_set_polakovicp_enhancements

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: 🟠 High confidence: high path: packages/ti_eset/changelog.yml:4

The 1.10.0 changelog entry is a literal placeholder with link: TODO; replace the description with a real user-facing summary and point the link at this PR before merge.

Details

The new 1.10.0 entry reads description: PLACEHOLDER - specified once all related PRs will be merged to branch ti_set_polakovicp_enhancements and link: TODO. Changelog descriptions are rendered verbatim as the package release notes in Fleet, and link is expected to be a pull request URL (every prior entry in this file, e.g. 1.9.0, uses https://github.com/elastic/integrations/pull/<n>). Shipping this text would publish PLACEHOLDER and TODO to users.

The entry also bundles several distinct user-visible changes under a single line: the new puaadware data stream and transform, and the relocation of the username/password variables in the root manifest (see the separate finding on manifest.yml). Each should get its own change entry.

Recommendation:

Replace the placeholder with real entries and a PR link:

- version: "1.10.0"
  changes:
    - description: Add new PUA adware data stream collected over CEL.
      type: enhancement
      link: https://github.com/elastic/integrations/pull/19132
    - description: Move ETI username and password to package level variables.
      type: breaking-change
      link: https://github.com/elastic/integrations/pull/19132

🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

Comment thread packages/ti_eset/manifest.yml Outdated
size: 96x96
type: image/svg+xml
vars:
- name: username

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: 🟠 High confidence: medium path: packages/ti_eset/manifest.yml:34

Moving username/password from the httpjson input to package level vars breaks upgrades of existing 1.9.0 policies; either keep the input level vars or record it as a breaking-change and document the reconfiguration.

Details

This diff deletes vars.username and vars.password from policy_templates[0].inputs[0] (the httpjson input) and re-declares them as package level vars, both required: true with no default.

Existing installations of 1.9.0 store those credential values on the httpjson input of the package policy. When Fleet upgrades a policy it reconciles variables by name within the same scope: input scoped variables that no longer exist in the new manifest are dropped, and the two new required package level variables have no value and no default to fall back on. The upgrade therefore surfaces as a conflict that the user must resolve by re-entering the ETI username and password; it is not a transparent upgrade.

Rendering is unaffected either way, since package level variables are also exposed to stream templates and the sibling templates (for example data_stream/files/agent/stream/httpjson.yml.hbs lines 9-10) reference plain {{username}} / {{password}}. The problem is purely the migration of already stored values.

Recommendation:

If the relocation is intentional (it is what lets the new cel input reuse the same credentials), keep it but classify it correctly in changelog.yml and call it out in the README setup section:

- version: "1.10.0"
  changes:
    - description: Move ETI username and password from the httpjson input to package level variables. Existing integration policies must re-enter the credentials after upgrading.
      type: breaking-change
      link: https://github.com/elastic/integrations/pull/19132

Otherwise, leave the existing httpjson input variables in place and declare the credentials on the new cel input as well, so no stored value is orphaned:

policy_templates:
  - name: eset
    inputs:
      - type: httpjson
        title: ETI Legacy Feeds (TAXII 2.1)
        description: Collection of ETI legacy feeds
        vars:
          - name: username
            type: text
            title: ETI Username
            required: true
            show_user: true
          - name: password
            type: password
            title: ETI Password
            multi: false
            required: true
            secret: true
            show_user: true

🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

@@ -0,0 +1,103 @@
title: PUA adware

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: 🟡 Medium confidence: high path: packages/ti_eset/data_stream/puaadware/manifest.yml:1

The new puaadware data stream is missing from _dev/build/docs/README.md, so it has no rendered field reference or sample event; add it to the three tables and the Logs section.

Details

This PR adds the puaadware data stream, its ILM policy, its sample_event.json and the puaadware_latest_ioc transform, but packages/ti_eset/_dev/build/docs/README.md is not touched by the PR. That file enumerates every data stream in four places, all of which still list only the seven pre-existing feeds:

  • the "Dataset / TAXII2 Collection name" table (lines 6-14)
  • the source datastream to destination index/alias table (lines 22-30)
  • the ILM "Deleted after / Expired after" table (lines 37-45)
  • the ## Logs section, whose {{fields "<name>"}} and {{event "<name>"}} directives generate the field reference and sample event for each data stream (lines 62-104)

Because there is no {{fields "puaadware"}} / {{event "puaadware"}} pair, the committed sample_event.json and the new eset.* field definitions are never rendered into the published documentation.

Recommendation:

Add puaadware to each table and append a Logs subsection in packages/ti_eset/_dev/build/docs/README.md, then rebuild the docs:

| Dataset | TAXII2 Collection name |
|--------:|:-----------------------|
|     apt | apt stix 2.1           |
| puaadware | PUA adware stix 2.1  |

| Source Datastream           | Destination Index Pattern            | Destination Alias             |
|:----------------------------|:-------------------------------------|-------------------------------|
| `logs-ti_eset.puaadware-*`  | logs-ti_eset_latest.dest_puaadware-* | logs-ti_eset_latest.puaadware |

|                    Index | Deleted after | Expired after |
|-------------------------:|:--------------|---------------|
| `logs-ti_eset.puaadware` | 7d            | 48h           |

### PUA adware

{{fields "puaadware"}}

{{event "puaadware"}}

🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

@@ -0,0 +1 @@
{"type": "indicator", "spec_version": "2.1", "id": "indicator--f7c7d9f1-e31d-4c34-9d4b-c4d89b6b51ba", "created_by_ref": "identity--55f6ea5e-51ac-4344-bc8c-4170950d210f", "created": "2025-03-04T18:10:25.000Z", "modified": "2025-03-04T18:10:25.000Z", "name": "Malware variant", "description": "Each of these file hashes indicates that a variant of a variant of Win32/Adware.OpenSUpdater.LC.gen application is present.", "pattern": "[file:hashes.'SHA-256' = 'ece8562d64dad4a1aa24d06be50db38caffeac428d49be9cc6ef6eabf9120a0b'] OR [file:hashes.'SHA-1' = '74ecd87132c2ed11b7ef822692cb5d6d88280e7e'] OR [file:hashes.'MD5' = 'eb9c9d6eb5398d0d223140f866ad769c']", "pattern_type": "stix", "pattern_version": "2.1", "valid_from": "2025-03-04T18:10:25Z", "valid_until": "2025-03-06T18:10:25Z", "labels": ["malicious-activity"], "confidence": 85, "object_marking_refs": ["marking-definition--f88d31f6-486f-44da-b317-01333bde0b82"]}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: 🟡 Medium confidence: medium path: packages/ti_eset/data_stream/puaadware/_dev/test/pipeline/test-eset-ndjson.log:1

The pipeline test fixture contains a single indicator, leaving most pipeline branches unexercised; add documents covering the other confidence labels and the non-indicator drop path.

Details

The fixture has one line, a malicious-activity indicator carrying all three hash types. The new pipeline has several branches that this document never reaches:

  • script_bce6660a (default.yml lines 100-115) maps four label values to confidence; only malicious-activity -> High is covered, and unwanted-activity -> Medium / benign -> Low are the ones actually relevant to a PUA adware feed.
  • drop_5a415bbc (line 47) drops documents whose eti.type != 'indicator'.
  • The if guards on eti.created, eti.modified, eti.valid_until and eti.name are never exercised with the field absent.
  • The foreach over eti._patterns is never exercised with a pattern containing only a subset of the three hash types.

By comparison, the sibling files data stream ships an 11-document fixture at packages/ti_eset/data_stream/files/_dev/test/pipeline/test-eset-ndjson.log.

Recommendation:

Extend the fixture with documents that hit the remaining branches, for example an unwanted-activity indicator with a single hash and no valid_until, plus a non-indicator object that must be dropped:

{"type": "indicator", "spec_version": "2.1", "id": "indicator--0f0e5b1a-1111-4222-8333-444455556666", "created": "2025-03-05T09:00:00.000Z", "modified": "2025-03-05T09:00:00.000Z", "name": "Adware variant", "pattern": "[file:hashes.'SHA-256' = '5f2b7c1d9e3a4b6c8d0e2f4a6b8c0d2e4f6a8b0c2d4e6f8a0b2c4d6e8f0a2b4c']", "pattern_type": "stix", "labels": ["unwanted-activity"]}
{"type": "identity", "spec_version": "2.1", "id": "identity--55f6ea5e-51ac-4344-bc8c-4170950d210f", "created": "2025-03-05T09:00:00.000Z", "modified": "2025-03-05T09:00:00.000Z", "name": "ESET"}

Then regenerate the expected file with elastic-package test pipeline --generate.


🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

@@ -0,0 +1,3 @@
- name: input.type
type: keyword
description: Input type. No newline at end of file

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: 🔵 Low confidence: low path: packages/ti_eset/data_stream/puaadware/fields/agent.yml:3

Three files added or edited by this PR end without a trailing newline; add one so elastic-package format does not rewrite them.

Details

packages/ti_eset/data_stream/puaadware/fields/agent.yml, packages/ti_eset/data_stream/puaadware/elasticsearch/ilm/default_policy.json and packages/ti_eset/_dev/deploy/docker/files/config.yml all end with \ No newline at end of file in the diff. For fields/agent.yml this is a regression introduced by the latest commit: the previous revision of the file ended with a newline and the trimming commit removed it. elastic-package format normalises the trailing newline on YAML and JSON files, so these files will show up as modified after a format run.

Recommendation:

Terminate each file with a newline, e.g. packages/ti_eset/data_stream/puaadware/fields/agent.yml:

- name: input.type
  type: keyword
  description: Input type.

(with a trailing newline after the last line), then run elastic-package format in packages/ti_eset to confirm nothing else changes.


🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.


### PUA adware

{{fields "puaadware"}}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: 🟠 High confidence: high path: packages/ti_eset/_dev/build/docs/README.md:105

The generated docs/README.md was never regenerated from this template, so the published docs contain no puaadware content at all. Run elastic-package build and commit the regenerated packages/ti_eset/docs/README.md.

Details

This template adds the ### PUA adware section (lines 103-107) plus a puaadware row in the dataset table (line 14), the transform table (line 31) and the ILM table (line 47). However, the generated file packages/ti_eset/docs/README.md — which IS included in this PR — contains zero occurrences of puaadware, puadualapps or PUA adware. Its dataset table still stops at url (docs/README.md lines 6-14), the transform table has only 7 rows (lines 22-30) and the ILM table only 7 rows (lines 37-45); no rendered PUA adware fields table or sample event exists. The only edits made to docs/README.md in this PR are refreshed agent.ephemeral_id / namespace / event.ingested values in the pre-existing sample events, which means the file was hand-edited or partially copied rather than produced by elastic-package build. Because elastic-package check regenerates docs/README.md from this template and fails when the committed copy differs, CI will fail on this package, and if it did merge the published integration docs would not document the new data stream.

Recommendation:

Regenerate the docs from the template and commit the result:

cd packages/ti_eset
elastic-package build
# commit the regenerated packages/ti_eset/docs/README.md

After regenerating, packages/ti_eset/docs/README.md should contain the ### PUA adware heading, the rendered field table, the puaadware sample event, and the puaadware rows in all three tables.


🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

| `logs-ti_eset.domains-*` | logs-ti_eset_latest.dest_domains-* | logs-ti_eset_latest.domains |
| `logs-ti_eset.files-*` | logs-ti_eset_latest.dest_files-* | logs-ti_eset_latest.files |
| `logs-ti_eset.ip-*` | logs-ti_eset_latest.dest_ip-* | logs-ti_eset_latest.ip |
| `logs-ti_eset.puadualapps-*` | logs-ti_eset_latest.dest_puadualapps-* | logs-ti_eset_latest.puadualapps |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: 🟡 Medium confidence: high path: packages/ti_eset/_dev/build/docs/README.md:31

The transform table row for the new data stream says puadualapps, but the data stream and transform are named puaadware, so the documented index pattern and alias do not exist. Correct the row to use puaadware.

Details

All three cells on this row use puadualapps, which appears nowhere else in the package. The actual transform at packages/ti_eset/elasticsearch/transform/puaadware_latest_ioc/transform.yml declares dest.index: logs-ti_eset_latest.dest_puaadware-1 (line 6) and dest.aliases[0].alias: logs-ti_eset_latest.puaadware (line 8), and its source is logs-ti_eset.puaadware-* (line 22). The ILM table two rows further down (line 47) correctly says logs-ti_eset.puaadware, and the dataset table (line 14) says puaadware, so this row is the odd one out. A user following the docs would query an index pattern and alias that are never created.

Recommendation:

Replace puadualapps with puaadware in all three cells so the row matches the transform's actual source index, destination index and alias. Remember to re-run elastic-package build afterwards so the change reaches docs/README.md.

Suggested change
| `logs-ti_eset.puadualapps-*` | logs-ti_eset_latest.dest_puadualapps-* | logs-ti_eset_latest.puadualapps |
| `logs-ti_eset.puaadware-*` | logs-ti_eset_latest.dest_puaadware-* | logs-ti_eset_latest.puaadware |

🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.


- version: "1.10.0"
changes:
- description: PLACEHOLDER - specified once all related PRs will be merged to branch ti_set_polakovicp_enhancements

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: 🟡 Medium confidence: high path: packages/ti_eset/changelog.yml:5

The 1.10.0 changelog entry is an unresolved placeholder with link: TODO, which is not a valid URL and would surface as literal placeholder text in the Fleet UI. Replace it with a real description and PR link before this reaches main.

Details

The description reads "PLACEHOLDER - specified once all related PRs will be merged to branch ti_set_polakovicp_enhancements" and line 7 sets link: TODO. Every other entry in this file uses a real https://github.com/elastic/integrations/pull/<n> link (e.g. line 12). I recognise this PR targets the feature branch ti_set_polakovicp_enhancements rather than main, so the placeholder is presumably intentional for now — this is a note that it must be resolved before the feature branch merges, since elastic-package check validates the changelog and the text is user-visible in the Fleet UI changelog. When you write the real entry, please also reconsider the version and type: this release narrows conditions.kibana.version from ^8.19.4 || ~9.0.7 || ^9.1.4 to ^9.4.0 (manifest.yml line 12), dropping support for 8.x and for 9.0-9.3 entirely. Dropping stack support is a breaking change, so type: enhancement on a minor 1.10.0 bump understates it; it warrants a type: breaking-change entry and, per the package-spec conventions, a major version bump.

Recommendation:

Replace the placeholder with a real entry, and classify the stack-support drop as breaking:

- version: "2.0.0"
  changes:
    - description: Add puaadware data stream and migrate existing feeds to the CEL input.
      type: enhancement
      link: https://github.com/elastic/integrations/pull/19132
    - description: Drop support for Kibana versions below 9.4.0.
      type: breaking-change
      link: https://github.com/elastic/integrations/pull/19132

🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

field: threat.indicator.name
copy_from: eti.name
if: ctx.eti?.name != null
- remove:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: 🟡 Medium confidence: medium path: packages/ti_eset/data_stream/puaadware/elasticsearch/ingest_pipeline/default.yml:142

The pipeline never removes event.original, so the raw STIX object is indexed for every document even when the user leaves preserve_original_event at its default of false. Add the standard conditional remove processor after the eti cleanup.

Details

Line 30 renames message into event.original unconditionally, and nothing later in the pipeline drops it. The data stream manifest declares a user-facing preserve_original_event variable (packages/ti_eset/data_stream/puaadware/manifest.yml lines 43-50) documented as "Preserves a raw copy of the original event, added to the field event.original", defaulting to false. As written that variable only controls whether the preserve_original_event tag is added to tags by the agent template (cel.yml.hbs lines 29-32); it has no effect on whether event.original is actually stored. The result is that every user gets the full raw STIX indicator duplicated into event.original regardless of the setting, roughly doubling storage for this feed and contradicting the documented behaviour of the variable. The standard Elastic pattern gates the removal on the tag. I am aware the seven existing ti_eset streams share this gap, but this is new code in a new data stream, so it is worth getting right here rather than inheriting it.

Recommendation:

Add the standard conditional removal after the remove of eti (line 142-146) and before script_to_drop_null_values:

- remove:
    field: event.original
    tag: remove_event_original
    if: ctx.tags == null || !(ctx.tags.contains('preserve_original_event'))
    ignore_missing: true

The existing pipeline test already sets the tag via _dev/test/pipeline/test-common-config.yml, so test-eset-ndjson.log-expected.json will keep its event.original and does not need regenerating.


🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

Comment thread packages/ti_eset/manifest.yml Outdated
title: Sample logo
size: 96x96
type: image/svg+xml
vars:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: 🟡 Medium confidence: low path: packages/ti_eset/manifest.yml:33

username and password move from the policy-template input to package level in this release; please confirm Fleet migrates already-configured credentials on upgrade rather than silently blanking them.

Details

In 1.9.0 these two variables were declared under policy_templates[0].inputs[0].vars (the httpjson input). This PR moves them to a package-level vars: block (lines 33-45) and leaves the httpjson and cel inputs with no vars at all. Existing installed package policies store the values against the input, so after upgrade Fleet has to relocate them to the package-level vars. Both are required: true with no default, and password is secret: true, so if the relocation is not handled the upgrade either fails validation or leaves every existing ESET integration policy without credentials — every feed stops collecting until an operator re-enters them. I could not verify Fleet's migration behaviour for input-to-package var moves from the checkout alone, so this is flagged for confirmation rather than as a proven defect. If the relocation is not migrated automatically, it needs to be called out as a breaking change with upgrade instructions in the README (see also finding 4).

Recommendation:

Verify the upgrade path on a stack with a 1.9.0 policy already configured, and if credentials do not carry over, document it in _dev/build/docs/README.md:

## Upgrading to 2.0.0

The ETI username and password moved from the input level to the integration level.
After upgrading, re-enter your ESET Threat Intelligence credentials in the
integration settings for each existing policy.

🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

ilm_policy: logs-ti_eset.puaadware-default_policy
streams:
- input: cel
enabled: false

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: 🔵 Low confidence: medium path: packages/ti_eset/data_stream/puaadware/manifest.yml:6

The puaadware stream is the only one in the package disabled by default, so the feed silently collects nothing after install. Drop enabled: false unless the opt-in default is deliberate.

Details

None of the seven existing data stream manifests (apt, botnet, cc, domains, files, ip, url) set enabled, so they all default to enabled. Only this new stream sets enabled: false. If that is deliberate — for example because the PUA adware collection requires a separate ESET entitlement — it is fine, but it is worth being explicit about, because a user who adds the integration expecting the documented puaadware data stream will get an empty logs-ti_eset.puaadware-* index and an idle puaadware_latest_ioc transform with no obvious indication why. Note the ILM policy, lifecycle.yml and transform are all installed regardless.

Recommendation:

If the feed should collect by default like its siblings, remove the line:

streams:
  - input: cel
    title: PUA adware
    description: Collect data from ETI PUA adware feed
    template_path: cel.yml.hbs

If the opt-in default is intentional, mention in _dev/build/docs/README.md that the PUA adware data stream must be enabled explicitly and why.


🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

@mergify

mergify Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request
@vera-review-bot

Copy link
Copy Markdown

🟢 Reviewed the latest commits 941fa6c — nothing new beyond already posted comments.

Review summary

Issues found across earlier commits e1893fe — 1 high, 4 medium, 1 low
  • 🟠 The generated docs/README.md was never regenerated from this template, so the published docs contain no puaadware content at all. Run elastic-package build and commit the regenerated packages/ti_eset/docs/README.md. (link) (Unresolved)
  • 🟡 The transform table row for the new data stream says puadualapps, but the data stream and transform are named puaadware, so the documented index pattern and alias do not exist. Correct the row to use puaadware. (link) (Unresolved)
  • 🟡 The 1.10.0 changelog entry is an unresolved placeholder with link: TODO, which is not a valid URL and would surface as literal placeholder text in the Fleet UI. Replace it with a real description and PR link before this reaches main. (link) (Unresolved)
  • 🟡 The pipeline never removes event.original, so the raw STIX object is indexed for every document even when the user leaves preserve_original_event at its default of false. Add the standard conditional remove processor after the eti cleanup. (link) (Unresolved)
  • 🟡 username and password move from the policy-template input to package level in this release (link) (Unresolved)
  • 🔵 The puaadware stream is the only one in the package disabled by default, so the feed silently collects nothing after install. Drop enabled: false unless the opt-in default is deliberate. (link) (Unresolved)
Issues found across earlier commits f1a5b34, f68bfc3 — 2 high, 2 medium, 1 low
  • 🟠 The 1.10.0 changelog entry is a literal placeholder with link: TODO (link) (Unresolved)
  • 🟠 Moving username/password from the httpjson input to package level vars breaks upgrades of existing 1.9.0 policies (link) (Unresolved)
  • 🟡 The new puaadware data stream is missing from _dev/build/docs/README.md, so it has no rendered field reference or sample event (link) (Unresolved)
  • 🟡 The pipeline test fixture contains a single indicator, leaving most pipeline branches unexercised (link) (Unresolved)
  • 🔵 Three files added or edited by this PR end without a trailing newline (link) (Unresolved)

Since this is a community PR, a new commit triggers another review — at most once every 30 minutes. I skip the PR while it's approved or has merge conflicts.

🤖 AI-Generated Review | Vera Review Bot - v0.2.7 | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Integration:ti_eset ESET Threat Intelligence (Partner supported) Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

2 participants