Skip to content

integration:cisco_secure_endpoint Fix noisy error log entry when pagination completes #10527

Description

@agmic

The Cisco Secure Endpoint integration collects event data via httpjson. The integration throws an error when it hits the last page, and this gets logged in the Elastic Agent error log with something similar to:

error processing response: template: :1:17: executing "" at <.last_response.body.metadata.links.next>: map has no entry for key "next"
This fills up the Elastic Agent dashboard with agent errors, as it is raised every time the Secure Endpoint data is polled (every 2 minutes/default) and the noise makes it more difficult to find actual errors.

I initially raised this in (elastic/beats#38291) and a new way of handling pagination was merged in (elastic/beats#39929)

The integration now needs to be updated to incorporate this new way of paginating.

existing code
https://github.com/elastic/integrations/blob/129752c5db99035ccc6c8f8e70606e54268b392a/packages/cisco_secure_endpoint/data_stream/event/agent/stream/httpjson.yml.hbs#L36C1-L41C1

response.pagination:
- set:
    target: url.value
    value: '[[ .last_response.body.metadata.links.next ]]'
    fail_on_template_error: true

example from 39929, may need to be adapted

response.pagination_until: '[[not (index .last_response.body.meta "cursor")]]'
response.pagination:
- set:
    target: url.params.cursor
    value: '[[.last_response.body.meta.cursor]]

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions