Skip to content

[Stack Monitoring] Add an "Excessive segment count" alert rule #231378

@geekpete

Description

@geekpete

Describe the feature:

A rules to monitor segment counts at shard/index/node levels and alert when the number of segments becomes excessive for each context.

Describe a specific use case for the feature:

A rule to alert on excessive segment counts that might indicate problems with successful merging affecting different cluster contexts.

Problems this might help avoid or alert on may include:

  • impact from bugs, eg Use local segment fieldInfos to lookup tsdb merge stats elasticsearch#132597
  • impact from resource over utilisation (eg too many indices/segments on a node)
  • best practice not being followed or edge cases that may lead to performance or stability impacts
    • lack of force merge for a use case where it's generally required
    • excessive scrolls holding segments up from merging out deleted lucene docs, leading to excessive segment counts
    • extreme update workloads where merge is throttling or just isn't keeping up for particular reasons leading to unhealth segment counts

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions