Skip to content

Restrict the number of spans created through trace_methods misconfiguration #2788

@eyalkoren

Description

@eyalkoren

We saw lots of cases where the trace_methods configuration was abused to capture way-too-many methods using wildcards on full packages. In other cases, the problem is not with the amount of methods being captured, but with capturing a low-latency method that is very frequently being executed.
The only safety measure we currently employ to deal with that is the limitation of spans per transaction, however this limitation still doesn't prevent additional spans from being created (only they will be dropped), because we still need them for breakdown metrics calculations and for context propagation.

So what we can do is simply use the co.elastic.apm.agent.impl.transaction.Transaction#isSpanLimitReached() within the TraceMethodInstrumentation.TraceMethodAdvice and if the limit is reached, simply return null instead of creating a span.

Possibly even better, we can stop creating trace_methods-related spans before the limit is reached, for example when 90% of the limit is reached, so that we won't drop the really "interesting" spans.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions