Skip to content

Log when a trace is too large to compact - #2105

Merged
mapno merged 1 commit into
grafana:mainfrom
scalalang2:scalalang2/log-when-traces-are-too-large
Feb 24, 2023
Merged

Log when a trace is too large to compact#2105
mapno merged 1 commit into
grafana:mainfrom
scalalang2:scalalang2/log-when-traces-are-too-large

Conversation

@scalalang2

@scalalang2 scalalang2 commented Feb 14, 2023

Copy link
Copy Markdown
Contributor

What this PR does:
Log when a trace is too large to compact.
If the total bytes of traces exceeds maximum limit, the following messages will be logged.

{
    "msg": "max size of trace exceeded",
    "tenant": "XXXXX",
    "traceId": "XXXXX",
    "discarded_span_count": 100
}

There will be some dicussions in this PR.
Please refer to following comments.

Which issue(s) this PR fixes:
Fixes #1931

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]
Comment thread modules/compactor/compactor.go Outdated

@joe-elliott joe-elliott left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

two comments. looking good!

Comment thread modules/compactor/compactor.go Outdated
Comment thread modules/compactor/compactor.go Outdated
@scalalang2
scalalang2 force-pushed the scalalang2/log-when-traces-are-too-large branch 2 times, most recently from 42a31fd to 9e848ef Compare February 15, 2023 02:17
Comment thread tempodb/encoding/vparquet/compactor.go

@joe-elliott joe-elliott left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is looking good. two small comments.

i forgot that v2 and vParquet have different paths for counting dropped spans. thank you for tracking this down. i suppose ultimately we'd prefer both act like vParquet does, but that's too much change for this PR.

Comment thread modules/compactor/compactor.go Outdated
Comment thread tempodb/encoding/vparquet/compactor.go
@scalalang2
scalalang2 force-pushed the scalalang2/log-when-traces-are-too-large branch from 9e848ef to df93a32 Compare February 23, 2023 02:36

@joe-elliott joe-elliott left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thanks for this improvement. It looks like there's a conflict in the changelog. If you can clean that up we'll merge.

@scalalang2
scalalang2 force-pushed the scalalang2/log-when-traces-are-too-large branch from df93a32 to 3dfe897 Compare February 24, 2023 01:24
@scalalang2

Copy link
Copy Markdown
Contributor Author

I cleaned changelog, thanks joe :)

@scalalang2
scalalang2 force-pushed the scalalang2/log-when-traces-are-too-large branch from 3dfe897 to 1e0a4fe Compare February 24, 2023 01:27
@mapno
mapno merged commit 24afc71 into grafana:main Feb 24, 2023
mattdurham pushed a commit to mattdurham/tempo that referenced this pull request Jun 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants