feat: add list_objects_version option to S3 backend

[kbelokon]

What this PR does:

Adds an optional list_objects_version setting to the S3 backend, letting operators select the V1 (ListObjects, marker-based) list API instead of ListObjectsV2. The default is unchanged — V2 — so existing deployments are unaffected; only backends that explicitly set list_objects_version: v1 change behavior.

Some S3-compatible stores return a truncated ListObjectsV2 response without a NextContinuationToken. The backend cannot paginate that response, so the blocklist poller fails in ListBlocks/Find and the block list is never built. Known affected backends include Ceph RGW (pre-Nautilus, e.g. mimic 13.2.2 — Ceph tracker #37801), Dell ECS (#5703, #558), and RustFS / some MinIO and ECS configurations (#6436). The V1 marker-based list API paginates correctly on these stores.

This is the same list_objects_version option already exposed across the Grafana stack — Mimir (#5099), Cortex (#6280), and Thanos/objstore (#3312) — with the same name and values, so the knob is consistent for operators running Tempo alongside them. Tempo calls minio-go's low-level Core API directly (it does not vendor objstore), so the switch is implemented at the two ListObjectsV2 call sites rather than passed through.

V1 is opt-in only and the default stays V2; V2 remains the recommended, more efficient path, and the option description notes the tradeoff. This is distinct from #6114, which set maxKeys=1 on the startup confirmation list — that change does not touch the blocklist poller's ListBlocks/Find pagination, which is what this option addresses.

Which issue(s) this PR fixes:
Related to #5703, #6436

Checklist

• Tests updated
• Documentation added
• Changelog entry added under .chloggen/ (run make chlog-new, or make chlog-new FILENAME=<name> to override the default branch-name file; see .chloggen/README.md)

[cla-assistant]

All committers have signed the CLA.

[penfree]

I encountered the same problem，hope this issue be merged soon

[knylander-grafana]

Thank you for adding docs!

[kbelokon]

Thank you for taking a look!

[knylander-grafana]

The manifest.md is a generated doc. Any changes that you make to it will be overwritten next time the file is generated.

[kbelokon]

Thanks for flagging! These lines weren't hand-edited — they come from running make generate-manifest (go run pkg/docsgen/generate_manifest.go) on the branch, so they match what the generator emits for the new option. I re-ran it after rebasing on latest main and the tree stays clean, so the next regeneration keeps them.

[evan361425]

An additional question regarding this: if limit = 0 is changed to a non-zero value (such as 100), what are the behavioral differences between v1 and v2?

I'm planning to open a PR to apply this limit, see #7598

[kbelokon]

Good question. With a non-zero max-keys the two versions behave the same from the caller's perspective: S3 caps every page at min(max-keys, 1000), so a smaller limit just means more round-trips. What differs is only the cursor mechanics, which this PR already normalizes: V2 advances via NextContinuationToken, V1 via marker — with a fallback to the last key of the page, because AWS only returns NextMarker when a delimiter is set. Since both Core.ListObjects and Core.ListObjectsV2 take maxKeys as their last argument and both call sites go through the unified listObjects helper here, wiring a max_keys option for #7598 after this merges should be a small, mechanical change.

[kbelokon]

@electron0zero would you have a moment to review? This is the pagination-side counterpart of the S3-compatible-store issues you triaged in #6436: some stores truncate ListObjectsV2 responses without a continuation token, which breaks the blocklist poller. The default stays v2 (a wire-level test asserts list-type=2 is still sent), v1 is opt-in and mirrors the option of the same name in Mimir, Cortex, and Thanos. In #5703 joe noted a config option is fine as long as default behavior stays unchanged — that's the case here. #7598 is also waiting on this to add max_keys on top. Rebased on latest main — the refreshed CI run is waiting on workflow approval (the previous run was fully green).