Skip to content

fix: broken get and replace for s3 client - #21163

Merged
sandeepsukhani merged 1 commit into
mainfrom
fix-s3-client-get-and-replace
Mar 16, 2026
Merged

fix: broken get and replace for s3 client#21163
sandeepsukhani merged 1 commit into
mainfrom
fix-s3-client-get-and-replace

Conversation

@sandeepsukhani

@sandeepsukhani sandeepsukhani commented Mar 13, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:
When GetAndReplace callbacks return opaque io.ReadCloser wrappers (such as io.NopCloser), objstore.TryToGetSize cannot determine the content size, causing minio-go to use a multipart upload path that silently reconstructs PutObjectOptions before CompleteMultipartUpload, dropping customHeaders and, therefore, the If-Match conditional write header. S3 then accepts the upload unconditionally, allowing a stale-ETag write to overwrite a newer version of the object.

To fix it, I have added sizedGetAndReplaceBucket, which wraps the caller's callback inside GetAndReplace and buffers the returned reader when its size is unknown, replacing it with an objstore.NopCloserWithSize-wrapped bytes.Reader. This gives minio-go a known content length, forcing the single-part PUT path where If-Match is preserved end-to-end.

GetAndReplace is only used to update TOC files, which are not that large, so switching them to a single-part upload should not cause any issues. We can try to fix the issue upstream or change the reader interface to match what objstore.TryToGetSize handles.

@sandeepsukhani
sandeepsukhani requested a review from a team as a code owner March 13, 2026 16:00
@sandeepsukhani
sandeepsukhani merged commit e6abd15 into main Mar 16, 2026
88 checks passed
@sandeepsukhani
sandeepsukhani deleted the fix-s3-client-get-and-replace branch March 16, 2026 07:09
@sandeepsukhani sandeepsukhani added type/bug Somehing is not working as expected backport k298 labels Mar 16, 2026
loki-gh-app Bot pushed a commit that referenced this pull request Mar 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport k298 size/M type/bug Somehing is not working as expected

2 participants