Receive: fix capnproto replication in endless loop

[lachruzam]

• I added CHANGELOG entry for this change.
• Change is not relevant to the end user.

Changes

Recreate Cap'n Proto client in case of failed bootstrap.

Verification

There are no new tests.
The issue was reproduced by artificially lowering the context timeout. The forwarding didn't stuck for the fixed version.

[GiedriusS]

Reading code a little bit and will get back to it soon. Sorry for the delay. Could you please add the DCO?

The issue was reproduced by artificially lowering the context timeout. The forwarding didn't stuck for the fixed version.

Where did you do it? In some (internal) fork of yours or some ad-hoc code? I'm interested to know this because I tried many different ways of reproducing this bug.

[lachruzam]

Reading code a little bit and will get back to it soon. Sorry for the delay. Could you please add the DCO?

The issue was reproduced by artificially lowering the context timeout. The forwarding didn't stuck for the fixed version.

Where did you do it? In some (internal) fork of yours or some ad-hoc code? I'm interested to know this because I tried many different ways of reproducing this bug.

I checked on an internal fork, lowering the timeout for bootstrap to 1ms.
In my understanding the specific problem that caused bootstrap to fail isn't important as in each case the client would be resolved to ErrorClient.

DCO was added and a slightly cleaner fix applied, however approach taken by @pedro-stanaka may be better.

[GiedriusS]

Is there some pull request I can't find? https://github.com/thanos-io/thanos/pulls/pedro-stanaka there's nothing here

[lachruzam]

Is there some pull request I can't find? https://github.com/thanos-io/thanos/pulls/pedro-stanaka there's nothing here

I have been referring to: receive: Recreate Cap’n Proto client when bootstrap fails (port of

The fix is rewritten to synchronously resolve the client.

[elohmeier]

Please see my related PR/comment here: #8534 (comment)