Support HTTP request trailers - #6278
Conversation
vietj
left a comment
There was a problem hiding this comment.
we need a test for HTTP/2 and HTTP/3 as well
HttpServerRequest had no way to access trailers, while HttpClientResponse exposes trailers() and getTrailer(). Trailers were parsed by Netty and then dropped: Http1ServerConnection carried a "//TODO chunk trailers" comment, and the HTTP/2 codec connection noted "not implemented yet (in api)". The public interface gains getTrailer(String) and trailers(), mirroring HttpClientResponse, and returning an empty map before the request has ended rather than null. For HTTP/1 the trailers travel through the request's inbound message queue, as the client already does, so they cannot be observed before the preceding data has been delivered. For HTTP/2 the codec connection now forwards the trailing HEADERS frame to the stream instead of signalling empty trailers; the multiplex, HTTP/3 and QUIC paths already forwarded them, and HttpServerRequestImpl already received the MultiMap and discarded it. Fixes eclipse-vertx#5253 Signed-off-by: jnbdz <jn@yaloub.com>
Completes eclipse-vertx#5253. The server side of the same commit lets a Vert.x server read request trailers; this lets a Vert.x client send them, so the feature no longer has to be tested by hand-writing chunked bytes onto a socket. HttpClientRequest gains trailers() and the four putTrailer overloads, declared exactly as on HttpServerResponse. Trailers ride on the terminating chunk, so a request carrying them is always sent chunked: Netty silently discards trailing headers under Content-Length framing, which would produce a request that looks fine and arrives without its trailers. HttpClientRequestImpl therefore drops Content-Length and forces Transfer-Encoding when trailers are set, writes the body with end=false, and emits the trailers as the terminating write — mirroring how HttpServerResponseImpl already writes response trailers. HttpClientStream gains writeHeaders(MultiMap, boolean), the counterpart of the method HttpServerStream already had. HTTP/2 needs no wire code because DefaultHttp2Stream already implements it; HTTP/3 reuses the server adapter; HTTP/1 is the only new wire path. HttpClientConnection gains newHttpTrailers() so the protocol-agnostic request can obtain a suitably backed map. Tests go in HttpTest, which Http1xTest, Http2Test and Http3Test all extend, so they run across all three protocols: trailers with a body, trailers without a body, and a request with none. Removing the chunked coercion fails testRequestTrailers rather than passing silently. The existing raw-socket and raw-Netty tests are kept deliberately: they prove the server accepts trailers from a non-Vert.x client, which a test driving our own client cannot show.
|
Thanks — those are in now; the review landed just before I pushed them. HTTP/2 — HTTP/3 — A second commit adds the client side ( The raw-socket and raw-Netty tests are kept deliberately: they show the server accepts trailers from a non-Vert.x client. Happy to split the client side into its own PR if you would rather review them separately. |
Address review comments: pull the shared writeHeaders(MultiMap, boolean) declaration up from HttpClientStream/HttpServerStream to HttpStream, drop redundant comments, and mirror the HTTP/2 request trailer tests in Http3ServerTest.
|
@vietj a scope question for a possible follow-up, not for this PR: With request trailers in, Vert.x carries the trailer fields in every direction, but neither side ever touches the Would you want Vert.x to emit |
Fixes #5253. Two commits: the server reading request trailers, and the client sending them.
HttpServerRequesthas no way to read trailers, whileHttpClientResponseexposestrailers()andgetTrailer(). The trailers do arrive — Netty parses them — but the server drops them, and the code already marked both spots:Http1ServerConnection.onContent://TODO chunk trailersHttp2ServerConnectionImpl.onHeadersRead:// Http server request trailer - not implemented yet (in api)HttpServerRequestImpl.handleTrailers(MultiMap)was already receiving the trailers and discarding them.Change
HttpServerRequestgainsgetTrailer(String)andtrailers(), declared exactly as onHttpClientResponse(@Nullableon the former,@CacheReturnon the latter). Before the request has ended, and for requests without trailers,trailers()returns an empty map rather than null — matching the client side.Http1ServerConnectionpassesLastHttpContent.trailingHeaders()down, andHttp1ServerRequestwrites them into the inbound message queue rather than setting a field directly, so they cannot be observed before the preceding data has been delivered. This mirrorsHttp1ClientConnection.onEnd(LastHttpContent).stream.onTrailers(headers)instead of signalling empty trailers, mirroringHttp2ClientConnectionImpl.HttpServerRequestWrappergains the two delegating overrides.Setting the trailers uses the same escape-safe merge as
HttpClientResponseImpl.handleTrailers: if the caller already obtained the lazily-allocated empty map, it is updated in place instead of replaced.Tests
Covering all three protocols:
Http1xTest.testRequestTrailers/testRequestNoTrailers, driven over a raw socket since the client cannot send trailers.Http2ServerTest.testRequestTrailers/testRequestNoTrailers, sending a trailing HEADERS frame via the Netty test client. BecauseHttp2MultiplexServerTest extends Http2ServerTest, these run against both the codec and multiplex connection implementations.Http3ServerTest.testTrailersalready sent request trailers but could only note// No API to get client trailers; it now asserts them.Http1xTest(504) andHttp2Test(359) pass. Each new assertion was checked by reverting the corresponding production change: dropping the HTTP/1 plumbing failsHttp1xTest.testRequestTrailers, and droppingstream.onTrailers(...)in the HTTP/2 codec failsHttp2ServerTest.testRequestTrailers.Note: five
testUpgradeToClearText*/testPriorKnowledgecases inHttp2ServerTestfail on my machine withBindException: Address already in useon port 8080, identically on unmodifiedmaster— a local port conflict, unrelated to this change.Second commit — client sending
HttpClientRequestgainstrailers()and the fourputTraileroverloads, declared exactly as onHttpServerResponse.Trailers ride on the terminating chunk, so a request carrying them is always sent chunked — Netty silently discards trailing headers under
Content-Lengthframing, which would yield a request that looks correct and arrives without its trailers.HttpClientRequestImpldropsContent-Length, forcesTransfer-Encoding, writes the body withend=falseand emits the trailers as the terminating write, mirroringHttpServerResponseImpl.HttpClientStreamgainswriteHeaders(MultiMap, boolean)— the counterpart of the methodHttpServerStreamalready had. HTTP/2 needs no new wire code (DefaultHttp2Streamalready implements it, covering codec and multiplex); HTTP/3 reuses the server adapter; HTTP/1 is the only new wire path.HttpClientConnectiongainsnewHttpTrailers()so the protocol-agnostic request can obtain a suitably backed map.Because the client can now send trailers, the new tests live in
HttpTestand therefore run underHttp1xTest,Http2TestandHttp3Test: trailers with a body, trailers without a body, and a request with none. Removing the chunked coercion failstestRequestTrailersrather than passing silently.The raw-socket and raw-Netty tests from the first commit are kept on purpose — they show the server accepts trailers from a non-Vert.x client, which a test driving our own client cannot demonstrate.
Full run across the affected classes: 1200 tests, 0 failures. (Two
BindExceptionerrors on my machine come from an unrelated process holding port 8080; they reproduce on unmodifiedmaster.)Concurrency and async
Nothing here blocks — no
await,executeBlocking, latch or synchronous result access is introduced on either side.HttpClientRequestImpl.doWrite, the trailers reference is captured inside the existingsynchronizedblock, alongside the other values used after it (chunked,writeEnd,writeHead), so it is safely published to whichever thread performs the write.end()completes only once the trailers are on the wire, and a failed body write short-circuits so trailers are never emitted onto a broken stream.writeTrailersgoes through the connection's outboundMessageWritequeue like an ordinary body write, so back-pressure behaviour is unchanged, and every path — stream reset, cancel, successful write — completes the promise. (A missed path there would hang a request rather than fail it.)Http2UpgradeClientConnection.UpgradingStream.writeHeadersre-dispatches to the event loop when invoked from another thread and still firesSEND_BUFFERED_MESSAGES_EVENTon end, mirroringwriteChunk.The
synchronized (conn)usage follows the existing idiom in these classes: short critical sections with no I/O performed under the lock.I also have this backported to
4.xif a maintenance backport is wanted.