Skip to content

fix(ext/node): keep writev buffer boundaries in TLS record framing - #36719

Open
tomas-zijdemans wants to merge 4 commits into
denoland:mainfrom
tomas-zijdemans:fix/node-tls-writev-record-boundaries
Open

fix(ext/node): keep writev buffer boundaries in TLS record framing#36719
tomas-zijdemans wants to merge 4 commits into
denoland:mainfrom
tomas-zijdemans:fix/node-tls-writev-record-boundaries

Conversation

@tomas-zijdemans

@tomas-zijdemans tomas-zijdemans commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Problem

npm:tedious (and therefore npm:mssql and @prisma/adapter-mssql) connections to SQL Server / Azure SQL fail with socket hang up / Connection lost - socket hang up whenever a query's TDS message spans 3 or more TDS packets (roughly > 8 KB with the default 4096-byte packet size). Small queries work; the same code works on Node. This is distinct from the implicit session-resumption issue fixed by #36592 and from the JSStreamSocket deadlock fixed for #33907 — with both of those fixes in place (Deno 2.9.6), large writes still kill the connection.

Mechanism, verified against a real SQL Server 2022 with a record-level wire capture:

  • tedious writes one TDS packet (4096 bytes) per stream write. Packet 1 flushes alone; while it is in flight, packets 2..n queue in the Writable and are submitted together through _writev.
  • Deno's TLSWrap.writev flattened all chunks into one slice before feeding rustls, and rustls fragments records per writer().write() call — so the resulting application-data record spanned two TDS packets. For a 3-packet message the capture shows Deno sending records of 4096 + 4150 plaintext bytes where Node sends 4096 + 4096 + 54.
  • Node encrypts each writev buffer with its own SSL_write (TLSWrap::DoWrite loops over the buffers), so records always align with TDS packet boundaries. SQL Server appears to require this alignment: in our captures it closes the connection (TCP FIN, no TLS alert) as soon as a record crosses a packet boundary. Capping the record size alone (rustls max_fragment_size = 4096) did not help — alignment is what matters, not size. tedious's setMaxSendFragment(packetSize) call points at the same server behavior.

The total plaintext Deno sent was byte-identical to Node's — only the record framing differed.

Fix

Track the end offset of each write chunk in pending_cleartext_boundaries, have writev record those offsets alongside the concatenated buffer instead of discarding them, and make clear_in never pass bytes from two chunks to a single rustls write() call. Since rustls fragments per write() call, each chunk now gets its own TLS record(s), matching Node's per-buffer SSL_write. Single-buffer writes and the MAX_CLEAR_IN rate-limiting behave exactly as before; a chunk that straddles a MAX_CLEAR_IN cut is still only split within itself, never merged with the next one.

Tradeoff: a writev batch of many small chunks now produces a record per chunk (~22 bytes overhead each) rather than one coalesced record. That is Node's behavior, and it is the property protocols like TDS depend on.

Testing

  • New regression test in tests/unit_node/tls_test.ts, run for both TLS 1.2 and 1.3, builds the same topology tedious uses (TLS client over a back-to-back Duplex pair), corks three writes (4096/4096/54) so they flow through one _writev batch, and asserts one application-data record per chunk by parsing the record headers off the encrypted side. Both versions fail without the fix (the three chunks arrive as one merged record) and pass with it.
  • New Rust unit test for the boundary walk, including resuming mid-chunk after a MAX_CLEAR_IN cut.
  • End-to-end repro against SQL Server 2022 in Docker (tedious 19.2.1, encrypt: true): before the change every INSERT whose TDS message needs > 2 packets fails with socket hang up (threshold bisected exactly to the 2-to-3 packet boundary); with the change, inserts up to 1 MB pass and the client's TLS record sizes match Node's byte-for-byte.
  • cargo test unit_node::tls_test and the existing tls_wrap Rust unit tests pass.

Likely also relevant to older reports of mssql/Prisma failures that persisted after #33914, e.g. the "more than N rows" symptom in #32271.

I used Claude Code to help investigate and write this change.

TLSWrap.writev flattened all buffers into one slice before feeding
rustls, which fragments records per write() call. Node encrypts each
writev buffer with its own SSL_write, so a record never spans two
buffers. Protocols that tunnel framed data over TLS depend on that:
SQL Server (TDS) closes the connection when a TLS record crosses a
TDS packet boundary, surfacing in tedious/mssql/@prisma/adapter-mssql
as "socket hang up" for any query spanning 3+ TDS packets.

Track chunk end offsets in pending_cleartext_boundaries and never
pass bytes from two chunks to a single rustls write() call.
Pass the writev buffers as one buffer plus chunk end offsets instead of
a Vec<Vec<u8>>, so each buffer is copied once as before rather than
twice. Walk the boundary list with a cursor (chunk_end) instead of
rescanning it per chunk.

Cover TLS 1.3 in the regression test as well: the framing is version
independent, and under 1.3 the client's Finished is also an outer
application_data record, so record capture now starts after a one-byte
round trip instead of at secureConnect. Add a Rust unit test for the
boundary walk, including resuming mid-chunk after a MAX_CLEAR_IN cut.
Comment thread ext/node/ops/tls_wrap.rs Outdated
op_state: &mut OpState,
) -> i32 {
let mut data = Vec::new();
let mut boundaries = Vec::new();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you preallocate this vec?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Fixed now

@bartlomieju bartlomieju left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants