sigil-http / treeRELEASE_NOTES_v0.18.3.md

Bound the TLS handshake and the request write, so an outbound call cannot block forever

An outbound HTTP call passes through six blocking segments. Four of them were unbounded, including the one that matters most: the TLS handshake. Once a peer has accepted the TCP connection the connect phase is over, so connect-timeout: never reached the handshake read that follows. A server that accepts and then never sends a ServerHello left that read blocking with no way out, and because Sigil's scheduler is cooperative that does not fail one request, it freezes the whole process.

That is not a hypothetical. It ran a production monitoring service for 55 days while its process, its service status and its listening port all reported it healthy.

Name resolution is still unbounded, and this release does not change that. See "Known limitations" below. Please do not read "sigil-http bounds its blocking segments" as "sigil-http cannot block".

Who this fixes without a code change

connect-timeout: now bounds the TLS handshake as well as the TCP connect, so anything already passing connect-timeout: is fixed by upgrading alone. In this estate that is courier, on both its Telegram poll and send paths, through sigil-telegram's connect-timeout field.

Everything else has to start passing it. A survey of the estate's repositories found no other caller supplying connect-timeout: today, so vigil, kiln, bureau, tally and fjo each need to add it to get the protection.

Changes

• connect-timeout: bounds the TLS handshake in addition to the TCP connect. Each phase gets the full value rather than a shared split, so a slow but working connect cannot consume the handshake's budget. The worst case before a request is sent is therefore about twice connect-timeout:.
• connect-timeout: now works for plain HTTP as well. It resolves the host and connects to each address under a deadline, giving each address a fair slice of what remains, so a blackholed first address cannot burn the whole budget and leave a working address untried.
• timeout: bounds the request write. A request usually fits the socket buffer, so the write usually returns at once, but a peer that never drains its receive queue fills the window and a large body then blocks indefinitely. A write that runs out of time raises HTTP request timed out: write deadline exceeded. That error is deliberately NOT marked ack-unconfirmed: a write deadline can only fire with bytes still unsent, so the server holds a partial request it cannot act on and retrying is safe.
• The response read keeps its own fresh deadline rather than sharing the write's, so the response window that timeout: has always meant is unchanged. The consequence is that timeout: is now a per-phase bound rather than a total: (http-get url timeout: 5) can spend up to 5 seconds writing and a further 5 reading. Sharing one deadline would have silently shortened the response window that existing callers sized their value against, which seemed the worse of the two changes, but callers who read timeout: as a total should halve it.

Fixed

• http-fetch-bytes was not bounded by its own timeout argument. Its idle deadline loops detect "no data yet" from an empty read, which only a non-blocking connection ever returns, and it never set one. So its reads blocked and every deadline check between them was unreachable: the argument looked like a safeguard and did nothing. The connection is now non-blocking, which is what those loops were written for, and they sleep between retries instead of spinning.

Known limitations

• http-download is not bounded at all. It takes no timeout keywords and is blocking end to end: connect, write, header read and body stream. It is the API most likely to be pointed at a large, slow remote, and it is the one still able to freeze the process. Bounding it means bounding a streaming read as well, which is a larger change than this release. Stated here rather than left to be discovered.
• DNS resolution is unbounded. getaddrinfo(3) is a blocking call with no deadline argument and no portable cancellation, and both paths into it are inside native code: the bounded plain-HTTP connect must resolve before it can connect per-address, and tls-connect resolves inside the same native call that performs its bounded connect. A resolver that stops answering still blocks a request for as long as the system resolver takes to give up, and neither timeout: nor connect-timeout: affects that.

Callers needing a hard bound across resolution too have to get it outside this library, by running the request somewhere it can be abandoned, or by resolving ahead of time and requesting by address. Fixing it properly means adopting the async resolver that sigil-socket 0.17 added, which raises the required floor for every consumer and is a separate change.

Upgrade notes

Requires sigil-tls 0.16.5, which carries the handshake bound itself. The pin is updated accordingly.

Every bound remains opt-in. With timeout: and connect-timeout: omitted the behaviour is exactly what it was: blocking connect, blocking write, blocking read. Nothing changes for a caller that passes neither.