TLS 1.3 KeyUpdate: should kTLS handle it instead of the tlshd upcall?

[martin-belanger]

@chucklever, @hreinecke — attached is a design document (revision 9) proposing that kTLS handles TLS 1.3 KeyUpdate itself, rather than through an upcall to tlshd. This note covers how I got there and what I am asking.
The document is attached as both Markdown and HTML. The text is identical; the HTML is easier to read in a browser.

tls-keyupdate-design-v9.html

tls-keyupdate-design-v9.md

How I got here

I started from a failure in NVMe/TCP. A colleague at Dell reproduced a silent, indefinite stall of an I/O queue when the peer sent a KeyUpdate, on an unmodified 6.17 kernel. Nothing was logged, and the controller kept reporting state = live.

That led to net/tls, where kTLS detects the record, sets key_update_pending and returns -EKEYEXPIRED with the comment "let userspace deal with it". For NVMe/TCP there is no userspace to deal with it.

Looking for prior work I found ktls-utils issue #29 and, through Hannes's comment there, Alistair Francis's series "nvme-tcp: Support receiving KeyUpdate requests" (v7, 2026-03-04). It solves the RX half through an upcall to tlshd, and is not merged. Reading it raised the question this document asks: whether the problem is better solved inside the kernel.

Two things turned out to be broader than the NVMe case. The TX half affects every kTLS user: the kernel owns the record count and exposes it only as a snapshot to poll, so no library enforces the RFC 8446 §5.5 limit under offload. And the RX half affects both in-tree consumers — sunrpc discards a received KeyUpdate as an unknown record type, in both the client and the server path. I have read that code but not tested it.

That second point is why I am raising this here rather than as an NVMe issue.

What I am asking

1. Is this a reasonable reading of the problem, or is there an architectural or kTLS-specific reason why KeyUpdate should not be handled in the kernel? I could not find that discussion in the v1 to v7 threads or in issue TLS session key re-negotiation #29. If there is a reason, that answer is more useful than the proposal.
2. Chuck — you wrote the record-type handling in both xprtsock.c and svcsock.c, so this one is really for you: is the sunrpc behavior in section 2.3 what you would expect, and has it been seen in practice?
3. Most of the work is in net/tls, where I have no established contact. Who should this go to on netdev, and would either of you be willing to introduce it? Sabrina Dubroca wrote the existing rekey support, so that seems like the right starting point.

Nothing here is meant to dismiss the existing series. It carries a Reviewed-by from both of you and it addresses a real bug. My question is about the long-term home for the functionality, not about the work already done.

How the document was produced

I used Claude to help analyze the code and draft the document. It worked across five trees — the kernel, ktls-utils, GnuTLS, OpenSSL and the v7 series — which is more cross-referencing than I would have done by hand, and it is how the sunrpc case surfaced at all.

I mention it because the document makes specific claims about code in projects I do not maintain. Every one is cited to a file and line number, the exact tree revisions are in Appendix D, and anything that was read but not tested says so. If a citation is wrong I would like to know; the argument should stand or fall on the code, not on how the text was assembled.

[chucklever]

is there an architectural or kTLS-specific reason why KeyUpdate should not be handled in the kernel?

We’re working on KeyUpdate support for kernel TLS consumers, but there are plenty of constraints and working with netdev (who maintains both net/tls/ and net/handshake/) takes time.

My experience says that netdev will reject such changes to net/tls/ out of hand because KeyUpdate does not apply at all to user space consumers of kTLS: user space handles KeyUpdate in whatever library is linked to the requesting application. By design, net/tls/ contains only the TLS Record protocol implementation.

The kernel consumers must therefore rely on an upcall to tlshd , which is the kernel equivalent of a TLS library implementation. It will eventually handle KeyUpdate when we get the pre-requisites in place.

The current implementation of NVMe is supposed to terminate the TLS session and drop TCP connection when it receives a KeyUpdate alert. If it is instead looping, that’s a bug and needs to be reported to the NVMe folks.

[martin-belanger]

Chuck — thanks for the quick reply, and for setting out the constraints.

The way I see the boundary is policy versus mechanical operation. The handshake has real policy decisions: which cipher suites are acceptable, which peer identities are trusted, which certificate or PSK to present, whether the chain verifies. Those need configuration and a trust store, and they belong in userspace.

KeyUpdate has none of that. There is no negotiation and no decision to make — a conforming implementation has exactly one way to advance the key schedule when it receives a valid KeyUpdate. It is a mechanical operation on keys the kernel already holds.

That is why I think it falls on a different side of the line than the handshake, even though it is formally a handshake-protocol message.

I may raise it with the net/tls maintainers. I understand the chances of it landing there are close to zero, but I would rather hear that from them than assume it.

Thanks again,
Martin

[hreinecke]

Please note that KeyUpdate with nvme-tcp currently does not work as the spec is actually wrong at this point; the only choice which we have it to reconnect to get the keys back in shape. So if we get a stall when receiving a KeyUpdate message that's a bug in the kernel code, and should be fixed there.

[martin-belanger]

@hreinecke: Hannes — could you say more about which part of the spec you consider wrong?

I checked all the ratified NVMe specifications and technical proposals: KeyUpdate is not mentioned anywhere. The only normative statement I can find is in the NVMe/TCP Transport Specification r1.3 §3.6.1 — "NVMe/TCP implementations that support TLS shall support TLS 1.3 (refer to RFC 8446)" — which appears to pull in RFC 8446 §4.6.3 as-is.

Is the problem that the spec incorporates TLS 1.3 wholesale without addressing KeyUpdate for a long-lived PSK transport, or is there specific text you think conflicts?

[hreinecke]

The problem lies with generated keys. The current spec has the 'REPLACETLSPSK' setting which is meant to refresh the keys used by the current TLS connection. The implication here was that this could be hooked up with the KeyUpdate mechanism to use the keys generated by a new authentication protocol run as the new keys for the existing connection (which you could to for TLS 1.2). But for TLS 1.3 this is no longer possible; either you use the KeyUpdate mechanism (and be completely invisible to the kernel as the kernel only knows about the TLS PSK, not the derived keys used for TLS 1.3 encryption) or you regenerate the keys via NVMe authentication and have to reset the connection to use that. So if you use KeyUpdate you essentially disabled secure concatenation, which is precisely there to update the key material if the previous one is deemed insecure. IE the very same case KeyUpdate is addressing.

[martin-belanger]

I think we may be talking about two different mechanisms.

TLS 1.3 §7.2 derives the next traffic secret as:

application_traffic_secret_N+1 =
    HKDF-Expand-Label(application_traffic_secret_N, "traffic upd", "", Hash.length)

The only input is the current traffic secret. KeyUpdate cannot introduce new key material, so it could never install a PSK generated by a fresh authentication run — in TLS 1.3 or any other version. What could do that was TLS 1.2 renegotiation, and TLS 1.3 forbids renegotiation outright (§4.1.2). So I agree with your conclusion that a replaced PSK can only take effect on a new connection — but I read that as a property of TLS 1.3, not a defect in the NVMe spec. The transport spec seems to accept it already: "TLS secure channels already set up with a generated PSK shall not be affected by a PSK replacement."

One clarification on the kernel side, because I think it locates the problem precisely. The kernel does know the derived keys — setsockopt(SOL_TLS, TLS_TX/TLS_RX) installs key, IV, salt and record sequence number, and that is what kTLS encrypts with. It also holds the PSK, in the keyring. What it does not have is the value in between: the application traffic secret. The derivation cannot be run backwards from the key, so the kernel cannot compute secret_N+1 even though it holds everything on either side of it. That single missing value is what blocks KeyUpdate in the kernel, and it is why handing the traffic secret down at handshake time would be enough to unblock it. tlshd already does exactly that for QUIC, using gnutls_handshake_set_secret_function().

If that is right, then KeyUpdate and REPLACETLSPSK are not alternatives and one does not disable the other. KeyUpdate does not touch the PSK, so secure concatenation is unaffected by it. What KeyUpdate does address is the RFC 8446 §5.5 limit on how much data one AES-GCM key may protect, and §4.6.3's requirement to send a KeyUpdate in response to a peer's update_requested before the next application data record. Neither of those can be satisfied by replacing the PSK.

Does that match your understanding?

[chucklever]

The way I see the boundary is policy versus mechanical operation. The handshake has real policy decisions: which cipher suites are acceptable, which peer identities are trusted, which certificate or PSK to present, whether the chain verifies. Those need configuration and a trust store, and they belong in userspace.

KeyUpdate has none of that. There is no negotiation and no decision to make — a conforming implementation has exactly one way to advance the key schedule when it receives a valid KeyUpdate. It is a mechanical operation on keys the kernel already holds.

That is why I think it falls on a different side of the line than the handshake, even though it is formally a handshake-protocol message.

I may raise it with the net/tls maintainers. I understand the chances of it landing there are close to zero, but I would rather hear that from them than assume it.

No-one is assuming anything. The netdev maintainers have repeatedly asserted the design that kTLS handles only TLS Records, and will not accept any changes that do not apply to kTLS's user space consumers. KeyUpdate for in-kernel consumers hits both of those prohibitions squarely.

You might have missed this subtlety however: although net/handshake/ is maintained by netdev, it falls outside of kTLS. If there is a place where a fully in-kernel KeyUpdate implementation might reside, it is there. What is currently missing is a reliable way to handle TLS Alerts on kernel sockets. That is coming together right now and will be the basis for KeyUpdate and other mechanisms.

[martin-belanger]

Thanks — the net/handshake pointer is useful, and I will follow the Alert work.

One correction on the premise, because I think it changes the calculus: the proposal is not specific to in-kernel consumers. It applies to every kTLS user, in both directions, and a userspace application benefits from either half independently — the traffic secret is registered per direction, so an application can opt into receive, transmit, or both.

On receive, userspace can hand KeyUpdate off to the kernel entirely. Today an application on an offloaded socket has to recognize -EKEYEXPIRED in the middle of a read loop, feed the record back into its TLS library, and install the new key itself. If it registers its receive traffic secret, none of that is needed.

On transmit, the RFC 8446 §5.5 limit is not enforced by anyone today:

• GnuTLS forces a rekey after 2^24 records by default, but gnutls_record_send() short-circuits to _gnutls_ktls_send() before reaching the check in _gnutls_send_tlen_int(). Enabling kTLS silently disables a protection that was on by default.
• OpenSSL never had a record-count trigger at all.
• The count lives in the kernel. An application can only sample it by polling getsockopt(SOL_TLS, TLS_TX), and the kernel closes records early on its own when the scatter-gather limit is hit without reporting it, so an application's own count drifts below the real one.

So this is a gap for ordinary userspace applications on kTLS, independent of NVMe or NFS, and the fix is available to them on the same terms as to kernel consumers.

I have a working prototype. Its interop test is exactly that case: an ordinary userspace program using kTLS on one end, OpenSSL doing userspace TLS on the other. KeyUpdate works in both directions, including the mandatory response to update_requested.

I am not trying to relitigate placement. If net/handshake is the more likely home I will look there. But "does not apply to userspace consumers" is not accurate, and that seemed worth correcting before a routing decision rests on it.

[hreinecke]

Well, there had been a previous implementation by @alistair23 , but that got shelved as it was based on an earlier version of TLS alert handling. If we were to do that properly TLS KeyUpdate needs to be handled in net/tls as we need to pace TX and RX key update with the respective KeyUpdate messages in each direction. Otherwise I don't see the point as then we'd have to update both keys at the same time, at which point we might as well reset the connection as the impact for upper layers will be about the same.