[zeenix]

The bus forwards the messages of each peer one at a time, and writes each of them to a client's socket with a system call of its own, however many messages of the peer are there already. This forwards them in batches instead, with zbus's new Connection::send_all (z-galaxy/zbus#2079):

• ⚡️ The task that serves a peer takes the messages that the peer sent after the one it waits for, as far as they are there already, up to 16 of them, ending with the one that brings them to 16 KiB or more, and forwards them as a batch. It picks the recipients of all of them under one hold of the locks, and sends each recipient its messages of the batch in one go, which zbus writes with one system call. Each recipient still gets the messages of each peer in order. A recipient may now get a message of a batch before another recipient gets an earlier one, which nothing ruled out before.
• The time limit of a send, SEND_TIMEOUT, now covers the messages that a client gets in one go, as well as the wait for those on their way to it, so a batch ends at 16 KiB: a client has to take less than 16 KiB more within it than the last of those messages alone.
• A client that didn't negotiate passing file descriptors still gets the messages of a batch that carry none: zbus turns the batch down before it writes any of it, and the bus sends it again without the others. A new test covers it.
• 📌 busd builds on zbus from the branch of ✨ zb: Add Connection::send_all zbus#2079, by commit, until a release of zbus has send_all. The earlier 📌 commits pinned the commits of that branch that the measurements below were made with.
• ✅ stalled_subscribers, which fails on CI on the main branch too, times the disconnections of the stalled clients while the flood goes on, rather than once all of it is sent. On CI, where every test logs all that it does, the rest of the flood took over 300 ms, and the test 853 ms on the main branch, beyond the 800 ms that it allows.

Measurements

On a 4-vCPU Xeon VM, against busd's main branch, its queue for each client before it, dbus-broker and dbus-daemon, with dbus-bench's clients of the signals and of the latencies' fan-out running each subscriber on a thread of its own, as processes of their own would (results):

• Against busd's main branch, the broadcast of 6000 signals to 20 subscribers takes 70.5% less time (0.283 s against 0.959 s) and a burst of signals to one subscriber 13.6% less. A signal of the latencies' fan-out reaches each of 20 subscribers 72% sooner at the median, and all of them are there 66% sooner. The bus uses 60% less CPU time on the signals, 79% less system time, and 16% less CPU time on the latencies.
• The calls, the round trips and the connection that 100 proxies share are level: within 5%, or inconclusive. The bus forwards those a message at a time, and callgrind counts 1.4% more instructions of it on the calls than with busd's main branch, with 4f346a3, whose forwarding is the same (details).
• Against dbus-broker, busd is now faster on the broadcast too, by 11.6%, and on the fan-out, and stays 38 to 42% faster with 20 connections or more. busd still takes 24% longer than dbus-broker on the burst to one subscriber, and uses 14 to 25% more CPU time on the calls, the latencies and the shared connection, and as much on the signals.
• No run of any broker failed.

On a Ryzen 7 7840U, with dbus-bench's earlier clients, which ran all subscribers on one thread (results), the burst and the broadcast were 6.9% and 6.7% faster than with busd's main branch, with 31.7% less CPU time of the bus on the signals, and 48.3% less system time. A run with the threaded clients there is to come.

Costs

• A client has to take a batch to it within the time limit, rather than a message: less than 16 KiB more than the last message of the batch alone, which can be bigger. In one run on the VM, with batches of up to 64 KiB, the bus disconnected the connection that 100 proxies share, as it started, once in 40 rounds. 200 starts of that client against each of busd with batches of up to 16 KiB, busd with batches of up to 64 KiB and busd's main branch couldn't reproduce it, so the bound of 16 KiB rests on the time limit's reasoning above.

Generated by Claude Opus 5.5.