Test harness: integration tier, refactor-protection unit tests, and CI

[jcheng5]

Phases 2–5 of the test-harness plan. Stacked on #598 — review that first;
this PR's base will retarget to master once it lands.

Goal isn't "test Express 5". It's to build the characterization net that makes
three pending changes safe — the Express 4→5 upgrade (#596), the Q removal, and
the TypeScript port — and to use Express 4→5 as its first customer.

The tiers

mocha isn't recursive, so every directory is named explicitly (in package.json
and Jenkinsfile — a new directory added in only one silently doesn't run).

Command	What	Needs
npm test	unit + fast integration	nothing
npm run test:r	real R processes	R + shiny

test/integration/ boots a real, complete server on an ephemeral port for
each test — ~15ms — and makes real HTTP requests. Only su and R are stubbed;
the real TcpTransport, endpoint, shared secret, connectEndpoint_p handshake
and proxy all stay live, which is the part an Express upgrade actually touches.
Covers __assets__ rewriting (including the ?foo case the explicit /?
rewrite exists for, and traversal attempts), send's directory handling and the
text/R MIME mutation, req._parsedUrl survival to the proxy, worker refcount
cleanup with compression on and off, X-Powered-By, and the access log's
reliance on originalUrl.

test/integration-r/ launches actual R through the real AppWorker. Runs as
the current user, since AppWorker only shells out to su when runAs differs
from the process user.

Unit tests for what the Q and TypeScript ports will touch, all previously at
zero coverage: qutil (forEachPromise_p is the router chain's control flow;
map_p's sequencing is load-bearing), ShinyProxy.httpListener's dispatch
contract, Scheduler.shutdown()/dump() (the only synchronous promise
inspection in the codebase — no native equivalent, so it needs a design change
during the Q removal), connectEndpoint_p, and the hand-written config
language, which the memory bank rates as the highest-value gap.

Findings, characterized rather than fixed

• qutil.serialized hands a queued caller the previous invocation's
  outcome. Q's .fin() settles with the original promise's value even when
  its callback returns a promise. Benign today only because the sole production
  user is loadConfig_p, whose queued caller is the SIGHUP handler, which
  .eat()s the result — but a live trap for the Q removal.
• NODE_ENV is never set, so Express runs in development mode and an
  unhandled throw returns a stack trace, with no 4-arg error handler to stop it.
  Asserted, so that adding one becomes a deliberate change. Possibly worth
  treating as security-adjacent independently of this work.
• connectEndpoint_p's timeoutId is assigned null and never set, so the
  clearTimeout on the success path is a no-op; the !isPending() guard is
  what actually stops the retry loop.
• manual.test/test-config-config.js fails on a stale expectation, not a
  product bug — run_as; with no users is legal, because the schema declares
  param String users....

Two flakiness traps worth reading

Both had symptoms pointing nowhere near the cause; together they took the suite
from ~12% flaky to 0 failures in 25 consecutive full runs.

• Ephemeral-port shadowing. TcpTransport allocates a worker port by
  binding 127.0.0.1:0, reading it, and closing again. A server listening on
  the wildcard :: can be handed that same port; the worker then binds
  127.0.0.1 on it, which succeeds — a specific-address bind is allowed
  alongside a wildcard one — and shadows the server for loopback traffic.
  Requests silently reach the worker instead of Shiny Server, surfacing as
  inexplicable 404s and Parse Error: Expected HTTP/. The harness pins its
  listener to 127.0.0.1.
• Socket pooling across recycled ports. fetch() pools keep-alive sockets
  per origin; test servers restart milliseconds apart and ports get recycled, so
  a socket belonging to a dead server gets handed to the next test — which then
  talks to the previous test's config. Connection: close is not a fix: fetch
  treats it as a forbidden header and drops it silently. Requests go through
  http.request with agent: false.

CI

.github/workflows/ci.yml — npm ci, tests and license check on Linux and
macOS, plus a build-freshness check (tsc then git diff --exit-code lib/).
Generated .js is committed next to the .ts and tests run against the
generated files, so editing a .ts without npm run build passes CI today
while changing nothing at runtime.

Node is pinned from .nvmrc: master's nan (^2.18.0) does not compile against
Node 24, so anything else fails in node-gyp rather than anywhere informative.
That's also a prerequisite for #596's Node bump. The license check runs
check-licenses.js and check-upstream.sh directly rather than
tools/preflight.sh, which invokes ./bin/node — a runtime that only exists
after the full CMake build.

docker/jenkins/Dockerfile.ubuntu-20.04 gains the shiny package; it had
r-base only, so the real-R tier would have skipped itself there. CI treats a
skip as a provisioning failure so it can't pass vacuously.

Verification

• npm test: 291 passing, 0 failing, 4.3s (macOS, Node 20.17.0).
• npm run test:r: 8 passing.
• 0 failures across 15 consecutive full runs; 25 across the integration tier.
• tsc produces no diff.
• The real-R assertions were negative-checked: deliberately breaking the config
  so R can't launch fails 6 of 8, and the two that still pass are the ones
  correctly independent of R.

Suggested next step

With this in place, #596 can be split into separately-attributable steps —
Node, then http-proxy-3, then express+send together, then TypeScript — running
this suite after each, so a status change at the express+send step is a genuine
breaking change rather than something to bisect by hand.

🤖 Generated with Claude Code