spike(npm): prototype npx stash proxy distribution - #398
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
c928bdf to
fde3ad2
Compare
Proof-of-concept for shipping the proxy via npm as `npx stash proxy`, using the esbuild/Biome/SWC pattern (per-platform packages + os/cpu-filtered optionalDependencies + a thin JS launcher) -- NOT native N-API bindings, since proxy is a standalone server we only need to distribute and launch. Verified end-to-end locally on darwin-arm64: npx -> stash shim -> exec native cipherstash-proxy binary, with --version/--help passthrough, correct exit-code forwarding (0 / clap's 2), signal forwarding, and os/cpu platform resolution. Binaries are git-ignored build artifacts (build-binaries.sh / demo.sh regenerate them). Packages are private + 0.0.0-prototype to prevent publish. See npm/README.md for how this maps to a production CI matrix and the code-signing rationale (skips notarization/Developer-ID; keeps free ad-hoc signing on Apple Silicon). Signed-off-by: James Sadler <james@cipherstash.com>
Signed-off-by: James Sadler <james@cipherstash.com>
`stash proxy --psql ...` starts the proxy, waits for it to report its listen address (parsing the OS-assigned port when the default is in use), then launches psql connected to the proxy with the target db/user/password. psql is the foreground session; the proxy is torn down when it exits. Falls back with a clear message if psql is not on PATH. Connection details are taken from --database-url, then --db-* flags, then CS_DATABASE__* env. Validated against a local dev DB. Signed-off-by: James Sadler <james@cipherstash.com>
When `--psql` is used and psql isn't on PATH (or STASH_USE_BUILTIN_SQL=1 is set), open a small built-in SQL shell (lib/repl.js) instead of failing. It uses the pure-JS `pg` driver (no native binaries) and runs SQL through the proxy with tabular output and a few meta-commands (\l, \dt, \d, \?, \q). Not a psql replacement -- a convenience fallback. Real psql is still preferred when installed. Validated end-to-end against a local dev DB via the proxy. Background: bundling real psql isn't viable off-the-shelf -- the @embedded-postgres/* packages ship initdb/pg_ctl/postgres but strip psql -- so a pure-JS shell is the pragmatic no-native-deps fallback. Signed-off-by: James Sadler <james@cipherstash.com>
Visually distinguishes a via-proxy session from a direct psql connection: the prompt becomes e.g. `stash:mydb=>` with "stash" in cyan (on a TTY). Applied to both real psql (via PROMPT1/PROMPT2 --set) and the built-in shell. Override with STASH_PSQL_PROMPT (set empty to use psql's default / ~/.psqlrc); colour honours NO_COLOR and is disabled off a TTY. Signed-off-by: James Sadler <james@cipherstash.com>
A literal ESC byte in PROMPT1 was stripped by psql's variable parser, so the prompt showed in the default colour. psql's own %033 octal escape produces the ESC reliably (verified: \001 ESC[36m \002 stash \001 ESC[0m \002 -- 'stash' wrapped in cyan). Signed-off-by: James Sadler <james@cipherstash.com>
Signed-off-by: James Sadler <james@cipherstash.com>
59bed52 to
6f82ea5
Compare
npx stash proxynpx stash proxy distribution
tobyhede
left a comment
There was a problem hiding this comment.
Review of the npm/ diff. Everything below was verified against this branch (6f82ea5b) by running it — process-group experiments against the real stash.js, the parsers exercised in node, and the proxy's own listen/TLS/signal paths read in packages/cipherstash-proxy/. Severities account for this being an unwired spike.
Worth fixing before this leaves spike status
1. Ctrl-C during a query tears down the proxy — bin/stash.js:83, :109
In the --psql path the proxy is spawned without detached, so proxy, psql and the launcher all sit in the shell's foreground process group. The tty delivers SIGINT to all three: psql traps it and cancels the query as expected, but the proxy takes it too, breaks its accept loop (main.rs:96-99) and drops the session ~2s later. The user's query-cancel gesture kills the server underneath a still-live client.
forwardSignals is not the cause — with it removed the proxy still received SIGINT at the same instant, and the launcher itself died, orphaning psql. Verified fix: spawn the proxy detached: true and don't forward SIGINT while psql owns the terminal. (Confirmed working: psql cancels, \q exits, proxy gets a clean SIGTERM.) This is separate from the exit-status/SIGQUIT work already done in this PR.
Worth a line before the sketch is wired into CI
2. release-workflow.example.yml:42-50 — the published binary would not be executable
upload-artifact@v4 normalises files to 644 and download-artifact restores them that way; the workflow packs the downloaded binary with no chmod. The only chmod +x in the repo is build-binaries.sh:28, which the workflow never calls, and npm won't add the bit (it only fixes bin entries; the platform packages declare none). Result would be spawn ... EACCES on every npx stash proxy. Add chmod +x to the assemble step, or tar the binary inside the artifact.
Worth flagging because it's the one gap the sketch doesn't already know about — the TODO at :65 covers versions and optionalDependencies but not this. (That TODO could also gain "clear private", though the private: true guard is deliberate and documented, so nothing is broken today.)
Minor / diagnostics
stash.js:132—PGHOSTis hardcoded to127.0.0.1. Any non-IPv4-loopback bind (CS_SERVER__HOST=localhostresolves to[::1]first on macOS) breaks--psqlregardless of anything else. Relatedly, the listen regex at:97is non-greedy and mis-parses the bracketed IPv6SocketAddrthe proxy prints (connect/mod.rs:105-109):[::1]:6432yields port 1. Given the hardcoded host, the regex mostly just degrades the error message — but both are worth a fix if IPv6 binds are in scope.stash.js:132—PGSSLMODE: "disable"is unconditional and written after theprocess.envspread, so there's no override.--psql --tlsdoes work (client TLS is negotiated viaSSLRequest;--tlsonly controls whether a bad TLS config is fatal), but the session fails underrequire_tls = true, and an explicit--tlsis silently downgraded to a plaintext loopback session.stash.js:116-123— proxy death mid-session is silent. Not a hang: the launcher exits when psql exits. But it prints nothing when the proxy dies under a live session, and swallows the proxy's exit code (measured: proxy exited 7, launcher exited 0). A diagnostic plus status propagation would help.stash.js:90-97— no timeout on the listen line. The early-exit case is handled, so this only bites when the proxy stays alive and silent (e.g.CS_DATABASE__CONNECTION_TIMEOUT=0), where the user waits indefinitely with no launcher-level message. Also worth gating the regex match on a\n— a chunk boundary inside the port digits would parse a truncated port. I couldn't reproduce it (0 partial lines in 921 chunks / 2.2MB; the message is well underPIPE_BUF, so writes are atomic) — one-line hardening, not a real defect.lib/repl.js:139-146—endsWith(";")splits dollar-quoted bodies.CREATE FUNCTION ... AS $$ ... ; ... $$;is sent as two fragments. Confined to the fallback shell used only whenpsqlis absent, and it fails loudly with a syntax error, so a note in the README caveats seems proportionate.
Checked and clean
usePsql/forceBuiltin warning condition; positionalDatabase's value-skipping (matches the clap flag set in cli/mod.rs); require.resolve of the platform subpath; stdout buffering (Rust println! is line-buffered when piped); demo.sh step 3c under set -e; no false-positive match against the other "listening on…" log lines.
One process note: npm publish --dry-run does not catch a private package in npm ≥12 — it exits 0 with a success line, so dry-run validation of the release sketch gives a false green.
I also looked at lib/connection.js:21-31: optionValue is genuinely option/value-unaware, but clap rejects the argv that would exploit it, and the accepted forms parse correctly. Only a database name literally starting with -u/-W after -- misparses, which fails loudly at auth. Noting it rather than raising it.
Prototypes distributing CipherStash Proxy through npm so it can be launched with:
Distribution design
stashmeta package resolves and launches the correct prebuilt Proxy binary while forwarding argv, stdio, exit status, and common termination signals.os/cpu-filtered optional platform packages cover macOS and Linux on arm64 and x64.npm/build-binaries.shbuilds and stages the current host binary.npm/release-workflow.example.ymlsketches a four-target build/publish matrix, including a working Linux arm64crosssetup.--psqlworkflowpsql, or a pure-JavaScript SQL shell whenpsqlis unavailable.stash:<db>=>prompt on interactive terminals.--flag=valueforms, positional database name, and environment fallbacks using the same CLI-over-URL-over-environment precedence as Proxy.Review fixes
set -eno longer terminates it prematurely.SIGQUITforwarding.Validation
node --checkfor all launcher modules.npm test: 3/3 connection-propagation tests pass.bash -n npm/build-binaries.sh npm/demo.sh.npm pack --dry-runfor the meta package and all four platform packages.npxversion/help passthrough, and exit-code validation.Status / decision
This remains a non-publishable prototype: packages are private and versioned
0.0.0-prototype; no release workflow is active and no platform package has been published.