Skip to content

spike(npm): prototype npx stash proxy distribution - #398

Draft
coderdan wants to merge 7 commits into
dan/cli-connection-flagsfrom
dan/npx-stash-proxy-prototype
Draft

spike(npm): prototype npx stash proxy distribution#398
coderdan wants to merge 7 commits into
dan/cli-connection-flagsfrom
dan/npx-stash-proxy-prototype

Conversation

@coderdan

@coderdan coderdan commented May 30, 2026

Copy link
Copy Markdown
Contributor

Spike / draft implementing CIP-3824. Stacked on #397 (dan/cli-connection-flags); the review diff is limited to npm/.

Prototypes distributing CipherStash Proxy through npm so it can be launched with:

npx stash proxy --database-url postgres://user:pass@host:5432/db
npx stash proxy --psql --database-url postgres://user:pass@host:5432/db

Distribution design

  • A thin pure-JavaScript stash meta package resolves and launches the correct prebuilt Proxy binary while forwarding argv, stdio, exit status, and common termination signals.
  • Four os/cpu-filtered optional platform packages cover macOS and Linux on arm64 and x64.
  • The macOS arm64 staging path applies an ad-hoc signature and fails if signing does not succeed.
  • npm/build-binaries.sh builds and stages the current host binary.
  • npm/release-workflow.example.yml sketches a four-target build/publish matrix, including a working Linux arm64 cross setup.

--psql workflow

  • Starts Proxy and waits for its reported listen address, including an OS-assigned fallback port.
  • Launches system psql, or a pure-JavaScript SQL shell when psql is unavailable.
  • Uses a cyan stash:<db>=> prompt on interactive terminals.
  • Propagates connection URL, individual long/short flags, --flag=value forms, positional database name, and environment fallbacks using the same CLI-over-URL-over-environment precedence as Proxy.

Review fixes

  • Fixed the demo's expected-failure assertion so set -e no longer terminates it prematurely.
  • Extracted connection interpretation from the launcher and added dependency-free unit tests.
  • Preserved child signal termination semantics and added SIGQUIT forwarding.
  • Rebased onto the reviewed feat(cli): connection flags + DX (quiet logging, random port, best-effort TLS) #397 head and added required DCO sign-offs to all seven commits.

Validation

  • node --check for all launcher modules.
  • npm test: 3/3 connection-propagation tests pass.
  • bash -n npm/build-binaries.sh npm/demo.sh.
  • npm pack --dry-run for the meta package and all four platform packages.
  • Full macOS arm64 proof: release build, ad-hoc signing, local npm install, correct platform-package resolution, npx version/help passthrough, and exit-code validation.

Status / decision

This remains a non-publishable prototype: packages are private and versioned 0.0.0-prototype; no release workflow is active and no platform package has been published.

  • Decide whether to productionize this distribution channel.
  • If yes, wire the example workflow into release CI and define npm access plus synchronized package versioning.

@coderabbitai

coderabbitai Bot commented May 30, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 47445073-8e8f-45e3-832e-7997c0071df3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch dan/npx-stash-proxy-prototype

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@freshtonic
freshtonic force-pushed the dan/cli-connection-flags branch from c928bdf to fde3ad2 Compare August 18, 2026 06:46
coderdan and others added 7 commits August 18, 2026 17:06
Proof-of-concept for shipping the proxy via npm as `npx stash proxy`, using the
esbuild/Biome/SWC pattern (per-platform packages + os/cpu-filtered
optionalDependencies + a thin JS launcher) -- NOT native N-API bindings, since
proxy is a standalone server we only need to distribute and launch.

Verified end-to-end locally on darwin-arm64: npx -> stash shim -> exec native
cipherstash-proxy binary, with --version/--help passthrough, correct exit-code
forwarding (0 / clap's 2), signal forwarding, and os/cpu platform resolution.

Binaries are git-ignored build artifacts (build-binaries.sh / demo.sh
regenerate them). Packages are private + 0.0.0-prototype to prevent publish.
See npm/README.md for how this maps to a production CI matrix and the
code-signing rationale (skips notarization/Developer-ID; keeps free ad-hoc
signing on Apple Silicon).

Signed-off-by: James Sadler <james@cipherstash.com>
Signed-off-by: James Sadler <james@cipherstash.com>
`stash proxy --psql ...` starts the proxy, waits for it to report its listen
address (parsing the OS-assigned port when the default is in use), then launches
psql connected to the proxy with the target db/user/password. psql is the
foreground session; the proxy is torn down when it exits. Falls back with a
clear message if psql is not on PATH.

Connection details are taken from --database-url, then --db-* flags, then
CS_DATABASE__* env. Validated against a local dev DB.

Signed-off-by: James Sadler <james@cipherstash.com>
When `--psql` is used and psql isn't on PATH (or STASH_USE_BUILTIN_SQL=1 is
set), open a small built-in SQL shell (lib/repl.js) instead of failing. It uses
the pure-JS `pg` driver (no native binaries) and runs SQL through the proxy with
tabular output and a few meta-commands (\l, \dt, \d, \?, \q).

Not a psql replacement -- a convenience fallback. Real psql is still preferred
when installed. Validated end-to-end against a local dev DB via the proxy.

Background: bundling real psql isn't viable off-the-shelf -- the
@embedded-postgres/* packages ship initdb/pg_ctl/postgres but strip psql -- so a
pure-JS shell is the pragmatic no-native-deps fallback.

Signed-off-by: James Sadler <james@cipherstash.com>
Visually distinguishes a via-proxy session from a direct psql connection: the
prompt becomes e.g. `stash:mydb=>` with "stash" in cyan (on a TTY). Applied to
both real psql (via PROMPT1/PROMPT2 --set) and the built-in shell.

Override with STASH_PSQL_PROMPT (set empty to use psql's default / ~/.psqlrc);
colour honours NO_COLOR and is disabled off a TTY.

Signed-off-by: James Sadler <james@cipherstash.com>
A literal ESC byte in PROMPT1 was stripped by psql's variable parser, so the
prompt showed in the default colour. psql's own %033 octal escape produces the
ESC reliably (verified: \001 ESC[36m \002 stash \001 ESC[0m \002 -- 'stash'
wrapped in cyan).

Signed-off-by: James Sadler <james@cipherstash.com>
Signed-off-by: James Sadler <james@cipherstash.com>
@freshtonic
freshtonic force-pushed the dan/npx-stash-proxy-prototype branch from 59bed52 to 6f82ea5 Compare August 18, 2026 07:06
@freshtonic freshtonic changed the title spike(npm): distribute proxy via npx stash proxy spike(npm): prototype npx stash proxy distribution Aug 18, 2026
@freshtonic
freshtonic marked this pull request as ready for review August 18, 2026 12:19
@freshtonic freshtonic self-assigned this Aug 18, 2026

@tobyhede tobyhede left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of the npm/ diff. Everything below was verified against this branch (6f82ea5b) by running it — process-group experiments against the real stash.js, the parsers exercised in node, and the proxy's own listen/TLS/signal paths read in packages/cipherstash-proxy/. Severities account for this being an unwired spike.

Worth fixing before this leaves spike status

1. Ctrl-C during a query tears down the proxy — bin/stash.js:83, :109

In the --psql path the proxy is spawned without detached, so proxy, psql and the launcher all sit in the shell's foreground process group. The tty delivers SIGINT to all three: psql traps it and cancels the query as expected, but the proxy takes it too, breaks its accept loop (main.rs:96-99) and drops the session ~2s later. The user's query-cancel gesture kills the server underneath a still-live client.

forwardSignals is not the cause — with it removed the proxy still received SIGINT at the same instant, and the launcher itself died, orphaning psql. Verified fix: spawn the proxy detached: true and don't forward SIGINT while psql owns the terminal. (Confirmed working: psql cancels, \q exits, proxy gets a clean SIGTERM.) This is separate from the exit-status/SIGQUIT work already done in this PR.

Worth a line before the sketch is wired into CI

2. release-workflow.example.yml:42-50 — the published binary would not be executable

upload-artifact@v4 normalises files to 644 and download-artifact restores them that way; the workflow packs the downloaded binary with no chmod. The only chmod +x in the repo is build-binaries.sh:28, which the workflow never calls, and npm won't add the bit (it only fixes bin entries; the platform packages declare none). Result would be spawn ... EACCES on every npx stash proxy. Add chmod +x to the assemble step, or tar the binary inside the artifact.

Worth flagging because it's the one gap the sketch doesn't already know about — the TODO at :65 covers versions and optionalDependencies but not this. (That TODO could also gain "clear private", though the private: true guard is deliberate and documented, so nothing is broken today.)

Minor / diagnostics

  • stash.js:132PGHOST is hardcoded to 127.0.0.1. Any non-IPv4-loopback bind (CS_SERVER__HOST=localhost resolves to [::1] first on macOS) breaks --psql regardless of anything else. Relatedly, the listen regex at :97 is non-greedy and mis-parses the bracketed IPv6 SocketAddr the proxy prints (connect/mod.rs:105-109): [::1]:6432 yields port 1. Given the hardcoded host, the regex mostly just degrades the error message — but both are worth a fix if IPv6 binds are in scope.
  • stash.js:132PGSSLMODE: "disable" is unconditional and written after the process.env spread, so there's no override. --psql --tls does work (client TLS is negotiated via SSLRequest; --tls only controls whether a bad TLS config is fatal), but the session fails under require_tls = true, and an explicit --tls is silently downgraded to a plaintext loopback session.
  • stash.js:116-123 — proxy death mid-session is silent. Not a hang: the launcher exits when psql exits. But it prints nothing when the proxy dies under a live session, and swallows the proxy's exit code (measured: proxy exited 7, launcher exited 0). A diagnostic plus status propagation would help.
  • stash.js:90-97 — no timeout on the listen line. The early-exit case is handled, so this only bites when the proxy stays alive and silent (e.g. CS_DATABASE__CONNECTION_TIMEOUT=0), where the user waits indefinitely with no launcher-level message. Also worth gating the regex match on a \n — a chunk boundary inside the port digits would parse a truncated port. I couldn't reproduce it (0 partial lines in 921 chunks / 2.2MB; the message is well under PIPE_BUF, so writes are atomic) — one-line hardening, not a real defect.
  • lib/repl.js:139-146endsWith(";") splits dollar-quoted bodies. CREATE FUNCTION ... AS $$ ... ; ... $$; is sent as two fragments. Confined to the fallback shell used only when psql is absent, and it fails loudly with a syntax error, so a note in the README caveats seems proportionate.

Checked and clean

usePsql/forceBuiltin warning condition; positionalDatabase's value-skipping (matches the clap flag set in cli/mod.rs); require.resolve of the platform subpath; stdout buffering (Rust println! is line-buffered when piped); demo.sh step 3c under set -e; no false-positive match against the other "listening on…" log lines.

One process note: npm publish --dry-run does not catch a private package in npm ≥12 — it exits 0 with a success line, so dry-run validation of the release sketch gives a false green.

I also looked at lib/connection.js:21-31: optionValue is genuinely option/value-unaware, but clap rejects the argv that would exploit it, and the accepted forms parse correctly. Only a database name literally starting with -u/-W after -- misparses, which fails loudly at auth. Noting it rather than raising it.

@freshtonic
freshtonic marked this pull request as draft August 19, 2026 01:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants