Fix deploy-cli.sh: bundle was completely broken on a real host

Caught live, deployed to the actual target -- the previous version
failed on the real host with "Cannot use import statement outside a
module" (Node 18, no ancestor package.json to signal ESM for an
extensionless file). That surfaced two stacked bugs my own testing had
missed by running the bundle from inside this repo's own directory
tree, which coincidentally supplied both things the standalone
artifact was silently depending on:

1. Module-format ambiguity: an extensionless file with no controlling
   package.json defaults to CommonJS on older Node (no auto-detection
   heuristic before recent versions). Fixed by naming the bundle
   output keep.mjs -- unconditionally ESM on any Node version,
   regardless of extension-less-file heuristics or nearby package.json.

2. The bigger one: crypto.ts's createRequire(import.meta.url) trick
   (needed because libsodium-wrappers' published ESM build follows a
   broken relative import) is opaque to esbuild's static bundler --
   it's a runtime-obtained require reference, not the literal `require`
   token esbuild's bundling recognizes. The previous "18kb bundle"
   never actually contained libsodium-wrappers at all; it silently
   relied on real node_modules being nearby on disk, which was only
   ever true by accident when testing from within this repo. On a
   clean host it threw "Cannot find module 'libsodium-wrappers'".

   Tried forcing static inlining via a literal require() call (esbuild
   does special-case that even in ESM-format output, unlike `import`,
   which is permanently pinned to the "import" resolution condition
   and can't be routed to the package's working CJS build no matter
   what --conditions/--main-fields/--alias combination is tried this
   was tried and confirmed exhaustively). That got real resolution and
   a real 1.7MB bundle, but broke at runtime with "No secure random
   number generator found" -- inlining the WASM engine breaks its own
   Node-crypto feature detection.

   Landed on: ship libsodium-wrappers (and its own dependency,
   libsodium) as real, unmodified package files alongside the ~18kb
   bundle, via a new copy-cli-deps.mjs step, rather than fighting
   further to force single-file inlining. ~1.6MB total, still one tar
   stream over SSH, still the entire deploy step.

deploy-cli.sh now tars dist-bundle/ (bundle + node_modules) instead of
catting a single file, extracts to ~/.keep-cli/ on the target, and
symlinks ~/bin/keep to the entry point inside it.

Verified this time in conditions that actually match the failure: full
isolation (mktemp'd HOME and install dir, no ancestor package.json, no
adjacent node_modules from this repo) for both `node keep.mjs` and
direct shebang execution, plus the exact tar/extract cycle
deploy-cli.sh performs.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

INTEGRATION.md

@@ -52,20 +52,31 @@ Symlinked rather than copied — a future `git pull && npm run build` is
 the entire upgrade story, no need to re-run the install script.
 
 **A deploy target you'd rather not clone a git repo onto** — bundle the
-CLI into a single portable file and ship just that, over SSH, the same
-way flit/waste-go's `deploy-*.sh` scripts ship a compiled binary:
+CLI and ship it over SSH, the same way flit/waste-go's `deploy-*.sh`
+scripts ship a compiled binary:
 
 ```bash
 HOST=user@your-vps ./deploy-cli.sh
 ```
 
-This bundles `src/cli` (esbuild, single file, ~18kb — the CLI only
-touches Node builtins and libsodium-wrappers, no native addons, so it's
-fully portable) and `cat`s it straight to `~/bin/keep` on the target over
-SSH — no git, no npm install, no node_modules on that machine at all,
-just a `node` runtime (already there, if it's running any of the other
-Node-based projects in this family) and the one file. Re-run it any time
-the CLI changes; it's a single file, safe to overwrite.
+Not quite a single file: the bundle itself (esbuild, ~18kb) covers
+everything except `libsodium-wrappers`, which ships alongside it as the
+real, unmodified package (~1.6MB total with its own dependency,
+`libsodium`) rather than being force-inlined. That package's published
+ESM build follows a broken relative import (see `src/shared/crypto.ts`'s
+own comment on this), and statically inlining it via esbuild — technically
+possible by routing through a literal `require()` call instead of
+`import` — breaks its WASM engine's own Node-crypto feature detection at
+runtime ("No secure random number generator found"), confirmed by
+testing, not assumed. Shipping the real package is more reliable than
+fighting that.
+
+The script tars `dist-bundle/` (the esbuild output plus that one
+dependency) and extracts it to `~/.keep-cli/` on the target over SSH,
+then symlinks `~/bin/keep` to the entry point inside it — no git, no npm
+install on that machine at all, just a `node` runtime (already there, if
+it's running any of the other Node-based projects in this family). Re-run
+it any time the CLI changes; safe to overwrite.
 
 ## Pattern: several docker-compose projects on one or more VPSes
 

deploy-cli.sh

@@ -1,23 +1,38 @@
 #!/usr/bin/env bash
-# deploy-cli.sh — bundle the keep CLI into a single portable file and ship
-# it straight to a host over SSH. No git clone, no npm install on the
-# target — the bundle only touches Node builtins and libsodium-wrappers
-# (pure JS/WASM, no native compilation), so the file itself is the whole
-# artifact. Mirrors flit/deploy-daemon.sh's ssh-cat-chmod pattern.
+# deploy-cli.sh — package the keep CLI (bundle + its one real runtime
+# dependency, libsodium-wrappers) and ship it straight to a host over SSH.
+# No git clone, no npm install on the target.
+#
+# Not a single file: libsodium-wrappers' published ESM build is broken
+# (a relative import that doesn't resolve outside a bundler-aware
+# context — see src/shared/crypto.ts), and statically force-inlining it
+# via esbuild breaks its own Node-crypto feature-detection at runtime
+# ("No secure random number generator found"). Shipping the real,
+# unmodified package alongside a small (~18kb) bundle for everything
+# else is more reliable than fighting that — the whole package is still
+# only ~1.6MB and this script is still the entire deploy step.
 #
 # Usage: HOST=user@host ./deploy-cli.sh
 set -euo pipefail
 
 HOST="${HOST:?usage: HOST=user@host ./deploy-cli.sh}"
 BIN_DIR="${BIN_DIR:-~/bin}"
+INSTALL_DIR="${INSTALL_DIR:-~/.keep-cli}"
 
 echo "→ bundling keep CLI…"
 npm run build:cli-bundle
 
-echo "→ uploading to $HOST:$BIN_DIR/keep…"
-ssh "$HOST" "mkdir -p $BIN_DIR && cat > $BIN_DIR/keep && chmod +x $BIN_DIR/keep" < dist-bundle/keep
+echo "→ packaging…"
+tar -C dist-bundle -czf /tmp/keep-cli.tar.gz .
+
+echo "→ uploading to $HOST:$INSTALL_DIR…"
+ssh "$HOST" "mkdir -p $INSTALL_DIR && tar -C $INSTALL_DIR -xzf -" < /tmp/keep-cli.tar.gz
+rm /tmp/keep-cli.tar.gz
+
+echo "→ linking $BIN_DIR/keep…"
+ssh "$HOST" "mkdir -p $BIN_DIR && chmod +x $INSTALL_DIR/keep.mjs && ln -sf $INSTALL_DIR/keep.mjs $BIN_DIR/keep"
 
 echo "→ verifying…"
 ssh "$HOST" "$BIN_DIR/keep --help" | head -1
 
-echo "✓ done — re-run this script any time the CLI changes; it's one file, safe to overwrite."
+echo "✓ done — re-run this script any time the CLI changes; safe to overwrite."

package.json

@@ -10,7 +10,7 @@
     "dev:server": "tsx watch --env-file=.env src/server/index.ts",
     "cli": "tsx src/cli/index.ts",
     "build": "tsc",
-    "build:cli-bundle": "esbuild src/cli/index.ts --bundle --platform=node --format=esm --outfile=dist-bundle/keep --external:node:*",
+    "build:cli-bundle": "esbuild src/cli/index.ts --bundle --platform=node --format=esm --outfile=dist-bundle/keep.mjs --external:node:* --external:libsodium-wrappers && node scripts/copy-cli-deps.mjs",
     "start": "node --env-file=.env dist/server/index.js"
   },
   "dependencies": {

scripts/copy-cli-deps.mjs

@@ -0,0 +1,32 @@
+// Copies the one real runtime dependency the CLI bundle can't statically
+// inline (libsodium-wrappers, and its own dependency libsodium) next to
+// the bundle, so dist-bundle/ is a small, complete, self-contained
+// directory -- no npm install needed wherever it ends up.
+//
+// Why not just bundle it: libsodium-wrappers' published ESM build follows
+// a relative import ("./libsodium.mjs") that's broken in that package's
+// own dist layout -- the exact bug crypto.ts's own comment documents.
+// Forcing static inlining via a literal require() (the only way to route
+// esbuild's resolver to the *working* CJS build instead) was tried and
+// technically resolved, but produced "No secure random number generator
+// found" at runtime -- inlining the WASM engine breaks its own Node
+// crypto feature-detection. Shipping the real, unmodified package files
+// is more reliable than fighting that.
+import { cpSync, mkdirSync, existsSync } from 'node:fs';
+import { join, dirname } from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+const root = join(dirname(fileURLToPath(import.meta.url)), '..');
+const outDir = join(root, 'dist-bundle', 'node_modules');
+mkdirSync(outDir, { recursive: true });
+
+for (const pkg of ['libsodium-wrappers', 'libsodium']) {
+  const src = join(root, 'node_modules', pkg);
+  if (!existsSync(src)) {
+    console.error(`missing node_modules/${pkg} -- run npm install first`);
+    process.exit(1);
+  }
+  cpSync(src, join(outDir, pkg), { recursive: true });
+}
+
+console.log('dist-bundle/node_modules populated (libsodium-wrappers, libsodium)');