Files
keep/INTEGRATION.md
Fredrik Johansson 1b4cd9b826 Fix deploy-cli.sh: bundle was completely broken on a real host
Caught live, deployed to the actual target -- the previous version
failed on the real host with "Cannot use import statement outside a
module" (Node 18, no ancestor package.json to signal ESM for an
extensionless file). That surfaced two stacked bugs my own testing had
missed by running the bundle from inside this repo's own directory
tree, which coincidentally supplied both things the standalone
artifact was silently depending on:

1. Module-format ambiguity: an extensionless file with no controlling
   package.json defaults to CommonJS on older Node (no auto-detection
   heuristic before recent versions). Fixed by naming the bundle
   output keep.mjs -- unconditionally ESM on any Node version,
   regardless of extension-less-file heuristics or nearby package.json.

2. The bigger one: crypto.ts's createRequire(import.meta.url) trick
   (needed because libsodium-wrappers' published ESM build follows a
   broken relative import) is opaque to esbuild's static bundler --
   it's a runtime-obtained require reference, not the literal `require`
   token esbuild's bundling recognizes. The previous "18kb bundle"
   never actually contained libsodium-wrappers at all; it silently
   relied on real node_modules being nearby on disk, which was only
   ever true by accident when testing from within this repo. On a
   clean host it threw "Cannot find module 'libsodium-wrappers'".

   Tried forcing static inlining via a literal require() call (esbuild
   does special-case that even in ESM-format output, unlike `import`,
   which is permanently pinned to the "import" resolution condition
   and can't be routed to the package's working CJS build no matter
   what --conditions/--main-fields/--alias combination is tried this
   was tried and confirmed exhaustively). That got real resolution and
   a real 1.7MB bundle, but broke at runtime with "No secure random
   number generator found" -- inlining the WASM engine breaks its own
   Node-crypto feature detection.

   Landed on: ship libsodium-wrappers (and its own dependency,
   libsodium) as real, unmodified package files alongside the ~18kb
   bundle, via a new copy-cli-deps.mjs step, rather than fighting
   further to force single-file inlining. ~1.6MB total, still one tar
   stream over SSH, still the entire deploy step.

deploy-cli.sh now tars dist-bundle/ (bundle + node_modules) instead of
catting a single file, extracts to ~/.keep-cli/ on the target, and
symlinks ~/bin/keep to the entry point inside it.

Verified this time in conditions that actually match the failure: full
isolation (mktemp'd HOME and install dir, no ancestor package.json, no
adjacent node_modules from this repo) for both `node keep.mjs` and
direct shebang execution, plus the exact tar/extract cycle
deploy-cli.sh performs.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 15:59:44 +02:00

6.9 KiB

Incorporating keep into an existing deploy setup

Concrete, by deploy pattern rather than by naming specific projects — the shape of the fit depends on how a project currently gets its config onto a target machine, not on what the project is.

The general shape

Wherever a deploy script or a manually-maintained .env exists today, the change is the same: replace "assume .env is already sitting there, hand-copied at some point" with a keep pull at the top of the script.

# before — a deploy script assumes .env already exists on this machine
set -euo pipefail
source .env   # or docker-compose reads it directly

# after
set -euo pipefail
keep pull <project>/<env> --out .env

The one-time setup cost per machine: keep identity init once, send the printed public key to whoever runs keep recipient add, then keep identity set-id <id> with what comes back. After that, every future deploy from that machine just works — no more "wait, what's the DB password again" when setting up a new box.

A deploy identity should almost always be granted read-only access (keep grant <vault> <deploy-recipient-id> --read-only) — it only ever needs to pull, and a read-only grant means a compromised deploy box can't also rotate the vault or add itself more recipients.

Getting the keep CLI onto a machine

Chicken-and-egg: every pattern below assumes keep is already a runnable command on the machine doing the pulling. Two ways to get there, pick based on whether you're fine with a git checkout existing on that machine:

A machine you're already developing on, or don't mind cloning onto (your laptop, a VPS you administer directly):

git clone <this repo>
cd keep
npm install
./scripts/install-cli.sh   # builds, symlinks dist/cli/index.js onto PATH as `keep`

Symlinked rather than copied — a future git pull && npm run build is the entire upgrade story, no need to re-run the install script.

A deploy target you'd rather not clone a git repo onto — bundle the CLI and ship it over SSH, the same way flit/waste-go's deploy-*.sh scripts ship a compiled binary:

HOST=user@your-vps ./deploy-cli.sh

Not quite a single file: the bundle itself (esbuild, ~18kb) covers everything except libsodium-wrappers, which ships alongside it as the real, unmodified package (~1.6MB total with its own dependency, libsodium) rather than being force-inlined. That package's published ESM build follows a broken relative import (see src/shared/crypto.ts's own comment on this), and statically inlining it via esbuild — technically possible by routing through a literal require() call instead of import — breaks its WASM engine's own Node-crypto feature detection at runtime ("No secure random number generator found"), confirmed by testing, not assumed. Shipping the real package is more reliable than fighting that.

The script tars dist-bundle/ (the esbuild output plus that one dependency) and extracts it to ~/.keep-cli/ on the target over SSH, then symlinks ~/bin/keep to the entry point inside it — no git, no npm install on that machine at all, just a node runtime (already there, if it's running any of the other Node-based projects in this family). Re-run it any time the CLI changes; safe to overwrite.

Pattern: several docker-compose projects on one or more VPSes

The common shape once more than one or two projects are on keep: each project lives in its own directory on the VPS (/opt/docker/<project>/docker-compose.yml), and "deploy" means pulling that project's env and running docker compose up -d in its directory. scripts/deploy.sh in this repo is a small generic wrapper for exactly that:

./deploy.sh myapp              # pulls myapp/production, runs compose in /opt/docker/myapp
./deploy.sh myapp staging      # pulls myapp/staging instead
DEPLOY_ROOT=/srv/apps ./deploy.sh myapp   # different project root

One-time setup per VPS: keep identity init on that machine, register its public key with keep recipient add, keep identity set-id, then grant it read-only access to each project vault it deploys (keep grant <project>/production <vps-recipient-id> --read-only). After that, ./deploy.sh <project> is the whole deploy step — no .env hand-copied onto the box first.

For "which machine can deploy which project," keep overview (admin) prints every vault and every recipient's access in one screen — the thing to reach for instead of checking each vault's grants one at a time.

Pattern: docker-compose reading a local .env

A common shape — docker compose up -d reads environment: values from a .env file sitting next to docker-compose.yml on the deploy machine, hand-maintained and containing real secrets.

keep pull myapp/production --out .env
docker compose up -d

No code changes needed in the target project — keep only changes how the .env file gets there, not what reads it. This is the easiest category to retrofit.

Pattern: cross-compile + scp + SSH-restart deploy scripts

A shape where a deploy script builds a binary locally, ships it to a VPS, and restarts a service over SSH — reading a local .env next to the script for things like shared secrets that shouldn't be hardcoded or re-typed on every invocation.

keep pull myapp/production --out .env
set -a && source .env && set +a
# ...rest of the deploy script unchanged

Same retrofit as the docker-compose case, just applied before whatever build/ship step already exists.

Pattern: a plaintext config file served to a browser

Not every config.js/similar file that looks like environment config actually holds secrets — some of it is public-by-design (an API base URL, a feature flag) that's explicitly meant to be visible to anyone loading the page. keep solves a confidentiality problem; a file with nothing confidential in it doesn't have one. Worth checking this distinction before wiring keep into anything — retrofitting a non-secret config file would be solving a problem that file doesn't have.

A vault server's own configuration

Whatever runs keep itself needs its own ADMIN_PASSWORD to be a real, manually-set secret on its own host — it can't bootstrap its own trust root by pulling itself from itself. This is the same "root of trust starts somewhere unmanaged" fact every secrets system has (Vault's own unseal keys aren't stored in Vault either). Not a gap, just worth being explicit about: keep's own deploy stays on a plain .env/ docker-compose.yml environment variables.

New projects going forward

The actual best use case isn't retrofitting existing deploys — it's never having the "hand-reconstruct a .env from memory" moment for a new deploy target in the first place. For whatever gets built next: register its production host as a keep recipient as part of first setup, write the deploy script to keep pull from day one, and the annoyance this was built to fix never has a chance to happen.