keep push now saves a comment/structure-preserving, value-redacted copy
of the source .env next to the encrypted secrets, so context like "why
this exists" or "get this from X" survives ingest instead of being
silently dropped. keep pull --template retrieves it. Old vaults pushed
before this existed keep working (flat payload, no template) and fail
with a clear error rather than crashing if --template is requested.
Surfaced by a real mix-up this session: KEEP_SERVER_URL was set to a
"dev"-named hostname, and it was unclear without checking DNS/proxy
config whether that was actually the same server as the one running
on the deploy VPS or a separate instance -- nothing in `overview`'s
output (vault names, recipient labels) says which server answered.
Defaults silently to localhost:3050 if the env var isn't set at all,
which is its own way to end up looking at the wrong thing.
Printed before the fetch, not after, so it still shows up on a failed
request ("error: fetch failed") -- the exact case where knowing the
target mattered most and, before this, showed nothing at all.
Verified against a local test server: shows the right URL on success,
and shows the (default, unset) URL immediately before a deliberately
failed fetch to a port nothing's listening on.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Was previously hardcoded to vault key "<project-dir-name>/<env>" with
no marker-file lookup at all -- a real gap versus the design already
written out in PROPOSAL-announce-and-agent.md §4, which this script
never actually caught up to. Repo names, deploy directory names, and
vault keys are three independent strings in practice (a repo can be
named one thing, its deploy directory another, entirely by history/
accident) -- exactly the situation the npm-statuspage rollout hit.
Resolution order: $dir/.keep-vault if present, else fall back to
"<project>/<env>". `keep pull` is now fully best-effort -- missing
vault, missing grant, or keep not installed at all is never fatal,
matches the proposal's explicit requirement that a keep failure means
"doesn't apply to this project (yet)," never "stop the deploy."
Existing/hand-copied .env is left alone if the pull doesn't happen.
Verified with stub keep/docker commands on PATH (no real server
needed for this): a marker-file project with a deliberately mismatched
vault key resolves and pulls correctly; a project with neither a
marker nor a matching vault falls through cleanly to docker compose up
-d without ever touching .env or failing the deploy.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Caught live, deployed to the actual target -- the previous version
failed on the real host with "Cannot use import statement outside a
module" (Node 18, no ancestor package.json to signal ESM for an
extensionless file). That surfaced two stacked bugs my own testing had
missed by running the bundle from inside this repo's own directory
tree, which coincidentally supplied both things the standalone
artifact was silently depending on:
1. Module-format ambiguity: an extensionless file with no controlling
package.json defaults to CommonJS on older Node (no auto-detection
heuristic before recent versions). Fixed by naming the bundle
output keep.mjs -- unconditionally ESM on any Node version,
regardless of extension-less-file heuristics or nearby package.json.
2. The bigger one: crypto.ts's createRequire(import.meta.url) trick
(needed because libsodium-wrappers' published ESM build follows a
broken relative import) is opaque to esbuild's static bundler --
it's a runtime-obtained require reference, not the literal `require`
token esbuild's bundling recognizes. The previous "18kb bundle"
never actually contained libsodium-wrappers at all; it silently
relied on real node_modules being nearby on disk, which was only
ever true by accident when testing from within this repo. On a
clean host it threw "Cannot find module 'libsodium-wrappers'".
Tried forcing static inlining via a literal require() call (esbuild
does special-case that even in ESM-format output, unlike `import`,
which is permanently pinned to the "import" resolution condition
and can't be routed to the package's working CJS build no matter
what --conditions/--main-fields/--alias combination is tried this
was tried and confirmed exhaustively). That got real resolution and
a real 1.7MB bundle, but broke at runtime with "No secure random
number generator found" -- inlining the WASM engine breaks its own
Node-crypto feature detection.
Landed on: ship libsodium-wrappers (and its own dependency,
libsodium) as real, unmodified package files alongside the ~18kb
bundle, via a new copy-cli-deps.mjs step, rather than fighting
further to force single-file inlining. ~1.6MB total, still one tar
stream over SSH, still the entire deploy step.
deploy-cli.sh now tars dist-bundle/ (bundle + node_modules) instead of
catting a single file, extracts to ~/.keep-cli/ on the target, and
symlinks ~/bin/keep to the entry point inside it.
Verified this time in conditions that actually match the failure: full
isolation (mktemp'd HOME and install dir, no ancestor package.json, no
adjacent node_modules from this repo) for both `node keep.mjs` and
direct shebang execution, plus the exact tar/extract cycle
deploy-cli.sh performs.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The CLI only touches Node builtins and libsodium-wrappers (no native
addons like better-sqlite3, which the server needs but the CLI never
imports) -- confirmed by checking its actual import graph, not assumed.
That makes it a good candidate for a single-file esbuild bundle rather
than requiring a full git checkout + npm install on every machine that
needs to run `keep pull`.
deploy-cli.sh bundles src/cli (~18kb) and ships it straight to a host
over SSH -- HOST=user@vps ./deploy-cli.sh -- mirroring flit's
deploy-daemon.sh ssh-cat-chmod pattern rather than inventing a new
convention. No git, no npm install, no node_modules on the target at
all, just the one file plus a `node` runtime already there from
whatever else is deployed on that box.
Caught a real bug while building this: --banner:js="#!/usr/bin/env
node" duplicated the shebang esbuild already preserves from the source
file automatically, landing it on line 2 instead of line 1 -- Node
only special-cases a shebang on line 1, so the bundle threw a syntax
error on every invocation until the redundant banner flag was removed.
Verified the bundle for real, not just that it builds: ran `keep
identity init` / `identity show` against it directly and confirmed a
real Ed25519 keypair comes out the other end -- the specific thing
worth checking given libsodium-wrappers needed a createRequire
workaround for its own broken ESM packaging, and bundling could easily
have broken that differently.
Also documents both CLI-provisioning paths in INTEGRATION.md
(scripts/install-cli.sh for a machine you're fine cloning onto,
deploy-cli.sh for one you'd rather not) as a "getting keep onto a
machine" prerequisite section, ahead of the existing deploy-pattern
docs that all assume it's already there.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
npm run cli -- <args> only works from inside this repo's directory,
which is awkward everywhere but especially on a deploy host running
this alongside a dozen other projects. scripts/install-cli.sh builds
the CLI and symlinks dist/cli/index.js onto PATH (~/.local/bin by
default, no root/global npm install needed) as a real `keep` command,
runnable from anywhere.
Symlinked rather than copied, so a future `git pull && npm run build`
is the entire upgrade story -- no need to re-run the install script
after the first time.
Verified: keep --help and keep identity show both work correctly from
unrelated directories (/tmp, $HOME) after running the install script,
confirming no hidden cwd dependency.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
SETUP.local.md.example is a fill-in-the-blanks runbook for rolling
keep out across real machines and projects (gitignored once copied to
SETUP.local.md — real vault keys/recipient ids/hostnames are fleet
topology, not something to commit). bootstrap-project.sh wraps push +
grant --read-only for one or more recipients into a single call for
onboarding a new project vault.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CI announces a new image tag to keep (any grant is enough, read-only
included); each deploy target polls for it locally instead of CI
holding standing SSH access to production. keep stays a pure relay —
the tag lives outside the encrypted vault payload in its own table,
and keep never executes anything itself.
Adds vault_announcements, POST/GET /vaults/:key/announce, and the
`keep announce`/`keep watch` CLI commands, per
PROPOSAL-announce-and-agent.md (now marked implemented). Verified live:
a read-only identity announcing successfully and being logged, watch
picking up the tag on its first poll, and watch exiting nonzero for an
unknown/ungranted vault.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Two changes based on further review:
- Explicitly considers and rejects letting keep's server execute the
redeploy directly on announce, even though co-location removes the
credential/network objection that sank the CI-SSHes-in alternative.
The reason it's still wrong: a fully compromised keep server today
can't do anything worse than leak ciphertext, since it never holds a
decryption key. Giving it exec capability breaks that property — a
routine web app bug becomes "triggers arbitrary redeploys," not
"leaks encrypted blobs." Also notes the concrete cost: doing this
from inside a container needs the Docker socket mounted in, which is
broadly equivalent to root on the host.
- Replaces the "vault key = deploy directory name" assumption with an
explicit .keep-vault marker file per project directory. Repo name,
deploy directory name, and vault key are three independent strings
in practice (confirmed by a real example already in this fleet) —
no naming convention should assume any two of them match. Bootstrap
is now: choose a vault key, keep push it, drop a one-line
.keep-vault file. CI's announce step uses the same deliberately
hardcoded key, never derived from the repo name automatically.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The original draft speced a hypothetical per-project agent.sh on a
per-project cron. Reality turned out different: there's already a
production bulk deploy script on the host (fixed project array,
docker compose up -d per project, no explicit pull — freshness comes
entirely from pull_policy: always) — so this patches that existing
loop in place instead of introducing a parallel per-project script
that would duplicate its array/failure-tracking logic.
Also folds in a real prerequisite gap found while reconciling the
proposal with the actual script: pull_policy: always isn't universal
across the fleet yet, independent of keep entirely — the bulk script
can't redeploy anything missing that line regardless of keep adoption.
One instance of this was already fixed this session (a sibling
project's docker-compose.yml missed the line when copying another
project's compose pattern).
Reframes §5's announce/watch benefit accordingly: since the real
script is one all-or-nothing sweep, not N independent watchers, the
natural fit is waking the whole sweep early (watch-any), not
per-project selective redeploy — still deliberately left unbuilt
until there's a real signal it's worth it over the already-scheduled
sweep.
Project names anonymized per policy — generic proj1..proj12
placeholders instead of the real fleet.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Design doc for closing the CI->deploy gap without reintroducing the
credential-concentration problem a naive "CI SSHes into the VPS"
approach would bring back. keep stays a pure relay — never executes
anything remotely. CI announces a new tag (a distinct verb from push/
pull, since a tag isn't a secret and doesn't belong in the encrypted
vault payload); each VPS polls and redeploys itself locally, using an
identity and grant it already holds.
Explicitly specs the not-yet-bootstrapped fallback: the agent script's
`keep pull` is allowed to fail for any reason, and on failure just runs
a plain `docker compose pull && up -d` against whatever's already on
disk — the same thing that would have happened before keep existed.
Ships the dumb-periodic-timer version of the agent first; the watch/
announce fast path is speced but deliberately left unbuilt until
there's a real signal that polling latency is worth the complexity.
Not implemented — design stage.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Was silently dropping updatedAt/updatedBy even though the server
already returned them — the CLI just never printed them. Also
resolves updatedBy to the recipient's label server-side (matching how
grants already show a label alongside the recipient id) instead of
printing a bare recipient_id.
Verified against a live server: overview now prints "demo/production
(updated <iso timestamp> by test-box)" instead of omitting that line
entirely.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
keep overview gives an admin a cross-vault view of every recipient and
grant in one screen, instead of walking vaults one at a time via
/vaults/:key/recipients. scripts/deploy.sh pulls a project's env from
keep before running docker compose up, for VPS deploys of several
docker-compose projects.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
keep has no web UI (API + CLI only), so there's no "server screenshot"
in the way wisp/npm-statuspage have one. Instead, a styled terminal
image showing a real round trip: identity registration, a push, a
read-only grant, and that read-only recipient correctly blocked from
pushing. Matches the project family's dark-terminal visual style.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
IMPLEMENTATION.md hadn't been updated after the second implementation
pass — the data model SQL was missing prev_ciphertext/prev_nonce,
can_write, and the entire vault_grants_previous table; the Rotation
section described the old "overwritten, not versioned" behavior
(the opposite of what got built); the crypto scheme section still
said "read implies write" while the Resolved design decisions section
below it said the opposite; the CLI surface listing was missing
--purge/--previous/--read-only; and "What was verified" only covered
the first verification pass, not the read-only/rollback/purge one.
README's Core model was missing any mention of the retention/purge
behavior or read/write grant scoping. INTEGRATION.md now recommends
--read-only for deploy identities specifically, now that the
capability exists.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Per-question resolution, per best practice rather than deferral:
- Push authorization: closed the least-privilege gap where read implied
write. vault_grants gained can_write (default true, so nothing
existing changes); enforced on push and grant (granting others is
itself a mutation of vault membership, so it needs write too, not
just read). 'keep grant --read-only' creates a read-only grant.
- Version history: not full history (conflates "undo a typo'd push"
with "this leaked, stop retaining it" into one mechanism). Retains
exactly one previous version as a rollback safety net
(vaults.prev_ciphertext/prev_nonce + a vault_grants_previous mirror
table so recipients can unwrap it), plus 'keep push --purge' for
compromise-driven rotations that explicitly skips retention and
wipes any existing previous version too.
- Per-secret-key granularity: resolved by NOT building it — documented
the escape hatch (split into more vaults) instead of adding partial-
decrypt complexity for a problem the existing primitive solves.
One more real bug caught during verification: the CLI's --previous flag
initially signed a path including its query string, but the server
verifies against req.originalUrl with the query stripped — a mismatch
that would have made every --previous request fail signature
verification. Fixed by splitting the signed path from the request URL
in signedFetch, signing only the former.
Verified end-to-end with three independent identities: read-only grant
correctly blocked from push and from granting others, write access and
read-only status both preserved correctly across a rotation, previous-
version pull working for a routine push and correctly unavailable to
every recipient after a purge push. Also re-verified against a fresh
Docker build.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Server: Express + better-sqlite3 (WAL), multi-recipient key-wrapping
per IMPLEMENTATION.md's design — vaults/recipients/vault_grants/
access_log. Two auth paths: ADMIN_PASSWORD header for recipient
management and revoke (pure metadata operations), signed-request
auth (Ed25519 signature over method+path+timestamp+body-hash) for
push/pull/grant, mirroring the spirit of this project family's other
signed-handshake patterns without naming them.
CLI: identity init/show/set-id, push/pull/grant/log, admin recipient
add/list/remove and revoke. Grant is a client-side crypto operation
(the granter unwraps the vault's current key locally and reseals it
for the new recipient) rather than a server-side operation, since the
server never holds an unwrapped key to grant with.
Verified end-to-end with two independent local identities against a
live server and separately against the built Docker image: register,
push, pull (granted and ungranted), grant without re-pushing, admin
revoke, a subsequent rotation confirming the revoked recipient stays
excluded, and rejection of missing/malformed signed-request auth.
Two real bugs caught during verification, not just written up:
- libsodium-wrappers' published ESM build does a relative import only
resolvable under bundler-style resolution — broken under plain Node
ESM. Fixed via createRequire to force the CJS build.
- Express's req.path inside a sub-router is relative to the mount
point, which would have silently mismatched a client signing the
full request path. Fixed by verifying against req.originalUrl.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
README covers the pitch and how this differs from Vault/Doppler (too
heavy) and a static age/sops-encrypted file in git (no live rotation,
no access log). IMPLEMENTATION covers the multi-recipient key-wrapping
scheme (age/sops-style, adapted to libsodium primitives already used
in flit/waste-go), the flat vault/recipient/grant/access-log schema,
rotation and revocation semantics (including the standard "revoke
doesn't retroactively unread already-decrypted secrets" caveat), CLI
surface, and deploy-script integration.
Motivated by a real, already-felt annoyance: every repo in this
project family (goonk, wisp, npm-statuspage, flit, waste-go) has its
own hand-copied .env today, with no rotation story and no record of
which machine has which secret.
No code yet — design stage. Admin auth mechanism flagged as needing a
decision before implementation starts.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
