Compare commits

...

14 Commits

Author SHA1 Message Date
Fredrik Johansson
a640bc1faf keep pull: fill saved template with real values by default
All checks were successful
Docker / build-and-push (push) Successful in 1m54s
A bare key/value dump throws away the comments/structure a template
preserves; now a plain pull reconstitutes the real .env from the
template when the vault has one, falling back to the flat dump
otherwise. --template still returns the redacted structure alone.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-16 16:52:02 +02:00
Fredrik Johansson
04b48e05e4 Document env templating in the integration guide
All checks were successful
Docker / build-and-push (push) Successful in 1m58s
2026-07-16 10:59:44 +02:00
Fredrik Johansson
8dcb675ad2 Save a redacted .env template alongside pushed secrets
All checks were successful
Docker / build-and-push (push) Successful in 1m51s
keep push now saves a comment/structure-preserving, value-redacted copy
of the source .env next to the encrypted secrets, so context like "why
this exists" or "get this from X" survives ingest instead of being
silently dropped. keep pull --template retrieves it. Old vaults pushed
before this existed keep working (flat payload, no template) and fail
with a clear error rather than crashing if --template is requested.
2026-07-15 20:41:45 +02:00
Fredrik Johansson
bb7f890a68 keep overview: print which server it's talking to
All checks were successful
Docker / build-and-push (push) Successful in 1m53s
Surfaced by a real mix-up this session: KEEP_SERVER_URL was set to a
"dev"-named hostname, and it was unclear without checking DNS/proxy
config whether that was actually the same server as the one running
on the deploy VPS or a separate instance -- nothing in `overview`'s
output (vault names, recipient labels) says which server answered.
Defaults silently to localhost:3050 if the env var isn't set at all,
which is its own way to end up looking at the wrong thing.

Printed before the fetch, not after, so it still shows up on a failed
request ("error: fetch failed") -- the exact case where knowing the
target mattered most and, before this, showed nothing at all.

Verified against a local test server: shows the right URL on success,
and shows the (default, unset) URL immediately before a deliberately
failed fetch to a port nothing's listening on.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 16:25:40 +02:00
Fredrik Johansson
74122b25cd deploy.sh: resolve vault key via .keep-vault marker, never fail on keep
Was previously hardcoded to vault key "<project-dir-name>/<env>" with
no marker-file lookup at all -- a real gap versus the design already
written out in PROPOSAL-announce-and-agent.md §4, which this script
never actually caught up to. Repo names, deploy directory names, and
vault keys are three independent strings in practice (a repo can be
named one thing, its deploy directory another, entirely by history/
accident) -- exactly the situation the npm-statuspage rollout hit.

Resolution order: $dir/.keep-vault if present, else fall back to
"<project>/<env>". `keep pull` is now fully best-effort -- missing
vault, missing grant, or keep not installed at all is never fatal,
matches the proposal's explicit requirement that a keep failure means
"doesn't apply to this project (yet)," never "stop the deploy."
Existing/hand-copied .env is left alone if the pull doesn't happen.

Verified with stub keep/docker commands on PATH (no real server
needed for this): a marker-file project with a deliberately mismatched
vault key resolves and pulls correctly; a project with neither a
marker nor a matching vault falls through cleanly to docker compose up
-d without ever touching .env or failing the deploy.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 16:12:57 +02:00
Fredrik Johansson
1b4cd9b826 Fix deploy-cli.sh: bundle was completely broken on a real host
Caught live, deployed to the actual target -- the previous version
failed on the real host with "Cannot use import statement outside a
module" (Node 18, no ancestor package.json to signal ESM for an
extensionless file). That surfaced two stacked bugs my own testing had
missed by running the bundle from inside this repo's own directory
tree, which coincidentally supplied both things the standalone
artifact was silently depending on:

1. Module-format ambiguity: an extensionless file with no controlling
   package.json defaults to CommonJS on older Node (no auto-detection
   heuristic before recent versions). Fixed by naming the bundle
   output keep.mjs -- unconditionally ESM on any Node version,
   regardless of extension-less-file heuristics or nearby package.json.

2. The bigger one: crypto.ts's createRequire(import.meta.url) trick
   (needed because libsodium-wrappers' published ESM build follows a
   broken relative import) is opaque to esbuild's static bundler --
   it's a runtime-obtained require reference, not the literal `require`
   token esbuild's bundling recognizes. The previous "18kb bundle"
   never actually contained libsodium-wrappers at all; it silently
   relied on real node_modules being nearby on disk, which was only
   ever true by accident when testing from within this repo. On a
   clean host it threw "Cannot find module 'libsodium-wrappers'".

   Tried forcing static inlining via a literal require() call (esbuild
   does special-case that even in ESM-format output, unlike `import`,
   which is permanently pinned to the "import" resolution condition
   and can't be routed to the package's working CJS build no matter
   what --conditions/--main-fields/--alias combination is tried this
   was tried and confirmed exhaustively). That got real resolution and
   a real 1.7MB bundle, but broke at runtime with "No secure random
   number generator found" -- inlining the WASM engine breaks its own
   Node-crypto feature detection.

   Landed on: ship libsodium-wrappers (and its own dependency,
   libsodium) as real, unmodified package files alongside the ~18kb
   bundle, via a new copy-cli-deps.mjs step, rather than fighting
   further to force single-file inlining. ~1.6MB total, still one tar
   stream over SSH, still the entire deploy step.

deploy-cli.sh now tars dist-bundle/ (bundle + node_modules) instead of
catting a single file, extracts to ~/.keep-cli/ on the target, and
symlinks ~/bin/keep to the entry point inside it.

Verified this time in conditions that actually match the failure: full
isolation (mktemp'd HOME and install dir, no ancestor package.json, no
adjacent node_modules from this repo) for both `node keep.mjs` and
direct shebang execution, plus the exact tar/extract cycle
deploy-cli.sh performs.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 15:59:44 +02:00
Fredrik Johansson
74bee56c54 Add deploy-cli.sh: ship the CLI as a single bundled file, no git clone needed
The CLI only touches Node builtins and libsodium-wrappers (no native
addons like better-sqlite3, which the server needs but the CLI never
imports) -- confirmed by checking its actual import graph, not assumed.
That makes it a good candidate for a single-file esbuild bundle rather
than requiring a full git checkout + npm install on every machine that
needs to run `keep pull`.

deploy-cli.sh bundles src/cli (~18kb) and ships it straight to a host
over SSH -- HOST=user@vps ./deploy-cli.sh -- mirroring flit's
deploy-daemon.sh ssh-cat-chmod pattern rather than inventing a new
convention. No git, no npm install, no node_modules on the target at
all, just the one file plus a `node` runtime already there from
whatever else is deployed on that box.

Caught a real bug while building this: --banner:js="#!/usr/bin/env
node" duplicated the shebang esbuild already preserves from the source
file automatically, landing it on line 2 instead of line 1 -- Node
only special-cases a shebang on line 1, so the bundle threw a syntax
error on every invocation until the redundant banner flag was removed.

Verified the bundle for real, not just that it builds: ran `keep
identity init` / `identity show` against it directly and confirmed a
real Ed25519 keypair comes out the other end -- the specific thing
worth checking given libsodium-wrappers needed a createRequire
workaround for its own broken ESM packaging, and bundling could easily
have broken that differently.

Also documents both CLI-provisioning paths in INTEGRATION.md
(scripts/install-cli.sh for a machine you're fine cloning onto,
deploy-cli.sh for one you'd rather not) as a "getting keep onto a
machine" prerequisite section, ahead of the existing deploy-pattern
docs that all assume it's already there.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 15:45:31 +02:00
Fredrik Johansson
b74564fd9b Add a real global keep CLI instead of npm run cli --
npm run cli -- <args> only works from inside this repo's directory,
which is awkward everywhere but especially on a deploy host running
this alongside a dozen other projects. scripts/install-cli.sh builds
the CLI and symlinks dist/cli/index.js onto PATH (~/.local/bin by
default, no root/global npm install needed) as a real `keep` command,
runnable from anywhere.

Symlinked rather than copied, so a future `git pull && npm run build`
is the entire upgrade story -- no need to re-run the install script
after the first time.

Verified: keep --help and keep identity show both work correctly from
unrelated directories (/tmp, $HOME) after running the install script,
confirming no hidden cwd dependency.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 15:38:12 +02:00
Fredrik Johansson
442c3b7cf5 Add fleet setup template and a one-shot project bootstrap script
All checks were successful
Docker / build-and-push (push) Successful in 1m58s
SETUP.local.md.example is a fill-in-the-blanks runbook for rolling
keep out across real machines and projects (gitignored once copied to
SETUP.local.md — real vault keys/recipient ids/hostnames are fleet
topology, not something to commit). bootstrap-project.sh wraps push +
grant --read-only for one or more recipients into a single call for
onboarding a new project vault.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-12 21:21:55 +02:00
Fredrik Johansson
58d5d787dd Implement announce/watch: a deploy signal that reverses CI's reach
All checks were successful
Docker / build-and-push (push) Successful in 1m58s
CI announces a new image tag to keep (any grant is enough, read-only
included); each deploy target polls for it locally instead of CI
holding standing SSH access to production. keep stays a pure relay —
the tag lives outside the encrypted vault payload in its own table,
and keep never executes anything itself.

Adds vault_announcements, POST/GET /vaults/:key/announce, and the
`keep announce`/`keep watch` CLI commands, per
PROPOSAL-announce-and-agent.md (now marked implemented). Verified live:
a read-only identity announcing successfully and being logged, watch
picking up the tag on its first poll, and watch exiting nonzero for an
unknown/ungranted vault.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-12 21:12:55 +02:00
Fredrik Johansson
674e408f09 Reject keep-executes-locally, switch to a .keep-vault marker file
Two changes based on further review:

- Explicitly considers and rejects letting keep's server execute the
  redeploy directly on announce, even though co-location removes the
  credential/network objection that sank the CI-SSHes-in alternative.
  The reason it's still wrong: a fully compromised keep server today
  can't do anything worse than leak ciphertext, since it never holds a
  decryption key. Giving it exec capability breaks that property — a
  routine web app bug becomes "triggers arbitrary redeploys," not
  "leaks encrypted blobs." Also notes the concrete cost: doing this
  from inside a container needs the Docker socket mounted in, which is
  broadly equivalent to root on the host.

- Replaces the "vault key = deploy directory name" assumption with an
  explicit .keep-vault marker file per project directory. Repo name,
  deploy directory name, and vault key are three independent strings
  in practice (confirmed by a real example already in this fleet) —
  no naming convention should assume any two of them match. Bootstrap
  is now: choose a vault key, keep push it, drop a one-line
  .keep-vault file. CI's announce step uses the same deliberately
  hardcoded key, never derived from the repo name automatically.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-12 21:07:08 +02:00
Fredrik Johansson
114f234cd3 Revise announce+agent proposal against the real deploy script
The original draft speced a hypothetical per-project agent.sh on a
per-project cron. Reality turned out different: there's already a
production bulk deploy script on the host (fixed project array,
docker compose up -d per project, no explicit pull — freshness comes
entirely from pull_policy: always) — so this patches that existing
loop in place instead of introducing a parallel per-project script
that would duplicate its array/failure-tracking logic.

Also folds in a real prerequisite gap found while reconciling the
proposal with the actual script: pull_policy: always isn't universal
across the fleet yet, independent of keep entirely — the bulk script
can't redeploy anything missing that line regardless of keep adoption.
One instance of this was already fixed this session (a sibling
project's docker-compose.yml missed the line when copying another
project's compose pattern).

Reframes §5's announce/watch benefit accordingly: since the real
script is one all-or-nothing sweep, not N independent watchers, the
natural fit is waking the whole sweep early (watch-any), not
per-project selective redeploy — still deliberately left unbuilt
until there's a real signal it's worth it over the already-scheduled
sweep.

Project names anonymized per policy — generic proj1..proj12
placeholders instead of the real fleet.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-12 20:50:15 +02:00
Fredrik Johansson
e505fbf9b4 Add proposal: announce + self-deploying agent, with a no-keep fallback
Design doc for closing the CI->deploy gap without reintroducing the
credential-concentration problem a naive "CI SSHes into the VPS"
approach would bring back. keep stays a pure relay — never executes
anything remotely. CI announces a new tag (a distinct verb from push/
pull, since a tag isn't a secret and doesn't belong in the encrypted
vault payload); each VPS polls and redeploys itself locally, using an
identity and grant it already holds.

Explicitly specs the not-yet-bootstrapped fallback: the agent script's
`keep pull` is allowed to fail for any reason, and on failure just runs
a plain `docker compose pull && up -d` against whatever's already on
disk — the same thing that would have happened before keep existed.
Ships the dumb-periodic-timer version of the agent first; the watch/
announce fast path is speced but deliberately left unbuilt until
there's a real signal that polling latency is worth the complexity.

Not implemented — design stage.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-12 20:43:51 +02:00
Fredrik Johansson
5f31eb76c7 overview: show when each vault was last updated, and by whom
Was silently dropping updatedAt/updatedBy even though the server
already returned them — the CLI just never printed them. Also
resolves updatedBy to the recipient's label server-side (matching how
grants already show a label alongside the recipient id) instead of
printing a bare recipient_id.

Verified against a live server: overview now prints "demo/production
(updated <iso timestamp> by test-box)" instead of omitting that line
entirely.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-12 20:30:02 +02:00
21 changed files with 1014 additions and 39 deletions

View File

@@ -1,6 +1,9 @@
PORT=3050
DATA_DIR=./data
# Image to run via docker compose, e.g. repo.example.com/owner/keep:latest
IMAGE=
# Gates recipient add/remove and revoke. Unset disables /api/admin/* (503)
# but push/pull/grant keep working for already-registered recipients.
ADMIN_PASSWORD=change-me

6
.gitignore vendored
View File

@@ -1,4 +1,10 @@
node_modules/
dist/
dist-bundle/
.env
data/
# Personal rollout notes — real vault keys, recipient ids, VPS hostnames.
# Not secrets themselves, but fleet topology that doesn't belong in a
# public repo. See SETUP.local.md.example for the template.
SETUP.local.md

View File

@@ -183,6 +183,8 @@ keep push <vault> [--file .env] [--purge] -- encrypt + upload; requires a WRITE
keep pull <vault> [--format env|json] [--out <path>] [--previous] -- fetch + decrypt; requires a grant
keep grant <vault> <recipient-id> [--read-only] -- add a recipient to a vault you have WRITE access to
keep log <vault> -- tail the access log for one vault
keep announce <vault> --tag <tag> -- record a deploy tag; requires ANY grant on the vault (read-only is enough)
keep watch <vault> [--interval 60] -- poll for a new announced tag; exits nonzero if the vault doesn't exist / no grant
# admin operations — gated by ADMIN_PASSWORD, see below
keep recipient add --label "..." --pubkey <hex>
@@ -215,6 +217,75 @@ Went with (2) — a global admin password is an acceptable trust model for
"one person or small team running their own homelab," which is the
actual scale this is built for.
## Announce/watch — a deploy signal, not a secret
CI producing a fresh image on every push still left deployment as a
manual step, and the obvious fix — CI SSHes into the VPS and redeploys —
was rejected: it means a standing SSH credential with production-deploy
rights sits in CI secrets, reachable by whatever's executing on a shared
runner across every project it builds. A compromised build step in one
project could then reach every other project's deploy target, which is
a real regression from `keep`'s own "read no longer implies write"
posture (see above).
The fix that doesn't reintroduce that problem: reverse the direction.
CI never reaches into a VPS — it tells `keep` "there's a new image for
this project" (`keep announce <vault> --tag <tag>`), and each VPS, which
already has to run *something* locally to redeploy itself, polls `keep`
for that signal (`keep watch <vault>`) and acts on it locally, using an
identity and grant it already legitimately holds. `keep`'s server never
executes anything remotely — it stays a pure relay, same posture as
everything else it does.
This was deliberately not built as "`keep` executes the redeploy
itself," even though `keep` typically runs on the same box as its
deploy targets and that would skip the polling indirection entirely.
Doing so would trade away the property that's been the throughline of
`keep`'s whole design: today, a fully compromised `keep` server — not
just a compromised client identity — can't do anything worse than leak
ciphertext and metadata, because the server never holds a decryption
key. The moment `keep` can also execute a command on `announce`, an
ordinary web app bug (not even a cryptographic break) becomes "attacker
triggers arbitrary redeploys" — a new class of bad outcome, not a bigger
version of an existing one.
An announced tag is stored outside the encrypted vault payload
(`vault_announcements`, one row per vault — latest known state, not a
history, same shape as `vaults.prev_ciphertext`) since an image tag
isn't a secret and doesn't belong inside a secrets blob alongside real
credentials. `POST /vaults/:key/announce` only requires *a* grant, not a
write grant — `hasGrant`, not `hasWriteGrant` — since announcing isn't a
mutation of a vault's secret content, just a signal; a CI identity for a
project can stay read-only even with this feature in play. `keep watch`
is deliberately dumb polling, not a websocket or long-poll — matches the
pattern already used elsewhere in this project family, and a 60-second
detection lag for a deploy is an acceptable tradeoff against the
complexity of push-based notification.
Two things worth stating plainly about what this does and doesn't
change:
- The tag itself is now server-visible, where previously not even the
fact that a new image existed was something `keep` needed to know.
Accepted tradeoff — a git short-SHA isn't sensitive, and the feature
only works if `keep` can see it.
- A compromised read-only CI identity could announce a *false* tag,
pointing a deploy target at an attacker-chosen image. `keep` never
pulls or runs that image itself, only relays the string — the actual
`docker compose pull` on the deploy target still pulls from whatever
registry it's configured for, using the tag it was told. This is
equivalent in severity to that CI identity's build step being
compromised in the first place (it could already push a bad image
under a legitimate tag), not a new capability.
A bulk multi-project deploy script (looping over a fixed project list,
`docker compose up -d` per project) adopts this by checking for a
per-project `.keep-vault` marker file before calling `keep pull`; a
missing marker falls through to exactly what the script did before
`keep` existed — never a hard failure, since not every project will be
bootstrapped onto `keep` at the same time. `scripts/deploy.sh` in this
repo is the single-project version of the same pattern.
## Deploy integration
The actual motivating use case — a deploy script pulling secrets instead

View File

@@ -31,6 +31,60 @@ A deploy identity should almost always be granted **read-only** access
needs to `pull`, and a read-only grant means a compromised deploy box
can't also rotate the vault or add itself more recipients.
When you do the first `keep push` for a project, push the *real*,
hand-maintained `.env` rather than a stripped-down one — any `# get this
from X` / `# rotate quarterly` comments in it get preserved (values
redacted) as a template alongside the secrets, retrievable later with
`keep pull <vault> --template`. That's the one moment this context is
available at all; a `.env` reconstructed later from memory won't have it.
## Getting the `keep` CLI onto a machine
Chicken-and-egg: every pattern below assumes `keep` is already a runnable
command on the machine doing the pulling. Two ways to get there, pick
based on whether you're fine with a git checkout existing on that
machine:
**A machine you're already developing on, or don't mind cloning onto**
(your laptop, a VPS you administer directly):
```bash
git clone <this repo>
cd keep
npm install
./scripts/install-cli.sh # builds, symlinks dist/cli/index.js onto PATH as `keep`
```
Symlinked rather than copied — a future `git pull && npm run build` is
the entire upgrade story, no need to re-run the install script.
**A deploy target you'd rather not clone a git repo onto** — bundle the
CLI and ship it over SSH, the same way flit/waste-go's `deploy-*.sh`
scripts ship a compiled binary:
```bash
HOST=user@your-vps ./deploy-cli.sh
```
Not quite a single file: the bundle itself (esbuild, ~18kb) covers
everything except `libsodium-wrappers`, which ships alongside it as the
real, unmodified package (~1.6MB total with its own dependency,
`libsodium`) rather than being force-inlined. That package's published
ESM build follows a broken relative import (see `src/shared/crypto.ts`'s
own comment on this), and statically inlining it via esbuild — technically
possible by routing through a literal `require()` call instead of
`import` — breaks its WASM engine's own Node-crypto feature detection at
runtime ("No secure random number generator found"), confirmed by
testing, not assumed. Shipping the real package is more reliable than
fighting that.
The script tars `dist-bundle/` (the esbuild output plus that one
dependency) and extracts it to `~/.keep-cli/` on the target over SSH,
then symlinks `~/bin/keep` to the entry point inside it — no git, no npm
install on that machine at all, just a `node` runtime (already there, if
it's running any of the other Node-based projects in this family). Re-run
it any time the CLI changes; safe to overwrite.
## Pattern: several docker-compose projects on one or more VPSes
The common shape once more than one or two projects are on `keep`:

View File

@@ -0,0 +1,369 @@
# Proposal: announce + a self-deploying agent, with a no-`keep` fallback
**Status:** implemented (§1§3, §6 shape). `announce`/`watch`, the
`vault_announcements` table, and the API routes described below are
built — see [IMPLEMENTATION.md](./IMPLEMENTATION.md)'s "Announce/watch"
section for the as-built version of the design rationale, and the CLI
surface list there for the exact commands. §4 (patching the real bulk
deploy script) and §6 (the CI workflow step) are per-project changes
made outside this repo, on the host and in each project's own
`.gitea/workflows/docker.yml` — not something this repo's code can
contain. This file is kept as the original design record, superseded by
IMPLEMENTATION.md as the source of truth for what's actually built.
## Context (read this first)
Full background on `keep` is in [README.md](./README.md) and
[IMPLEMENTATION.md](./IMPLEMENTATION.md) — read both before touching
code. The relevant existing pieces for this proposal:
- `src/server/db.ts` — SQLite (better-sqlite3, WAL mode). `vaults`,
`recipients`, `vault_grants`, `vault_grants_previous`, `access_log`.
- `src/server/routes.ts` — recipient-authenticated (signed-request)
routes: `push`, `pull`, `grant`, `log`, plus `exists` and
`recipients` (both used by the CLI to compute who to re-wrap for).
- `src/server/auth.ts``requireRecipientAuth` (Ed25519 signature over
method+path+timestamp+body-hash, verified against `req.originalUrl`
with the query string stripped — see IMPLEMENTATION.md's "What was
verified end-to-end" for a bug that already bit this exact mechanism
once) and `requireAdminAuth` (`X-Admin-Password` header).
- `src/cli/``identity.ts`, `recipient.ts`, `vault.ts`,
`signedFetch.ts` (note: `signedFetch`'s `path` param must never
include a query string — see the comment there for why).
- `scripts/deploy.sh` (this repo) — pulls `<project>/<env>` from `keep`
into `$DEPLOY_ROOT/<project>/.env`, then `docker compose up -d` for
one project. Assumes the vault already exists and this machine
already has a grant on it. **Superseded by the real deploy script
below as the thing this proposal actually patches** — kept in this
repo as a single-project example/fallback, not the production path.
- **The real, already-running deploy script — on the host, not in any
repo.** A fixed-array bulk script, already in production use, looping
over roughly a dozen project directories:
```bash
ROOT="$(cd "$(dirname "$0")" && pwd)"
PROJECTS=(proj1 proj2 proj3 proj4 proj5 proj6 proj7 proj8 proj9 proj10 proj11 proj12)
for project in "${PROJECTS[@]}"; do
dir="$ROOT/$project"
[ -f "$dir/docker-compose.yml" ] || { echo "skip: $project"; continue; }
(cd "$dir" && docker compose up -d) || FAILED+=("$project")
done
docker image prune -f
```
Two things about it that matter for this proposal: **it never runs
`docker compose pull`** — freshness comes entirely from each
project's own `pull_policy: always` in its `docker-compose.yml`
(`up -d` re-pulls implicitly). And **it loops over a fixed, explicit
array**, not directory auto-discovery — adding a project means
editing this array by hand. This is the actual thing to patch, not a
new per-project script; see Design §4.
- **Prerequisite gap, found while writing this proposal, not yet fixed
everywhere**: `pull_policy: always` isn't universal across the
fleet — confirmed present in roughly half the projects checked,
confirmed missing in several others (one of which was fixed in this
session — a sibling project's `docker-compose.yml` didn't have it, a
regression from copying another project's compose file without
carrying this line over), and unchecked for the rest (no local
checkout available at proposal time). **This is independent of
`keep` entirely** — the bulk script can't redeploy any project
missing this line regardless of whether that project ever adopts
`keep`, and it should be swept fleet-wide before or alongside this
work, not as part of it.
- `.gitea/workflows/docker.yml` (present in `keep` and other sibling
projects) — builds and pushes an image to the Gitea container
registry on every push to `main`. Does **not** currently deploy
anything — that's the manual/scheduled bulk script above today.
## Motivation
The CI workflow already produces a fresh image on every push. Deployment
is still a manual "SSH in and run the deploy script" step. The obvious
fix — CI SSHes into the VPS and runs the deploy — was considered and
rejected: it means a standing SSH credential with production-deploy
rights sits in Gitea's CI secrets, reachable by whatever's executing on
the shared runner across every project it builds. A compromised build
step in any one project could then reach every other project's deploy
target. That's a real regression from where `keep` already left things
(see IMPLEMENTATION.md's resolved "read no longer implies write" — the
whole point was *not* concentrating reach into one identity).
The fix that doesn't reintroduce that problem: **reverse the direction.**
CI never reaches into a VPS. Instead, CI tells `keep` "there's a new
image for this project," and each VPS — which already has to run
*something* locally to redeploy itself — polls `keep` for that signal
and acts on it locally, using an identity and grant it already legitimately
holds. `keep`'s server never executes anything remotely; it stays a pure
relay, same posture as everything else it does.
**Not every project will have been migrated to `keep` yet.** Bootstrapping
a project (register a recipient, grant it, push the first vault) is a
real, deliberate step per project, and there'll be a period — maybe a
long one — where some projects have it and some don't. The redeploy path
has to work identically well for both, defaulting to exactly what
happens today (a plain `docker compose up -d`, relying on that project's
own `pull_policy: always`) for anything not yet bootstrapped, rather than
either blocking deploys entirely or hard-failing on missing vaults.
### Considered and rejected: `keep` executes the redeploy itself
Since `keep`'s own server will run on the same VPS as the deploy targets,
it's tempting to skip the polling indirection entirely and have `keep`
shell out to the deploy script directly when it receives an `announce` —
no new SSH credential, no new network-reachable pathway, since nothing
is reaching *into* the box that wasn't already running on it.
Rejected anyway, because it trades away a property that's been the
throughline of `keep`'s whole design: **today, a fully compromised
`keep` server — not just a compromised client identity — can't do
anything worse than leak ciphertext and metadata**, because the server
never holds a decryption key. The moment `keep` can also execute a
command on `announce`, that stops being true — a bug in the HTTP layer
(not even a cryptographic break; an ordinary web app bug, an auth check
wrong on some edge case) becomes "attacker triggers arbitrary
redeploys," not "attacker sees encrypted blobs they can't read." That's
a new class of bad outcome, not a bigger version of an existing one, and
it's not worth trading for skipping a polling loop.
There's also a concrete operational cost, not just a principle: actually
running `docker compose up -d` from inside `keep`'s own process would
need either the Docker socket mounted into `keep`'s container
(`/var/run/docker.sock` — widely treated as equivalent to handing that
container root on the host) or running `keep` outside a container
entirely, throwing away the isolation its own `Dockerfile` already
provides today.
`keep` stays a pure relay regardless of where it's deployed. Something
*outside* `keep`'s own process boundary does the polling and the
executing — see §4.
## Design
### 1. `announce` — a distinct verb from `push`/`pull`
An image tag isn't a secret. It doesn't belong inside the encrypted vault
payload alongside real credentials — that's a category mismatch (deploy
metadata living inside a secrets blob), not just an implementation
convenience either way. `announce` is its own small mechanism:
```sql
CREATE TABLE IF NOT EXISTS vault_announcements (
vault_key TEXT PRIMARY KEY,
tag TEXT NOT NULL,
announced_at INTEGER NOT NULL,
announced_by TEXT NOT NULL -- recipient_id
);
```
One row per vault — like the vault's own one-previous-version retention,
this is "latest known state," not a history. If a real audit trail of
every announce is wanted later, `access_log` already records the action;
this table is just the fast-path "what's the current tag" read, kept
separate from the log on purpose (reading current state back out of an
append-only log is the wrong tool for that job — same reasoning
`IMPLEMENTATION.md` already gives for why `vault_grants_previous` is a
real table and not a log replay).
### 2. API
```
POST /api/vaults/:key/announce -- { tag: string } — requires ANY grant on the vault (read-only is enough; announcing isn't a secrets mutation)
GET /api/vaults/:key/announce -- { tag, announcedAt, announcedBy } | 404 if nothing announced yet — requires a grant, same as pull
```
Both go through `requireRecipientAuth`, identical shape to the existing
routes. `POST` only requires *a* grant (not write) — reuse `hasGrant`,
not `hasWriteGrant` — since this isn't a mutation of the vault's secret
content, just a signal, and CI's identity for a project should be able to
stay read-only even after this feature exists. (This is worth stating
explicitly since a naive implementation might reach for `hasWriteGrant`
by analogy with `push` — resist that; it would silently widen CI's
required permissions for no real reason.)
### 3. CLI
```
keep announce <vault> --tag <tag> -- POST, requires a grant on <vault>
keep watch <vault> [--interval 60] -- polls GET .../announce every N seconds, prints the tag on change, exits nonzero if the vault doesn't exist / no grant (see fallback below)
```
`keep watch` is deliberately dumb — plain polling, not a websocket or
long-poll. Matches the polling pattern already used elsewhere in this
project family, and a 60-second detection lag for a deploy is fine.
Don't build push-based notification for this; it's more machinery for a
problem polling already solves adequately.
### 4. Patch the existing loop — don't build a parallel script
This is the part the fallback requirement is actually about. The real
deploy script already does the one thing that matters (a loop over a
fixed project list, `docker compose up -d` per project, relying on
`pull_policy: always` for freshness) — the change is inserting one
`keep`-with-fallback step per iteration, in place, not writing a new
per-project `agent.sh` that duplicates the loop/array/failure-tracking
logic the real script already has:
```bash
for project in "${PROJECTS[@]}"; do
dir="$ROOT/$project"
[ -f "$dir/docker-compose.yml" ] || { echo "skip: $project"; continue; }
echo ""
echo "▶ $project"
# A vault key is NOT assumed to match the deploy-directory name — repo
# names, deploy directory names, and vault keys are three independent
# strings in practice (a repo can be named one thing, its deploy
# directory another, entirely by history/accident). .keep-vault is an
# explicit, one-time-per-project marker: if it's missing, this project
# was never bootstrapped onto keep (or never will be — a static site
# with no secrets), and the loop falls through to exactly what it did
# before keep existed. Every keep call here is allowed to fail;
# failure means "keep doesn't apply to this project," never "stop the
# deploy."
if [ -f "$dir/.keep-vault" ]; then
vault="$(cat "$dir/.keep-vault")"
if keep pull "$vault" --out "$dir/.env.new" 2>/dev/null; then
mv "$dir/.env.new" "$dir/.env"
else
rm -f "$dir/.env.new"
fi
fi
if (cd "$dir" && docker compose up -d 2>&1); then
echo " ✓ $project up"
else
echo " ✗ $project failed"
FAILED+=("$project")
fi
done
```
Bootstrapping a project onto `keep` becomes: choose a vault key (no
naming constraint — doesn't need to match the repo or the directory),
`keep push` it, grant whichever recipients need it, and
`echo "<the vault key>" > "$dir/.keep-vault"`. One file, human-readable,
`cat`-able to check what's configured, no second array in this script to
keep in sync by hand as directories get renamed or reorganized.
Notice `announce`/`watch` aren't actually load-bearing in this version —
whatever already triggers this script (cron, systemd timer, or still
run by hand) keeps triggering it exactly as before, and `keep pull`
either updates `.env` or silently doesn't. That's deliberate: **the
existing script, patched this way, is already a complete, correct
baseline** — keep-bootstrapped projects get fresh secrets on every run,
everything else behaves identically to today. `announce` sits on top as
a latency optimization *for however this script gets triggered*, not a
requirement for this patch to be complete — see the next section.
### 5. Where `announce`/`watch` actually earn their keep
Two independent benefits, worth keeping conceptually separate — and the
second one looks different than it would for a one-agent-per-project
design, because the real script is one bulk all-or-nothing sweep, not N
independent watchers:
- **Secrets sourcing** — `keep pull` instead of a hand-placed `.env`.
Available per-project the moment its vault exists and this machine
has a grant on it. This is what §4's patch already delivers, on its
own, with zero dependency on `announce`.
- **Waking the sweep early** — however the bulk script currently gets
triggered (cron, systemd timer, or still by hand), `announce` could
let it wake immediately on a push instead of waiting for the next
scheduled run. The natural fit given the script's actual shape: `keep
watch-any <vault1> <vault2> ...` blocking until *any* watched vault's
tag changes, used as the trigger for a full sweep — not a per-project
selective redeploy. Selective ("only redeploy the one project that
actually changed") would require reshaping the script away from its
current fixed-array-sweep model into something that takes a target
project as an argument, which is a real behavior change to something
already in production use, not a small addition — out of scope here.
Deliberately **not** building the wake-the-sweep-early mechanism in this
pass. Ship §4's patch first — it already satisfies the fallback
requirement completely and delivers the secrets-sourcing benefit on its
own. Whether waiting up to one scheduled interval for a redeploy is
actually a problem worth solving with `announce`/`watch-any` should be
judged after living with the patched bulk script for a while, not
designed in advance of any real signal that the wait matters.
### 6. CI workflow addition
One step appended to the existing `docker.yml`, after the existing
build-and-push step, best-effort (must not fail the build if this
project hasn't been bootstrapped onto `keep` yet):
```yaml
- name: announce to keep
continue-on-error: true
run: keep announce myapp/production --tag ${{ steps.meta.outputs.short_sha }}
```
The vault key here (`myapp/production`) is a **literal, hardcoded string
in this specific repo's own workflow file** — never derived from
`${{ github.repository }}` or any other automatic source. Same reasoning
as §4's `.keep-vault` file: the repo's own name, its deploy directory's
name, and its vault key are three independent strings that happen to
often look similar and are not guaranteed to match (see Context — a real
example already exists in this fleet). Whoever bootstraps a project onto
`keep` should paste the same deliberately-chosen vault key into both this
workflow step and that project's `.keep-vault` file, not rely on either
side inferring it from something else.
`continue-on-error: true` is load-bearing, not incidental — a project
without a `keep` identity registered yet must still build and push its
image successfully. This step failing should be silent/expected for any
not-yet-bootstrapped project, not a red X on the workflow.
## Security considerations
- `announce` leaks a little more than `push`/`pull` did: the *tag*
itself is now server-visible (previously, if you were being strict,
even the fact that a new image existed wasn't something `keep` needed
to know). This is an intentional, accepted tradeoff — a git short-SHA
or version tag isn't sensitive, and the whole feature only works if
`keep` can see it.
- Requiring only a read grant (not write) to `announce` means a
compromised read-only CI identity could announce a *false* tag,
pointing the fleet at an attacker-chosen image — worth flagging even
though `keep` never pulls or runs that image itself, only relays the
string. The actual `docker compose pull` on each VPS still pulls from
the configured registry using the tag it was told, so this is
equivalent in severity to that CI identity's build step being
compromised in the first place (it could already push a bad image
under a legitimate tag) — not a new capability, just worth stating
plainly rather than leaving implicit.
- The fallback path must never silently downgrade a *bootstrapped*
project to running with a stale `.env` because of a transient `keep`
outage. As written, `keep pull` failing for any reason (network
blip, `keep` server down, not just "not bootstrapped") falls through
to "deploy with whatever `.env` is already on disk" — for an
already-bootstrapped project, that's the *previous* `keep pull`'s
output, still valid, not a downgrade. This only becomes a real problem
if a bootstrapped project's very first deploy attempt coincides with
`keep` being unreachable, in which case there's no `.env` on disk yet
at all and `docker compose up` will fail for the mundane reason that
required env vars are missing — a loud, obvious failure, not a silent
wrong-secrets one.
## Open questions
- **`.keep-vault` format** — a bare vault-key string is enough for
today's one-environment-per-directory reality. If a directory ever
needs to deploy more than one environment from the same
`docker-compose.yml` (unlikely given the current fleet shape, but not
impossible), a bare string stops being enough and it'd need to become
a small key=value file or similar. Not designing that ahead of an
actual need — ship the bare-string version.
- **Fleet-wide `pull_policy: always` sweep** — a prerequisite this
proposal depends on but doesn't itself fix (see Context). Should
happen before or alongside this work, as its own small pass, not
bundled into this one.
- **Does `watch`/`watch-any` ever get used**, or does the
already-scheduled bulk sweep turn out to be good enough forever?
Deliberately left unbuilt (see §5) until there's a real signal the
wait between scheduled runs is worth solving.
- **Multi-VPS fan-out for one project** — if a project ever runs on more
than one box, `announce` already supports that for free (any number of
watchers can poll the same vault's announce endpoint), but this hasn't
been exercised and might reveal something not accounted for here.

View File

@@ -91,24 +91,53 @@ cp .env.example .env
npm install
npm run dev:server
# 2. Each machine/person that needs access generates its own identity
npm run cli -- identity init
# 2. Build the CLI and link it onto PATH as a real `keep` command --
# no npm run cli --, no cd-ing into this repo from wherever you
# actually need it (a deploy directory on some VPS, for instance).
# Symlinked, not copied: a future `git pull && npm run build` here
# is the entire upgrade story, no need to re-run this.
./scripts/install-cli.sh
# 3. Each machine/person that needs access generates its own identity
keep identity init
# → prints a public key
# 3. An admin registers that public key as a recipient
KEEP_ADMIN_PASSWORD=... npm run cli -- recipient add --label "my-laptop" --pubkey <hex>
# 4. An admin registers that public key as a recipient
KEEP_ADMIN_PASSWORD=... keep recipient add --label "my-laptop" --pubkey <hex>
# → prints a recipient id
# 4. The machine that generated the identity records its assigned id
npm run cli -- identity set-id <recipient-id>
# 5. The machine that generated the identity records its assigned id
keep identity set-id <recipient-id>
# 5. Push a vault — the pusher is automatically its first recipient
npm run cli -- push myapp/production --file .env.production
# 6. Push a vault — the pusher is automatically its first recipient
keep push myapp/production --file .env.production
# 6. Anyone else with a grant can pull it, decrypted, ready to use
npm run cli -- pull myapp/production > .env
# 7. Anyone else with a grant can pull it, decrypted, ready to use
keep pull myapp/production > .env
```
## Templates
`keep push` also saves a redacted copy of the source `.env` alongside
the secrets — same keys, comments, and blank lines, values stripped.
Comments explaining *where a value comes from* or *why it exists* are
exactly the kind of context a bare key/value pair throws away, so this
keeps it without keeping the secret itself around unencrypted anywhere.
A plain `keep pull` uses that template by default: if the vault has one,
you get real values filled back into it, comments and all — not just a
bare key/value dump. Ask for the redacted structure itself with:
```bash
keep pull myapp/production --template > .env.example
```
Vaults pushed before this existed just don't have a saved template —
`keep pull` on one of those falls back to a plain key/value dump, and
`keep pull --template` fails with a clear error instead of silently
returning nothing. No migration needed; push again and the template
gets backfilled from whatever `.env` you push next.
## Granting, revoking, rotating
```bash
@@ -141,6 +170,29 @@ Pushing to an existing vault requires a *write* grant — a read-only
recipient can pull but can't push or grant others access. A brand-new
vault's first push is always read-write for the pusher.
## Announcing deploys
`keep` can also relay a small "there's a new image" signal alongside a
vault, so CI never needs standing SSH access to a deploy target — CI
announces a tag, and whatever already redeploys that box locally polls
for it instead of being reached into from outside:
```bash
# CI, after a successful build/push — any grant is enough, read-only included:
keep announce myapp/production --tag <short-sha>
# The deploy target polls for a change and prints it:
keep watch myapp/production --interval 60
```
An announced tag isn't a secret — it lives outside the encrypted vault
payload, in its own small table, and is visible to the server (same as
vault names and access timestamps already are). `keep` never pulls or
runs the announced image itself; it only relays the string. See
[IMPLEMENTATION.md](./IMPLEMENTATION.md) for how this fits into a bulk
deploy script with a no-`keep` fallback for projects not yet bootstrapped
onto it.
## Docker
```bash
@@ -177,12 +229,16 @@ CLI-side: `KEEP_SERVER_URL` (default `http://localhost:3050`),
Implemented: identity, push/pull/grant/revoke/log, read/write grant
scoping, one-step rollback with an explicit purge mode for
compromise-driven rotations, admin recipient management, Docker deploy.
Verified end-to-end — two and three independent identities, read-only
grants correctly blocked from push/grant, grants preserved across
rotation, previous-version pull working and correctly wiped by
`--purge` for every recipient, malformed-auth rejection — against both
a local server and the built Docker image. See
compromise-driven rotations, admin recipient management, an admin
cross-vault `overview`, deploy-signal `announce`/`watch`, a
`scripts/deploy.sh` wrapper, and Docker deploy. Verified end-to-end —
two and three independent identities, read-only grants correctly
blocked from push/grant, grants preserved across rotation,
previous-version pull working and correctly wiped by `--purge` for
every recipient, a read-only identity successfully announcing while
still being logged, `watch` picking up an announced tag and exiting
nonzero for an unknown/ungranted vault, malformed-auth rejection —
against both a local server and the built Docker image. See
[IMPLEMENTATION.md](./IMPLEMENTATION.md) for the full design and its
resolved decisions (per-secret-key granularity deliberately not built —
use multiple vaults instead).

80
SETUP.local.md.example Normal file
View File

@@ -0,0 +1,80 @@
# keep fleet setup — personal notes (copy to SETUP.local.md, gitignored)
Fill this in as you actually run the commands. It's your own record of
what recipient id belongs to which machine and which vault backs which
project — none of this is secret by itself (a recipient id or a vault
key doesn't grant access without the matching private key), but it's
real fleet topology that doesn't belong in a public repo.
## 0. Server
- [ ] Deployed at: `___________________` (e.g. https://keep.example.internal)
- [ ] `ADMIN_PASSWORD` stored in: `___________________` (password manager entry, not here)
- [ ] `KEEP_SERVER_URL` exported in shell profile on every machine below
## 1. Identities
One row per machine. `keep identity init` prints the public key; an
admin runs `keep recipient add`, which prints back a recipient id; that
id gets recorded locally with `keep identity set-id <id>`.
| Machine | Recipient id | Grant type (per vault, see §3) |
|---|---|---|
| this laptop | `___________________` | rw on everything it pushes |
| vps-1 (`___________________` hostname) | `___________________` | r- (read-only) |
| vps-2 | `___________________` | r- (read-only) |
Commands, run once per machine:
```bash
keep identity init # on the machine itself
KEEP_ADMIN_PASSWORD=... keep recipient add --label "<name>" --pubkey <hex> # from an admin machine
keep identity set-id <recipient-id> # back on the machine itself
```
## 2. Projects → vaults
One vault per project+environment. Vault key is a free-form string —
it does NOT need to match the repo name or the deploy directory name
(see IMPLEMENTATION.md — these are three independent strings in this
fleet on purpose).
| Project | Deploy dir | Vault key | `.keep-vault` written? |
|---|---|---|---|
| myapp | `/opt/docker/myapp` | `myapp/production` | [ ] |
| otherapp | `/opt/docker/otherapp` | `otherapp/production` | [ ] |
Bootstrap a project:
```bash
keep push myapp/production --file .env.production # from the laptop, becomes rw grantee
keep grant myapp/production <vps-1-recipient-id> --read-only
echo "myapp/production" > /opt/docker/myapp/.keep-vault # on vps-1, next to its docker-compose.yml
```
Or use `scripts/bootstrap-project.sh` (see below) to do the push+grant
step in one call for multiple VPS recipients at once.
## 3. CI announce (optional, per project)
Only needed if you want `keep watch` on a VPS to skip waiting for the
next scheduled sweep. Add to that project's own
`.gitea/workflows/docker.yml`, after the build-and-push step:
```yaml
- name: announce to keep
continue-on-error: true
run: keep announce myapp/production --tag ${{ steps.meta.outputs.short_sha }}
```
The vault key here must be the literal string from §2's table — not
derived from the repo name.
## 4. Sanity check
```bash
KEEP_ADMIN_PASSWORD=... keep overview
```
Confirm every project vault shows up with exactly the grants you
expect — laptop `rw`, each deploying VPS `r-`, nothing else.

38
deploy-cli.sh Executable file
View File

@@ -0,0 +1,38 @@
#!/usr/bin/env bash
# deploy-cli.sh — package the keep CLI (bundle + its one real runtime
# dependency, libsodium-wrappers) and ship it straight to a host over SSH.
# No git clone, no npm install on the target.
#
# Not a single file: libsodium-wrappers' published ESM build is broken
# (a relative import that doesn't resolve outside a bundler-aware
# context — see src/shared/crypto.ts), and statically force-inlining it
# via esbuild breaks its own Node-crypto feature-detection at runtime
# ("No secure random number generator found"). Shipping the real,
# unmodified package alongside a small (~18kb) bundle for everything
# else is more reliable than fighting that — the whole package is still
# only ~1.6MB and this script is still the entire deploy step.
#
# Usage: HOST=user@host ./deploy-cli.sh
set -euo pipefail
HOST="${HOST:?usage: HOST=user@host ./deploy-cli.sh}"
BIN_DIR="${BIN_DIR:-~/bin}"
INSTALL_DIR="${INSTALL_DIR:-~/.keep-cli}"
echo "→ bundling keep CLI…"
npm run build:cli-bundle
echo "→ packaging…"
tar -C dist-bundle -czf /tmp/keep-cli.tar.gz .
echo "→ uploading to $HOST:$INSTALL_DIR"
ssh "$HOST" "mkdir -p $INSTALL_DIR && tar -C $INSTALL_DIR -xzf -" < /tmp/keep-cli.tar.gz
rm /tmp/keep-cli.tar.gz
echo "→ linking $BIN_DIR/keep…"
ssh "$HOST" "mkdir -p $BIN_DIR && chmod +x $INSTALL_DIR/keep.mjs && ln -sf $INSTALL_DIR/keep.mjs $BIN_DIR/keep"
echo "→ verifying…"
ssh "$HOST" "$BIN_DIR/keep --help" | head -1
echo "✓ done — re-run this script any time the CLI changes; safe to overwrite."

1
package-lock.json generated
View File

@@ -21,6 +21,7 @@
"@types/express": "^4.17.21",
"@types/libsodium-wrappers": "^0.7.14",
"@types/node": "^22.0.0",
"esbuild": "^0.28.1",
"tsx": "^4.15.0",
"typescript": "^5.5.0"
}

View File

@@ -10,18 +10,20 @@
"dev:server": "tsx watch --env-file=.env src/server/index.ts",
"cli": "tsx src/cli/index.ts",
"build": "tsc",
"build:cli-bundle": "esbuild src/cli/index.ts --bundle --platform=node --format=esm --outfile=dist-bundle/keep.mjs --external:node:* --external:libsodium-wrappers && node scripts/copy-cli-deps.mjs",
"start": "node --env-file=.env dist/server/index.js"
},
"dependencies": {
"express": "^4.19.2",
"better-sqlite3": "^11.3.0",
"express": "^4.19.2",
"libsodium-wrappers": "^0.7.15"
},
"devDependencies": {
"@types/express": "^4.17.21",
"@types/node": "^22.0.0",
"@types/better-sqlite3": "^7.6.11",
"@types/express": "^4.17.21",
"@types/libsodium-wrappers": "^0.7.14",
"@types/node": "^22.0.0",
"esbuild": "^0.28.1",
"tsx": "^4.15.0",
"typescript": "^5.5.0"
}

31
scripts/bootstrap-project.sh Executable file
View File

@@ -0,0 +1,31 @@
#!/usr/bin/env bash
# Pushes a project's first env file and grants read-only access to one
# or more already-registered recipients (typically the VPS boxes that
# will deploy it), in one call.
#
# Usage:
# ./bootstrap-project.sh <vault> <env-file> <recipient-id> [recipient-id...]
#
# The pusher (this machine's identity) becomes the vault's first,
# read-write recipient automatically — see IMPLEMENTATION.md. Every
# recipient-id argument here is granted read-only; run `keep grant`
# by hand afterward for anyone who needs write access too.
set -euo pipefail
vault="${1:?usage: bootstrap-project.sh <vault> <env-file> <recipient-id> [recipient-id...]}"
file="${2:?usage: bootstrap-project.sh <vault> <env-file> <recipient-id> [recipient-id...]}"
shift 2
if [ "$#" -eq 0 ]; then
echo "error: at least one recipient id is required" >&2
exit 1
fi
keep push "$vault" --file "$file"
for recipient in "$@"; do
keep grant "$vault" "$recipient" --read-only
done
echo ""
echo "done. verify with: KEEP_ADMIN_PASSWORD=... keep overview"

32
scripts/copy-cli-deps.mjs Normal file
View File

@@ -0,0 +1,32 @@
// Copies the one real runtime dependency the CLI bundle can't statically
// inline (libsodium-wrappers, and its own dependency libsodium) next to
// the bundle, so dist-bundle/ is a small, complete, self-contained
// directory -- no npm install needed wherever it ends up.
//
// Why not just bundle it: libsodium-wrappers' published ESM build follows
// a relative import ("./libsodium.mjs") that's broken in that package's
// own dist layout -- the exact bug crypto.ts's own comment documents.
// Forcing static inlining via a literal require() (the only way to route
// esbuild's resolver to the *working* CJS build instead) was tried and
// technically resolved, but produced "No secure random number generator
// found" at runtime -- inlining the WASM engine breaks its own Node
// crypto feature-detection. Shipping the real, unmodified package files
// is more reliable than fighting that.
import { cpSync, mkdirSync, existsSync } from 'node:fs';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
const root = join(dirname(fileURLToPath(import.meta.url)), '..');
const outDir = join(root, 'dist-bundle', 'node_modules');
mkdirSync(outDir, { recursive: true });
for (const pkg of ['libsodium-wrappers', 'libsodium']) {
const src = join(root, 'node_modules', pkg);
if (!existsSync(src)) {
console.error(`missing node_modules/${pkg} -- run npm install first`);
process.exit(1);
}
cpSync(src, join(outDir, pkg), { recursive: true });
}
console.log('dist-bundle/node_modules populated (libsodium-wrappers, libsodium)');

View File

@@ -1,14 +1,29 @@
#!/usr/bin/env bash
# Pulls a project's env from keep, then brings its docker compose stack up.
# Pulls a project's env from keep (if it's bootstrapped onto keep), then
# brings its docker compose stack up.
#
# Usage:
# ./deploy.sh <project> [env]
#
# Expects, per project, a directory at $DEPLOY_ROOT/<project>/ containing
# a docker-compose.yml. Vault key pulled is "<project>/<env>" (env
# defaults to "production"). Requires a keep identity already registered
# and granted read access to that vault on this machine
# (see INTEGRATION.md).
# Expects a directory at $DEPLOY_ROOT/<project>/ containing a
# docker-compose.yml.
#
# The vault key is NOT assumed to match the deploy-directory name — repo
# names, deploy directory names, and vault keys are three independent
# strings in practice (a repo can be named one thing, its deploy
# directory another, entirely by history/accident). Resolution:
# 1. $dir/.keep-vault, if present — a one-time-per-project marker
# (`echo "<vault key>" > "$dir/.keep-vault"` when bootstrapping),
# one line, human-readable, cat-able to check what's configured.
# 2. Otherwise falls back to "<project>/<env>" — works when the vault
# key genuinely does match, and is also the graceful "not
# bootstrapped yet" case for a keep pull that's expected to fail.
#
# `keep pull` is always best-effort: a missing vault, a missing grant, or
# `keep` not being installed at all is never fatal here — it means "keep
# doesn't apply to this project (yet)," never "stop the deploy." Whatever
# .env already exists on disk (hand-copied, or from a previous keep pull)
# is what docker compose reads either way.
set -euo pipefail
project="${1:?usage: deploy.sh <project> [env]}"
@@ -21,5 +36,18 @@ if [ ! -d "$dir" ]; then
exit 1
fi
keep pull "$project/$env" --out "$dir/.env"
if [ -f "$dir/.keep-vault" ]; then
vault="$(cat "$dir/.keep-vault")"
else
vault="$project/$env"
fi
if command -v keep >/dev/null 2>&1 && keep pull "$vault" --out "$dir/.env.new" 2>/dev/null; then
mv "$dir/.env.new" "$dir/.env"
echo " ✓ pulled $vault"
else
rm -f "$dir/.env.new"
echo " · keep pull skipped or failed for $vault (using existing .env, if any)"
fi
(cd "$dir" && docker compose up -d)

23
scripts/install-cli.sh Executable file
View File

@@ -0,0 +1,23 @@
#!/bin/sh
# Builds keep and symlinks the CLI onto PATH as a real `keep` command --
# no `npm run cli --` from inside this repo, no npm global install (which
# needs root/write access to the system Node install on most hosts).
#
# Since this is a symlink to dist/cli/index.js rather than a copy, a
# future `git pull && npm run build` here is the entire upgrade story --
# no need to re-run this script.
set -eu
cd "$(dirname "$0")/.."
BIN_DIR="${KEEP_BIN_DIR:-$HOME/.local/bin}"
mkdir -p "$BIN_DIR"
npm run build
chmod +x dist/cli/index.js
ln -sf "$(pwd)/dist/cli/index.js" "$BIN_DIR/keep"
echo "keep CLI linked: $BIN_DIR/keep -> $(pwd)/dist/cli/index.js"
case ":$PATH:" in
*":$BIN_DIR:"*) echo "$BIN_DIR is already on PATH — try: keep --help" ;;
*) echo "warning: $BIN_DIR is not on PATH — add it to your shell profile" ;;
esac

View File

@@ -1,4 +1,4 @@
import { adminFetch, expectOk } from '../signedFetch.js';
import { adminFetch, expectOk, serverUrl } from '../signedFetch.js';
export async function recipientAdd(label: string, publicKey: string): Promise<void> {
const data = await expectOk(await adminFetch('POST', '/api/admin/recipients', { label, publicKey }));
@@ -20,9 +20,18 @@ export async function recipientRemove(recipientId: string): Promise<void> {
}
export async function adminOverview(): Promise<void> {
// Printed before the fetch, not after -- which server this hit is exactly
// the thing that's ambiguous from the rest of the output alone (no vault
// or recipient here says "production" or "dev"), and defaults silently
// to localhost:3050 if KEEP_SERVER_URL isn't set. Printing it first means
// it still shows up even when the fetch itself fails ("error: fetch
// failed") -- the exact case where knowing the target mattered most and
// previously showed nothing at all.
console.log(`server: ${serverUrl()}`);
const data = await expectOk(await adminFetch('GET', '/api/admin/overview'));
console.log('recipients:');
console.log('\nrecipients:');
if (!data.recipients.length) console.log(' none yet.');
for (const r of data.recipients) {
console.log(` ${r.recipientId} ${r.label}`);
@@ -31,7 +40,9 @@ export async function adminOverview(): Promise<void> {
console.log('\nvaults:');
if (!data.vaults.length) console.log(' none yet.');
for (const v of data.vaults) {
console.log(` ${v.vaultKey}`);
const updated = v.updatedAt ? new Date(v.updatedAt * 1000).toISOString() : 'unknown';
const by = v.updatedByLabel ?? v.updatedBy ?? 'unknown';
console.log(` ${v.vaultKey} (updated ${updated} by ${by})`);
for (const g of v.grants) {
console.log(` ${g.canWrite ? 'rw' : 'r-'} ${g.recipientId} ${g.label ?? '(unknown recipient)'}`);
}

View File

@@ -1,7 +1,7 @@
import fs from 'node:fs';
import { requireIdentityAndRecipientId } from '../identityStore.js';
import { signedFetch, adminFetch, expectOk } from '../signedFetch.js';
import { parseEnv, serializeEnv } from '../envFormat.js';
import { parseEnv, serializeEnv, buildEnvTemplate, fillEnvTemplate } from '../envFormat.js';
import {
ready,
Identity,
@@ -16,8 +16,13 @@ export async function vaultPush(vaultKey: string, file: string, purge: boolean):
const { identity, recipientId } = await requireIdentityAndRecipientId();
if (!fs.existsSync(file)) throw new Error(`file not found: ${file}`);
const values = parseEnv(fs.readFileSync(file, 'utf8'));
const payload = enc.fromUtf8(JSON.stringify(values));
const raw = fs.readFileSync(file, 'utf8');
const values = parseEnv(raw);
const template = buildEnvTemplate(raw);
// __keepEnvelope marks the new wrapped shape so vaultPull can tell it
// apart from an old vault pushed before templates existed, which is
// just the flat { KEY: value } object with no wrapper at all.
const payload = enc.fromUtf8(JSON.stringify({ __keepEnvelope: true, values, template }));
const existsRes = await expectOk(await signedFetch(identity, recipientId, 'GET', `/api/vaults/${encodeURIComponent(vaultKey)}/exists`));
@@ -55,7 +60,7 @@ export async function vaultPush(vaultKey: string, file: string, purge: boolean):
console.log(`pushed '${vaultKey}' — ${Object.keys(values).length} key${Object.keys(values).length === 1 ? '' : 's'}, wrapped for ${wrapFor.size} recipient${wrapFor.size === 1 ? '' : 's'}${purge ? ' (purged previous version)' : ''}.`);
}
export async function vaultPull(vaultKey: string, format: 'env' | 'json', outFile: string | undefined, previous: boolean): Promise<void> {
export async function vaultPull(vaultKey: string, format: 'env' | 'json', outFile: string | undefined, previous: boolean, wantTemplate: boolean): Promise<void> {
await ready();
const { identity, recipientId } = await requireIdentityAndRecipientId();
@@ -66,9 +71,27 @@ export async function vaultPull(vaultKey: string, format: 'env' | 'json', outFil
if (!symmetricKey) throw new Error('failed to unwrap this vault\'s key — wrong identity, or the grant is stale');
const plaintext = secretboxDecrypt(symmetricKey, enc.fromBase64(data.nonce), enc.fromBase64(data.ciphertext));
const values = JSON.parse(enc.toUtf8(plaintext)) as Record<string, string>;
const parsed = JSON.parse(enc.toUtf8(plaintext)) as Record<string, unknown>;
const out = format === 'json' ? JSON.stringify(values, null, 2) : serializeEnv(values);
// Vaults pushed before templates existed are just the flat { KEY: value }
// object with no wrapper — __keepEnvelope tells the two shapes apart.
const values = (parsed.__keepEnvelope ? parsed.values : parsed) as Record<string, string>;
const template = parsed.__keepEnvelope ? (parsed.template as string | undefined) : undefined;
if (wantTemplate) {
if (template === undefined) throw new Error(`vault '${vaultKey}' has no saved template (pushed before template support, or the source file had no comments to preserve)`);
if (outFile) {
fs.writeFileSync(outFile, template);
console.error(`wrote template to ${outFile}${previous ? ' (previous version)' : ''}`);
} else {
process.stdout.write(template);
}
return;
}
const out = format === 'json'
? JSON.stringify(values, null, 2)
: template !== undefined ? fillEnvTemplate(template, values) : serializeEnv(values);
if (outFile) {
fs.writeFileSync(outFile, out);
console.error(`wrote ${Object.keys(values).length} key(s) to ${outFile}${previous ? ' (previous version)' : ''}`);
@@ -114,3 +137,35 @@ export async function vaultLog(vaultKey: string): Promise<void> {
console.log(`${new Date(e.accessed_at * 1000).toISOString()} ${e.action.padEnd(6)} ${e.recipient_id}`);
}
}
export async function vaultAnnounce(vaultKey: string, tag: string): Promise<void> {
await ready();
const { identity, recipientId } = await requireIdentityAndRecipientId();
await expectOk(await signedFetch(identity, recipientId, 'POST', `/api/vaults/${encodeURIComponent(vaultKey)}/announce`, { tag }));
console.log(`announced '${vaultKey}' @ ${tag}`);
}
async function fetchAnnouncement(vaultKey: string): Promise<{ tag: string; announcedAt: number; announcedBy: string } | null> {
const { identity, recipientId } = await requireIdentityAndRecipientId();
const res = await signedFetch(identity, recipientId, 'GET', `/api/vaults/${encodeURIComponent(vaultKey)}/announce`);
if (res.status === 404) return null;
return await expectOk(res);
}
// Deliberately dumb — plain polling, not a websocket or long-poll. A
// 60s detection lag for a deploy signal is fine; don't build push-based
// notification for this.
export async function vaultWatch(vaultKey: string, intervalSeconds: number): Promise<void> {
await ready();
console.log(`watching '${vaultKey}' every ${intervalSeconds}s (ctrl-c to stop)...`);
let lastTag: string | null = null;
for (;;) {
const announcement = await fetchAnnouncement(vaultKey);
if (announcement && announcement.tag !== lastTag) {
lastTag = announcement.tag;
console.log(`${new Date(announcement.announcedAt * 1000).toISOString()} ${vaultKey} ${announcement.tag} (by ${announcement.announcedBy})`);
}
await new Promise(resolve => setTimeout(resolve, intervalSeconds * 1000));
}
}

View File

@@ -20,3 +20,38 @@ export function serializeEnv(values: Record<string, string>): string {
.map(([k, v]) => `${k}=${/[\s#"']/.test(v) ? JSON.stringify(v) : v}`)
.join('\n') + '\n';
}
// Redacts values while keeping comments, blank lines, and key order intact
// — the whole point is to preserve the "why"/"where to get this" context
// that plain parseEnv() throws away, without keeping the actual secret
// around in the template.
// Inverse of buildEnvTemplate: substitutes real values back into a
// template's `KEY=` lines while leaving comments/blank lines/order intact.
export function fillEnvTemplate(template: string, values: Record<string, string>): string {
return template
.split('\n')
.map(line => {
const trimmed = line.trim();
if (!trimmed || trimmed.startsWith('#')) return line;
const eq = line.indexOf('=');
if (eq === -1) return line;
const key = line.slice(0, eq);
const value = values[key.trim()] ?? '';
return `${key}=${/[\s#"']/.test(value) ? JSON.stringify(value) : value}`;
})
.join('\n');
}
export function buildEnvTemplate(text: string): string {
return text
.split('\n')
.map(rawLine => {
const line = rawLine.trimEnd();
const trimmed = line.trim();
if (!trimmed || trimmed.startsWith('#')) return line;
const eq = line.indexOf('=');
if (eq === -1) return line;
return `${line.slice(0, eq)}=`;
})
.join('\n');
}

View File

@@ -1,7 +1,7 @@
#!/usr/bin/env node
import { identityInit, identityShow, identitySetId } from './commands/identity.js';
import { recipientAdd, recipientList, recipientRemove, adminOverview } from './commands/recipient.js';
import { vaultPush, vaultPull, vaultGrant, vaultRevoke, vaultLog } from './commands/vault.js';
import { vaultPush, vaultPull, vaultGrant, vaultRevoke, vaultLog, vaultAnnounce, vaultWatch } from './commands/vault.js';
function flag(args: string[], name: string, fallback?: string): string | undefined {
const i = args.indexOf(`--${name}`);
@@ -31,10 +31,12 @@ async function main(): Promise<void> {
if (cmd === 'overview') return await adminOverview();
if (cmd === 'push') return await vaultPush(sub, flag(rest, 'file', '.env')!, boolFlag(rest, 'purge'));
if (cmd === 'pull') return await vaultPull(sub, (flag(rest, 'format', 'env') as 'env' | 'json'), flag(rest, 'out'), boolFlag(rest, 'previous'));
if (cmd === 'pull') return await vaultPull(sub, (flag(rest, 'format', 'env') as 'env' | 'json'), flag(rest, 'out'), boolFlag(rest, 'previous'), boolFlag(rest, 'template'));
if (cmd === 'grant') return await vaultGrant(sub, rest[0], boolFlag(rest, 'read-only'));
if (cmd === 'revoke') return await vaultRevoke(sub, rest[0]);
if (cmd === 'log') return await vaultLog(sub);
if (cmd === 'announce') return await vaultAnnounce(sub, flag(rest, 'tag')!);
if (cmd === 'watch') return await vaultWatch(sub, Number(flag(rest, 'interval', '60')));
printUsage();
process.exitCode = cmd ? 1 : 0;
@@ -52,10 +54,14 @@ function printUsage(): void {
keep identity set-id <recipient-id>
keep push <vault> [--file .env] [--purge] (--purge: don't retain the outgoing version as rollback)
keep pull <vault> [--format env|json] [--out <path>] [--previous]
keep pull <vault> [--format env|json] [--out <path>] [--previous] [--template]
(fills the saved template with real values by default, if one exists;
--template: comments/structure with values redacted, not the secrets)
keep grant <vault> <recipient-id> [--read-only]
keep revoke <vault> <recipient-id> (admin — needs KEEP_ADMIN_PASSWORD)
keep log <vault>
keep announce <vault> --tag <tag> (requires any grant, read-only is enough)
keep watch <vault> [--interval 60] (polls for a new announced tag; ctrl-c to stop)
keep recipient add --label <label> --pubkey <hex> (admin)
keep recipient list (admin)

View File

@@ -57,6 +57,7 @@ router.get('/overview', (_req, res) => {
vaultKey: v.vault_key,
updatedAt: v.updated_at,
updatedBy: v.updated_by,
updatedByLabel: byId.get(v.updated_by)?.label ?? null,
grants: listGrantsForVault(v.vault_key).map(g => ({
recipientId: g.recipient_id,
label: byId.get(g.recipient_id)?.label ?? null,

View File

@@ -61,6 +61,17 @@ db.exec(`
action TEXT NOT NULL,
accessed_at INTEGER NOT NULL
);
-- Latest known deploy tag for a vault — "current state", not a
-- history, same one-row-per-vault shape as the vaults table's own
-- prev_ciphertext retention. An image tag isn't a secret, so this
-- lives outside the encrypted payload entirely.
CREATE TABLE IF NOT EXISTS vault_announcements (
vault_key TEXT PRIMARY KEY,
tag TEXT NOT NULL,
announced_at INTEGER NOT NULL,
announced_by TEXT NOT NULL
);
`);
export interface VaultRow {
@@ -88,6 +99,13 @@ export interface GrantRow {
granted_at: number;
}
export interface AnnouncementRow {
vault_key: string;
tag: string;
announced_at: number;
announced_by: string;
}
export function getVault(vaultKey: string): VaultRow | undefined {
return db.prepare(`SELECT * FROM vaults WHERE vault_key = ?`).get(vaultKey) as VaultRow | undefined;
}
@@ -225,7 +243,7 @@ export function deleteGrant(vaultKey: string, recipientId: string): void {
db.prepare(`DELETE FROM vault_grants WHERE vault_key = ? AND recipient_id = ?`).run(vaultKey, recipientId);
}
export function logAccess(vaultKey: string, recipientId: string, action: 'pull' | 'push' | 'grant' | 'revoke'): void {
export function logAccess(vaultKey: string, recipientId: string, action: 'pull' | 'push' | 'grant' | 'revoke' | 'announce'): void {
db.prepare(
`INSERT INTO access_log (vault_key, recipient_id, action, accessed_at) VALUES (?, ?, ?, ?)`,
).run(vaultKey, recipientId, action, Math.floor(Date.now() / 1000));
@@ -236,3 +254,17 @@ export function getAccessLog(vaultKey: string, limit = 50): { recipient_id: stri
`SELECT recipient_id, action, accessed_at FROM access_log WHERE vault_key = ? ORDER BY accessed_at DESC LIMIT ?`,
).all(vaultKey, limit) as { recipient_id: string; action: string; accessed_at: number }[];
}
// One row per vault — "latest known tag", not a history. Upsert.
export function setAnnouncement(vaultKey: string, tag: string, announcedBy: string): void {
db.prepare(
`INSERT INTO vault_announcements (vault_key, tag, announced_at, announced_by)
VALUES (?, ?, ?, ?)
ON CONFLICT(vault_key) DO UPDATE SET
tag = excluded.tag, announced_at = excluded.announced_at, announced_by = excluded.announced_by`,
).run(vaultKey, tag, Math.floor(Date.now() / 1000), announcedBy);
}
export function getAnnouncement(vaultKey: string): AnnouncementRow | undefined {
return db.prepare(`SELECT * FROM vault_announcements WHERE vault_key = ?`).get(vaultKey) as AnnouncementRow | undefined;
}

View File

@@ -11,6 +11,8 @@ import {
logAccess,
getAccessLog,
listRecipients,
setAnnouncement,
getAnnouncement,
} from './db.js';
import { requireRecipientAuth, type AuthedRequest } from './auth.js';
@@ -179,3 +181,42 @@ router.get('/vaults/:key/log', (req: AuthedRequest, res) => {
}
res.json(getAccessLog(vaultKey));
});
// A deploy tag isn't a secret — it's not stored in the encrypted payload,
// just a small side table. Only requires ANY grant (read-only is enough)
// since announcing isn't a mutation of vault secrets — a CI identity
// should never need write access on a vault just to say "there's a new
// image". Resist reaching for hasWriteGrant here by analogy with push.
router.post('/vaults/:key/announce', (req: AuthedRequest, res) => {
const vaultKey = req.params.key;
const recipientId = req.recipientId!;
if (!hasGrant(vaultKey, recipientId)) {
res.status(403).json({ error: 'no access to this vault' });
return;
}
const body = req.body as { tag?: string };
if (!body.tag) {
res.status(400).json({ error: 'tag is required' });
return;
}
setAnnouncement(vaultKey, body.tag, recipientId);
logAccess(vaultKey, recipientId, 'announce');
res.json({ ok: true });
});
router.get('/vaults/:key/announce', (req: AuthedRequest, res) => {
const vaultKey = req.params.key;
if (!hasGrant(vaultKey, req.recipientId!)) {
res.status(403).json({ error: 'no access to this vault' });
return;
}
const announcement = getAnnouncement(vaultKey);
if (!announcement) {
res.status(404).json({ error: 'nothing announced yet' });
return;
}
res.json({ tag: announcement.tag, announcedAt: announcement.announced_at, announcedBy: announcement.announced_by });
});