Compare commits
5 Commits
f6a1ac13f2
...
58d5d787dd
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
58d5d787dd | ||
|
|
674e408f09 | ||
|
|
114f234cd3 | ||
|
|
e505fbf9b4 | ||
|
|
5f31eb76c7 |
@@ -183,6 +183,8 @@ keep push <vault> [--file .env] [--purge] -- encrypt + upload; requires a WRITE
|
||||
keep pull <vault> [--format env|json] [--out <path>] [--previous] -- fetch + decrypt; requires a grant
|
||||
keep grant <vault> <recipient-id> [--read-only] -- add a recipient to a vault you have WRITE access to
|
||||
keep log <vault> -- tail the access log for one vault
|
||||
keep announce <vault> --tag <tag> -- record a deploy tag; requires ANY grant on the vault (read-only is enough)
|
||||
keep watch <vault> [--interval 60] -- poll for a new announced tag; exits nonzero if the vault doesn't exist / no grant
|
||||
|
||||
# admin operations — gated by ADMIN_PASSWORD, see below
|
||||
keep recipient add --label "..." --pubkey <hex>
|
||||
@@ -215,6 +217,75 @@ Went with (2) — a global admin password is an acceptable trust model for
|
||||
"one person or small team running their own homelab," which is the
|
||||
actual scale this is built for.
|
||||
|
||||
## Announce/watch — a deploy signal, not a secret
|
||||
|
||||
CI producing a fresh image on every push still left deployment as a
|
||||
manual step, and the obvious fix — CI SSHes into the VPS and redeploys —
|
||||
was rejected: it means a standing SSH credential with production-deploy
|
||||
rights sits in CI secrets, reachable by whatever's executing on a shared
|
||||
runner across every project it builds. A compromised build step in one
|
||||
project could then reach every other project's deploy target, which is
|
||||
a real regression from `keep`'s own "read no longer implies write"
|
||||
posture (see above).
|
||||
|
||||
The fix that doesn't reintroduce that problem: reverse the direction.
|
||||
CI never reaches into a VPS — it tells `keep` "there's a new image for
|
||||
this project" (`keep announce <vault> --tag <tag>`), and each VPS, which
|
||||
already has to run *something* locally to redeploy itself, polls `keep`
|
||||
for that signal (`keep watch <vault>`) and acts on it locally, using an
|
||||
identity and grant it already legitimately holds. `keep`'s server never
|
||||
executes anything remotely — it stays a pure relay, same posture as
|
||||
everything else it does.
|
||||
|
||||
This was deliberately not built as "`keep` executes the redeploy
|
||||
itself," even though `keep` typically runs on the same box as its
|
||||
deploy targets and that would skip the polling indirection entirely.
|
||||
Doing so would trade away the property that's been the throughline of
|
||||
`keep`'s whole design: today, a fully compromised `keep` server — not
|
||||
just a compromised client identity — can't do anything worse than leak
|
||||
ciphertext and metadata, because the server never holds a decryption
|
||||
key. The moment `keep` can also execute a command on `announce`, an
|
||||
ordinary web app bug (not even a cryptographic break) becomes "attacker
|
||||
triggers arbitrary redeploys" — a new class of bad outcome, not a bigger
|
||||
version of an existing one.
|
||||
|
||||
An announced tag is stored outside the encrypted vault payload
|
||||
(`vault_announcements`, one row per vault — latest known state, not a
|
||||
history, same shape as `vaults.prev_ciphertext`) since an image tag
|
||||
isn't a secret and doesn't belong inside a secrets blob alongside real
|
||||
credentials. `POST /vaults/:key/announce` only requires *a* grant, not a
|
||||
write grant — `hasGrant`, not `hasWriteGrant` — since announcing isn't a
|
||||
mutation of a vault's secret content, just a signal; a CI identity for a
|
||||
project can stay read-only even with this feature in play. `keep watch`
|
||||
is deliberately dumb polling, not a websocket or long-poll — matches the
|
||||
pattern already used elsewhere in this project family, and a 60-second
|
||||
detection lag for a deploy is an acceptable tradeoff against the
|
||||
complexity of push-based notification.
|
||||
|
||||
Two things worth stating plainly about what this does and doesn't
|
||||
change:
|
||||
|
||||
- The tag itself is now server-visible, where previously not even the
|
||||
fact that a new image existed was something `keep` needed to know.
|
||||
Accepted tradeoff — a git short-SHA isn't sensitive, and the feature
|
||||
only works if `keep` can see it.
|
||||
- A compromised read-only CI identity could announce a *false* tag,
|
||||
pointing a deploy target at an attacker-chosen image. `keep` never
|
||||
pulls or runs that image itself, only relays the string — the actual
|
||||
`docker compose pull` on the deploy target still pulls from whatever
|
||||
registry it's configured for, using the tag it was told. This is
|
||||
equivalent in severity to that CI identity's build step being
|
||||
compromised in the first place (it could already push a bad image
|
||||
under a legitimate tag), not a new capability.
|
||||
|
||||
A bulk multi-project deploy script (looping over a fixed project list,
|
||||
`docker compose up -d` per project) adopts this by checking for a
|
||||
per-project `.keep-vault` marker file before calling `keep pull`; a
|
||||
missing marker falls through to exactly what the script did before
|
||||
`keep` existed — never a hard failure, since not every project will be
|
||||
bootstrapped onto `keep` at the same time. `scripts/deploy.sh` in this
|
||||
repo is the single-project version of the same pattern.
|
||||
|
||||
## Deploy integration
|
||||
|
||||
The actual motivating use case — a deploy script pulling secrets instead
|
||||
|
||||
369
PROPOSAL-announce-and-agent.md
Normal file
369
PROPOSAL-announce-and-agent.md
Normal file
@@ -0,0 +1,369 @@
|
||||
# Proposal: announce + a self-deploying agent, with a no-`keep` fallback
|
||||
|
||||
**Status:** implemented (§1–§3, §6 shape). `announce`/`watch`, the
|
||||
`vault_announcements` table, and the API routes described below are
|
||||
built — see [IMPLEMENTATION.md](./IMPLEMENTATION.md)'s "Announce/watch"
|
||||
section for the as-built version of the design rationale, and the CLI
|
||||
surface list there for the exact commands. §4 (patching the real bulk
|
||||
deploy script) and §6 (the CI workflow step) are per-project changes
|
||||
made outside this repo, on the host and in each project's own
|
||||
`.gitea/workflows/docker.yml` — not something this repo's code can
|
||||
contain. This file is kept as the original design record, superseded by
|
||||
IMPLEMENTATION.md as the source of truth for what's actually built.
|
||||
|
||||
## Context (read this first)
|
||||
|
||||
Full background on `keep` is in [README.md](./README.md) and
|
||||
[IMPLEMENTATION.md](./IMPLEMENTATION.md) — read both before touching
|
||||
code. The relevant existing pieces for this proposal:
|
||||
|
||||
- `src/server/db.ts` — SQLite (better-sqlite3, WAL mode). `vaults`,
|
||||
`recipients`, `vault_grants`, `vault_grants_previous`, `access_log`.
|
||||
- `src/server/routes.ts` — recipient-authenticated (signed-request)
|
||||
routes: `push`, `pull`, `grant`, `log`, plus `exists` and
|
||||
`recipients` (both used by the CLI to compute who to re-wrap for).
|
||||
- `src/server/auth.ts` — `requireRecipientAuth` (Ed25519 signature over
|
||||
method+path+timestamp+body-hash, verified against `req.originalUrl`
|
||||
with the query string stripped — see IMPLEMENTATION.md's "What was
|
||||
verified end-to-end" for a bug that already bit this exact mechanism
|
||||
once) and `requireAdminAuth` (`X-Admin-Password` header).
|
||||
- `src/cli/` — `identity.ts`, `recipient.ts`, `vault.ts`,
|
||||
`signedFetch.ts` (note: `signedFetch`'s `path` param must never
|
||||
include a query string — see the comment there for why).
|
||||
- `scripts/deploy.sh` (this repo) — pulls `<project>/<env>` from `keep`
|
||||
into `$DEPLOY_ROOT/<project>/.env`, then `docker compose up -d` for
|
||||
one project. Assumes the vault already exists and this machine
|
||||
already has a grant on it. **Superseded by the real deploy script
|
||||
below as the thing this proposal actually patches** — kept in this
|
||||
repo as a single-project example/fallback, not the production path.
|
||||
- **The real, already-running deploy script — on the host, not in any
|
||||
repo.** A fixed-array bulk script, already in production use, looping
|
||||
over roughly a dozen project directories:
|
||||
|
||||
```bash
|
||||
ROOT="$(cd "$(dirname "$0")" && pwd)"
|
||||
PROJECTS=(proj1 proj2 proj3 proj4 proj5 proj6 proj7 proj8 proj9 proj10 proj11 proj12)
|
||||
for project in "${PROJECTS[@]}"; do
|
||||
dir="$ROOT/$project"
|
||||
[ -f "$dir/docker-compose.yml" ] || { echo "skip: $project"; continue; }
|
||||
(cd "$dir" && docker compose up -d) || FAILED+=("$project")
|
||||
done
|
||||
docker image prune -f
|
||||
```
|
||||
|
||||
Two things about it that matter for this proposal: **it never runs
|
||||
`docker compose pull`** — freshness comes entirely from each
|
||||
project's own `pull_policy: always` in its `docker-compose.yml`
|
||||
(`up -d` re-pulls implicitly). And **it loops over a fixed, explicit
|
||||
array**, not directory auto-discovery — adding a project means
|
||||
editing this array by hand. This is the actual thing to patch, not a
|
||||
new per-project script; see Design §4.
|
||||
- **Prerequisite gap, found while writing this proposal, not yet fixed
|
||||
everywhere**: `pull_policy: always` isn't universal across the
|
||||
fleet — confirmed present in roughly half the projects checked,
|
||||
confirmed missing in several others (one of which was fixed in this
|
||||
session — a sibling project's `docker-compose.yml` didn't have it, a
|
||||
regression from copying another project's compose file without
|
||||
carrying this line over), and unchecked for the rest (no local
|
||||
checkout available at proposal time). **This is independent of
|
||||
`keep` entirely** — the bulk script can't redeploy any project
|
||||
missing this line regardless of whether that project ever adopts
|
||||
`keep`, and it should be swept fleet-wide before or alongside this
|
||||
work, not as part of it.
|
||||
- `.gitea/workflows/docker.yml` (present in `keep` and other sibling
|
||||
projects) — builds and pushes an image to the Gitea container
|
||||
registry on every push to `main`. Does **not** currently deploy
|
||||
anything — that's the manual/scheduled bulk script above today.
|
||||
|
||||
## Motivation
|
||||
|
||||
The CI workflow already produces a fresh image on every push. Deployment
|
||||
is still a manual "SSH in and run the deploy script" step. The obvious
|
||||
fix — CI SSHes into the VPS and runs the deploy — was considered and
|
||||
rejected: it means a standing SSH credential with production-deploy
|
||||
rights sits in Gitea's CI secrets, reachable by whatever's executing on
|
||||
the shared runner across every project it builds. A compromised build
|
||||
step in any one project could then reach every other project's deploy
|
||||
target. That's a real regression from where `keep` already left things
|
||||
(see IMPLEMENTATION.md's resolved "read no longer implies write" — the
|
||||
whole point was *not* concentrating reach into one identity).
|
||||
|
||||
The fix that doesn't reintroduce that problem: **reverse the direction.**
|
||||
CI never reaches into a VPS. Instead, CI tells `keep` "there's a new
|
||||
image for this project," and each VPS — which already has to run
|
||||
*something* locally to redeploy itself — polls `keep` for that signal
|
||||
and acts on it locally, using an identity and grant it already legitimately
|
||||
holds. `keep`'s server never executes anything remotely; it stays a pure
|
||||
relay, same posture as everything else it does.
|
||||
|
||||
**Not every project will have been migrated to `keep` yet.** Bootstrapping
|
||||
a project (register a recipient, grant it, push the first vault) is a
|
||||
real, deliberate step per project, and there'll be a period — maybe a
|
||||
long one — where some projects have it and some don't. The redeploy path
|
||||
has to work identically well for both, defaulting to exactly what
|
||||
happens today (a plain `docker compose up -d`, relying on that project's
|
||||
own `pull_policy: always`) for anything not yet bootstrapped, rather than
|
||||
either blocking deploys entirely or hard-failing on missing vaults.
|
||||
|
||||
### Considered and rejected: `keep` executes the redeploy itself
|
||||
|
||||
Since `keep`'s own server will run on the same VPS as the deploy targets,
|
||||
it's tempting to skip the polling indirection entirely and have `keep`
|
||||
shell out to the deploy script directly when it receives an `announce` —
|
||||
no new SSH credential, no new network-reachable pathway, since nothing
|
||||
is reaching *into* the box that wasn't already running on it.
|
||||
|
||||
Rejected anyway, because it trades away a property that's been the
|
||||
throughline of `keep`'s whole design: **today, a fully compromised
|
||||
`keep` server — not just a compromised client identity — can't do
|
||||
anything worse than leak ciphertext and metadata**, because the server
|
||||
never holds a decryption key. The moment `keep` can also execute a
|
||||
command on `announce`, that stops being true — a bug in the HTTP layer
|
||||
(not even a cryptographic break; an ordinary web app bug, an auth check
|
||||
wrong on some edge case) becomes "attacker triggers arbitrary
|
||||
redeploys," not "attacker sees encrypted blobs they can't read." That's
|
||||
a new class of bad outcome, not a bigger version of an existing one, and
|
||||
it's not worth trading for skipping a polling loop.
|
||||
|
||||
There's also a concrete operational cost, not just a principle: actually
|
||||
running `docker compose up -d` from inside `keep`'s own process would
|
||||
need either the Docker socket mounted into `keep`'s container
|
||||
(`/var/run/docker.sock` — widely treated as equivalent to handing that
|
||||
container root on the host) or running `keep` outside a container
|
||||
entirely, throwing away the isolation its own `Dockerfile` already
|
||||
provides today.
|
||||
|
||||
`keep` stays a pure relay regardless of where it's deployed. Something
|
||||
*outside* `keep`'s own process boundary does the polling and the
|
||||
executing — see §4.
|
||||
|
||||
## Design
|
||||
|
||||
### 1. `announce` — a distinct verb from `push`/`pull`
|
||||
|
||||
An image tag isn't a secret. It doesn't belong inside the encrypted vault
|
||||
payload alongside real credentials — that's a category mismatch (deploy
|
||||
metadata living inside a secrets blob), not just an implementation
|
||||
convenience either way. `announce` is its own small mechanism:
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS vault_announcements (
|
||||
vault_key TEXT PRIMARY KEY,
|
||||
tag TEXT NOT NULL,
|
||||
announced_at INTEGER NOT NULL,
|
||||
announced_by TEXT NOT NULL -- recipient_id
|
||||
);
|
||||
```
|
||||
|
||||
One row per vault — like the vault's own one-previous-version retention,
|
||||
this is "latest known state," not a history. If a real audit trail of
|
||||
every announce is wanted later, `access_log` already records the action;
|
||||
this table is just the fast-path "what's the current tag" read, kept
|
||||
separate from the log on purpose (reading current state back out of an
|
||||
append-only log is the wrong tool for that job — same reasoning
|
||||
`IMPLEMENTATION.md` already gives for why `vault_grants_previous` is a
|
||||
real table and not a log replay).
|
||||
|
||||
### 2. API
|
||||
|
||||
```
|
||||
POST /api/vaults/:key/announce -- { tag: string } — requires ANY grant on the vault (read-only is enough; announcing isn't a secrets mutation)
|
||||
GET /api/vaults/:key/announce -- { tag, announcedAt, announcedBy } | 404 if nothing announced yet — requires a grant, same as pull
|
||||
```
|
||||
|
||||
Both go through `requireRecipientAuth`, identical shape to the existing
|
||||
routes. `POST` only requires *a* grant (not write) — reuse `hasGrant`,
|
||||
not `hasWriteGrant` — since this isn't a mutation of the vault's secret
|
||||
content, just a signal, and CI's identity for a project should be able to
|
||||
stay read-only even after this feature exists. (This is worth stating
|
||||
explicitly since a naive implementation might reach for `hasWriteGrant`
|
||||
by analogy with `push` — resist that; it would silently widen CI's
|
||||
required permissions for no real reason.)
|
||||
|
||||
### 3. CLI
|
||||
|
||||
```
|
||||
keep announce <vault> --tag <tag> -- POST, requires a grant on <vault>
|
||||
keep watch <vault> [--interval 60] -- polls GET .../announce every N seconds, prints the tag on change, exits nonzero if the vault doesn't exist / no grant (see fallback below)
|
||||
```
|
||||
|
||||
`keep watch` is deliberately dumb — plain polling, not a websocket or
|
||||
long-poll. Matches the polling pattern already used elsewhere in this
|
||||
project family, and a 60-second detection lag for a deploy is fine.
|
||||
Don't build push-based notification for this; it's more machinery for a
|
||||
problem polling already solves adequately.
|
||||
|
||||
### 4. Patch the existing loop — don't build a parallel script
|
||||
|
||||
This is the part the fallback requirement is actually about. The real
|
||||
deploy script already does the one thing that matters (a loop over a
|
||||
fixed project list, `docker compose up -d` per project, relying on
|
||||
`pull_policy: always` for freshness) — the change is inserting one
|
||||
`keep`-with-fallback step per iteration, in place, not writing a new
|
||||
per-project `agent.sh` that duplicates the loop/array/failure-tracking
|
||||
logic the real script already has:
|
||||
|
||||
```bash
|
||||
for project in "${PROJECTS[@]}"; do
|
||||
dir="$ROOT/$project"
|
||||
[ -f "$dir/docker-compose.yml" ] || { echo "skip: $project"; continue; }
|
||||
|
||||
echo ""
|
||||
echo "▶ $project"
|
||||
|
||||
# A vault key is NOT assumed to match the deploy-directory name — repo
|
||||
# names, deploy directory names, and vault keys are three independent
|
||||
# strings in practice (a repo can be named one thing, its deploy
|
||||
# directory another, entirely by history/accident). .keep-vault is an
|
||||
# explicit, one-time-per-project marker: if it's missing, this project
|
||||
# was never bootstrapped onto keep (or never will be — a static site
|
||||
# with no secrets), and the loop falls through to exactly what it did
|
||||
# before keep existed. Every keep call here is allowed to fail;
|
||||
# failure means "keep doesn't apply to this project," never "stop the
|
||||
# deploy."
|
||||
if [ -f "$dir/.keep-vault" ]; then
|
||||
vault="$(cat "$dir/.keep-vault")"
|
||||
if keep pull "$vault" --out "$dir/.env.new" 2>/dev/null; then
|
||||
mv "$dir/.env.new" "$dir/.env"
|
||||
else
|
||||
rm -f "$dir/.env.new"
|
||||
fi
|
||||
fi
|
||||
|
||||
if (cd "$dir" && docker compose up -d 2>&1); then
|
||||
echo " ✓ $project up"
|
||||
else
|
||||
echo " ✗ $project failed"
|
||||
FAILED+=("$project")
|
||||
fi
|
||||
done
|
||||
```
|
||||
|
||||
Bootstrapping a project onto `keep` becomes: choose a vault key (no
|
||||
naming constraint — doesn't need to match the repo or the directory),
|
||||
`keep push` it, grant whichever recipients need it, and
|
||||
`echo "<the vault key>" > "$dir/.keep-vault"`. One file, human-readable,
|
||||
`cat`-able to check what's configured, no second array in this script to
|
||||
keep in sync by hand as directories get renamed or reorganized.
|
||||
|
||||
Notice `announce`/`watch` aren't actually load-bearing in this version —
|
||||
whatever already triggers this script (cron, systemd timer, or still
|
||||
run by hand) keeps triggering it exactly as before, and `keep pull`
|
||||
either updates `.env` or silently doesn't. That's deliberate: **the
|
||||
existing script, patched this way, is already a complete, correct
|
||||
baseline** — keep-bootstrapped projects get fresh secrets on every run,
|
||||
everything else behaves identically to today. `announce` sits on top as
|
||||
a latency optimization *for however this script gets triggered*, not a
|
||||
requirement for this patch to be complete — see the next section.
|
||||
|
||||
### 5. Where `announce`/`watch` actually earn their keep
|
||||
|
||||
Two independent benefits, worth keeping conceptually separate — and the
|
||||
second one looks different than it would for a one-agent-per-project
|
||||
design, because the real script is one bulk all-or-nothing sweep, not N
|
||||
independent watchers:
|
||||
|
||||
- **Secrets sourcing** — `keep pull` instead of a hand-placed `.env`.
|
||||
Available per-project the moment its vault exists and this machine
|
||||
has a grant on it. This is what §4's patch already delivers, on its
|
||||
own, with zero dependency on `announce`.
|
||||
- **Waking the sweep early** — however the bulk script currently gets
|
||||
triggered (cron, systemd timer, or still by hand), `announce` could
|
||||
let it wake immediately on a push instead of waiting for the next
|
||||
scheduled run. The natural fit given the script's actual shape: `keep
|
||||
watch-any <vault1> <vault2> ...` blocking until *any* watched vault's
|
||||
tag changes, used as the trigger for a full sweep — not a per-project
|
||||
selective redeploy. Selective ("only redeploy the one project that
|
||||
actually changed") would require reshaping the script away from its
|
||||
current fixed-array-sweep model into something that takes a target
|
||||
project as an argument, which is a real behavior change to something
|
||||
already in production use, not a small addition — out of scope here.
|
||||
|
||||
Deliberately **not** building the wake-the-sweep-early mechanism in this
|
||||
pass. Ship §4's patch first — it already satisfies the fallback
|
||||
requirement completely and delivers the secrets-sourcing benefit on its
|
||||
own. Whether waiting up to one scheduled interval for a redeploy is
|
||||
actually a problem worth solving with `announce`/`watch-any` should be
|
||||
judged after living with the patched bulk script for a while, not
|
||||
designed in advance of any real signal that the wait matters.
|
||||
|
||||
### 6. CI workflow addition
|
||||
|
||||
One step appended to the existing `docker.yml`, after the existing
|
||||
build-and-push step, best-effort (must not fail the build if this
|
||||
project hasn't been bootstrapped onto `keep` yet):
|
||||
|
||||
```yaml
|
||||
- name: announce to keep
|
||||
continue-on-error: true
|
||||
run: keep announce myapp/production --tag ${{ steps.meta.outputs.short_sha }}
|
||||
```
|
||||
|
||||
The vault key here (`myapp/production`) is a **literal, hardcoded string
|
||||
in this specific repo's own workflow file** — never derived from
|
||||
`${{ github.repository }}` or any other automatic source. Same reasoning
|
||||
as §4's `.keep-vault` file: the repo's own name, its deploy directory's
|
||||
name, and its vault key are three independent strings that happen to
|
||||
often look similar and are not guaranteed to match (see Context — a real
|
||||
example already exists in this fleet). Whoever bootstraps a project onto
|
||||
`keep` should paste the same deliberately-chosen vault key into both this
|
||||
workflow step and that project's `.keep-vault` file, not rely on either
|
||||
side inferring it from something else.
|
||||
|
||||
`continue-on-error: true` is load-bearing, not incidental — a project
|
||||
without a `keep` identity registered yet must still build and push its
|
||||
image successfully. This step failing should be silent/expected for any
|
||||
not-yet-bootstrapped project, not a red X on the workflow.
|
||||
|
||||
## Security considerations
|
||||
|
||||
- `announce` leaks a little more than `push`/`pull` did: the *tag*
|
||||
itself is now server-visible (previously, if you were being strict,
|
||||
even the fact that a new image existed wasn't something `keep` needed
|
||||
to know). This is an intentional, accepted tradeoff — a git short-SHA
|
||||
or version tag isn't sensitive, and the whole feature only works if
|
||||
`keep` can see it.
|
||||
- Requiring only a read grant (not write) to `announce` means a
|
||||
compromised read-only CI identity could announce a *false* tag,
|
||||
pointing the fleet at an attacker-chosen image — worth flagging even
|
||||
though `keep` never pulls or runs that image itself, only relays the
|
||||
string. The actual `docker compose pull` on each VPS still pulls from
|
||||
the configured registry using the tag it was told, so this is
|
||||
equivalent in severity to that CI identity's build step being
|
||||
compromised in the first place (it could already push a bad image
|
||||
under a legitimate tag) — not a new capability, just worth stating
|
||||
plainly rather than leaving implicit.
|
||||
- The fallback path must never silently downgrade a *bootstrapped*
|
||||
project to running with a stale `.env` because of a transient `keep`
|
||||
outage. As written, `keep pull` failing for any reason (network
|
||||
blip, `keep` server down, not just "not bootstrapped") falls through
|
||||
to "deploy with whatever `.env` is already on disk" — for an
|
||||
already-bootstrapped project, that's the *previous* `keep pull`'s
|
||||
output, still valid, not a downgrade. This only becomes a real problem
|
||||
if a bootstrapped project's very first deploy attempt coincides with
|
||||
`keep` being unreachable, in which case there's no `.env` on disk yet
|
||||
at all and `docker compose up` will fail for the mundane reason that
|
||||
required env vars are missing — a loud, obvious failure, not a silent
|
||||
wrong-secrets one.
|
||||
|
||||
## Open questions
|
||||
|
||||
- **`.keep-vault` format** — a bare vault-key string is enough for
|
||||
today's one-environment-per-directory reality. If a directory ever
|
||||
needs to deploy more than one environment from the same
|
||||
`docker-compose.yml` (unlikely given the current fleet shape, but not
|
||||
impossible), a bare string stops being enough and it'd need to become
|
||||
a small key=value file or similar. Not designing that ahead of an
|
||||
actual need — ship the bare-string version.
|
||||
- **Fleet-wide `pull_policy: always` sweep** — a prerequisite this
|
||||
proposal depends on but doesn't itself fix (see Context). Should
|
||||
happen before or alongside this work, as its own small pass, not
|
||||
bundled into this one.
|
||||
- **Does `watch`/`watch-any` ever get used**, or does the
|
||||
already-scheduled bulk sweep turn out to be good enough forever?
|
||||
Deliberately left unbuilt (see §5) until there's a real signal the
|
||||
wait between scheduled runs is worth solving.
|
||||
- **Multi-VPS fan-out for one project** — if a project ever runs on more
|
||||
than one box, `announce` already supports that for free (any number of
|
||||
watchers can poll the same vault's announce endpoint), but this hasn't
|
||||
been exercised and might reveal something not accounted for here.
|
||||
39
README.md
39
README.md
@@ -141,6 +141,29 @@ Pushing to an existing vault requires a *write* grant — a read-only
|
||||
recipient can pull but can't push or grant others access. A brand-new
|
||||
vault's first push is always read-write for the pusher.
|
||||
|
||||
## Announcing deploys
|
||||
|
||||
`keep` can also relay a small "there's a new image" signal alongside a
|
||||
vault, so CI never needs standing SSH access to a deploy target — CI
|
||||
announces a tag, and whatever already redeploys that box locally polls
|
||||
for it instead of being reached into from outside:
|
||||
|
||||
```bash
|
||||
# CI, after a successful build/push — any grant is enough, read-only included:
|
||||
keep announce myapp/production --tag <short-sha>
|
||||
|
||||
# The deploy target polls for a change and prints it:
|
||||
keep watch myapp/production --interval 60
|
||||
```
|
||||
|
||||
An announced tag isn't a secret — it lives outside the encrypted vault
|
||||
payload, in its own small table, and is visible to the server (same as
|
||||
vault names and access timestamps already are). `keep` never pulls or
|
||||
runs the announced image itself; it only relays the string. See
|
||||
[IMPLEMENTATION.md](./IMPLEMENTATION.md) for how this fits into a bulk
|
||||
deploy script with a no-`keep` fallback for projects not yet bootstrapped
|
||||
onto it.
|
||||
|
||||
## Docker
|
||||
|
||||
```bash
|
||||
@@ -177,12 +200,16 @@ CLI-side: `KEEP_SERVER_URL` (default `http://localhost:3050`),
|
||||
|
||||
Implemented: identity, push/pull/grant/revoke/log, read/write grant
|
||||
scoping, one-step rollback with an explicit purge mode for
|
||||
compromise-driven rotations, admin recipient management, Docker deploy.
|
||||
Verified end-to-end — two and three independent identities, read-only
|
||||
grants correctly blocked from push/grant, grants preserved across
|
||||
rotation, previous-version pull working and correctly wiped by
|
||||
`--purge` for every recipient, malformed-auth rejection — against both
|
||||
a local server and the built Docker image. See
|
||||
compromise-driven rotations, admin recipient management, an admin
|
||||
cross-vault `overview`, deploy-signal `announce`/`watch`, a
|
||||
`scripts/deploy.sh` wrapper, and Docker deploy. Verified end-to-end —
|
||||
two and three independent identities, read-only grants correctly
|
||||
blocked from push/grant, grants preserved across rotation,
|
||||
previous-version pull working and correctly wiped by `--purge` for
|
||||
every recipient, a read-only identity successfully announcing while
|
||||
still being logged, `watch` picking up an announced tag and exiting
|
||||
nonzero for an unknown/ungranted vault, malformed-auth rejection —
|
||||
against both a local server and the built Docker image. See
|
||||
[IMPLEMENTATION.md](./IMPLEMENTATION.md) for the full design and its
|
||||
resolved decisions (per-secret-key granularity deliberately not built —
|
||||
use multiple vaults instead).
|
||||
|
||||
@@ -31,7 +31,9 @@ export async function adminOverview(): Promise<void> {
|
||||
console.log('\nvaults:');
|
||||
if (!data.vaults.length) console.log(' none yet.');
|
||||
for (const v of data.vaults) {
|
||||
console.log(` ${v.vaultKey}`);
|
||||
const updated = v.updatedAt ? new Date(v.updatedAt * 1000).toISOString() : 'unknown';
|
||||
const by = v.updatedByLabel ?? v.updatedBy ?? 'unknown';
|
||||
console.log(` ${v.vaultKey} (updated ${updated} by ${by})`);
|
||||
for (const g of v.grants) {
|
||||
console.log(` ${g.canWrite ? 'rw' : 'r-'} ${g.recipientId} ${g.label ?? '(unknown recipient)'}`);
|
||||
}
|
||||
|
||||
@@ -114,3 +114,35 @@ export async function vaultLog(vaultKey: string): Promise<void> {
|
||||
console.log(`${new Date(e.accessed_at * 1000).toISOString()} ${e.action.padEnd(6)} ${e.recipient_id}`);
|
||||
}
|
||||
}
|
||||
|
||||
export async function vaultAnnounce(vaultKey: string, tag: string): Promise<void> {
|
||||
await ready();
|
||||
const { identity, recipientId } = await requireIdentityAndRecipientId();
|
||||
await expectOk(await signedFetch(identity, recipientId, 'POST', `/api/vaults/${encodeURIComponent(vaultKey)}/announce`, { tag }));
|
||||
console.log(`announced '${vaultKey}' @ ${tag}`);
|
||||
}
|
||||
|
||||
async function fetchAnnouncement(vaultKey: string): Promise<{ tag: string; announcedAt: number; announcedBy: string } | null> {
|
||||
const { identity, recipientId } = await requireIdentityAndRecipientId();
|
||||
const res = await signedFetch(identity, recipientId, 'GET', `/api/vaults/${encodeURIComponent(vaultKey)}/announce`);
|
||||
if (res.status === 404) return null;
|
||||
return await expectOk(res);
|
||||
}
|
||||
|
||||
// Deliberately dumb — plain polling, not a websocket or long-poll. A
|
||||
// 60s detection lag for a deploy signal is fine; don't build push-based
|
||||
// notification for this.
|
||||
export async function vaultWatch(vaultKey: string, intervalSeconds: number): Promise<void> {
|
||||
await ready();
|
||||
console.log(`watching '${vaultKey}' every ${intervalSeconds}s (ctrl-c to stop)...`);
|
||||
|
||||
let lastTag: string | null = null;
|
||||
for (;;) {
|
||||
const announcement = await fetchAnnouncement(vaultKey);
|
||||
if (announcement && announcement.tag !== lastTag) {
|
||||
lastTag = announcement.tag;
|
||||
console.log(`${new Date(announcement.announcedAt * 1000).toISOString()} ${vaultKey} ${announcement.tag} (by ${announcement.announcedBy})`);
|
||||
}
|
||||
await new Promise(resolve => setTimeout(resolve, intervalSeconds * 1000));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env node
|
||||
import { identityInit, identityShow, identitySetId } from './commands/identity.js';
|
||||
import { recipientAdd, recipientList, recipientRemove, adminOverview } from './commands/recipient.js';
|
||||
import { vaultPush, vaultPull, vaultGrant, vaultRevoke, vaultLog } from './commands/vault.js';
|
||||
import { vaultPush, vaultPull, vaultGrant, vaultRevoke, vaultLog, vaultAnnounce, vaultWatch } from './commands/vault.js';
|
||||
|
||||
function flag(args: string[], name: string, fallback?: string): string | undefined {
|
||||
const i = args.indexOf(`--${name}`);
|
||||
@@ -35,6 +35,8 @@ async function main(): Promise<void> {
|
||||
if (cmd === 'grant') return await vaultGrant(sub, rest[0], boolFlag(rest, 'read-only'));
|
||||
if (cmd === 'revoke') return await vaultRevoke(sub, rest[0]);
|
||||
if (cmd === 'log') return await vaultLog(sub);
|
||||
if (cmd === 'announce') return await vaultAnnounce(sub, flag(rest, 'tag')!);
|
||||
if (cmd === 'watch') return await vaultWatch(sub, Number(flag(rest, 'interval', '60')));
|
||||
|
||||
printUsage();
|
||||
process.exitCode = cmd ? 1 : 0;
|
||||
@@ -56,6 +58,8 @@ function printUsage(): void {
|
||||
keep grant <vault> <recipient-id> [--read-only]
|
||||
keep revoke <vault> <recipient-id> (admin — needs KEEP_ADMIN_PASSWORD)
|
||||
keep log <vault>
|
||||
keep announce <vault> --tag <tag> (requires any grant, read-only is enough)
|
||||
keep watch <vault> [--interval 60] (polls for a new announced tag; ctrl-c to stop)
|
||||
|
||||
keep recipient add --label <label> --pubkey <hex> (admin)
|
||||
keep recipient list (admin)
|
||||
|
||||
@@ -57,6 +57,7 @@ router.get('/overview', (_req, res) => {
|
||||
vaultKey: v.vault_key,
|
||||
updatedAt: v.updated_at,
|
||||
updatedBy: v.updated_by,
|
||||
updatedByLabel: byId.get(v.updated_by)?.label ?? null,
|
||||
grants: listGrantsForVault(v.vault_key).map(g => ({
|
||||
recipientId: g.recipient_id,
|
||||
label: byId.get(g.recipient_id)?.label ?? null,
|
||||
|
||||
@@ -61,6 +61,17 @@ db.exec(`
|
||||
action TEXT NOT NULL,
|
||||
accessed_at INTEGER NOT NULL
|
||||
);
|
||||
|
||||
-- Latest known deploy tag for a vault — "current state", not a
|
||||
-- history, same one-row-per-vault shape as the vaults table's own
|
||||
-- prev_ciphertext retention. An image tag isn't a secret, so this
|
||||
-- lives outside the encrypted payload entirely.
|
||||
CREATE TABLE IF NOT EXISTS vault_announcements (
|
||||
vault_key TEXT PRIMARY KEY,
|
||||
tag TEXT NOT NULL,
|
||||
announced_at INTEGER NOT NULL,
|
||||
announced_by TEXT NOT NULL
|
||||
);
|
||||
`);
|
||||
|
||||
export interface VaultRow {
|
||||
@@ -88,6 +99,13 @@ export interface GrantRow {
|
||||
granted_at: number;
|
||||
}
|
||||
|
||||
export interface AnnouncementRow {
|
||||
vault_key: string;
|
||||
tag: string;
|
||||
announced_at: number;
|
||||
announced_by: string;
|
||||
}
|
||||
|
||||
export function getVault(vaultKey: string): VaultRow | undefined {
|
||||
return db.prepare(`SELECT * FROM vaults WHERE vault_key = ?`).get(vaultKey) as VaultRow | undefined;
|
||||
}
|
||||
@@ -225,7 +243,7 @@ export function deleteGrant(vaultKey: string, recipientId: string): void {
|
||||
db.prepare(`DELETE FROM vault_grants WHERE vault_key = ? AND recipient_id = ?`).run(vaultKey, recipientId);
|
||||
}
|
||||
|
||||
export function logAccess(vaultKey: string, recipientId: string, action: 'pull' | 'push' | 'grant' | 'revoke'): void {
|
||||
export function logAccess(vaultKey: string, recipientId: string, action: 'pull' | 'push' | 'grant' | 'revoke' | 'announce'): void {
|
||||
db.prepare(
|
||||
`INSERT INTO access_log (vault_key, recipient_id, action, accessed_at) VALUES (?, ?, ?, ?)`,
|
||||
).run(vaultKey, recipientId, action, Math.floor(Date.now() / 1000));
|
||||
@@ -236,3 +254,17 @@ export function getAccessLog(vaultKey: string, limit = 50): { recipient_id: stri
|
||||
`SELECT recipient_id, action, accessed_at FROM access_log WHERE vault_key = ? ORDER BY accessed_at DESC LIMIT ?`,
|
||||
).all(vaultKey, limit) as { recipient_id: string; action: string; accessed_at: number }[];
|
||||
}
|
||||
|
||||
// One row per vault — "latest known tag", not a history. Upsert.
|
||||
export function setAnnouncement(vaultKey: string, tag: string, announcedBy: string): void {
|
||||
db.prepare(
|
||||
`INSERT INTO vault_announcements (vault_key, tag, announced_at, announced_by)
|
||||
VALUES (?, ?, ?, ?)
|
||||
ON CONFLICT(vault_key) DO UPDATE SET
|
||||
tag = excluded.tag, announced_at = excluded.announced_at, announced_by = excluded.announced_by`,
|
||||
).run(vaultKey, tag, Math.floor(Date.now() / 1000), announcedBy);
|
||||
}
|
||||
|
||||
export function getAnnouncement(vaultKey: string): AnnouncementRow | undefined {
|
||||
return db.prepare(`SELECT * FROM vault_announcements WHERE vault_key = ?`).get(vaultKey) as AnnouncementRow | undefined;
|
||||
}
|
||||
|
||||
@@ -11,6 +11,8 @@ import {
|
||||
logAccess,
|
||||
getAccessLog,
|
||||
listRecipients,
|
||||
setAnnouncement,
|
||||
getAnnouncement,
|
||||
} from './db.js';
|
||||
import { requireRecipientAuth, type AuthedRequest } from './auth.js';
|
||||
|
||||
@@ -179,3 +181,42 @@ router.get('/vaults/:key/log', (req: AuthedRequest, res) => {
|
||||
}
|
||||
res.json(getAccessLog(vaultKey));
|
||||
});
|
||||
|
||||
// A deploy tag isn't a secret — it's not stored in the encrypted payload,
|
||||
// just a small side table. Only requires ANY grant (read-only is enough)
|
||||
// since announcing isn't a mutation of vault secrets — a CI identity
|
||||
// should never need write access on a vault just to say "there's a new
|
||||
// image". Resist reaching for hasWriteGrant here by analogy with push.
|
||||
router.post('/vaults/:key/announce', (req: AuthedRequest, res) => {
|
||||
const vaultKey = req.params.key;
|
||||
const recipientId = req.recipientId!;
|
||||
|
||||
if (!hasGrant(vaultKey, recipientId)) {
|
||||
res.status(403).json({ error: 'no access to this vault' });
|
||||
return;
|
||||
}
|
||||
|
||||
const body = req.body as { tag?: string };
|
||||
if (!body.tag) {
|
||||
res.status(400).json({ error: 'tag is required' });
|
||||
return;
|
||||
}
|
||||
|
||||
setAnnouncement(vaultKey, body.tag, recipientId);
|
||||
logAccess(vaultKey, recipientId, 'announce');
|
||||
res.json({ ok: true });
|
||||
});
|
||||
|
||||
router.get('/vaults/:key/announce', (req: AuthedRequest, res) => {
|
||||
const vaultKey = req.params.key;
|
||||
if (!hasGrant(vaultKey, req.recipientId!)) {
|
||||
res.status(403).json({ error: 'no access to this vault' });
|
||||
return;
|
||||
}
|
||||
const announcement = getAnnouncement(vaultKey);
|
||||
if (!announcement) {
|
||||
res.status(404).json({ error: 'nothing announced yet' });
|
||||
return;
|
||||
}
|
||||
res.json({ tag: announcement.tag, announcedAt: announcement.announced_at, announcedBy: announcement.announced_by });
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user