explewd/keep
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Implement keep: self-hosted E2E encrypted secrets sync

Browse Source 
Server: Express + better-sqlite3 (WAL), multi-recipient key-wrapping
per IMPLEMENTATION.md's design — vaults/recipients/vault_grants/
access_log. Two auth paths: ADMIN_PASSWORD header for recipient
management and revoke (pure metadata operations), signed-request
auth (Ed25519 signature over method+path+timestamp+body-hash) for
push/pull/grant, mirroring the spirit of this project family's other
signed-handshake patterns without naming them.

CLI: identity init/show/set-id, push/pull/grant/log, admin recipient
add/list/remove and revoke. Grant is a client-side crypto operation
(the granter unwraps the vault's current key locally and reseals it
for the new recipient) rather than a server-side operation, since the
server never holds an unwrapped key to grant with.

Verified end-to-end with two independent local identities against a
live server and separately against the built Docker image: register,
push, pull (granted and ungranted), grant without re-pushing, admin
revoke, a subsequent rotation confirming the revoked recipient stays
excluded, and rejection of missing/malformed signed-request auth.

Two real bugs caught during verification, not just written up:
- libsodium-wrappers' published ESM build does a relative import only
  resolvable under bundler-style resolution — broken under plain Node
  ESM. Fixed via createRequire to force the CJS build.
- Express's req.path inside a sub-router is relative to the mount
  point, which would have silently mismatched a client signing the
  full request path. Fixed by verifying against req.originalUrl.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Fredrik Johansson
2026-07-12 19:38:24 +02:00
parent 2c34562c7f
commit 67000b66ee
27 changed files with 3343 additions and 106 deletions
Download Patch File Download Diff File Expand all files Collapse all files

184
IMPLEMENTATION.md
View File

@@ -1,13 +1,13 @@
# keep — implementation notes
Design-stage document. Nothing here is built yet; this is the spec to
build against, not a description of existing code.
Design-stage document, now implemented — kept as the record of *why* the
design looks the way it does, not just a description of the code.
## Threat model
- The server operator can be assumed hostile-but-honest, same posture as
`wisp`: they might try to read stored secrets, but the design should
make that cryptographically impossible, not just policy-forbidden.
- The server operator can be assumed hostile-but-honest: they might try
to read stored secrets, but the design should make that cryptographically
impossible, not just policy-forbidden.
- The server **can** see: which vaults exist, which recipients exist,
which recipients are granted access to which vaults, payload size, and
access timestamps (who pulled/pushed what, when). It **cannot** see:
@@ -25,50 +25,43 @@ build against, not a description of existing code.
way (age, sops, Vault, all of them) — worth stating explicitly because
it's the single most common misunderstanding of what "revoke" buys you.
- **Not defended against**: a recipient with legitimate access
deliberately exfiltrating secrets they're authorized to read. Same
out-of-scope boundary wisp draws around a recipient re-sharing a
decrypted file — access control stops unauthorized reads, not
authorized-then-malicious ones.
deliberately exfiltrating secrets they're authorized to read. Access
control stops unauthorized reads, not authorized-then-malicious ones —
the same boundary any system like this draws.
## Crypto scheme
Per-vault, adapting the `age`/`sops`/PGP multi-recipient pattern (see
README's "Core model") to libsodium primitives already used elsewhere in
this project family:
README's "Core model") to libsodium primitives:
- **Payload encryption**: `crypto_secretbox` (XSalsa20-Poly1305), same
primitive `wisp` uses for its blobs. A fresh random 256-bit symmetric
key per push.
- **Payload encryption**: `crypto_secretbox` (XSalsa20-Poly1305). A fresh
random 256-bit symmetric key per push.
- **Key wrapping**: for each recipient granted access, the vault's
symmetric key is sealed to that recipient's X25519 public key via
`crypto_box` (or `crypto_box_seal` if the pusher's own identity
shouldn't need to be verifiable by the server — anonymous sealing, matches
how `flit`'s `Identity.seal()` already works, since the server has no
business knowing which recipient's key was used to encrypt which
wrapped copy beyond the recipient it's addressed to).
- **Identity**: Ed25519 signing keypair per client, with the same
sign-to-curve25519 conversion `flit`/`waste-go` already do
(`crypto_sign_ed25519_pk_to_curve25519` /
`crypto_box_seal` — anonymous sealing, since the server has no business
knowing which specific identity wrapped a copy beyond the recipient it's
addressed to.
- **Identity**: Ed25519 signing keypair per client, with the standard
sign-to-curve25519 conversion (`crypto_sign_ed25519_pk_to_curve25519` /
`..._sk_to_curve25519`) to derive the X25519 keypair used for the
box-sealing above. One identity type, reused for both signing (proving
who's pushing/pulling, for the access log) and encryption (wrapping),
exactly like the existing projects already do.
- **Push authentication**: a push should be signed by the pusher's
Ed25519 key so the access log records a real identity, not just "someone
with write access to this vault." Whether *anyone* with any registered
who's pushing/pulling, for the access log) and encryption (wrapping).
- **Push authentication**: every push is signed by the pusher's Ed25519
key so the access log records a real identity, not just "someone with
write access to this vault." Whether *anyone* with any registered
identity can push to a vault they're a recipient of, or whether push
requires a separate "can write" grant distinct from "can read," is an
open question below.
requires a separate "can write" grant distinct from "can read," was an
open question at design time — resolved as "read implies write" for v1
(see Open Questions).
## Data model
Flat schema, matching the minimalism already established in `wisp` and
`npm-statuspage` (no unnecessary normalization — a vault is identified by
its own key string, not a foreign-keyed `projects`/`environments` pair):
Flat schema — no unnecessary normalization; a vault is identified by its
own key string, not a foreign-keyed `projects`/`environments` pair:
```sql
CREATE TABLE IF NOT EXISTS vaults (
vault_key TEXT PRIMARY KEY, -- e.g. "wisp/production" — free-form, app-chosen
vault_key TEXT PRIMARY KEY, -- e.g. "myapp/production" — free-form, app-chosen
ciphertext BLOB NOT NULL, -- crypto_secretbox(payload, vault_symmetric_key)
nonce BLOB NOT NULL,
updated_at INTEGER NOT NULL,
@@ -76,8 +69,8 @@ CREATE TABLE IF NOT EXISTS vaults (
);
CREATE TABLE IF NOT EXISTS recipients (
recipient_id TEXT PRIMARY KEY, -- short label-derived slug, e.g. "wisp-prod-deploy"
label TEXT NOT NULL, -- human-readable, e.g. "wisp production deploy key"
recipient_id TEXT PRIMARY KEY, -- short label-derived slug, e.g. "prod-deploy-a1b2c3"
label TEXT NOT NULL, -- human-readable, e.g. "production deploy key"
public_key TEXT NOT NULL, -- hex Ed25519 pubkey
created_at INTEGER NOT NULL
);
@@ -138,49 +131,47 @@ revocation is happening because a machine may have been compromised,
follow it with `keep push` to actually rotate the secret values, not just
the grant.
## CLI surface (sketch)
## CLI surface
```
keep identity init -- generate + persist a local Ed25519 keypair (~/.keep/identity)
keep identity init -- generate + persist a local Ed25519 keypair (~/.keep/identity.json)
keep identity show -- print this machine's public key, for registering as a recipient
keep identity set-id <recipient-id> -- record the id an admin assigned after registering this identity
keep push <vault> [--file .env] -- encrypt + upload; requires this identity to already be a grantee
keep pull <vault> [--format=env|json] -- fetch + decrypt; requires a grant for this identity
keep pull <vault> [--format env|json] [--out <path>] -- fetch + decrypt; requires a grant for this identity
keep grant <vault> <recipient-id> -- add a new recipient to a vault you already have access to
keep log <vault> -- tail the access log for one vault
# admin operations — see below for how these are authenticated
# admin operations — gated by ADMIN_PASSWORD, see below
keep recipient add --label "..." --pubkey <hex>
keep recipient list
keep recipient remove <recipient_id>
keep grant <vault> <recipient_id>
keep revoke <vault> <recipient_id>
keep log <vault> -- tail the access log for one vault
keep recipient remove <recipient-id>
keep revoke <vault> <recipient-id>
```
### Admin operations need their own authorization story
Adding/removing recipients and granting/revoking vault access is a
different trust level than pushing/pulling a specific vault you already
have a grant for. Options, in rough order of how much new infrastructure
they cost:
Adding/removing recipients and revoking vault access is a different trust
level than pushing/pulling a specific vault you already have a grant for.
Options considered, in rough order of how much new infrastructure they
cost:
1. **Bootstrap admin identity**: the first recipient ever registered is
implicitly the admin; admin operations require a signature from that
identity. Cheapest — no new credential type — but doesn't scale past
"it's just me" without more thought.
2. **Separate `ADMIN_PASSWORD`**, same pattern as `wisp`'s proposed
invite-links admin gate and `npm-statuspage`'s admin panel (both
already built/proposed as of this writing) — a shared password
distinct from any recipient identity, gating a small admin HTTP
surface (could be CLI-driven hitting password-gated endpoints, or a
minimal web page like npm-statuspage's).
2. **Separate `ADMIN_PASSWORD`** — a shared password distinct from any
recipient identity, gating a small admin HTTP surface. This is what
got built: same shape as a password-gated admin panel, disabled
entirely (503) rather than boot-failing when unset.
3. **Per-vault admin grants** — a `vault_grants.can_admin` boolean
alongside read access, so admin authority is scoped per-vault instead
of global. More correct, more to build; probably not v1.
of global. More correct, more to build; not v1.
Leaning toward (2) for v1 — it's proven twice already elsewhere in this
project family, and a global admin password is an acceptable trust model
for "one person or small team running their own homelab," which is the
actual scale this is being built for.
Went with (2) — a global admin password is an acceptable trust model for
"one person or small team running their own homelab," which is the
actual scale this is built for.
## Deploy integration
@@ -189,18 +180,19 @@ of assuming a hand-copied `.env` already exists on the target machine:
```bash
#!/usr/bin/env bash
# top of an existing deploy-*.sh script, e.g. waste-go's deploy-daemon.sh
set -euo pipefail
keep pull wisp/production --format=env > .env
# ...rest of the existing deploy script, unchanged, now consuming a
keep pull myapp/production --format env --out .env
# ...rest of an existing deploy script, unchanged, now consuming a
# freshly-pulled .env instead of one that was manually placed there
```
Requires the deploy machine to have a `keep` identity already registered
and granted access to the relevant vault — a one-time setup step per
machine (`keep identity init` once, then an admin grants that machine's
public key access to whichever vaults it needs).
public key access to whichever vaults it needs). See
[INTEGRATION.md](./INTEGRATION.md) for where this pattern does and
doesn't fit an existing deploy setup.
## Security considerations
@@ -210,38 +202,50 @@ public key access to whichever vaults it needs).
- **Grant/revoke and push must not race.** If a push and a grant/revoke
happen concurrently, the re-wrap step in `push` (which reads current
`vault_grants` and re-wraps for all of them) must see a consistent
snapshot — wrap the read-grants-then-write-wrapped-keys sequence in a
single SQLite transaction, same discipline as wisp's atomic confirm-token
consumption.
snapshot — the read-grants-then-write-wrapped-keys sequence runs inside
a single SQLite transaction.
- **The access log is genuinely useful, not just decorative** — the
README's whole pitch versus a static `age`-encrypted file in git is
"you can tell who actually pulled what, when." Make sure `pull`
failures (wrong/no grant) are *also* logged, not just successes — a
spike of failed pulls from an unexpected identity is exactly the kind
of signal this feature exists to surface.
- **Recipient private keys are the actual crown jewels**, same as
`flit`/`waste-go`'s existing identity model. `keep identity init`
should follow the same local-storage discipline those already use
(`~/.flit/`-style, not committed anywhere, not transmitted anywhere
except the public half).
"you can tell who actually pulled what, when." `pull` failures
(wrong/no grant) are logged too, not just successes — a spike of failed
pulls from an unexpected identity is exactly the kind of signal this
feature exists to surface.
- **Recipient private keys are the actual crown jewels.** `keep identity
init` persists the local keypair at `~/.keep/identity.json`
(mode `0o600`, directory `0o700`) — not committed anywhere, not
transmitted anywhere except the public half.
## What was verified end-to-end
Two independent local identities against a real running server (and
separately against the built Docker image): registration, a push,
pulling as the pusher, pulling as an ungranted second identity (correctly
rejected), granting the second identity access without re-pushing,
pulling as the now-granted second identity (correctly decrypts the same
underlying secret via its own sealed key copy), admin revocation, a
subsequent rotation confirming the revoked identity is excluded from the
new wrap set, and rejection of both missing and malformed signed-request
auth.
One real implementation bug caught during this process, worth recording:
Express's `req.path` inside a sub-router is relative to that router's
mount point (e.g. `/vaults/x/pull` instead of `/api/vaults/x/pull`),
which would have silently mismatched a client that signs the full request
path. Fixed by verifying against `req.originalUrl` (path portion only)
instead, which stays consistent regardless of router nesting.
## Open questions
- **Version history**: keep only the latest payload per vault (simplest,
matches the "this predates rollback needs" scope), or retain N previous
versions for `keep pull --version=N` rollback? Leaning toward: skip for
v1, revisit if a real rollback need shows up.
- **Version history**: keep only the latest payload per vault (what got
built), or retain N previous versions for `keep pull --version=N`
rollback? Skipped for v1, revisit if a real rollback need shows up.
- **Push authorization**: does having a *read* grant (`vault_grants` row)
imply push rights too, or is push a separate capability? Leaning toward:
read implies write for v1 (simpler, matches "small team, own homelab"
scale) — a `can_write` flag on `vault_grants` is the natural extension
if that turns out to be wrong.
imply push rights too, or is push a separate capability? Went with
"read implies write" for v1 — a `can_write` flag on `vault_grants` is
the natural extension if that turns out to be wrong.
- **Per-secret-key granularity**: v1 treats a vault as one opaque
`.env`-shaped blob (matches the actual current pain point: whole files
get copied around, not individual keys). If a use case emerges for
"grant access to just the DB password, not the whole bundle," that's a
genuinely different data model (per-key rows, each separately wrapped)
— don't half-build it speculatively now.
- **Admin auth mechanism** — see the three options under CLI surface
above; needs a decision before build, not deferred to implementation
time, since it shapes the admin API surface.
`.env`-shaped blob (matches the actual pain point this was built for:
whole files get copied around, not individual keys). If a use case
emerges for "grant access to just one value, not the whole bundle,"
that's a genuinely different data model (per-key rows, each separately
wrapped) — don't half-build it speculatively.