Initial design: self-hosted E2E encrypted secrets sync
README covers the pitch and how this differs from Vault/Doppler (too heavy) and a static age/sops-encrypted file in git (no live rotation, no access log). IMPLEMENTATION covers the multi-recipient key-wrapping scheme (age/sops-style, adapted to libsodium primitives already used in flit/waste-go), the flat vault/recipient/grant/access-log schema, rotation and revocation semantics (including the standard "revoke doesn't retroactively unread already-decrypted secrets" caveat), CLI surface, and deploy-script integration. Motivated by a real, already-felt annoyance: every repo in this project family (goonk, wisp, npm-statuspage, flit, waste-go) has its own hand-copied .env today, with no rotation story and no record of which machine has which secret. No code yet — design stage. Admin auth mechanism flagged as needing a decision before implementation starts. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Fredrik Johansson
247
IMPLEMENTATION.md
Normal file
247
IMPLEMENTATION.md
Normal file
@@ -0,0 +1,247 @@
|
|||||||
|
# keep — implementation notes
|
||||||
|
|
||||||
|
Design-stage document. Nothing here is built yet; this is the spec to
|
||||||
|
build against, not a description of existing code.
|
||||||
|
|
||||||
|
## Threat model
|
||||||
|
|
||||||
|
- The server operator can be assumed hostile-but-honest, same posture as
|
||||||
|
`wisp`: they might try to read stored secrets, but the design should
|
||||||
|
make that cryptographically impossible, not just policy-forbidden.
|
||||||
|
- The server **can** see: which vaults exist, which recipients exist,
|
||||||
|
which recipients are granted access to which vaults, payload size, and
|
||||||
|
access timestamps (who pulled/pushed what, when). It **cannot** see:
|
||||||
|
secret values, the symmetric key that encrypts any payload, or any
|
||||||
|
recipient's private key.
|
||||||
|
- **Not defended against**: a compromised recipient machine. If a
|
||||||
|
deploy box with legitimate access to a vault is compromised, the
|
||||||
|
attacker gets everything that machine's identity can decrypt — same as
|
||||||
|
any secrets manager, self-hosted or not. Revoking that recipient stops
|
||||||
|
*future* pulls; it does not retroactively invalidate secrets already
|
||||||
|
decrypted and sitting in that machine's memory/disk/logs. **The fix for
|
||||||
|
a compromised recipient is always rotating the underlying secrets
|
||||||
|
(`keep push` with new values), not just revoking the recipient.** This
|
||||||
|
is a standard, correct caveat that applies to every system built this
|
||||||
|
way (age, sops, Vault, all of them) — worth stating explicitly because
|
||||||
|
it's the single most common misunderstanding of what "revoke" buys you.
|
||||||
|
- **Not defended against**: a recipient with legitimate access
|
||||||
|
deliberately exfiltrating secrets they're authorized to read. Same
|
||||||
|
out-of-scope boundary wisp draws around a recipient re-sharing a
|
||||||
|
decrypted file — access control stops unauthorized reads, not
|
||||||
|
authorized-then-malicious ones.
|
||||||
|
|
||||||
|
## Crypto scheme
|
||||||
|
|
||||||
|
Per-vault, adapting the `age`/`sops`/PGP multi-recipient pattern (see
|
||||||
|
README's "Core model") to libsodium primitives already used elsewhere in
|
||||||
|
this project family:
|
||||||
|
|
||||||
|
- **Payload encryption**: `crypto_secretbox` (XSalsa20-Poly1305), same
|
||||||
|
primitive `wisp` uses for its blobs. A fresh random 256-bit symmetric
|
||||||
|
key per push.
|
||||||
|
- **Key wrapping**: for each recipient granted access, the vault's
|
||||||
|
symmetric key is sealed to that recipient's X25519 public key via
|
||||||
|
`crypto_box` (or `crypto_box_seal` if the pusher's own identity
|
||||||
|
shouldn't need to be verifiable by the server — anonymous sealing, matches
|
||||||
|
how `flit`'s `Identity.seal()` already works, since the server has no
|
||||||
|
business knowing which recipient's key was used to encrypt which
|
||||||
|
wrapped copy beyond the recipient it's addressed to).
|
||||||
|
- **Identity**: Ed25519 signing keypair per client, with the same
|
||||||
|
sign-to-curve25519 conversion `flit`/`waste-go` already do
|
||||||
|
(`crypto_sign_ed25519_pk_to_curve25519` /
|
||||||
|
`..._sk_to_curve25519`) to derive the X25519 keypair used for the
|
||||||
|
box-sealing above. One identity type, reused for both signing (proving
|
||||||
|
who's pushing/pulling, for the access log) and encryption (wrapping),
|
||||||
|
exactly like the existing projects already do.
|
||||||
|
- **Push authentication**: a push should be signed by the pusher's
|
||||||
|
Ed25519 key so the access log records a real identity, not just "someone
|
||||||
|
with write access to this vault." Whether *anyone* with any registered
|
||||||
|
identity can push to a vault they're a recipient of, or whether push
|
||||||
|
requires a separate "can write" grant distinct from "can read," is an
|
||||||
|
open question below.
|
||||||
|
|
||||||
|
## Data model
|
||||||
|
|
||||||
|
Flat schema, matching the minimalism already established in `wisp` and
|
||||||
|
`npm-statuspage` (no unnecessary normalization — a vault is identified by
|
||||||
|
its own key string, not a foreign-keyed `projects`/`environments` pair):
|
||||||
|
|
||||||
|
```sql
|
||||||
|
CREATE TABLE IF NOT EXISTS vaults (
|
||||||
|
vault_key TEXT PRIMARY KEY, -- e.g. "wisp/production" — free-form, app-chosen
|
||||||
|
ciphertext BLOB NOT NULL, -- crypto_secretbox(payload, vault_symmetric_key)
|
||||||
|
nonce BLOB NOT NULL,
|
||||||
|
updated_at INTEGER NOT NULL,
|
||||||
|
updated_by TEXT NOT NULL -- recipient_id of whoever last pushed
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS recipients (
|
||||||
|
recipient_id TEXT PRIMARY KEY, -- short label-derived slug, e.g. "wisp-prod-deploy"
|
||||||
|
label TEXT NOT NULL, -- human-readable, e.g. "wisp production deploy key"
|
||||||
|
public_key TEXT NOT NULL, -- hex Ed25519 pubkey
|
||||||
|
created_at INTEGER NOT NULL
|
||||||
|
);
|
||||||
|
|
||||||
|
-- The multi-recipient key-wrapping table. One row per (vault, recipient)
|
||||||
|
-- pair that currently has access. Deleting a row IS the revocation —
|
||||||
|
-- no separate "enabled" flag needed.
|
||||||
|
CREATE TABLE IF NOT EXISTS vault_grants (
|
||||||
|
vault_key TEXT NOT NULL REFERENCES vaults(vault_key),
|
||||||
|
recipient_id TEXT NOT NULL REFERENCES recipients(recipient_id),
|
||||||
|
wrapped_key BLOB NOT NULL, -- crypto_box_seal(vault_symmetric_key, recipient_pubkey)
|
||||||
|
granted_at INTEGER NOT NULL,
|
||||||
|
PRIMARY KEY (vault_key, recipient_id)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Access log — metadata only, exactly what the threat model says the
|
||||||
|
-- server is allowed to know.
|
||||||
|
CREATE TABLE IF NOT EXISTS access_log (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
vault_key TEXT NOT NULL,
|
||||||
|
recipient_id TEXT NOT NULL,
|
||||||
|
action TEXT NOT NULL, -- 'pull' | 'push'
|
||||||
|
accessed_at INTEGER NOT NULL
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
A vault with zero rows in `vault_grants` is unreadable by anyone —
|
||||||
|
including, notably, whoever pushed it, unless they granted themselves
|
||||||
|
access first. This is intentional: creating a vault and granting access
|
||||||
|
to it are separate steps, so a vault can't accidentally be pushed
|
||||||
|
"open" to nobody or to a stale recipient list left over from a template.
|
||||||
|
|
||||||
|
## Rotation
|
||||||
|
|
||||||
|
`keep push <vault>` with new secret values:
|
||||||
|
|
||||||
|
1. Generate a fresh random symmetric key.
|
||||||
|
2. Encrypt the new payload under it.
|
||||||
|
3. Look up every current row in `vault_grants` for this vault.
|
||||||
|
4. Re-wrap the new symmetric key for every one of those recipients'
|
||||||
|
public keys (already have them via `recipients.public_key`).
|
||||||
|
5. Replace `vaults.ciphertext`/`nonce` and all `vault_grants.wrapped_key`
|
||||||
|
rows for this vault, atomically (single transaction).
|
||||||
|
|
||||||
|
The old symmetric key and old wrapped copies are simply gone —
|
||||||
|
overwritten, not versioned, in v1 (see Open Questions on whether
|
||||||
|
version history is worth adding).
|
||||||
|
|
||||||
|
## Revocation
|
||||||
|
|
||||||
|
Deleting one `vault_grants` row (`vault_key`, `recipient_id`) removes
|
||||||
|
that recipient's ability to `pull` — the very next pull attempt with
|
||||||
|
their identity finds no wrapped-key row and is rejected. No re-encryption
|
||||||
|
of the payload needed; every other recipient's access is untouched.
|
||||||
|
|
||||||
|
As stated in the threat model: this stops *future* pulls only. If
|
||||||
|
revocation is happening because a machine may have been compromised,
|
||||||
|
follow it with `keep push` to actually rotate the secret values, not just
|
||||||
|
the grant.
|
||||||
|
|
||||||
|
## CLI surface (sketch)
|
||||||
|
|
||||||
|
```
|
||||||
|
keep identity init -- generate + persist a local Ed25519 keypair (~/.keep/identity)
|
||||||
|
keep identity show -- print this machine's public key, for registering as a recipient
|
||||||
|
|
||||||
|
keep push <vault> [--file .env] -- encrypt + upload; requires this identity to already be a grantee
|
||||||
|
keep pull <vault> [--format=env|json] -- fetch + decrypt; requires a grant for this identity
|
||||||
|
|
||||||
|
# admin operations — see below for how these are authenticated
|
||||||
|
keep recipient add --label "..." --pubkey <hex>
|
||||||
|
keep recipient list
|
||||||
|
keep recipient remove <recipient_id>
|
||||||
|
keep grant <vault> <recipient_id>
|
||||||
|
keep revoke <vault> <recipient_id>
|
||||||
|
keep log <vault> -- tail the access log for one vault
|
||||||
|
```
|
||||||
|
|
||||||
|
### Admin operations need their own authorization story
|
||||||
|
|
||||||
|
Adding/removing recipients and granting/revoking vault access is a
|
||||||
|
different trust level than pushing/pulling a specific vault you already
|
||||||
|
have a grant for. Options, in rough order of how much new infrastructure
|
||||||
|
they cost:
|
||||||
|
|
||||||
|
1. **Bootstrap admin identity**: the first recipient ever registered is
|
||||||
|
implicitly the admin; admin operations require a signature from that
|
||||||
|
identity. Cheapest — no new credential type — but doesn't scale past
|
||||||
|
"it's just me" without more thought.
|
||||||
|
2. **Separate `ADMIN_PASSWORD`**, same pattern as `wisp`'s proposed
|
||||||
|
invite-links admin gate and `npm-statuspage`'s admin panel (both
|
||||||
|
already built/proposed as of this writing) — a shared password
|
||||||
|
distinct from any recipient identity, gating a small admin HTTP
|
||||||
|
surface (could be CLI-driven hitting password-gated endpoints, or a
|
||||||
|
minimal web page like npm-statuspage's).
|
||||||
|
3. **Per-vault admin grants** — a `vault_grants.can_admin` boolean
|
||||||
|
alongside read access, so admin authority is scoped per-vault instead
|
||||||
|
of global. More correct, more to build; probably not v1.
|
||||||
|
|
||||||
|
Leaning toward (2) for v1 — it's proven twice already elsewhere in this
|
||||||
|
project family, and a global admin password is an acceptable trust model
|
||||||
|
for "one person or small team running their own homelab," which is the
|
||||||
|
actual scale this is being built for.
|
||||||
|
|
||||||
|
## Deploy integration
|
||||||
|
|
||||||
|
The actual motivating use case — a deploy script pulling secrets instead
|
||||||
|
of assuming a hand-copied `.env` already exists on the target machine:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
# top of an existing deploy-*.sh script, e.g. waste-go's deploy-daemon.sh
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
keep pull wisp/production --format=env > .env
|
||||||
|
# ...rest of the existing deploy script, unchanged, now consuming a
|
||||||
|
# freshly-pulled .env instead of one that was manually placed there
|
||||||
|
```
|
||||||
|
|
||||||
|
Requires the deploy machine to have a `keep` identity already registered
|
||||||
|
and granted access to the relevant vault — a one-time setup step per
|
||||||
|
machine (`keep identity init` once, then an admin grants that machine's
|
||||||
|
public key access to whichever vaults it needs).
|
||||||
|
|
||||||
|
## Security considerations
|
||||||
|
|
||||||
|
- Every point already covered under Threat Model applies; this section
|
||||||
|
covers implementation-level details that aren't strictly part of the
|
||||||
|
threat model but matter for correctness.
|
||||||
|
- **Grant/revoke and push must not race.** If a push and a grant/revoke
|
||||||
|
happen concurrently, the re-wrap step in `push` (which reads current
|
||||||
|
`vault_grants` and re-wraps for all of them) must see a consistent
|
||||||
|
snapshot — wrap the read-grants-then-write-wrapped-keys sequence in a
|
||||||
|
single SQLite transaction, same discipline as wisp's atomic confirm-token
|
||||||
|
consumption.
|
||||||
|
- **The access log is genuinely useful, not just decorative** — the
|
||||||
|
README's whole pitch versus a static `age`-encrypted file in git is
|
||||||
|
"you can tell who actually pulled what, when." Make sure `pull`
|
||||||
|
failures (wrong/no grant) are *also* logged, not just successes — a
|
||||||
|
spike of failed pulls from an unexpected identity is exactly the kind
|
||||||
|
of signal this feature exists to surface.
|
||||||
|
- **Recipient private keys are the actual crown jewels**, same as
|
||||||
|
`flit`/`waste-go`'s existing identity model. `keep identity init`
|
||||||
|
should follow the same local-storage discipline those already use
|
||||||
|
(`~/.flit/`-style, not committed anywhere, not transmitted anywhere
|
||||||
|
except the public half).
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
- **Version history**: keep only the latest payload per vault (simplest,
|
||||||
|
matches the "this predates rollback needs" scope), or retain N previous
|
||||||
|
versions for `keep pull --version=N` rollback? Leaning toward: skip for
|
||||||
|
v1, revisit if a real rollback need shows up.
|
||||||
|
- **Push authorization**: does having a *read* grant (`vault_grants` row)
|
||||||
|
imply push rights too, or is push a separate capability? Leaning toward:
|
||||||
|
read implies write for v1 (simpler, matches "small team, own homelab"
|
||||||
|
scale) — a `can_write` flag on `vault_grants` is the natural extension
|
||||||
|
if that turns out to be wrong.
|
||||||
|
- **Per-secret-key granularity**: v1 treats a vault as one opaque
|
||||||
|
`.env`-shaped blob (matches the actual current pain point: whole files
|
||||||
|
get copied around, not individual keys). If a use case emerges for
|
||||||
|
"grant access to just the DB password, not the whole bundle," that's a
|
||||||
|
genuinely different data model (per-key rows, each separately wrapped)
|
||||||
|
— don't half-build it speculatively now.
|
||||||
|
- **Admin auth mechanism** — see the three options under CLI surface
|
||||||
|
above; needs a decision before build, not deferred to implementation
|
||||||
|
time, since it shapes the admin API surface.
|
||||||
77
README.md
Normal file
77
README.md
Normal file
@@ -0,0 +1,77 @@
|
|||||||
|
# keep
|
||||||
|
|
||||||
|
A self-hosted, end-to-end encrypted secrets store for deploy scripts.
|
||||||
|
`keep pull wisp/production` gets you a `.env`-shaped bundle of secrets,
|
||||||
|
decrypted locally — the server never sees plaintext values, and never
|
||||||
|
sees the key that would let it.
|
||||||
|
|
||||||
|
## Why
|
||||||
|
|
||||||
|
Across this project family — `goonk`, `wisp`, `npm-statuspage`, `flit`,
|
||||||
|
`waste-go` — every repo has its own `.env`/`.env.example` pair, and every
|
||||||
|
one of those `.env` files exists today by being hand-copied onto whatever
|
||||||
|
machine needs it. That's a real, already-felt annoyance, not a
|
||||||
|
hypothetical one:
|
||||||
|
|
||||||
|
- Rotating a credential means finding and updating every machine that has
|
||||||
|
a copy, by hand.
|
||||||
|
- There's no record of which machine has which secret, or when it was
|
||||||
|
last updated.
|
||||||
|
- Onboarding a new deploy target means manually reconstructing a `.env`
|
||||||
|
from memory, a password manager note, or asking "hey, what's the value
|
||||||
|
for X."
|
||||||
|
|
||||||
|
`keep` is a small always-on service that holds encrypted secret bundles
|
||||||
|
and lets authorized machines/people pull them, decrypted locally, without
|
||||||
|
ever exposing plaintext to the server in between.
|
||||||
|
|
||||||
|
## What it isn't
|
||||||
|
|
||||||
|
- **Not Vault or Doppler.** Those are the established, heavier answers —
|
||||||
|
full policy engines, dynamic secrets, enterprise everything. `keep` is
|
||||||
|
the "already-solved problem, but self-hosted and scoped to what this
|
||||||
|
actually needs" answer, closer in spirit to `age`/`sops` (multi-recipient
|
||||||
|
encrypted files) than to a full secrets-management platform.
|
||||||
|
- **Not a static encrypted file committed to git**, which is what
|
||||||
|
`age`/`sops` typically produce. `keep` exists specifically for the two
|
||||||
|
things a committed encrypted file can't give you without re-running a
|
||||||
|
tool and re-committing: **live rotation** (push once, everyone with
|
||||||
|
access can pull the update) and **an access log** (who/what actually
|
||||||
|
fetched a secret bundle, and when — a git history shows content
|
||||||
|
changes, not access).
|
||||||
|
- **Not wisp.** wisp is ephemeral and single-retrieval by design — the
|
||||||
|
opposite of what a secret needs to be. A deploy script needs to fetch
|
||||||
|
the same bundle on every deploy, not once and then have it vanish.
|
||||||
|
|
||||||
|
## Core model
|
||||||
|
|
||||||
|
Adapts the pattern `age`/`sops`/PGP already use for multi-recipient
|
||||||
|
encrypted files — a symmetric key encrypts the actual payload, and that
|
||||||
|
symmetric key is separately "wrapped" (encrypted) once per authorized
|
||||||
|
recipient's public key — into a live service instead of a static file:
|
||||||
|
|
||||||
|
1. Each client (a developer's machine, a deploy script running on a VPS)
|
||||||
|
has its own Ed25519/X25519 keypair — same identity model already
|
||||||
|
built twice over in `flit` and `waste-go`.
|
||||||
|
2. A vault (`wisp/production`, `goonk/production`, whatever key) holds one
|
||||||
|
encrypted blob — the actual secret bundle, encrypted under a random
|
||||||
|
symmetric key.
|
||||||
|
3. That symmetric key is wrapped separately for every recipient granted
|
||||||
|
access to that vault. The server stores N small wrapped-key blobs, one
|
||||||
|
per recipient, alongside the one encrypted payload.
|
||||||
|
4. `keep pull <vault>` fetches the payload plus this identity's wrapped
|
||||||
|
key, unwraps locally with the private key, decrypts, and prints (or
|
||||||
|
writes) the `.env`-shaped result.
|
||||||
|
5. Revoking a recipient's access is deleting their one wrapped-key row —
|
||||||
|
the payload and every other recipient's access are untouched.
|
||||||
|
6. Rotating a secret is `keep push <vault>` again — a new random
|
||||||
|
symmetric key, re-wrapped for every currently-granted recipient. Cheap,
|
||||||
|
since the recipient list rarely changes.
|
||||||
|
|
||||||
|
Full design — data model, the atomic-grant/revoke details, the CLI
|
||||||
|
surface, and what this explicitly does and doesn't protect against — is
|
||||||
|
in [IMPLEMENTATION.md](./IMPLEMENTATION.md).
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
Design only. No code yet.
|
||||||
Reference in New Issue
Block a user