Initial design: async single-retrieval encrypted file drop
README covers the pitch and how this differs from flit (synchronous, zero server storage) and zipline (persistent hosting). IMPLEMENTATION covers the crypto scheme (key-in-fragment, same primitives as flit/waste-go), storage shape, and the deletion-race problem — delete-on-first-byte-served is the wrong default (see Firefox Send's history with this exact bug); confirm-then-delete with a TTL backstop is the fix. No code yet — design stage. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
148
IMPLEMENTATION.md
Normal file
148
IMPLEMENTATION.md
Normal file
@@ -0,0 +1,148 @@
|
||||
# wisp — implementation notes
|
||||
|
||||
Design-stage document. Nothing here is built yet; this is the spec to build
|
||||
against, not a description of existing code.
|
||||
|
||||
## Threat model
|
||||
|
||||
- The server operator can be assumed hostile-but-honest: they might try to
|
||||
read stored files, but the design should make that cryptographically
|
||||
impossible, not just policy-forbidden.
|
||||
- The server **can** see: upload/download timing, file size (ciphertext
|
||||
length), IP addresses of uploader/downloader, and that a given blob id
|
||||
was retrieved. It **cannot** see: filename, file contents, or the
|
||||
encryption key.
|
||||
- Not defended against: a malicious *recipient* who screenshots or
|
||||
re-uploads the file elsewhere after decrypting it. Nothing can defend
|
||||
against that — it's out of scope, same as it is for flit.
|
||||
- Not defended against: a network observer between sender and the person
|
||||
they share the link with (e.g. if the link is sent over an insecure
|
||||
channel). The link itself is the credential; treat it like a password.
|
||||
|
||||
## Crypto scheme
|
||||
|
||||
- Client generates a random symmetric key (32 bytes, libsodium
|
||||
`crypto_secretbox` or `crypto_aead_xchacha20poly1305_ietf` — same
|
||||
primitives already used across flit/waste-go, no new crypto library to
|
||||
vet).
|
||||
- File is encrypted client-side before any network request. For files
|
||||
larger than a single-shot encrypt is comfortable with in a browser tab
|
||||
(say, >50MB), chunk into fixed-size blocks, each sealed independently
|
||||
with a per-chunk nonce derived from a chunk index — avoids holding the
|
||||
whole plaintext or ciphertext in memory at once. (flit already solved
|
||||
streamed chunked transfer over DataChannels; the chunking logic is
|
||||
reusable even though the transport here is HTTP upload, not WebRTC.)
|
||||
- The key is placed in the URL fragment: `https://wisp.example/d/<blob-id>#k=<key-b64>`.
|
||||
Fragments are never sent to the server in an HTTP request — this is the
|
||||
same trick Firefox Send and PrivateBin used, and it's the load-bearing
|
||||
property that makes "server never sees plaintext, ever" actually true
|
||||
rather than just a policy.
|
||||
- `blob-id` is a random, unguessable identifier (128-bit, base62), separate
|
||||
from the key. Knowing the id alone (e.g. from server logs) does not grant
|
||||
access to the file — you need the fragment too, which never reaches the
|
||||
server.
|
||||
|
||||
## Storage
|
||||
|
||||
- Ciphertext blobs on local disk, one file per upload, named by `blob-id`.
|
||||
- Metadata in SQLite (matches the `npm-statuspage`/general project
|
||||
convention — small, file-based, no separate DB server to run):
|
||||
|
||||
```sql
|
||||
CREATE TABLE drops (
|
||||
id TEXT PRIMARY KEY, -- blob-id, 128-bit base62
|
||||
size_bytes INTEGER NOT NULL,
|
||||
created_at INTEGER NOT NULL, -- unix seconds
|
||||
expires_at INTEGER NOT NULL, -- created_at + TTL
|
||||
downloaded_at INTEGER, -- set once, on confirmed decrypt
|
||||
burned INTEGER NOT NULL DEFAULT 0 -- 1 once deleted
|
||||
);
|
||||
```
|
||||
|
||||
- No filename, no content-type, nothing plaintext-adjacent stored server
|
||||
side. If the client wants to preserve a filename across the transfer, it
|
||||
goes *inside* the encrypted payload (e.g. a small JSON header before the
|
||||
file bytes, encrypted along with everything else), not in the database.
|
||||
|
||||
## The deletion race — why "delete on first byte served" is wrong
|
||||
|
||||
The naive version: server deletes the blob as soon as the download request
|
||||
starts (or completes, from the server's own point of view). Two failure
|
||||
modes:
|
||||
|
||||
1. **Network drops mid-download.** The recipient's connection dies at 80%.
|
||||
The server already considers the file "served" and deletes it. The
|
||||
recipient has an unrecoverable partial file and no way to retry — the
|
||||
link is now dead. This is the exact bug that made Firefox Send
|
||||
frustrating in practice.
|
||||
2. **Decrypt fails client-side** (corrupted download, browser crash mid-
|
||||
decrypt, wrong key from a mistyped/truncated link). Same outcome: file
|
||||
is gone from the server, but the recipient never actually got it.
|
||||
|
||||
**The fix**: the server serves the blob but does not delete it yet. The
|
||||
*client* decrypts and verifies (the AEAD tag on the ciphertext gives you
|
||||
integrity for free — if decrypt succeeds, the bytes are correct). Only on
|
||||
successful decrypt does the client send a short authenticated "confirm"
|
||||
request back to the server:
|
||||
|
||||
```
|
||||
POST /api/drops/<blob-id>/confirm
|
||||
Authorization: Bearer <confirm-token> -- returned alongside the download, single-use
|
||||
```
|
||||
|
||||
The confirm-token is issued at download time, tied to that specific
|
||||
download attempt, and single-use — so a network observer who sees the
|
||||
download request can't independently trigger deletion without also having
|
||||
successfully completed (or at least intercepted) the actual file transfer.
|
||||
On confirm, the server deletes the blob and disk file, sets `burned = 1`.
|
||||
|
||||
If confirm never arrives (dropped connection, closed tab, whatever), the
|
||||
blob **stays available for retry** — the recipient can just reload the
|
||||
link and try again — right up until the TTL backstop expires it anyway.
|
||||
This trades "guaranteed single-view" for "no silent unrecoverable
|
||||
failures," which is the right tradeoff: the TTL already bounds how long a
|
||||
never-confirmed file survives, so the failure mode of "someone could
|
||||
technically re-download before confirming" is bounded in time and no worse
|
||||
than a slightly-too-generous TTL.
|
||||
|
||||
## TTL and cleanup
|
||||
|
||||
- Default TTL: configurable, a few days seems reasonable for "someone else
|
||||
needs to get around to opening this."
|
||||
- A cron/interval job (same pattern as `npm-statuspage`'s scheduled checks)
|
||||
sweeps `drops` where `expires_at < now AND burned = 0`, deletes the disk
|
||||
file, marks burned.
|
||||
- Rows are kept after burning (not hard-deleted from SQLite) for basic
|
||||
abuse/debugging visibility, pruned separately on a much longer schedule.
|
||||
|
||||
## API surface (sketch)
|
||||
|
||||
```
|
||||
POST /api/drops -- multipart or raw body upload; returns { id, ttl }
|
||||
GET /api/drops/:id -- streams ciphertext; issues a confirm-token
|
||||
POST /api/drops/:id/confirm -- single-use, triggers deletion
|
||||
```
|
||||
|
||||
No auth on upload initially (matches flit/waste-go's "no accounts" ethos),
|
||||
but needs abuse guardrails since — unlike flit — this one actually costs
|
||||
disk space per upload:
|
||||
|
||||
- Max file size cap (needs a number; start conservative, e.g. a few hundred
|
||||
MB, revisit based on actual use).
|
||||
- Per-IP rate limit on uploads.
|
||||
- Maybe: require the uploader to have a known waste-go/flit identity
|
||||
(Ed25519 pubkey signs the upload request) to unlock a larger size limit
|
||||
or bypass rate limits — reuses identity infra that already exists rather
|
||||
than building new auth. Optional, not required for v1.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Exact TTL default and max file size — pick numbers once there's a rough
|
||||
sense of real usage, not before.
|
||||
- Whether to support multi-recipient (same link retrievable by N people
|
||||
before burning) — v1 should be strictly single-retrieval; multi-retrieval
|
||||
is a distinct feature with different deletion semantics, not a variant.
|
||||
- Whether filename/content-type belongs inside the encrypted envelope
|
||||
(current lean: yes) or should be a separate unencrypted-but-harmless
|
||||
field — leaning toward "inside," since even a filename can leak
|
||||
information the sender didn't intend to share with the server operator.
|
||||
Reference in New Issue
Block a user