Initial design: async single-retrieval encrypted file drop
README covers the pitch and how this differs from flit (synchronous, zero server storage) and zipline (persistent hosting). IMPLEMENTATION covers the crypto scheme (key-in-fragment, same primitives as flit/waste-go), storage shape, and the deletion-race problem — delete-on-first-byte-served is the wrong default (see Firefox Send's history with this exact bug); confirm-then-delete with a TTL backstop is the fix. No code yet — design stage. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
148
IMPLEMENTATION.md
Normal file
148
IMPLEMENTATION.md
Normal file
@@ -0,0 +1,148 @@
|
|||||||
|
# wisp — implementation notes
|
||||||
|
|
||||||
|
Design-stage document. Nothing here is built yet; this is the spec to build
|
||||||
|
against, not a description of existing code.
|
||||||
|
|
||||||
|
## Threat model
|
||||||
|
|
||||||
|
- The server operator can be assumed hostile-but-honest: they might try to
|
||||||
|
read stored files, but the design should make that cryptographically
|
||||||
|
impossible, not just policy-forbidden.
|
||||||
|
- The server **can** see: upload/download timing, file size (ciphertext
|
||||||
|
length), IP addresses of uploader/downloader, and that a given blob id
|
||||||
|
was retrieved. It **cannot** see: filename, file contents, or the
|
||||||
|
encryption key.
|
||||||
|
- Not defended against: a malicious *recipient* who screenshots or
|
||||||
|
re-uploads the file elsewhere after decrypting it. Nothing can defend
|
||||||
|
against that — it's out of scope, same as it is for flit.
|
||||||
|
- Not defended against: a network observer between sender and the person
|
||||||
|
they share the link with (e.g. if the link is sent over an insecure
|
||||||
|
channel). The link itself is the credential; treat it like a password.
|
||||||
|
|
||||||
|
## Crypto scheme
|
||||||
|
|
||||||
|
- Client generates a random symmetric key (32 bytes, libsodium
|
||||||
|
`crypto_secretbox` or `crypto_aead_xchacha20poly1305_ietf` — same
|
||||||
|
primitives already used across flit/waste-go, no new crypto library to
|
||||||
|
vet).
|
||||||
|
- File is encrypted client-side before any network request. For files
|
||||||
|
larger than a single-shot encrypt is comfortable with in a browser tab
|
||||||
|
(say, >50MB), chunk into fixed-size blocks, each sealed independently
|
||||||
|
with a per-chunk nonce derived from a chunk index — avoids holding the
|
||||||
|
whole plaintext or ciphertext in memory at once. (flit already solved
|
||||||
|
streamed chunked transfer over DataChannels; the chunking logic is
|
||||||
|
reusable even though the transport here is HTTP upload, not WebRTC.)
|
||||||
|
- The key is placed in the URL fragment: `https://wisp.example/d/<blob-id>#k=<key-b64>`.
|
||||||
|
Fragments are never sent to the server in an HTTP request — this is the
|
||||||
|
same trick Firefox Send and PrivateBin used, and it's the load-bearing
|
||||||
|
property that makes "server never sees plaintext, ever" actually true
|
||||||
|
rather than just a policy.
|
||||||
|
- `blob-id` is a random, unguessable identifier (128-bit, base62), separate
|
||||||
|
from the key. Knowing the id alone (e.g. from server logs) does not grant
|
||||||
|
access to the file — you need the fragment too, which never reaches the
|
||||||
|
server.
|
||||||
|
|
||||||
|
## Storage
|
||||||
|
|
||||||
|
- Ciphertext blobs on local disk, one file per upload, named by `blob-id`.
|
||||||
|
- Metadata in SQLite (matches the `npm-statuspage`/general project
|
||||||
|
convention — small, file-based, no separate DB server to run):
|
||||||
|
|
||||||
|
```sql
|
||||||
|
CREATE TABLE drops (
|
||||||
|
id TEXT PRIMARY KEY, -- blob-id, 128-bit base62
|
||||||
|
size_bytes INTEGER NOT NULL,
|
||||||
|
created_at INTEGER NOT NULL, -- unix seconds
|
||||||
|
expires_at INTEGER NOT NULL, -- created_at + TTL
|
||||||
|
downloaded_at INTEGER, -- set once, on confirmed decrypt
|
||||||
|
burned INTEGER NOT NULL DEFAULT 0 -- 1 once deleted
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
- No filename, no content-type, nothing plaintext-adjacent stored server
|
||||||
|
side. If the client wants to preserve a filename across the transfer, it
|
||||||
|
goes *inside* the encrypted payload (e.g. a small JSON header before the
|
||||||
|
file bytes, encrypted along with everything else), not in the database.
|
||||||
|
|
||||||
|
## The deletion race — why "delete on first byte served" is wrong
|
||||||
|
|
||||||
|
The naive version: server deletes the blob as soon as the download request
|
||||||
|
starts (or completes, from the server's own point of view). Two failure
|
||||||
|
modes:
|
||||||
|
|
||||||
|
1. **Network drops mid-download.** The recipient's connection dies at 80%.
|
||||||
|
The server already considers the file "served" and deletes it. The
|
||||||
|
recipient has an unrecoverable partial file and no way to retry — the
|
||||||
|
link is now dead. This is the exact bug that made Firefox Send
|
||||||
|
frustrating in practice.
|
||||||
|
2. **Decrypt fails client-side** (corrupted download, browser crash mid-
|
||||||
|
decrypt, wrong key from a mistyped/truncated link). Same outcome: file
|
||||||
|
is gone from the server, but the recipient never actually got it.
|
||||||
|
|
||||||
|
**The fix**: the server serves the blob but does not delete it yet. The
|
||||||
|
*client* decrypts and verifies (the AEAD tag on the ciphertext gives you
|
||||||
|
integrity for free — if decrypt succeeds, the bytes are correct). Only on
|
||||||
|
successful decrypt does the client send a short authenticated "confirm"
|
||||||
|
request back to the server:
|
||||||
|
|
||||||
|
```
|
||||||
|
POST /api/drops/<blob-id>/confirm
|
||||||
|
Authorization: Bearer <confirm-token> -- returned alongside the download, single-use
|
||||||
|
```
|
||||||
|
|
||||||
|
The confirm-token is issued at download time, tied to that specific
|
||||||
|
download attempt, and single-use — so a network observer who sees the
|
||||||
|
download request can't independently trigger deletion without also having
|
||||||
|
successfully completed (or at least intercepted) the actual file transfer.
|
||||||
|
On confirm, the server deletes the blob and disk file, sets `burned = 1`.
|
||||||
|
|
||||||
|
If confirm never arrives (dropped connection, closed tab, whatever), the
|
||||||
|
blob **stays available for retry** — the recipient can just reload the
|
||||||
|
link and try again — right up until the TTL backstop expires it anyway.
|
||||||
|
This trades "guaranteed single-view" for "no silent unrecoverable
|
||||||
|
failures," which is the right tradeoff: the TTL already bounds how long a
|
||||||
|
never-confirmed file survives, so the failure mode of "someone could
|
||||||
|
technically re-download before confirming" is bounded in time and no worse
|
||||||
|
than a slightly-too-generous TTL.
|
||||||
|
|
||||||
|
## TTL and cleanup
|
||||||
|
|
||||||
|
- Default TTL: configurable, a few days seems reasonable for "someone else
|
||||||
|
needs to get around to opening this."
|
||||||
|
- A cron/interval job (same pattern as `npm-statuspage`'s scheduled checks)
|
||||||
|
sweeps `drops` where `expires_at < now AND burned = 0`, deletes the disk
|
||||||
|
file, marks burned.
|
||||||
|
- Rows are kept after burning (not hard-deleted from SQLite) for basic
|
||||||
|
abuse/debugging visibility, pruned separately on a much longer schedule.
|
||||||
|
|
||||||
|
## API surface (sketch)
|
||||||
|
|
||||||
|
```
|
||||||
|
POST /api/drops -- multipart or raw body upload; returns { id, ttl }
|
||||||
|
GET /api/drops/:id -- streams ciphertext; issues a confirm-token
|
||||||
|
POST /api/drops/:id/confirm -- single-use, triggers deletion
|
||||||
|
```
|
||||||
|
|
||||||
|
No auth on upload initially (matches flit/waste-go's "no accounts" ethos),
|
||||||
|
but needs abuse guardrails since — unlike flit — this one actually costs
|
||||||
|
disk space per upload:
|
||||||
|
|
||||||
|
- Max file size cap (needs a number; start conservative, e.g. a few hundred
|
||||||
|
MB, revisit based on actual use).
|
||||||
|
- Per-IP rate limit on uploads.
|
||||||
|
- Maybe: require the uploader to have a known waste-go/flit identity
|
||||||
|
(Ed25519 pubkey signs the upload request) to unlock a larger size limit
|
||||||
|
or bypass rate limits — reuses identity infra that already exists rather
|
||||||
|
than building new auth. Optional, not required for v1.
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
- Exact TTL default and max file size — pick numbers once there's a rough
|
||||||
|
sense of real usage, not before.
|
||||||
|
- Whether to support multi-recipient (same link retrievable by N people
|
||||||
|
before burning) — v1 should be strictly single-retrieval; multi-retrieval
|
||||||
|
is a distinct feature with different deletion semantics, not a variant.
|
||||||
|
- Whether filename/content-type belongs inside the encrypted envelope
|
||||||
|
(current lean: yes) or should be a separate unencrypted-but-harmless
|
||||||
|
field — leaning toward "inside," since even a filename can leak
|
||||||
|
information the sender didn't intend to share with the server operator.
|
||||||
49
README.md
Normal file
49
README.md
Normal file
@@ -0,0 +1,49 @@
|
|||||||
|
# wisp
|
||||||
|
|
||||||
|
Short-lived, end-to-end encrypted file sharing. Send a file to someone who
|
||||||
|
isn't online right now — they get a link, they get the file once, and then
|
||||||
|
it's gone. The server never has the ability to read what it's holding.
|
||||||
|
|
||||||
|
## Why
|
||||||
|
|
||||||
|
`flit` already does encrypted file transfer between devices, but it's
|
||||||
|
synchronous — both sides have to be online at the same time for the WebRTC
|
||||||
|
handshake. That's fine for "send this to my other laptop right now," and
|
||||||
|
wrong for "here's a file for you, grab it whenever."
|
||||||
|
|
||||||
|
wisp is the asynchronous counterpart: upload once, the recipient retrieves
|
||||||
|
later, on their own schedule. That async gap is the one thing flit's
|
||||||
|
architecture (zero server storage, direct P2P) structurally can't do — it
|
||||||
|
requires the server to actually hold ciphertext for a while, which is new
|
||||||
|
territory for this project family, not a variant of something already built.
|
||||||
|
|
||||||
|
## What it isn't
|
||||||
|
|
||||||
|
- **Not zipline.** Zipline is persistent link/screenshot hosting — you keep
|
||||||
|
the URL, the file stays. wisp is the opposite: single retrieval, then the
|
||||||
|
file is unrecoverable, by design.
|
||||||
|
- **Not flit.** Flit needs both peers online simultaneously and never stores
|
||||||
|
anything server-side. wisp is deliberately async and requires temporary
|
||||||
|
server-side ciphertext storage.
|
||||||
|
|
||||||
|
## Core model
|
||||||
|
|
||||||
|
1. Sender encrypts the file client-side (symmetric, libsodium secretbox —
|
||||||
|
same primitive flit's browser client already uses).
|
||||||
|
2. Ciphertext uploads to the server. The encryption key never leaves the
|
||||||
|
sender's browser except inside the URL fragment (`#key=...`), which
|
||||||
|
browsers never transmit in HTTP requests. The server only ever sees
|
||||||
|
opaque bytes.
|
||||||
|
3. Sender shares the link through any channel they like.
|
||||||
|
4. Recipient opens the link, downloads the ciphertext, decrypts client-side.
|
||||||
|
5. On successful decrypt, the client sends a short confirmation to the
|
||||||
|
server. Only then does the server delete the blob.
|
||||||
|
6. A TTL (default: a few days) is the backstop for links nobody ever opens,
|
||||||
|
so nothing sits on disk forever.
|
||||||
|
|
||||||
|
Full design, including why "delete on first byte served" is the wrong
|
||||||
|
default, is in [IMPLEMENTATION.md](./IMPLEMENTATION.md).
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
Design only. No code yet.
|
||||||
Reference in New Issue
Block a user