Initial build: stash server + browser extension
Server (Express + better-sqlite3 + FTS5, mirroring keep's stack): /api/capture (upsert-on-url, per PROPOSAL.md's dedup decision), /api/articles (list/search/mark-read/delete), plus a minimal server-rendered reading list and reader view at / and /read/:id (token passed as ?t= there since browser navigation can't set an Authorization header). Bearer-token auth generated via `npm run token` — no login flow, matching keep's one-time identity registration pattern. Extension (Manifest V3, sideloaded, no store distribution): toolbar click or context menu "Send to stash" runs @mozilla/readability (vendored into extension/lib/) against the current page, falling back to raw rendered HTML when extraction comes back too thin. Options page for one-time server URL + token entry. Verified end-to-end with the real extension loaded in a live Chromium instance (not just unit-tested extraction logic): capture against a real page, dedup-on-recapture, mark-read, FTS5 search, 401 on missing auth, and both server-rendered views all confirmed working. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
137
PROPOSAL.md
Normal file
137
PROPOSAL.md
Normal file
@@ -0,0 +1,137 @@
|
||||
# Proposal: `stash` — self-hosted read-later with one-click capture
|
||||
|
||||
**Status:** proposed, not built. Self-contained — written so a fresh agent
|
||||
with no prior conversation context can pick this up and implement it
|
||||
without anything explained first. No repo exists yet; this file is the
|
||||
seed of one.
|
||||
|
||||
## Motivation
|
||||
|
||||
Read-later tools (Pocket, Instapaper, and their many clones) all share the
|
||||
same shape as every other cloud convenience this household of projects has
|
||||
been replacing: a third party ends up holding a full copy of everything
|
||||
you've ever wanted to read, indefinitely, for a feature that's really just
|
||||
"save this page, let me read it later, let me search it eventually."
|
||||
|
||||
The actual want: click a button in the browser, the current page's article
|
||||
content gets pulled out and stored on infrastructure already run
|
||||
(alongside `wisp`, `flit`, `keep`), with a plain reading-list view and full
|
||||
text search over what's been captured.
|
||||
|
||||
A read-later tool with no one-click capture is just a bookmarks file with
|
||||
extra steps — so this is explicitly a two-piece project: a browser
|
||||
extension that does the sending, and a small server that does the storing.
|
||||
Neither piece is useful alone.
|
||||
|
||||
## Architecture
|
||||
|
||||
Single-process server + SQLite, the same pattern already validated by
|
||||
`galr` (a photo gallery proving SQLite can carry an entire app's storage
|
||||
and query needs without a separate database service). No queue, no
|
||||
multi-service split — one process, one file on disk.
|
||||
|
||||
### Data model
|
||||
|
||||
One table, `articles`:
|
||||
|
||||
| column | type | notes |
|
||||
|---|---|---|
|
||||
| `id` | integer PK | |
|
||||
| `url` | text, unique | dedup key — see "Open questions" |
|
||||
| `title` | text | |
|
||||
| `content_html` | text | cleaned article body, for the reader view |
|
||||
| `content_text` | text | plain-text extract, indexed via FTS5 for search |
|
||||
| `author` | text, nullable | best-effort, from extraction |
|
||||
| `published_at` | text, nullable | best-effort, from extraction |
|
||||
| `captured_at` | text | when the extension sent it |
|
||||
| `read_at` | text, nullable | null = unread |
|
||||
| `tags` | text | comma-separated or JSON array, keep simple |
|
||||
|
||||
Full-text search via SQLite's FTS5 virtual table over `content_text` +
|
||||
`title` — no external search service, matches the "one process" bet.
|
||||
|
||||
### Server endpoints
|
||||
|
||||
- `POST /api/capture` — body: `{ url, title, content_html, content_text,
|
||||
author?, published_at? }`. Auth via a bearer token (see auth below).
|
||||
Upserts on `url` (see dedup, below).
|
||||
- `GET /api/articles?unread=true&q=<search>` — list/search, most recent
|
||||
capture first.
|
||||
- `GET /api/articles/:id` — full record, for the reader view.
|
||||
- `PATCH /api/articles/:id` — `{ read_at }` (mark read/unread), `{ tags }`.
|
||||
- `DELETE /api/articles/:id`
|
||||
- A minimal server-rendered reader view (e.g. `/read/:id`) that serves
|
||||
`content_html` in a plain readable template — the point of capturing
|
||||
cleanly is a nicer reading experience than the original page, not just
|
||||
an archive.
|
||||
|
||||
### Browser extension (Manifest V3)
|
||||
|
||||
- Toolbar icon + context-menu entry: "Send to stash"
|
||||
- On click: injects a content script that runs Readability.js-style
|
||||
extraction (Mozilla's `@mozilla/readability` is the standard library for
|
||||
this — strips nav, ads, comments, keeps article body/title/author/date)
|
||||
against the current DOM
|
||||
- If extraction confidence is low (Readability returns null or a very
|
||||
short result — common on heavily JS-rendered pages), fall back to
|
||||
capturing the page's raw rendered HTML rather than failing outright.
|
||||
Worse capture beats no capture for a personal tool.
|
||||
- POSTs the extracted `{ url, title, content_html, content_text, author,
|
||||
published_at }` to `POST /api/capture`
|
||||
- Toolbar badge or toast confirms success/failure so a silent failure
|
||||
doesn't look like a successful capture
|
||||
|
||||
### Auth
|
||||
|
||||
No login UI in the extension — that's overkill for a single-user tool.
|
||||
Instead: a settings page (extension's options page) with one field, a
|
||||
token pasted in once. The server generates that token via a CLI command
|
||||
or a one-time setup endpoint, the same "generate once, paste once" shape
|
||||
`keep`'s identity registration already uses. Stored in
|
||||
`chrome.storage.local`, sent as `Authorization: Bearer <token>` on every
|
||||
capture.
|
||||
|
||||
## What this deliberately isn't
|
||||
|
||||
- **Not multi-user, no accounts, no sync protocol** — one server, one
|
||||
reader (you), one writer (the extension on your own browser profiles).
|
||||
If two browsers both have the extension, they both hit the same server;
|
||||
there's no per-device state to reconcile.
|
||||
- **Not a visual/screenshot archive** — text and article structure only
|
||||
(à la archive.org's Wayback Machine, which this is not trying to be).
|
||||
Keeps storage small and keeps full-text search meaningful — searching
|
||||
rendered screenshots isn't a thing.
|
||||
- **Not published to the Chrome Web Store or Firefox Add-ons** —
|
||||
sideloaded/unpacked only (`chrome://extensions` → "Load unpacked", or
|
||||
Firefox's `about:debugging` temporary add-on). Store review, a
|
||||
developer account, and update-approval friction are all overhead for a
|
||||
tool with exactly one user. Tradeoff: Firefox's temporary add-ons don't
|
||||
survive a browser restart, so Chrome/Chromium (persistent unpacked
|
||||
extensions) is the more convenient target if only picking one browser
|
||||
to support first.
|
||||
|
||||
## Open questions
|
||||
|
||||
- **Dedup on re-capture** — a `url` unique constraint plus upsert is the
|
||||
simplest behavior (re-sending the same URL just refreshes the stored
|
||||
copy). The alternative — versioning every capture — adds real
|
||||
complexity for a benefit ("see how this article changed over time")
|
||||
that's unlikely to matter for a personal reading list. Recommend
|
||||
upsert-on-url as the default; revisit only if it turns out to matter in
|
||||
practice.
|
||||
- **JS-heavy / paywalled sites** — capture happens against whatever the
|
||||
DOM looks like at click-time, in the browser, as the logged-in user
|
||||
already sees it. This sidesteps needing server-side headless rendering
|
||||
entirely (no Puppeteer on the server, no cookie-jar management) — the
|
||||
extension captures exactly what's already rendered in front of the
|
||||
user. The limitation: if the page is still loading content
|
||||
asynchronously when "Send to stash" is clicked, that content won't be
|
||||
in the DOM yet. Not worth engineering around for a first cut; a
|
||||
"capture is incomplete, try again once the page fully loads" is an
|
||||
acceptable rough edge.
|
||||
- **Where does this live relative to `keep`** — once `keep` is rolled out
|
||||
further (see `goonk/FUTURE.md`'s "Roll out `keep` to the rest of the
|
||||
stack"), `stash`'s server should probably pull its own bearer-token
|
||||
secret from `keep` rather than a hand-copied `.env`, the same way any
|
||||
new project should from here on. Not a blocker for a first version,
|
||||
since `keep`'s rollout is itself still in progress.
|
||||
Reference in New Issue
Block a user