README: document actual usage, not just quickstart
All checks were successful
Docker / test-build-push (push) Successful in 37s
All checks were successful
Docker / test-build-push (push) Successful in 37s
Full walkthrough of the private incident notebook (timeline, hypotheses, publishing), an endpoint reference table for the receipts API, real curl examples for both receipt types, a config reference, and an honest note about the dead pre-v1 /api/receipts endpoint instead of leaving it undocumented. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
116
README.md
116
README.md
@@ -1,20 +1,120 @@
|
|||||||
# trace
|
# trace
|
||||||
|
|
||||||
Private incident notebook, Git fix-candidate inbox, deployment-receipt evidence store, and reviewed Astro Markdown exporter for goonk's `/failures` page.
|
Private incident notebook, Git fix-candidate inbox, deployment-receipt evidence store, and reviewed Astro Markdown exporter for goonk's `/failures` page. One ~70-line HTTP server (`src/server.mjs`), Node's built-in `node:sqlite`, no framework, no runtime dependencies.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cp .env.example .env
|
cp .env.example .env # set TRACE_PASSWORD and TRACE_RECEIPT_TOKENS at minimum
|
||||||
npm start
|
npm start
|
||||||
```
|
```
|
||||||
|
|
||||||
Node 24+, no runtime dependencies. Uses built-in `node:sqlite`. Set `TRACE_PASSWORD`. Incidents remain private until a human explicitly publishes them. Published projections are available through `GET /api/v1/public/failures`, `GET /api/v1/public/failures/:slug`, and `GET /api/v1/public/failures/:slug.md` for runtime and build-time consumers.
|
Node 24+. `npm run dev` for `--watch` during local work; `npm test` runs the receipt-contract tests; `npm run build` is a syntax check (`node --check`), not a bundle — there's nothing to bundle.
|
||||||
|
|
||||||
Git candidate scanning is optional and disabled by default. If an installation deliberately mounts read-only repository mirrors, list their container paths in `TRACE_REPOS`; production receipt ingestion does not require repository access.
|
## What trace actually is
|
||||||
|
|
||||||
Production can scan repositories without Git checkouts through Gitea's read-only API. Set `TRACE_GITEA_URL` and a comma-separated `TRACE_GITEA_REPOS` list such as `explewd/flit`. Public repositories require no token; `TRACE_GITEA_TOKEN` is optional for private repositories. A failed repository is skipped rather than failing the inbox.
|
Two things that get conflated everywhere else, kept structurally separate on purpose:
|
||||||
|
|
||||||
Producer onboarding, shared-runner examples, deployment-host responsibilities, and verification are documented in [`docs/INTEGRATING_PROJECTS.md`](docs/INTEGRATING_PROJECTS.md).
|
1. **A private incident notebook.** Record what broke, what you tried, what was actually true, and what you'd do differently — while you're still in it, not reconstructed from memory afterward. Nothing here is public until a human explicitly publishes it.
|
||||||
|
2. **An operational receipts ledger.** CI says what it built; a deploy host says what it actually started and verified. Trace stores both as separate, versioned, authenticated event types and never lets a green build masquerade as a live deployment.
|
||||||
|
|
||||||
## Deployment receipts v1
|
The two features share a database and a process, but not a purpose — the incident notebook is where a human writes a story, the receipts ledger is where machines report bare facts.
|
||||||
|
|
||||||
`POST /api/v1/events` accepts versioned build/deployment receipts using a project-scoped bearer token configured as `TRACE_RECEIPT_TOKENS=goonk:token,retur:other-token`. Event IDs are idempotent. `GET /api/v1/public/projects/:project/latest` returns an explicit allowlisted projection; raw payloads remain private.
|
## The private incident notebook
|
||||||
|
|
||||||
|
Everything under `/` (Basic Auth, `trace:$TRACE_PASSWORD`) is private by default:
|
||||||
|
|
||||||
|
- **`/`** — every incident, newest first, with a live status breakdown.
|
||||||
|
- **`/incidents/new`** → **`/incidents/:id`** — title, project, status (`investigating` / `mitigated` / `resolved` / `abandoned`), severity (`annoyance` → `security-risk`), and the fields that actually make a postmortem useful: symptom, impact, root cause, confidence, fix, verification, prevention, remaining risk, lesson. Confidence is tracked separately from status on purpose — "resolved, high confidence" and "resolved, we're not totally sure why" are different claims and shouldn't collapse into one field.
|
||||||
|
- **Timeline** — append-only entries per incident, each tagged `observation` / `hypothesis` / `action` / `result` / `decision` / `deploy` / `note`. This is the actual investigation log, in order, not reconstructed after the fact.
|
||||||
|
- **Hypotheses** — a statement, a status (`untested` / `supported` / `rejected` / `inconclusive`), and evidence. Explicitly tracking rejected hypotheses is the point — "we thought it was X, it wasn't, here's how we knew" is worth as much as the real answer.
|
||||||
|
- **`/candidates`** (the Git inbox) — click "scan configured repositories" to pull commits whose subject line matches fix-shaped keywords (`fix`, `bug`, `broken`, `race`, `crash`, `regression`, `rollback`, `restore`, `duplicate`, `auth`, `fail`) from either locally mounted read-only repo mirrors (`TRACE_REPOS`) or Gitea's API with no checkout at all (`TRACE_GITEA_URL` + `TRACE_GITEA_REPOS`). Link a candidate to an incident's UUID to connect "here's the bug" with "here's the commit that caused or fixed it."
|
||||||
|
|
||||||
|
### Publishing a real postmortem
|
||||||
|
|
||||||
|
An incident page has an **export public draft** action (writes reviewed Markdown to `data/exports/` for hand-editing) and, once you're happy with it, **publish / update reviewed article** — this sets a public slug and makes the incident visible through the read-only public API below. **Unpublish** reverses it instantly. Nothing crosses from private to public without that explicit action; a `symptom` field with an internal hostname in it never accidentally becomes a blog post.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -u trace:$TRACE_PASSWORD -X POST http://localhost:3082/incidents/new \
|
||||||
|
--data-urlencode "title=under-the-hood showed a stale deploy commit" \
|
||||||
|
--data-urlencode "project=goonk" \
|
||||||
|
--data-urlencode "detected_at=2026-07-21T01:00" \
|
||||||
|
--data-urlencode "status=resolved" \
|
||||||
|
--data-urlencode "severity=annoyance" \
|
||||||
|
--data-urlencode "symptom=Deployed showed an older commit and earlier timestamp than Built." \
|
||||||
|
--data-urlencode "root_cause=Nothing on the deploy host ever submitted a deployment receipt." \
|
||||||
|
--data-urlencode "fix=Added best-effort receipt submission to the shared deploy wrapper." \
|
||||||
|
--data-urlencode "public_summary=A deploy dashboard kept showing an old commit as current."
|
||||||
|
# → 303 redirect to /incidents/<new-uuid>
|
||||||
|
```
|
||||||
|
|
||||||
|
The form works the same from a browser; this is just the same POST scripted, useful for backfilling incidents from notes taken elsewhere.
|
||||||
|
|
||||||
|
## Deployment receipts (v1)
|
||||||
|
|
||||||
|
Two separate authenticated write paths, one shared authenticated read-only view:
|
||||||
|
|
||||||
|
| Endpoint | Method | Auth | Purpose |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `/api/v1/events` | `POST` | `Authorization: Bearer <project-scoped token>` | Submit a build/deployment/release/rollback receipt |
|
||||||
|
| `/events` | `GET` | Basic (`trace:$TRACE_PASSWORD`) | Private, human-readable feed of every received event, raw payload included |
|
||||||
|
| `/api/v1/public/projects/:project/latest` | `GET` | none | Latest **deployment** event only, allowlisted fields |
|
||||||
|
| `/api/v1/public/projects` | `GET` | none | Latest deployment per project, same allowlisted shape |
|
||||||
|
| `/api/health` | `GET` | none | Liveness probe (used by the Dockerfile's `HEALTHCHECK`) |
|
||||||
|
|
||||||
|
A receipt (`schemaVersion: 1`) needs `eventType` (`build`/`deployment`/`release`/`rollback`), `eventId`, `project`, `occurredAt` (ISO-8601), `result` (`success`/`failed`/`cancelled`/`unknown`), and `source.commit` (a 7–64 char hex SHA). `eventId` is the idempotency key — resubmitting the same one returns `{"ok":true,"duplicate":true}` rather than a second row, which is what makes "retry the CI step that failed to reach trace" safe.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# CI, right after pushing images (a build receipt)
|
||||||
|
curl -H "Authorization: Bearer $TRACE_RECEIPT_TOKEN" -H "Content-Type: application/json" \
|
||||||
|
--data-binary '{"schemaVersion":1,"eventType":"build","eventId":"goonk-build-713-3777586","project":"goonk",
|
||||||
|
"occurredAt":"2026-07-21T03:06:00Z","result":"success",
|
||||||
|
"source":{"commit":"37775868ab303f9f8a7da09b354dd0cc31bb0c6f","branch":"main"},
|
||||||
|
"artifacts":[{"component":"web","digest":"sha256:fe9e15f73804..."}],
|
||||||
|
"verification":{"build":"success"}}' \
|
||||||
|
https://trace.dev.xplwd.com/api/v1/events
|
||||||
|
|
||||||
|
# A deploy host, after docker compose up -d and a real health check
|
||||||
|
curl -H "Authorization: Bearer $TRACE_RECEIPT_TOKEN" -H "Content-Type: application/json" \
|
||||||
|
--data-binary '{"schemaVersion":1,"eventType":"deployment","eventId":"goonk-production-3777586-20260721T013412Z",
|
||||||
|
"project":"goonk","environment":"production","occurredAt":"2026-07-21T01:34:12Z","result":"success",
|
||||||
|
"source":{"commit":"37775868ab303f9f8a7da09b354dd0cc31bb0c6f","branch":"main"},
|
||||||
|
"verification":{"health":"healthy"}}' \
|
||||||
|
https://trace.dev.xplwd.com/api/v1/events
|
||||||
|
|
||||||
|
# Check what a public consumer sees
|
||||||
|
curl https://trace.dev.xplwd.com/api/v1/public/projects/goonk/latest
|
||||||
|
```
|
||||||
|
|
||||||
|
Full producer onboarding — registering a project, per-project tokens, the exact CI step shape, and the deploy-host responsibilities above — is in [`docs/INTEGRATING_PROJECTS.md`](docs/INTEGRATING_PROJECTS.md). `keep/scripts/deploy.sh` is the reference deploy-host implementation of the second `curl` above: after `docker compose up -d`, it polls the app's own embedded build identity until healthy, then submits exactly this receipt — opt-in per project via a `.trace-meta` marker file, never assumed.
|
||||||
|
|
||||||
|
### A known rough edge
|
||||||
|
|
||||||
|
`POST /api/receipts` (Basic Auth, no `/v1`, no bearer token) still exists in `src/server.mjs` as a leftover from before the v1 events API was built. It writes to a `receipts` table nothing else ever reads from — dead code, not a second real API. Use `/api/v1/events` for everything; the old path is scheduled for removal, not a documented feature.
|
||||||
|
|
||||||
|
## Configuration reference
|
||||||
|
|
||||||
|
| Variable | Required | Purpose |
|
||||||
|
|---|---|---|
|
||||||
|
| `PORT` | no (default `3082`) | HTTP listen port |
|
||||||
|
| `TRACE_PASSWORD` | effectively yes | Basic Auth password for `/`, `/events`, and the publish/unpublish actions. Empty disables auth entirely — fine for a throwaway local instance, never for anything reachable outside localhost |
|
||||||
|
| `TRACE_RECEIPT_TOKENS` | yes, for receipt ingestion | `project:token,project2:token2` — one random token per producer (`openssl rand -hex 32`), never shared across projects |
|
||||||
|
| `TRACE_REPOS` | no | `id:/absolute/path,id2:/absolute/path2` — locally mounted **read-only** repo mirrors for the Git inbox |
|
||||||
|
| `TRACE_GITEA_URL` | no (default `https://repo.explewd.com`) | Base URL for Gitea-API-based candidate scanning, no checkout needed |
|
||||||
|
| `TRACE_GITEA_REPOS` | no | Comma-separated `owner/repo` list to scan via the Gitea API |
|
||||||
|
| `TRACE_GITEA_TOKEN` | no | Only needed for private Gitea repositories; public ones scan with no token |
|
||||||
|
|
||||||
|
## Deploying
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp .env.example .env # fill in real values
|
||||||
|
docker compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
Single service, a named volume for `data/` (SQLite + exports), a `HEALTHCHECK` hitting `/api/health`. CI (`.gitea/workflows/docker.yml`) runs `npm test` and `npm run build` before it will push an image — a broken receipt-validation contract or a syntax error never reaches the registry.
|
||||||
|
|
||||||
|
## Security boundaries
|
||||||
|
|
||||||
|
- One bearer token per project; a `retur` token cannot submit a `goonk` receipt (checked with `crypto.timingSafeEqual`, not `===`).
|
||||||
|
- `TRACE_PASSWORD` gates every private route in one place — the notebook, the raw events feed, and publish/unpublish.
|
||||||
|
- The public API (`/api/v1/public/*`) never includes raw payloads, tokens, internal hostnames, or full registry references — only the allowlisted fields in `publicEvent()`/`publicIncident()`.
|
||||||
|
- `TRACE_REPOS` should be empty in any deployment with no deliberate read-only mirrors mounted; receipt ingestion never requires repository access at all.
|
||||||
|
- Trace being unreachable should never break a producer's build or deploy — every documented integration point treats submission as best-effort.
|
||||||
|
|||||||
Reference in New Issue
Block a user