diff --git a/README.md b/README.md index 04d4dfa..a8e18f3 100644 --- a/README.md +++ b/README.md @@ -11,6 +11,8 @@ Node 24+, no runtime dependencies. Uses built-in `node:sqlite`. Set `TRACE_PASSW Git candidate scanning is optional and disabled by default. If an installation deliberately mounts read-only repository mirrors, list their container paths in `TRACE_REPOS`; production receipt ingestion does not require repository access. +Producer onboarding, shared-runner examples, deployment-host responsibilities, and verification are documented in [`docs/INTEGRATING_PROJECTS.md`](docs/INTEGRATING_PROJECTS.md). + ## Deployment receipts v1 `POST /api/v1/events` accepts versioned build/deployment receipts using a project-scoped bearer token configured as `TRACE_RECEIPT_TOKENS=goonk:token,retur:other-token`. Event IDs are idempotent. `GET /api/v1/public/projects/:project/latest` returns an explicit allowlisted projection; raw payloads remain private. diff --git a/docs/INTEGRATING_PROJECTS.md b/docs/INTEGRATING_PROJECTS.md new file mode 100644 index 0000000..c989e05 --- /dev/null +++ b/docs/INTEGRATING_PROJECTS.md @@ -0,0 +1,272 @@ +# Integrating a project with Trace receipts + +Trace records two different facts: + +- a **build receipt** says what CI successfully produced; +- a **deployment receipt** says what a host actually started and verified. + +Keeping those separate prevents a green image build from being presented as a successful production deployment. + +```text +source commit → shared runner → immutable image digests → build receipt + ↓ +deploy host → pull exact images → start → health checks → deployment receipt + ↓ + Trace public projection / private history +``` + +The goonk integration is the reference implementation. Trace does not need access to any production Git repository. + +## 1. Register the project + +Add `ops/project.json` to the producer repository: + +```json +{ + "schemaVersion": 1, + "project": "retur", + "displayName": "Retur", + "repository": "explewd/retur", + "components": ["web"], + "environments": ["production"], + "publicUrl": "https://retur.games.goonk.se" +} +``` + +`project` is the stable machine ID. Use the same value in CI, Trace's token mapping, deployment receipts, and public API requests. + +## 2. Create a project-specific credential + +Generate a separate random token for every producer: + +```sh +openssl rand -hex 32 +``` + +Configure the Trace runtime through `keep`: + +```env +TRACE_RECEIPT_TOKENS=goonk:,retur:,wisp: +``` + +Configure the producer repository's Gitea Actions secrets: + +```text +TRACE_EVENTS_URL=https://trace.dev.xplwd.com/api/v1/events +TRACE_RECEIPT_TOKEN= +``` + +`TRACE_EVENTS_URL` may be an organization-level secret. `TRACE_RECEIPT_TOKEN` must remain per-project. Do not include the `project:` prefix in the repository secret. + +## 3. Generate a build receipt on the shared runner + +Generate the receipt after images have been pushed, when their immutable digests are available: + +```json +{ + "schemaVersion": 1, + "eventType": "build", + "eventId": "retur-build-713-17b6b042e751", + "project": "retur", + "occurredAt": "2026-07-20T19:35:13.180Z", + "result": "success", + "source": { + "repository": "explewd/retur", + "commit": "17b6b042e751b3158e4ab54fc3b318d104b581a1", + "branch": "main" + }, + "ci": { + "system": "gitea-actions", + "runId": "713" + }, + "artifacts": [ + { + "component": "web", + "digest": "sha256:" + } + ], + "verification": { + "build": "success" + } +} +``` + +Event IDs must be deterministic for a single CI run. Trace treats repeat submissions of the same event ID as successful duplicates. + +The goonk helper at `scripts/generate-build-receipt.mjs` demonstrates deriving source and runner identity. It currently assumes `web` and optional `api` components; adapt the component list for the producer rather than falsely reporting nonexistent artifacts. + +## 4. Preserve and submit the receipt + +Use `upload-artifact@v3` with the current Gitea Actions installation. Version 4 requires GitHub's newer artifact service and fails here. + +```yaml +- name: Preserve build receipt + uses: actions/upload-artifact@v3 + with: + name: operational-build-receipt + path: artifacts/build-receipt.json + +- name: Submit build receipt to Trace + if: always() + continue-on-error: true + env: + TRACE_EVENTS_URL: ${{ secrets.TRACE_EVENTS_URL }} + TRACE_RECEIPT_TOKEN: ${{ secrets.TRACE_RECEIPT_TOKEN }} + run: | + if [[ ! -f artifacts/build-receipt.json ]]; then + echo "No build receipt was generated." + exit 0 + fi + curl --fail-with-body --retry 2 \ + -H "Authorization: Bearer ${TRACE_RECEIPT_TOKEN}" \ + -H "Content-Type: application/json" \ + --data-binary @artifacts/build-receipt.json \ + "${TRACE_EVENTS_URL}" +``` + +Trace being unavailable should not invalidate a good application image. Preserve the artifact and retry later. Receipt generation failing because its inputs are malformed is different and should be visible in CI. + +## 5. Embed build identity when useful + +Web applications may bake a safe manifest into the image before building: + +```json +{ + "schemaVersion": 1, + "project": "retur", + "commit": "17b6b042e751b3158e4ab54fc3b318d104b581a1", + "branch": "main", + "builtAt": "2026-07-20T19:34:03.496Z", + "runId": "713" +} +``` + +This proves which source the running artifact identifies as. It normally cannot contain final image digests because those do not exist until after the image is built; the finalized Trace receipt remains the authority for digests. + +Never embed credentials, private registry locations, runner identities, or raw receipt payloads in a public manifest. + +## 6. Emit a deployment receipt from the deploy host + +The deploy host should: + +1. pull the intended immutable image or known SHA tag; +2. reconcile Compose; +3. run real application health checks; +4. submit a deployment receipt only after the result is known. + +The deployment host does not need a Git checkout. It may retain or download the build receipt and change only the deployment fields: + +```json +{ + "schemaVersion": 1, + "eventType": "deployment", + "eventId": "retur-production-17b6b04-20260720T194200Z", + "project": "retur", + "environment": "production", + "occurredAt": "2026-07-20T19:42:00Z", + "result": "success", + "source": { + "repository": "explewd/retur", + "commit": "17b6b042e751b3158e4ab54fc3b318d104b581a1", + "branch": "main" + }, + "artifacts": [ + { + "component": "web", + "digest": "sha256:" + } + ], + "verification": { + "health": "healthy" + } +} +``` + +Submit it to the same `/api/v1/events` endpoint using the same project-scoped bearer token. A failed deployment or rollback should also create an event with an honest `result`; absence of a receipt means unknown, never success. + +For now this last POST belongs in the actual deployment wrapper. Do not require an application repository or ad-hoc Node script on a Compose-only production host. + +## 7. Read the result + +Private operational history is visible at: + +```text +https://trace.dev.xplwd.com/events +``` + +It uses Trace's Basic Auth and includes raw private receipts. + +The public allowlisted projection is: + +```sh +curl https://trace.dev.xplwd.com/api/v1/public/projects/retur/latest +``` + +Example: + +```json +{ + "project": "retur", + "deployment": { + "schemaVersion": 1, + "eventType": "deployment", + "project": "retur", + "environment": "production", + "commit": "17b6b042e751b3158e4ab54fc3b318d104b581a1", + "result": "success", + "health": "healthy", + "occurredAt": "2026-07-20T19:42:00Z" + } +} +``` + +The projection never includes the stored raw payload, tokens, internal hostnames, logs, or full registry references. + +## Recommended health checks + +Choose checks that prove the deployed application shape, not merely that a container process exists. + +| Project shape | Minimum checks | +| --- | --- | +| Static site/game | homepage, essential asset, embedded commit | +| Web + API | homepage, API health, embedded commit | +| Stateful service | HTTP health, database access, safe read/write probe if appropriate | +| CLI release | tests, packaged artifact checksum; use a `release` event rather than `deployment` | + +Keep detailed responses private. The public receipt needs only a bounded result such as `healthy`, `degraded`, or `failed`. + +## Verification checklist + +After onboarding a project: + +1. The workflow artifact contains the correct commit and immutable digests. +2. Reposting the same build returns `duplicate: true`. +3. `/events` shows the build separately from deployments. +4. Before deployment, the public endpoint returns `deployment: null`. +5. After deployment, the public endpoint reports the deployed commit and health. +6. The embedded build identity matches the deployment receipt's commit. +7. Removing Trace connectivity does not stop the application from serving its static/core experience. + +## Security boundaries + +- One token per project; a `retur` token cannot submit as `goonk`. +- Store runtime token mappings in `keep`, not Git. +- Gitea secrets exist only in runner jobs; deploy hosts need their own `keep`-supplied token. +- Trace's Basic Auth password is unrelated to bearer-token ingestion. +- `TRACE_REPOS` is optional and should be empty in production when no deliberate read-only mirrors exist. +- Publication to `/failures` remains a separate human review and redaction process. + +## Onboarding summary + +```text +□ add ops/project.json +□ create project token and add it to Trace through keep +□ add the raw token to that repository's Gitea secrets +□ generate a versioned build receipt after image push +□ preserve it with upload-artifact@v3 +□ submit it best-effort to Trace +□ embed safe build identity if the application can expose it +□ add post-deploy health checks +□ submit deployment receipts from the real deploy mechanism +□ verify private history and public projection +```