trace/docs/INTEGRATING_PROJECTS.md

# Integrating a project with Trace receipts

Trace records two different facts:

- a **build receipt** says what CI successfully produced;
- a **deployment receipt** says what a host actually started and verified.

Keeping those separate prevents a green image build from being presented as a successful production deployment.

```text
source commit → shared runner → immutable image digests → build receipt
                                                       ↓
deploy host → pull exact images → start → health checks → deployment receipt
                                                       ↓
                                      Trace public projection / private history
```

The goonk integration is the reference implementation. Trace does not need access to any production Git repository.

## 1. Register the project

Add `ops/project.json` to the producer repository:

```json
{
  "schemaVersion": 1,
  "project": "retur",
  "displayName": "Retur",
  "repository": "explewd/retur",
  "components": ["web"],
  "environments": ["production"],
  "publicUrl": "https://retur.games.goonk.se"
}
```

`project` is the stable machine ID. Use the same value in CI, Trace's token mapping, deployment receipts, and public API requests.

## 2. Create a project-specific credential

Generate a separate random token for every producer:

```sh
openssl rand -hex 32
```

Configure the Trace runtime through `keep`:

```env
TRACE_RECEIPT_TOKENS=goonk:<goonk-token>,retur:<retur-token>,wisp:<wisp-token>
```

Configure the producer repository's Gitea Actions secrets:

```text
TRACE_EVENTS_URL=https://trace.dev.xplwd.com/api/v1/events
TRACE_RECEIPT_TOKEN=<only this project's raw token>
```

`TRACE_EVENTS_URL` may be an organization-level secret. `TRACE_RECEIPT_TOKEN` must remain per-project. Do not include the `project:` prefix in the repository secret.

## 3. Generate a build receipt on the shared runner

Generate the receipt after images have been pushed, when their immutable digests are available:

```json
{
  "schemaVersion": 1,
  "eventType": "build",
  "eventId": "retur-build-713-17b6b042e751",
  "project": "retur",
  "occurredAt": "2026-07-20T19:35:13.180Z",
  "result": "success",
  "source": {
    "repository": "explewd/retur",
    "commit": "17b6b042e751b3158e4ab54fc3b318d104b581a1",
    "branch": "main"
  },
  "ci": {
    "system": "gitea-actions",
    "runId": "713"
  },
  "artifacts": [
    {
      "component": "web",
      "digest": "sha256:<immutable-image-digest>"
    }
  ],
  "verification": {
    "build": "success"
  }
}
```

Event IDs must be deterministic for a single CI run. Trace treats repeat submissions of the same event ID as successful duplicates.

The goonk helper at `scripts/generate-build-receipt.mjs` demonstrates deriving source and runner identity. It currently assumes `web` and optional `api` components; adapt the component list for the producer rather than falsely reporting nonexistent artifacts.

## 4. Preserve and submit the receipt

Use `upload-artifact@v3` with the current Gitea Actions installation. Version 4 requires GitHub's newer artifact service and fails here.

```yaml
- name: Preserve build receipt
  uses: actions/upload-artifact@v3
  with:
    name: operational-build-receipt
    path: artifacts/build-receipt.json

- name: Submit build receipt to Trace
  if: always()
  continue-on-error: true
  env:
    TRACE_EVENTS_URL: ${{ secrets.TRACE_EVENTS_URL }}
    TRACE_RECEIPT_TOKEN: ${{ secrets.TRACE_RECEIPT_TOKEN }}
  run: |
    if [[ ! -f artifacts/build-receipt.json ]]; then
      echo "No build receipt was generated."
      exit 0
    fi
    curl --fail-with-body --retry 2 \
      -H "Authorization: Bearer ${TRACE_RECEIPT_TOKEN}" \
      -H "Content-Type: application/json" \
      --data-binary @artifacts/build-receipt.json \
      "${TRACE_EVENTS_URL}"
```

Trace being unavailable should not invalidate a good application image. Preserve the artifact and retry later. Receipt generation failing because its inputs are malformed is different and should be visible in CI.

## 5. Embed build identity when useful

Web applications may bake a safe manifest into the image before building:

```json
{
  "schemaVersion": 1,
  "project": "retur",
  "commit": "17b6b042e751b3158e4ab54fc3b318d104b581a1",
  "branch": "main",
  "builtAt": "2026-07-20T19:34:03.496Z",
  "runId": "713"
}
```

This proves which source the running artifact identifies as. It normally cannot contain final image digests because those do not exist until after the image is built; the finalized Trace receipt remains the authority for digests.

Never embed credentials, private registry locations, runner identities, or raw receipt payloads in a public manifest.

## 6. Emit a deployment receipt from the deploy host

The deploy host should:

1. pull the intended immutable image or known SHA tag;
2. reconcile Compose;
3. run real application health checks;
4. submit a deployment receipt only after the result is known.

The deployment host does not need a Git checkout. It may retain or download the build receipt and change only the deployment fields:

```json
{
  "schemaVersion": 1,
  "eventType": "deployment",
  "eventId": "retur-production-17b6b04-20260720T194200Z",
  "project": "retur",
  "environment": "production",
  "occurredAt": "2026-07-20T19:42:00Z",
  "result": "success",
  "source": {
    "repository": "explewd/retur",
    "commit": "17b6b042e751b3158e4ab54fc3b318d104b581a1",
    "branch": "main"
  },
  "artifacts": [
    {
      "component": "web",
      "digest": "sha256:<same-digest-as-build>"
    }
  ],
  "verification": {
    "health": "healthy"
  }
}
```

Submit it to the same `/api/v1/events` endpoint using the same project-scoped bearer token. A failed deployment or rollback should also create an event with an honest `result`; absence of a receipt means unknown, never success.

For now this last POST belongs in the actual deployment wrapper. Do not require an application repository or ad-hoc Node script on a Compose-only production host.

## 7. Read the result

Private operational history is visible at:

```text
https://trace.dev.xplwd.com/events
```

It uses Trace's Basic Auth and includes raw private receipts.

The public allowlisted projection is:

```sh
curl https://trace.dev.xplwd.com/api/v1/public/projects/retur/latest
```

Example:

```json
{
  "project": "retur",
  "deployment": {
    "schemaVersion": 1,
    "eventType": "deployment",
    "project": "retur",
    "environment": "production",
    "commit": "17b6b042e751b3158e4ab54fc3b318d104b581a1",
    "result": "success",
    "health": "healthy",
    "occurredAt": "2026-07-20T19:42:00Z"
  }
}
```

The projection never includes the stored raw payload, tokens, internal hostnames, logs, or full registry references.

## Recommended health checks

Choose checks that prove the deployed application shape, not merely that a container process exists.

| Project shape | Minimum checks |
| --- | --- |
| Static site/game | homepage, essential asset, embedded commit |
| Web + API | homepage, API health, embedded commit |
| Stateful service | HTTP health, database access, safe read/write probe if appropriate |
| CLI release | tests, packaged artifact checksum; use a `release` event rather than `deployment` |

Keep detailed responses private. The public receipt needs only a bounded result such as `healthy`, `degraded`, or `failed`.

## Verification checklist

After onboarding a project:

1. The workflow artifact contains the correct commit and immutable digests.
2. Reposting the same build returns `duplicate: true`.
3. `/events` shows the build separately from deployments.
4. Before deployment, the public endpoint returns `deployment: null`.
5. After deployment, the public endpoint reports the deployed commit and health.
6. The embedded build identity matches the deployment receipt's commit.
7. Removing Trace connectivity does not stop the application from serving its static/core experience.

## Security boundaries

- One token per project; a `retur` token cannot submit as `goonk`.
- Store runtime token mappings in `keep`, not Git.
- Gitea secrets exist only in runner jobs; deploy hosts need their own `keep`-supplied token.
- Trace's Basic Auth password is unrelated to bearer-token ingestion.
- `TRACE_REPOS` is optional and should be empty in production when no deliberate read-only mirrors exist.
- Publication to `/failures` remains a separate human review and redaction process.

## Onboarding summary

```text
□ add ops/project.json
□ create project token and add it to Trace through keep
□ add the raw token to that repository's Gitea secrets
□ generate a versioned build receipt after image push
□ preserve it with upload-artifact@v3
□ submit it best-effort to Trace
□ embed safe build identity if the application can expose it
□ add post-deploy health checks
□ submit deployment receipts from the real deploy mechanism
□ verify private history and public projection
```