Files
waste-go/MIGRATION.md
Fredrik Johansson 3e058bee9b Migrate to YAW/2 protocol: WebRTC transport, anchor signaling, nacl/box crypto
Replace the original raw-TCP peer connections and Blowfish/RSA-era design with
the YAW/2 protocol stack:

Transport
- internal/mesh/peer.go: raw TCP + manual ECDH replaced with pion/webrtc
  DataChannels (ICE, DTLS, SCTP). Peers negotiate via sealed offer/answer/
  candidate exchange rather than direct dialling.
- internal/nat: deleted — ICE subsumes everything this package was going to do.

Signaling
- cmd/anchor: new WebSocket signaling server (replaces cmd/relay). Verifies
  Ed25519 challenge/join signatures, routes opaque nacl/box blobs, broadcasts
  peer-join/peer-leave events. Never sees plaintext signaling content.
- internal/anchor: new anchor client. Manages PeerConnection lifecycle,
  decides offerer by peer-id comparison, seals/opens signaling payloads with
  nacl/box, implements mesh.Anchor.

Crypto (internal/crypto)
- PeerID encoding: base64url → lowercase hex (64 chars, YAW/2 §2).
- Sign/Verify: hex signatures throughout.
- Added CurvePublicKey/CurvePrivateKey: X25519 keys derived from Ed25519
  identity via Montgomery conversion (filippo.io/edwards25519), matching
  libsodium's crypto_sign_ed25519_*_to_curve25519.
- Added SignalingBox/SignalingOpen: nacl/box (XSalsa20-Poly1305) seal/open
  for signaling payloads.

Wire types (internal/proto)
- ChatMessage gains Mid (random 16-byte hex) for mesh deduplication (§8).
- FileChunk removed from PeerMessage — chunks go on a separate binary
  DataChannel labelled "f:<xid>"; FileDone added.
- New types: SignalingPayload, AnchorMessage, HelloMessage, HelloBindString.
- PeerID.Short() now returns first 16 hex chars grouped in 4s.
- IPC: CmdConnect removed; CmdJoinNetwork/CmdLeaveNetwork added;
  EvtSessionReady added (fires when DataChannel opens and hello is received).

IPC (internal/ipc)
- join_network/leave_network replace connect (direct TCP dial).
- Network joins are context-scoped: leave_network or client disconnect
  cancels the anchor connection cleanly.

README updated to reflect new project layout, getting-started commands,
IPC protocol, and roadmap state.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 17:48:14 +02:00

425 lines
14 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Migrating waste-go → YAW/2
This document describes every change needed to make waste-go speak the YAW/2
protocol and interoperate with the reference implementation. Read it top to
bottom before touching any code — the changes have an order that matters.
---
## What stays, what goes, what changes
| Package | Fate | Reason |
|---|---|---|
| `internal/crypto` | **Keep, modify** | Ed25519 is correct; swap encoding + add nacl/box |
| `internal/proto` | **Keep, extend** | Message shapes are close; add `mid`, tweak file transfer |
| `internal/mesh` (state) | **Keep, modify** | Peer map, event fan-out, IPC subscriptions all survive |
| `internal/mesh/peer.go` | **Replace** | Raw TCP → WebRTC DataChannel |
| `internal/ipc` | **Keep, minor tweaks** | IPC shape doesn't change |
| `internal/nat` | **Delete** | ICE/STUN inside pion/webrtc replaces this entirely |
| `cmd/relay` | **Replace** | TCP forwarder → WebSocket signaling anchor |
---
## Step 1 — New dependency: pion/webrtc
Add to `go.mod`:
```
github.com/pion/webrtc/v3 v3.x.x
golang.org/x/crypto (already present)
```
Run `go mod tidy` after editing.
`pion/webrtc` is a pure-Go implementation of the WebRTC stack. It gives you ICE
(host + server-reflexive candidates), DTLS, SCTP, and DataChannels — everything
YAW/2's transport layer assumes. You no longer implement any of that yourself.
---
## Step 2 — `internal/crypto`: three changes
### 2a. Identity encoding: base64url → hex
YAW/2 `id` is lowercase hex of the 32-byte Ed25519 public key (64 chars).
```go
// Before
import "encoding/base64"
var b64 = base64.RawURLEncoding
func (id *Identity) PeerID() proto.PeerID {
return proto.PeerID(b64.EncodeToString(id.PublicKey))
}
// After
import "encoding/hex"
func (id *Identity) PeerID() proto.PeerID {
return proto.PeerID(hex.EncodeToString(id.PublicKey))
}
```
Update `Verify` to decode hex instead of base64url. This is a **wire-breaking
change** — existing identity files still work (they store the raw key bytes),
only the over-the-wire representation changes.
### 2b. Signaling crypto: add nacl/box
YAW/2 seals signaling payloads (offer/answer/candidates) with libsodium's
`crypto_box` — XSalsa20-Poly1305. In Go this is `golang.org/x/crypto/nacl/box`,
which is byte-identical to libsodium.
Add to `internal/crypto/crypto.go`:
```go
import "golang.org/x/crypto/nacl/box"
// SignalingBox seals a plaintext payload for a recipient.
// Returns base64(nonce(24) || ciphertext) as the YAW/2 spec requires.
func SignalingBox(plaintext []byte, recipientPub, senderPriv *[32]byte) string {
var nonce [24]byte
rand.Read(nonce[:])
ct := box.Seal(nonce[:], plaintext, &nonce, recipientPub, senderPriv)
return base64.StdEncoding.EncodeToString(ct)
}
// SignalingOpen opens a sealed box from a sender.
func SignalingOpen(b64box string, senderPub, recipientPriv *[32]byte) ([]byte, error) {
raw, err := base64.StdEncoding.DecodeString(b64box)
if err != nil || len(raw) < 24 {
return nil, errors.New("invalid box")
}
var nonce [24]byte
copy(nonce[:], raw[:24])
out, ok := box.Open(nil, raw[24:], &nonce, senderPub, recipientPriv)
if !ok {
return nil, errors.New("box open failed")
}
return out, nil
}
```
### 2c. X25519 keys must be derived from the Ed25519 identity
YAW/2 requires that signaling boxes are sealed using X25519 keys derived from
the Ed25519 identity — not independently generated. This is the libsodium
`crypto_sign_ed25519_pk_to_curve25519` / `crypto_sign_ed25519_sk_to_curve25519`
conversion.
Add to `Identity`:
```go
import "filippo.io/edwards25519"
// CurvePublicKey returns the X25519 public key derived from this identity.
// Used for sealing signaling boxes (YAW/2 §3).
func (id *Identity) CurvePublicKey() *[32]byte {
edPoint, _ := new(edwards25519.Point).SetBytes(id.PublicKey)
mont := edPoint.BytesMontgomery()
var out [32]byte
copy(out[:], mont)
return &out
}
// CurvePrivateKey returns the X25519 private key derived from this identity.
func (id *Identity) CurvePrivateKey() *[32]byte {
// Ed25519 private key is 64 bytes: scalar(32) || pubkey(32)
// The X25519 scalar is the clamped SHA-512 first half.
h := sha512.Sum512(id.privateKey[:32])
h[0] &= 248
h[31] &= 127
h[31] |= 64
var out [32]byte
copy(out[:], h[:32])
return &out
}
```
Add `filippo.io/edwards25519` to `go.mod` (it's a Go standard library
dependency already pulled in transitively by `golang.org/x/crypto` — just
import it directly).
---
## Step 3 — `internal/proto`: wire format updates
### 3a. PeerID is now hex
```go
// PeerID is lowercase hex of the 32-byte Ed25519 public key (64 chars).
type PeerID string
func (p PeerID) Short() string {
// YAW/2 short id: first 16 hex chars, grouped in 4s: "a1b2 c3d4 e5f6 0718"
s := string(p)
if len(s) < 16 {
return s
}
return s[0:4] + " " + s[4:8] + " " + s[8:12] + " " + s[12:16]
}
```
### 3b. Every application message needs a `mid`
YAW/2 requires a `mid` (random 16-byte hex) on every DataChannel message for
deduplication in the full mesh (§8).
```go
type ChatMessage struct {
Mid string `json:"mid"` // add this — random 16-byte hex
ID string `json:"id"` // can keep for internal use
From PeerID `json:"from"`
// ... rest unchanged
}
```
Generate `mid` the same way as message IDs: `hex.EncodeToString(randomBytes(16))`.
### 3c. Signaling payload types (new)
Add a new group of types for the signaling layer — these go *inside* the sealed
boxes exchanged over the WebSocket anchor connection:
```go
// SignalingKind identifies the kind of sealed signaling payload.
type SignalingKind string
const (
SigOffer SignalingKind = "offer"
SigAnswer SignalingKind = "answer"
SigCandidate SignalingKind = "candidate"
SigBye SignalingKind = "bye"
)
// SignalingPayload is the JSON plaintext sealed inside a crypto_box.
type SignalingPayload struct {
Kind SignalingKind `json:"kind"`
SDP string `json:"sdp,omitempty"` // offer / answer
Cand string `json:"cand,omitempty"` // trickle ICE candidate line
Mid string `json:"mid,omitempty"` // media stream id for candidate
MLine int `json:"mline,omitempty"`
}
```
### 3d. Anchor WebSocket wire types (new)
The WebSocket connection to the anchor speaks its own JSON protocol (§5).
Add these alongside the existing IPC types:
```go
// AnchorMsgType identifies anchor WebSocket messages.
type AnchorMsgType string
const (
AnchorChallenge AnchorMsgType = "challenge"
AnchorJoin AnchorMsgType = "join"
AnchorJoined AnchorMsgType = "joined"
AnchorPeerJoin AnchorMsgType = "peer-join"
AnchorPeerLeave AnchorMsgType = "peer-leave"
AnchorTo AnchorMsgType = "to"
AnchorFrom AnchorMsgType = "from"
AnchorNoPeer AnchorMsgType = "no-peer"
)
// AnchorMessage covers all WebSocket frames to/from the anchor.
type AnchorMessage struct {
Type AnchorMsgType `json:"type"`
Nonce string `json:"nonce,omitempty"` // challenge nonce, hex
ID string `json:"id,omitempty"` // peer hex id
Net string `json:"net,omitempty"` // hashed network name
Sig string `json:"sig,omitempty"` // ed25519 sig, hex
Peers []string `json:"peers,omitempty"` // joined response
To string `json:"to,omitempty"` // sealed relay target
From string `json:"from,omitempty"` // sealed relay sender
Box string `json:"box,omitempty"` // base64 crypto_box
}
```
### 3e. DataChannel `hello` type (new)
The first message on every opened DataChannel is a mandatory identity
confirmation (§6). Add:
```go
// HelloMessage is the first message sent on the "yaw" DataChannel.
// The sig binds this identity to the specific DTLS session.
type HelloMessage struct {
Type string `json:"type"` // always "hello"
ID string `json:"id"` // hex pubkey
Nick string `json:"nick"` // alias
Caps []string `json:"caps"` // capability list, e.g. ["chat","file"]
Sig string `json:"sig"` // hex ed25519 sig over bind string (see below)
}
// HelloBindString returns the bytes the hello signature covers:
// "yaw/2 bind" || localDTLSFP(32 bytes) || remoteDTLSFP(32 bytes)
func HelloBindString(localFP, remoteFP []byte) []byte {
buf := []byte("yaw/2 bind")
buf = append(buf, localFP...)
buf = append(buf, remoteFP...)
return buf
}
```
### 3f. File transfer: dedicated channel, no chunk type in main channel
Remove `FileChunk` from `PeerMessage` — chunks go over a separate binary
DataChannel labeled `f:<xid>`. Keep `FileOffer`, `FileResponse`, and add
`FileDone`:
```go
type FileDone struct {
Mid string `json:"mid"`
Xid string `json:"xid"` // transfer id
SHA256 string `json:"sha256"` // hex, for receiver to verify
}
```
---
## Step 4 — `internal/mesh/peer.go`: replace with pion/webrtc
This is the largest single change. The current file (raw TCP dial/accept,
manual ECDH, encrypt/decrypt loop) is replaced entirely.
The new shape:
```go
// ConnectToPeer initiates a WebRTC session with a peer.
// Called when we are the offerer (our id < their id, lexicographically).
func ConnectToPeer(peerID proto.PeerID, m *Mesh, anchor Anchor) error {
pc, err := webrtc.NewPeerConnection(webrtc.Configuration{
ICEServers: []webrtc.ICEServer{{URLs: []string{"stun:stun.fnlr.se:3478"}}},
})
// create "yaw" DataChannel
dc, _ := pc.CreateDataChannel("yaw", &webrtc.DataChannelInit{Ordered: ptr(true)})
// wire up ICE candidate trickle → seal → send via anchor
pc.OnICECandidate(func(c *webrtc.ICECandidate) { /* seal and send */ })
// create offer, set local description, seal it, send via anchor
offer, _ := pc.CreateOffer(nil)
pc.SetLocalDescription(offer)
// seal offer → anchor.SendTo(peerID, sealedOffer)
// ... dc.OnOpen → sendHello, dc.OnMessage → handleMessage
}
// HandleAnswer processes a sealed answer received from the anchor.
func HandleAnswer(sealed string, fromID proto.PeerID, pc *webrtc.PeerConnection, ...) error {
// open the nacl box, unmarshal SignalingPayload, set remote description
}
```
Key things pion handles for you that you were going to write manually:
- All ICE candidate gathering (host + server-reflexive via STUN)
- The DTLS handshake and key material
- SCTP framing over UDP
- DataChannel reliable/ordered delivery
Key things you still write:
- Sealing/opening signaling payloads with `nacl/box` before sending to anchor
- The `hello` confirmation on DataChannel open (mandatory per §6)
- Verifying the hello signature against DTLS fingerprints
- Deciding who offers (smaller id offers — one `strings.Compare` call)
---
## Step 5 — `internal/nat`: delete it
Remove the package entirely. ICE does what this package was going to do, and
does it correctly for both host-to-host and NAT-traversal cases. The only
"NAT" configuration you provide is the STUN server URL passed to
`webrtc.NewPeerConnection`.
Update `cmd/daemon/main.go` to remove the `nat.Run(...)` goroutine and the
`-relay` flag.
---
## Step 6 — `cmd/relay` → `cmd/anchor`
Rename the binary and replace its internals. The anchor is a WebSocket server,
not a TCP server. It never reads the content of `box` fields.
Rough structure:
```go
// cmd/anchor/main.go
// Uses golang.org/x/net/websocket or nhooyr.io/websocket
// Per-connection state
type client struct {
id string // hex peer id, set after join
net string // hashed network name
ch chan []byte // outbound message queue
}
// On "join": verify ed25519 sig over (nonce || net), register (net, id) → client
// On "to": look up (net, to) → forward {type:"from", from:senderID, box:...}
// On disconnect: broadcast {type:"peer-leave", id:...} to net members
```
The anchor also runs (or points to) a STUN server. The simplest approach is
to run `coturn` on your Hetzner VPS in STUN-only mode alongside the anchor
binary, or use a public STUN server during development (`stun:stun.l.google.com:19302`).
The network name is never stored in plaintext:
```go
func hashNetName(name string) string {
h := sha256.Sum256([]byte("yaw2-net:" + name))
return hex.EncodeToString(h[:])
}
```
---
## Step 7 — `internal/ipc`: minor additions
The IPC layer mostly survives. Add two new commands:
```go
// Connect via anchor (replaces direct TCP connect)
CmdJoinNetwork IpcMsgType = "join_network" // fields: network_name (plaintext)
CmdLeaveNetwork IpcMsgType = "leave_network"
// New event when a peer's DataChannel opens and hello is verified
EvtSessionReady IpcMsgType = "session_ready" // fields: peer_id, nick
```
Remove `CmdConnect` (direct TCP dial) — there's no direct dialing in YAW/2,
only joining a named network via the anchor.
---
## Order to implement
Do these in order — each step produces something testable before moving to the next.
1. **Crypto changes** (Step 2) — unit-testable, no network involved. Write a
test that round-trips a `SignalingBox` seal/open and confirms the hex id format.
2. **Proto additions** (Step 3) — add types, confirm it compiles.
3. **Anchor server** (Step 6) — build this first so you have something to connect
to. Test with `websocat` or a browser `WebSocket` console.
4. **WebRTC peer connection** (Step 4) — start with two daemons on the same LAN,
anchor running locally. Confirm ICE succeeds and the `hello` bind check passes.
5. **Delete nat, update ipc, update daemon main** (Steps 5 + 7) — cleanup after
the above works.
6. **File transfer** — once chat works, add the `f:<xid>` DataChannel flow.
---
## What interoperability actually means in practice
Once this is done, your Go daemon and your friend's web/Tauri/Python client can
be in the same named network. They'll connect to the same anchor WebSocket,
exchange sealed offers/answers, and open DataChannels directly to each other.
The `hello` message format, the signaling box format, and the DataChannel
message types are the shared surface — as long as those match §5§9 of the
YAW/2 spec, the implementations are interoperable regardless of language.
The reference STUN and signaling endpoints in the spec are `stun.fnlr.se:3478`
and `wss://fnlr.se/...` (path TBD) — coordinate with your friend on the final
WebSocket path before wiring it in.