Files
waste-go/MIGRATION.md

425 lines
14 KiB
Markdown
Raw Permalink Normal View History

Migrate to YAW/2 protocol: WebRTC transport, anchor signaling, nacl/box crypto Replace the original raw-TCP peer connections and Blowfish/RSA-era design with the YAW/2 protocol stack: Transport - internal/mesh/peer.go: raw TCP + manual ECDH replaced with pion/webrtc DataChannels (ICE, DTLS, SCTP). Peers negotiate via sealed offer/answer/ candidate exchange rather than direct dialling. - internal/nat: deleted — ICE subsumes everything this package was going to do. Signaling - cmd/anchor: new WebSocket signaling server (replaces cmd/relay). Verifies Ed25519 challenge/join signatures, routes opaque nacl/box blobs, broadcasts peer-join/peer-leave events. Never sees plaintext signaling content. - internal/anchor: new anchor client. Manages PeerConnection lifecycle, decides offerer by peer-id comparison, seals/opens signaling payloads with nacl/box, implements mesh.Anchor. Crypto (internal/crypto) - PeerID encoding: base64url → lowercase hex (64 chars, YAW/2 §2). - Sign/Verify: hex signatures throughout. - Added CurvePublicKey/CurvePrivateKey: X25519 keys derived from Ed25519 identity via Montgomery conversion (filippo.io/edwards25519), matching libsodium's crypto_sign_ed25519_*_to_curve25519. - Added SignalingBox/SignalingOpen: nacl/box (XSalsa20-Poly1305) seal/open for signaling payloads. Wire types (internal/proto) - ChatMessage gains Mid (random 16-byte hex) for mesh deduplication (§8). - FileChunk removed from PeerMessage — chunks go on a separate binary DataChannel labelled "f:<xid>"; FileDone added. - New types: SignalingPayload, AnchorMessage, HelloMessage, HelloBindString. - PeerID.Short() now returns first 16 hex chars grouped in 4s. - IPC: CmdConnect removed; CmdJoinNetwork/CmdLeaveNetwork added; EvtSessionReady added (fires when DataChannel opens and hello is received). IPC (internal/ipc) - join_network/leave_network replace connect (direct TCP dial). - Network joins are context-scoped: leave_network or client disconnect cancels the anchor connection cleanly. README updated to reflect new project layout, getting-started commands, IPC protocol, and roadmap state. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 17:48:14 +02:00
# Migrating waste-go → YAW/2
This document describes every change needed to make waste-go speak the YAW/2
protocol and interoperate with the reference implementation. Read it top to
bottom before touching any code — the changes have an order that matters.
---
## What stays, what goes, what changes
| Package | Fate | Reason |
|---|---|---|
| `internal/crypto` | **Keep, modify** | Ed25519 is correct; swap encoding + add nacl/box |
| `internal/proto` | **Keep, extend** | Message shapes are close; add `mid`, tweak file transfer |
| `internal/mesh` (state) | **Keep, modify** | Peer map, event fan-out, IPC subscriptions all survive |
| `internal/mesh/peer.go` | **Replace** | Raw TCP → WebRTC DataChannel |
| `internal/ipc` | **Keep, minor tweaks** | IPC shape doesn't change |
| `internal/nat` | **Delete** | ICE/STUN inside pion/webrtc replaces this entirely |
| `cmd/relay` | **Replace** | TCP forwarder → WebSocket signaling anchor |
---
## Step 1 — New dependency: pion/webrtc
Add to `go.mod`:
```
github.com/pion/webrtc/v3 v3.x.x
golang.org/x/crypto (already present)
```
Run `go mod tidy` after editing.
`pion/webrtc` is a pure-Go implementation of the WebRTC stack. It gives you ICE
(host + server-reflexive candidates), DTLS, SCTP, and DataChannels — everything
YAW/2's transport layer assumes. You no longer implement any of that yourself.
---
## Step 2 — `internal/crypto`: three changes
### 2a. Identity encoding: base64url → hex
YAW/2 `id` is lowercase hex of the 32-byte Ed25519 public key (64 chars).
```go
// Before
import "encoding/base64"
var b64 = base64.RawURLEncoding
func (id *Identity) PeerID() proto.PeerID {
return proto.PeerID(b64.EncodeToString(id.PublicKey))
}
// After
import "encoding/hex"
func (id *Identity) PeerID() proto.PeerID {
return proto.PeerID(hex.EncodeToString(id.PublicKey))
}
```
Update `Verify` to decode hex instead of base64url. This is a **wire-breaking
change** — existing identity files still work (they store the raw key bytes),
only the over-the-wire representation changes.
### 2b. Signaling crypto: add nacl/box
YAW/2 seals signaling payloads (offer/answer/candidates) with libsodium's
`crypto_box` — XSalsa20-Poly1305. In Go this is `golang.org/x/crypto/nacl/box`,
which is byte-identical to libsodium.
Add to `internal/crypto/crypto.go`:
```go
import "golang.org/x/crypto/nacl/box"
// SignalingBox seals a plaintext payload for a recipient.
// Returns base64(nonce(24) || ciphertext) as the YAW/2 spec requires.
func SignalingBox(plaintext []byte, recipientPub, senderPriv *[32]byte) string {
var nonce [24]byte
rand.Read(nonce[:])
ct := box.Seal(nonce[:], plaintext, &nonce, recipientPub, senderPriv)
return base64.StdEncoding.EncodeToString(ct)
}
// SignalingOpen opens a sealed box from a sender.
func SignalingOpen(b64box string, senderPub, recipientPriv *[32]byte) ([]byte, error) {
raw, err := base64.StdEncoding.DecodeString(b64box)
if err != nil || len(raw) < 24 {
return nil, errors.New("invalid box")
}
var nonce [24]byte
copy(nonce[:], raw[:24])
out, ok := box.Open(nil, raw[24:], &nonce, senderPub, recipientPriv)
if !ok {
return nil, errors.New("box open failed")
}
return out, nil
}
```
### 2c. X25519 keys must be derived from the Ed25519 identity
YAW/2 requires that signaling boxes are sealed using X25519 keys derived from
the Ed25519 identity — not independently generated. This is the libsodium
`crypto_sign_ed25519_pk_to_curve25519` / `crypto_sign_ed25519_sk_to_curve25519`
conversion.
Add to `Identity`:
```go
import "filippo.io/edwards25519"
// CurvePublicKey returns the X25519 public key derived from this identity.
// Used for sealing signaling boxes (YAW/2 §3).
func (id *Identity) CurvePublicKey() *[32]byte {
edPoint, _ := new(edwards25519.Point).SetBytes(id.PublicKey)
mont := edPoint.BytesMontgomery()
var out [32]byte
copy(out[:], mont)
return &out
}
// CurvePrivateKey returns the X25519 private key derived from this identity.
func (id *Identity) CurvePrivateKey() *[32]byte {
// Ed25519 private key is 64 bytes: scalar(32) || pubkey(32)
// The X25519 scalar is the clamped SHA-512 first half.
h := sha512.Sum512(id.privateKey[:32])
h[0] &= 248
h[31] &= 127
h[31] |= 64
var out [32]byte
copy(out[:], h[:32])
return &out
}
```
Add `filippo.io/edwards25519` to `go.mod` (it's a Go standard library
dependency already pulled in transitively by `golang.org/x/crypto` — just
import it directly).
---
## Step 3 — `internal/proto`: wire format updates
### 3a. PeerID is now hex
```go
// PeerID is lowercase hex of the 32-byte Ed25519 public key (64 chars).
type PeerID string
func (p PeerID) Short() string {
// YAW/2 short id: first 16 hex chars, grouped in 4s: "a1b2 c3d4 e5f6 0718"
s := string(p)
if len(s) < 16 {
return s
}
return s[0:4] + " " + s[4:8] + " " + s[8:12] + " " + s[12:16]
}
```
### 3b. Every application message needs a `mid`
YAW/2 requires a `mid` (random 16-byte hex) on every DataChannel message for
deduplication in the full mesh (§8).
```go
type ChatMessage struct {
Mid string `json:"mid"` // add this — random 16-byte hex
ID string `json:"id"` // can keep for internal use
From PeerID `json:"from"`
// ... rest unchanged
}
```
Generate `mid` the same way as message IDs: `hex.EncodeToString(randomBytes(16))`.
### 3c. Signaling payload types (new)
Add a new group of types for the signaling layer — these go *inside* the sealed
boxes exchanged over the WebSocket anchor connection:
```go
// SignalingKind identifies the kind of sealed signaling payload.
type SignalingKind string
const (
SigOffer SignalingKind = "offer"
SigAnswer SignalingKind = "answer"
SigCandidate SignalingKind = "candidate"
SigBye SignalingKind = "bye"
)
// SignalingPayload is the JSON plaintext sealed inside a crypto_box.
type SignalingPayload struct {
Kind SignalingKind `json:"kind"`
SDP string `json:"sdp,omitempty"` // offer / answer
Cand string `json:"cand,omitempty"` // trickle ICE candidate line
Mid string `json:"mid,omitempty"` // media stream id for candidate
MLine int `json:"mline,omitempty"`
}
```
### 3d. Anchor WebSocket wire types (new)
The WebSocket connection to the anchor speaks its own JSON protocol (§5).
Add these alongside the existing IPC types:
```go
// AnchorMsgType identifies anchor WebSocket messages.
type AnchorMsgType string
const (
AnchorChallenge AnchorMsgType = "challenge"
AnchorJoin AnchorMsgType = "join"
AnchorJoined AnchorMsgType = "joined"
AnchorPeerJoin AnchorMsgType = "peer-join"
AnchorPeerLeave AnchorMsgType = "peer-leave"
AnchorTo AnchorMsgType = "to"
AnchorFrom AnchorMsgType = "from"
AnchorNoPeer AnchorMsgType = "no-peer"
)
// AnchorMessage covers all WebSocket frames to/from the anchor.
type AnchorMessage struct {
Type AnchorMsgType `json:"type"`
Nonce string `json:"nonce,omitempty"` // challenge nonce, hex
ID string `json:"id,omitempty"` // peer hex id
Net string `json:"net,omitempty"` // hashed network name
Sig string `json:"sig,omitempty"` // ed25519 sig, hex
Peers []string `json:"peers,omitempty"` // joined response
To string `json:"to,omitempty"` // sealed relay target
From string `json:"from,omitempty"` // sealed relay sender
Box string `json:"box,omitempty"` // base64 crypto_box
}
```
### 3e. DataChannel `hello` type (new)
The first message on every opened DataChannel is a mandatory identity
confirmation (§6). Add:
```go
// HelloMessage is the first message sent on the "yaw" DataChannel.
// The sig binds this identity to the specific DTLS session.
type HelloMessage struct {
Type string `json:"type"` // always "hello"
ID string `json:"id"` // hex pubkey
Nick string `json:"nick"` // alias
Caps []string `json:"caps"` // capability list, e.g. ["chat","file"]
Sig string `json:"sig"` // hex ed25519 sig over bind string (see below)
}
// HelloBindString returns the bytes the hello signature covers:
// "yaw/2 bind" || localDTLSFP(32 bytes) || remoteDTLSFP(32 bytes)
func HelloBindString(localFP, remoteFP []byte) []byte {
buf := []byte("yaw/2 bind")
buf = append(buf, localFP...)
buf = append(buf, remoteFP...)
return buf
}
```
### 3f. File transfer: dedicated channel, no chunk type in main channel
Remove `FileChunk` from `PeerMessage` — chunks go over a separate binary
DataChannel labeled `f:<xid>`. Keep `FileOffer`, `FileResponse`, and add
`FileDone`:
```go
type FileDone struct {
Mid string `json:"mid"`
Xid string `json:"xid"` // transfer id
SHA256 string `json:"sha256"` // hex, for receiver to verify
}
```
---
## Step 4 — `internal/mesh/peer.go`: replace with pion/webrtc
This is the largest single change. The current file (raw TCP dial/accept,
manual ECDH, encrypt/decrypt loop) is replaced entirely.
The new shape:
```go
// ConnectToPeer initiates a WebRTC session with a peer.
// Called when we are the offerer (our id < their id, lexicographically).
func ConnectToPeer(peerID proto.PeerID, m *Mesh, anchor Anchor) error {
pc, err := webrtc.NewPeerConnection(webrtc.Configuration{
ICEServers: []webrtc.ICEServer{{URLs: []string{"stun:stun.fnlr.se:3478"}}},
})
// create "yaw" DataChannel
dc, _ := pc.CreateDataChannel("yaw", &webrtc.DataChannelInit{Ordered: ptr(true)})
// wire up ICE candidate trickle → seal → send via anchor
pc.OnICECandidate(func(c *webrtc.ICECandidate) { /* seal and send */ })
// create offer, set local description, seal it, send via anchor
offer, _ := pc.CreateOffer(nil)
pc.SetLocalDescription(offer)
// seal offer → anchor.SendTo(peerID, sealedOffer)
// ... dc.OnOpen → sendHello, dc.OnMessage → handleMessage
}
// HandleAnswer processes a sealed answer received from the anchor.
func HandleAnswer(sealed string, fromID proto.PeerID, pc *webrtc.PeerConnection, ...) error {
// open the nacl box, unmarshal SignalingPayload, set remote description
}
```
Key things pion handles for you that you were going to write manually:
- All ICE candidate gathering (host + server-reflexive via STUN)
- The DTLS handshake and key material
- SCTP framing over UDP
- DataChannel reliable/ordered delivery
Key things you still write:
- Sealing/opening signaling payloads with `nacl/box` before sending to anchor
- The `hello` confirmation on DataChannel open (mandatory per §6)
- Verifying the hello signature against DTLS fingerprints
- Deciding who offers (smaller id offers — one `strings.Compare` call)
---
## Step 5 — `internal/nat`: delete it
Remove the package entirely. ICE does what this package was going to do, and
does it correctly for both host-to-host and NAT-traversal cases. The only
"NAT" configuration you provide is the STUN server URL passed to
`webrtc.NewPeerConnection`.
Update `cmd/daemon/main.go` to remove the `nat.Run(...)` goroutine and the
`-relay` flag.
---
## Step 6 — `cmd/relay` → `cmd/anchor`
Rename the binary and replace its internals. The anchor is a WebSocket server,
not a TCP server. It never reads the content of `box` fields.
Rough structure:
```go
// cmd/anchor/main.go
// Uses golang.org/x/net/websocket or nhooyr.io/websocket
// Per-connection state
type client struct {
id string // hex peer id, set after join
net string // hashed network name
ch chan []byte // outbound message queue
}
// On "join": verify ed25519 sig over (nonce || net), register (net, id) → client
// On "to": look up (net, to) → forward {type:"from", from:senderID, box:...}
// On disconnect: broadcast {type:"peer-leave", id:...} to net members
```
The anchor also runs (or points to) a STUN server. The simplest approach is
to run `coturn` on your Hetzner VPS in STUN-only mode alongside the anchor
binary, or use a public STUN server during development (`stun:stun.l.google.com:19302`).
The network name is never stored in plaintext:
```go
func hashNetName(name string) string {
h := sha256.Sum256([]byte("yaw2-net:" + name))
return hex.EncodeToString(h[:])
}
```
---
## Step 7 — `internal/ipc`: minor additions
The IPC layer mostly survives. Add two new commands:
```go
// Connect via anchor (replaces direct TCP connect)
CmdJoinNetwork IpcMsgType = "join_network" // fields: network_name (plaintext)
CmdLeaveNetwork IpcMsgType = "leave_network"
// New event when a peer's DataChannel opens and hello is verified
EvtSessionReady IpcMsgType = "session_ready" // fields: peer_id, nick
```
Remove `CmdConnect` (direct TCP dial) — there's no direct dialing in YAW/2,
only joining a named network via the anchor.
---
## Order to implement
Do these in order — each step produces something testable before moving to the next.
1. **Crypto changes** (Step 2) — unit-testable, no network involved. Write a
test that round-trips a `SignalingBox` seal/open and confirms the hex id format.
2. **Proto additions** (Step 3) — add types, confirm it compiles.
3. **Anchor server** (Step 6) — build this first so you have something to connect
to. Test with `websocat` or a browser `WebSocket` console.
4. **WebRTC peer connection** (Step 4) — start with two daemons on the same LAN,
anchor running locally. Confirm ICE succeeds and the `hello` bind check passes.
5. **Delete nat, update ipc, update daemon main** (Steps 5 + 7) — cleanup after
the above works.
6. **File transfer** — once chat works, add the `f:<xid>` DataChannel flow.
---
## What interoperability actually means in practice
Once this is done, your Go daemon and your friend's web/Tauri/Python client can
be in the same named network. They'll connect to the same anchor WebSocket,
exchange sealed offers/answers, and open DataChannels directly to each other.
The `hello` message format, the signaling box format, and the DataChannel
message types are the shared surface — as long as those match §5§9 of the
YAW/2 spec, the implementations are interoperable regardless of language.
The reference STUN and signaling endpoints in the spec are `stun.fnlr.se:3478`
and `wss://fnlr.se/...` (path TBD) — coordinate with your friend on the final
WebSocket path before wiring it in.