Replace the original raw-TCP peer connections and Blowfish/RSA-era design with the YAW/2 protocol stack: Transport - internal/mesh/peer.go: raw TCP + manual ECDH replaced with pion/webrtc DataChannels (ICE, DTLS, SCTP). Peers negotiate via sealed offer/answer/ candidate exchange rather than direct dialling. - internal/nat: deleted — ICE subsumes everything this package was going to do. Signaling - cmd/anchor: new WebSocket signaling server (replaces cmd/relay). Verifies Ed25519 challenge/join signatures, routes opaque nacl/box blobs, broadcasts peer-join/peer-leave events. Never sees plaintext signaling content. - internal/anchor: new anchor client. Manages PeerConnection lifecycle, decides offerer by peer-id comparison, seals/opens signaling payloads with nacl/box, implements mesh.Anchor. Crypto (internal/crypto) - PeerID encoding: base64url → lowercase hex (64 chars, YAW/2 §2). - Sign/Verify: hex signatures throughout. - Added CurvePublicKey/CurvePrivateKey: X25519 keys derived from Ed25519 identity via Montgomery conversion (filippo.io/edwards25519), matching libsodium's crypto_sign_ed25519_*_to_curve25519. - Added SignalingBox/SignalingOpen: nacl/box (XSalsa20-Poly1305) seal/open for signaling payloads. Wire types (internal/proto) - ChatMessage gains Mid (random 16-byte hex) for mesh deduplication (§8). - FileChunk removed from PeerMessage — chunks go on a separate binary DataChannel labelled "f:<xid>"; FileDone added. - New types: SignalingPayload, AnchorMessage, HelloMessage, HelloBindString. - PeerID.Short() now returns first 16 hex chars grouped in 4s. - IPC: CmdConnect removed; CmdJoinNetwork/CmdLeaveNetwork added; EvtSessionReady added (fires when DataChannel opens and hello is received). IPC (internal/ipc) - join_network/leave_network replace connect (direct TCP dial). - Network joins are context-scoped: leave_network or client disconnect cancels the anchor connection cleanly. README updated to reflect new project layout, getting-started commands, IPC protocol, and roadmap state. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
425 lines
14 KiB
Markdown
425 lines
14 KiB
Markdown
# Migrating waste-go → YAW/2
|
||
|
||
This document describes every change needed to make waste-go speak the YAW/2
|
||
protocol and interoperate with the reference implementation. Read it top to
|
||
bottom before touching any code — the changes have an order that matters.
|
||
|
||
---
|
||
|
||
## What stays, what goes, what changes
|
||
|
||
| Package | Fate | Reason |
|
||
|---|---|---|
|
||
| `internal/crypto` | **Keep, modify** | Ed25519 is correct; swap encoding + add nacl/box |
|
||
| `internal/proto` | **Keep, extend** | Message shapes are close; add `mid`, tweak file transfer |
|
||
| `internal/mesh` (state) | **Keep, modify** | Peer map, event fan-out, IPC subscriptions all survive |
|
||
| `internal/mesh/peer.go` | **Replace** | Raw TCP → WebRTC DataChannel |
|
||
| `internal/ipc` | **Keep, minor tweaks** | IPC shape doesn't change |
|
||
| `internal/nat` | **Delete** | ICE/STUN inside pion/webrtc replaces this entirely |
|
||
| `cmd/relay` | **Replace** | TCP forwarder → WebSocket signaling anchor |
|
||
|
||
---
|
||
|
||
## Step 1 — New dependency: pion/webrtc
|
||
|
||
Add to `go.mod`:
|
||
|
||
```
|
||
github.com/pion/webrtc/v3 v3.x.x
|
||
golang.org/x/crypto (already present)
|
||
```
|
||
|
||
Run `go mod tidy` after editing.
|
||
|
||
`pion/webrtc` is a pure-Go implementation of the WebRTC stack. It gives you ICE
|
||
(host + server-reflexive candidates), DTLS, SCTP, and DataChannels — everything
|
||
YAW/2's transport layer assumes. You no longer implement any of that yourself.
|
||
|
||
---
|
||
|
||
## Step 2 — `internal/crypto`: three changes
|
||
|
||
### 2a. Identity encoding: base64url → hex
|
||
|
||
YAW/2 `id` is lowercase hex of the 32-byte Ed25519 public key (64 chars).
|
||
|
||
```go
|
||
// Before
|
||
import "encoding/base64"
|
||
var b64 = base64.RawURLEncoding
|
||
func (id *Identity) PeerID() proto.PeerID {
|
||
return proto.PeerID(b64.EncodeToString(id.PublicKey))
|
||
}
|
||
|
||
// After
|
||
import "encoding/hex"
|
||
func (id *Identity) PeerID() proto.PeerID {
|
||
return proto.PeerID(hex.EncodeToString(id.PublicKey))
|
||
}
|
||
```
|
||
|
||
Update `Verify` to decode hex instead of base64url. This is a **wire-breaking
|
||
change** — existing identity files still work (they store the raw key bytes),
|
||
only the over-the-wire representation changes.
|
||
|
||
### 2b. Signaling crypto: add nacl/box
|
||
|
||
YAW/2 seals signaling payloads (offer/answer/candidates) with libsodium's
|
||
`crypto_box` — XSalsa20-Poly1305. In Go this is `golang.org/x/crypto/nacl/box`,
|
||
which is byte-identical to libsodium.
|
||
|
||
Add to `internal/crypto/crypto.go`:
|
||
|
||
```go
|
||
import "golang.org/x/crypto/nacl/box"
|
||
|
||
// SignalingBox seals a plaintext payload for a recipient.
|
||
// Returns base64(nonce(24) || ciphertext) as the YAW/2 spec requires.
|
||
func SignalingBox(plaintext []byte, recipientPub, senderPriv *[32]byte) string {
|
||
var nonce [24]byte
|
||
rand.Read(nonce[:])
|
||
ct := box.Seal(nonce[:], plaintext, &nonce, recipientPub, senderPriv)
|
||
return base64.StdEncoding.EncodeToString(ct)
|
||
}
|
||
|
||
// SignalingOpen opens a sealed box from a sender.
|
||
func SignalingOpen(b64box string, senderPub, recipientPriv *[32]byte) ([]byte, error) {
|
||
raw, err := base64.StdEncoding.DecodeString(b64box)
|
||
if err != nil || len(raw) < 24 {
|
||
return nil, errors.New("invalid box")
|
||
}
|
||
var nonce [24]byte
|
||
copy(nonce[:], raw[:24])
|
||
out, ok := box.Open(nil, raw[24:], &nonce, senderPub, recipientPriv)
|
||
if !ok {
|
||
return nil, errors.New("box open failed")
|
||
}
|
||
return out, nil
|
||
}
|
||
```
|
||
|
||
### 2c. X25519 keys must be derived from the Ed25519 identity
|
||
|
||
YAW/2 requires that signaling boxes are sealed using X25519 keys derived from
|
||
the Ed25519 identity — not independently generated. This is the libsodium
|
||
`crypto_sign_ed25519_pk_to_curve25519` / `crypto_sign_ed25519_sk_to_curve25519`
|
||
conversion.
|
||
|
||
Add to `Identity`:
|
||
|
||
```go
|
||
import "filippo.io/edwards25519"
|
||
|
||
// CurvePublicKey returns the X25519 public key derived from this identity.
|
||
// Used for sealing signaling boxes (YAW/2 §3).
|
||
func (id *Identity) CurvePublicKey() *[32]byte {
|
||
edPoint, _ := new(edwards25519.Point).SetBytes(id.PublicKey)
|
||
mont := edPoint.BytesMontgomery()
|
||
var out [32]byte
|
||
copy(out[:], mont)
|
||
return &out
|
||
}
|
||
|
||
// CurvePrivateKey returns the X25519 private key derived from this identity.
|
||
func (id *Identity) CurvePrivateKey() *[32]byte {
|
||
// Ed25519 private key is 64 bytes: scalar(32) || pubkey(32)
|
||
// The X25519 scalar is the clamped SHA-512 first half.
|
||
h := sha512.Sum512(id.privateKey[:32])
|
||
h[0] &= 248
|
||
h[31] &= 127
|
||
h[31] |= 64
|
||
var out [32]byte
|
||
copy(out[:], h[:32])
|
||
return &out
|
||
}
|
||
```
|
||
|
||
Add `filippo.io/edwards25519` to `go.mod` (it's a Go standard library
|
||
dependency already pulled in transitively by `golang.org/x/crypto` — just
|
||
import it directly).
|
||
|
||
---
|
||
|
||
## Step 3 — `internal/proto`: wire format updates
|
||
|
||
### 3a. PeerID is now hex
|
||
|
||
```go
|
||
// PeerID is lowercase hex of the 32-byte Ed25519 public key (64 chars).
|
||
type PeerID string
|
||
|
||
func (p PeerID) Short() string {
|
||
// YAW/2 short id: first 16 hex chars, grouped in 4s: "a1b2 c3d4 e5f6 0718"
|
||
s := string(p)
|
||
if len(s) < 16 {
|
||
return s
|
||
}
|
||
return s[0:4] + " " + s[4:8] + " " + s[8:12] + " " + s[12:16]
|
||
}
|
||
```
|
||
|
||
### 3b. Every application message needs a `mid`
|
||
|
||
YAW/2 requires a `mid` (random 16-byte hex) on every DataChannel message for
|
||
deduplication in the full mesh (§8).
|
||
|
||
```go
|
||
type ChatMessage struct {
|
||
Mid string `json:"mid"` // add this — random 16-byte hex
|
||
ID string `json:"id"` // can keep for internal use
|
||
From PeerID `json:"from"`
|
||
// ... rest unchanged
|
||
}
|
||
```
|
||
|
||
Generate `mid` the same way as message IDs: `hex.EncodeToString(randomBytes(16))`.
|
||
|
||
### 3c. Signaling payload types (new)
|
||
|
||
Add a new group of types for the signaling layer — these go *inside* the sealed
|
||
boxes exchanged over the WebSocket anchor connection:
|
||
|
||
```go
|
||
// SignalingKind identifies the kind of sealed signaling payload.
|
||
type SignalingKind string
|
||
|
||
const (
|
||
SigOffer SignalingKind = "offer"
|
||
SigAnswer SignalingKind = "answer"
|
||
SigCandidate SignalingKind = "candidate"
|
||
SigBye SignalingKind = "bye"
|
||
)
|
||
|
||
// SignalingPayload is the JSON plaintext sealed inside a crypto_box.
|
||
type SignalingPayload struct {
|
||
Kind SignalingKind `json:"kind"`
|
||
SDP string `json:"sdp,omitempty"` // offer / answer
|
||
Cand string `json:"cand,omitempty"` // trickle ICE candidate line
|
||
Mid string `json:"mid,omitempty"` // media stream id for candidate
|
||
MLine int `json:"mline,omitempty"`
|
||
}
|
||
```
|
||
|
||
### 3d. Anchor WebSocket wire types (new)
|
||
|
||
The WebSocket connection to the anchor speaks its own JSON protocol (§5).
|
||
Add these alongside the existing IPC types:
|
||
|
||
```go
|
||
// AnchorMsgType identifies anchor WebSocket messages.
|
||
type AnchorMsgType string
|
||
|
||
const (
|
||
AnchorChallenge AnchorMsgType = "challenge"
|
||
AnchorJoin AnchorMsgType = "join"
|
||
AnchorJoined AnchorMsgType = "joined"
|
||
AnchorPeerJoin AnchorMsgType = "peer-join"
|
||
AnchorPeerLeave AnchorMsgType = "peer-leave"
|
||
AnchorTo AnchorMsgType = "to"
|
||
AnchorFrom AnchorMsgType = "from"
|
||
AnchorNoPeer AnchorMsgType = "no-peer"
|
||
)
|
||
|
||
// AnchorMessage covers all WebSocket frames to/from the anchor.
|
||
type AnchorMessage struct {
|
||
Type AnchorMsgType `json:"type"`
|
||
Nonce string `json:"nonce,omitempty"` // challenge nonce, hex
|
||
ID string `json:"id,omitempty"` // peer hex id
|
||
Net string `json:"net,omitempty"` // hashed network name
|
||
Sig string `json:"sig,omitempty"` // ed25519 sig, hex
|
||
Peers []string `json:"peers,omitempty"` // joined response
|
||
To string `json:"to,omitempty"` // sealed relay target
|
||
From string `json:"from,omitempty"` // sealed relay sender
|
||
Box string `json:"box,omitempty"` // base64 crypto_box
|
||
}
|
||
```
|
||
|
||
### 3e. DataChannel `hello` type (new)
|
||
|
||
The first message on every opened DataChannel is a mandatory identity
|
||
confirmation (§6). Add:
|
||
|
||
```go
|
||
// HelloMessage is the first message sent on the "yaw" DataChannel.
|
||
// The sig binds this identity to the specific DTLS session.
|
||
type HelloMessage struct {
|
||
Type string `json:"type"` // always "hello"
|
||
ID string `json:"id"` // hex pubkey
|
||
Nick string `json:"nick"` // alias
|
||
Caps []string `json:"caps"` // capability list, e.g. ["chat","file"]
|
||
Sig string `json:"sig"` // hex ed25519 sig over bind string (see below)
|
||
}
|
||
|
||
// HelloBindString returns the bytes the hello signature covers:
|
||
// "yaw/2 bind" || localDTLSFP(32 bytes) || remoteDTLSFP(32 bytes)
|
||
func HelloBindString(localFP, remoteFP []byte) []byte {
|
||
buf := []byte("yaw/2 bind")
|
||
buf = append(buf, localFP...)
|
||
buf = append(buf, remoteFP...)
|
||
return buf
|
||
}
|
||
```
|
||
|
||
### 3f. File transfer: dedicated channel, no chunk type in main channel
|
||
|
||
Remove `FileChunk` from `PeerMessage` — chunks go over a separate binary
|
||
DataChannel labeled `f:<xid>`. Keep `FileOffer`, `FileResponse`, and add
|
||
`FileDone`:
|
||
|
||
```go
|
||
type FileDone struct {
|
||
Mid string `json:"mid"`
|
||
Xid string `json:"xid"` // transfer id
|
||
SHA256 string `json:"sha256"` // hex, for receiver to verify
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## Step 4 — `internal/mesh/peer.go`: replace with pion/webrtc
|
||
|
||
This is the largest single change. The current file (raw TCP dial/accept,
|
||
manual ECDH, encrypt/decrypt loop) is replaced entirely.
|
||
|
||
The new shape:
|
||
|
||
```go
|
||
// ConnectToPeer initiates a WebRTC session with a peer.
|
||
// Called when we are the offerer (our id < their id, lexicographically).
|
||
func ConnectToPeer(peerID proto.PeerID, m *Mesh, anchor Anchor) error {
|
||
pc, err := webrtc.NewPeerConnection(webrtc.Configuration{
|
||
ICEServers: []webrtc.ICEServer{{URLs: []string{"stun:stun.fnlr.se:3478"}}},
|
||
})
|
||
// create "yaw" DataChannel
|
||
dc, _ := pc.CreateDataChannel("yaw", &webrtc.DataChannelInit{Ordered: ptr(true)})
|
||
// wire up ICE candidate trickle → seal → send via anchor
|
||
pc.OnICECandidate(func(c *webrtc.ICECandidate) { /* seal and send */ })
|
||
// create offer, set local description, seal it, send via anchor
|
||
offer, _ := pc.CreateOffer(nil)
|
||
pc.SetLocalDescription(offer)
|
||
// seal offer → anchor.SendTo(peerID, sealedOffer)
|
||
// ... dc.OnOpen → sendHello, dc.OnMessage → handleMessage
|
||
}
|
||
|
||
// HandleAnswer processes a sealed answer received from the anchor.
|
||
func HandleAnswer(sealed string, fromID proto.PeerID, pc *webrtc.PeerConnection, ...) error {
|
||
// open the nacl box, unmarshal SignalingPayload, set remote description
|
||
}
|
||
```
|
||
|
||
Key things pion handles for you that you were going to write manually:
|
||
- All ICE candidate gathering (host + server-reflexive via STUN)
|
||
- The DTLS handshake and key material
|
||
- SCTP framing over UDP
|
||
- DataChannel reliable/ordered delivery
|
||
|
||
Key things you still write:
|
||
- Sealing/opening signaling payloads with `nacl/box` before sending to anchor
|
||
- The `hello` confirmation on DataChannel open (mandatory per §6)
|
||
- Verifying the hello signature against DTLS fingerprints
|
||
- Deciding who offers (smaller id offers — one `strings.Compare` call)
|
||
|
||
---
|
||
|
||
## Step 5 — `internal/nat`: delete it
|
||
|
||
Remove the package entirely. ICE does what this package was going to do, and
|
||
does it correctly for both host-to-host and NAT-traversal cases. The only
|
||
"NAT" configuration you provide is the STUN server URL passed to
|
||
`webrtc.NewPeerConnection`.
|
||
|
||
Update `cmd/daemon/main.go` to remove the `nat.Run(...)` goroutine and the
|
||
`-relay` flag.
|
||
|
||
---
|
||
|
||
## Step 6 — `cmd/relay` → `cmd/anchor`
|
||
|
||
Rename the binary and replace its internals. The anchor is a WebSocket server,
|
||
not a TCP server. It never reads the content of `box` fields.
|
||
|
||
Rough structure:
|
||
|
||
```go
|
||
// cmd/anchor/main.go
|
||
// Uses golang.org/x/net/websocket or nhooyr.io/websocket
|
||
|
||
// Per-connection state
|
||
type client struct {
|
||
id string // hex peer id, set after join
|
||
net string // hashed network name
|
||
ch chan []byte // outbound message queue
|
||
}
|
||
|
||
// On "join": verify ed25519 sig over (nonce || net), register (net, id) → client
|
||
// On "to": look up (net, to) → forward {type:"from", from:senderID, box:...}
|
||
// On disconnect: broadcast {type:"peer-leave", id:...} to net members
|
||
```
|
||
|
||
The anchor also runs (or points to) a STUN server. The simplest approach is
|
||
to run `coturn` on your Hetzner VPS in STUN-only mode alongside the anchor
|
||
binary, or use a public STUN server during development (`stun:stun.l.google.com:19302`).
|
||
|
||
The network name is never stored in plaintext:
|
||
|
||
```go
|
||
func hashNetName(name string) string {
|
||
h := sha256.Sum256([]byte("yaw2-net:" + name))
|
||
return hex.EncodeToString(h[:])
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## Step 7 — `internal/ipc`: minor additions
|
||
|
||
The IPC layer mostly survives. Add two new commands:
|
||
|
||
```go
|
||
// Connect via anchor (replaces direct TCP connect)
|
||
CmdJoinNetwork IpcMsgType = "join_network" // fields: network_name (plaintext)
|
||
CmdLeaveNetwork IpcMsgType = "leave_network"
|
||
|
||
// New event when a peer's DataChannel opens and hello is verified
|
||
EvtSessionReady IpcMsgType = "session_ready" // fields: peer_id, nick
|
||
```
|
||
|
||
Remove `CmdConnect` (direct TCP dial) — there's no direct dialing in YAW/2,
|
||
only joining a named network via the anchor.
|
||
|
||
---
|
||
|
||
## Order to implement
|
||
|
||
Do these in order — each step produces something testable before moving to the next.
|
||
|
||
1. **Crypto changes** (Step 2) — unit-testable, no network involved. Write a
|
||
test that round-trips a `SignalingBox` seal/open and confirms the hex id format.
|
||
|
||
2. **Proto additions** (Step 3) — add types, confirm it compiles.
|
||
|
||
3. **Anchor server** (Step 6) — build this first so you have something to connect
|
||
to. Test with `websocat` or a browser `WebSocket` console.
|
||
|
||
4. **WebRTC peer connection** (Step 4) — start with two daemons on the same LAN,
|
||
anchor running locally. Confirm ICE succeeds and the `hello` bind check passes.
|
||
|
||
5. **Delete nat, update ipc, update daemon main** (Steps 5 + 7) — cleanup after
|
||
the above works.
|
||
|
||
6. **File transfer** — once chat works, add the `f:<xid>` DataChannel flow.
|
||
|
||
---
|
||
|
||
## What interoperability actually means in practice
|
||
|
||
Once this is done, your Go daemon and your friend's web/Tauri/Python client can
|
||
be in the same named network. They'll connect to the same anchor WebSocket,
|
||
exchange sealed offers/answers, and open DataChannels directly to each other.
|
||
The `hello` message format, the signaling box format, and the DataChannel
|
||
message types are the shared surface — as long as those match §5–§9 of the
|
||
YAW/2 spec, the implementations are interoperable regardless of language.
|
||
|
||
The reference STUN and signaling endpoints in the spec are `stun.fnlr.se:3478`
|
||
and `wss://fnlr.se/...` (path TBD) — coordinate with your friend on the final
|
||
WebSocket path before wiring it in. |