Migrate to YAW/2 protocol: WebRTC transport, anchor signaling, nacl/box crypto
Replace the original raw-TCP peer connections and Blowfish/RSA-era design with the YAW/2 protocol stack: Transport - internal/mesh/peer.go: raw TCP + manual ECDH replaced with pion/webrtc DataChannels (ICE, DTLS, SCTP). Peers negotiate via sealed offer/answer/ candidate exchange rather than direct dialling. - internal/nat: deleted — ICE subsumes everything this package was going to do. Signaling - cmd/anchor: new WebSocket signaling server (replaces cmd/relay). Verifies Ed25519 challenge/join signatures, routes opaque nacl/box blobs, broadcasts peer-join/peer-leave events. Never sees plaintext signaling content. - internal/anchor: new anchor client. Manages PeerConnection lifecycle, decides offerer by peer-id comparison, seals/opens signaling payloads with nacl/box, implements mesh.Anchor. Crypto (internal/crypto) - PeerID encoding: base64url → lowercase hex (64 chars, YAW/2 §2). - Sign/Verify: hex signatures throughout. - Added CurvePublicKey/CurvePrivateKey: X25519 keys derived from Ed25519 identity via Montgomery conversion (filippo.io/edwards25519), matching libsodium's crypto_sign_ed25519_*_to_curve25519. - Added SignalingBox/SignalingOpen: nacl/box (XSalsa20-Poly1305) seal/open for signaling payloads. Wire types (internal/proto) - ChatMessage gains Mid (random 16-byte hex) for mesh deduplication (§8). - FileChunk removed from PeerMessage — chunks go on a separate binary DataChannel labelled "f:<xid>"; FileDone added. - New types: SignalingPayload, AnchorMessage, HelloMessage, HelloBindString. - PeerID.Short() now returns first 16 hex chars grouped in 4s. - IPC: CmdConnect removed; CmdJoinNetwork/CmdLeaveNetwork added; EvtSessionReady added (fires when DataChannel opens and hello is received). IPC (internal/ipc) - join_network/leave_network replace connect (direct TCP dial). - Network joins are context-scoped: leave_network or client disconnect cancels the anchor connection cleanly. README updated to reflect new project layout, getting-started commands, IPC protocol, and roadmap state. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Fredrik Johansson
425
MIGRATION.md
Normal file
425
MIGRATION.md
Normal file
@@ -0,0 +1,425 @@
|
||||
# Migrating waste-go → YAW/2
|
||||
|
||||
This document describes every change needed to make waste-go speak the YAW/2
|
||||
protocol and interoperate with the reference implementation. Read it top to
|
||||
bottom before touching any code — the changes have an order that matters.
|
||||
|
||||
---
|
||||
|
||||
## What stays, what goes, what changes
|
||||
|
||||
| Package | Fate | Reason |
|
||||
|---|---|---|
|
||||
| `internal/crypto` | **Keep, modify** | Ed25519 is correct; swap encoding + add nacl/box |
|
||||
| `internal/proto` | **Keep, extend** | Message shapes are close; add `mid`, tweak file transfer |
|
||||
| `internal/mesh` (state) | **Keep, modify** | Peer map, event fan-out, IPC subscriptions all survive |
|
||||
| `internal/mesh/peer.go` | **Replace** | Raw TCP → WebRTC DataChannel |
|
||||
| `internal/ipc` | **Keep, minor tweaks** | IPC shape doesn't change |
|
||||
| `internal/nat` | **Delete** | ICE/STUN inside pion/webrtc replaces this entirely |
|
||||
| `cmd/relay` | **Replace** | TCP forwarder → WebSocket signaling anchor |
|
||||
|
||||
---
|
||||
|
||||
## Step 1 — New dependency: pion/webrtc
|
||||
|
||||
Add to `go.mod`:
|
||||
|
||||
```
|
||||
github.com/pion/webrtc/v3 v3.x.x
|
||||
golang.org/x/crypto (already present)
|
||||
```
|
||||
|
||||
Run `go mod tidy` after editing.
|
||||
|
||||
`pion/webrtc` is a pure-Go implementation of the WebRTC stack. It gives you ICE
|
||||
(host + server-reflexive candidates), DTLS, SCTP, and DataChannels — everything
|
||||
YAW/2's transport layer assumes. You no longer implement any of that yourself.
|
||||
|
||||
---
|
||||
|
||||
## Step 2 — `internal/crypto`: three changes
|
||||
|
||||
### 2a. Identity encoding: base64url → hex
|
||||
|
||||
YAW/2 `id` is lowercase hex of the 32-byte Ed25519 public key (64 chars).
|
||||
|
||||
```go
|
||||
// Before
|
||||
import "encoding/base64"
|
||||
var b64 = base64.RawURLEncoding
|
||||
func (id *Identity) PeerID() proto.PeerID {
|
||||
return proto.PeerID(b64.EncodeToString(id.PublicKey))
|
||||
}
|
||||
|
||||
// After
|
||||
import "encoding/hex"
|
||||
func (id *Identity) PeerID() proto.PeerID {
|
||||
return proto.PeerID(hex.EncodeToString(id.PublicKey))
|
||||
}
|
||||
```
|
||||
|
||||
Update `Verify` to decode hex instead of base64url. This is a **wire-breaking
|
||||
change** — existing identity files still work (they store the raw key bytes),
|
||||
only the over-the-wire representation changes.
|
||||
|
||||
### 2b. Signaling crypto: add nacl/box
|
||||
|
||||
YAW/2 seals signaling payloads (offer/answer/candidates) with libsodium's
|
||||
`crypto_box` — XSalsa20-Poly1305. In Go this is `golang.org/x/crypto/nacl/box`,
|
||||
which is byte-identical to libsodium.
|
||||
|
||||
Add to `internal/crypto/crypto.go`:
|
||||
|
||||
```go
|
||||
import "golang.org/x/crypto/nacl/box"
|
||||
|
||||
// SignalingBox seals a plaintext payload for a recipient.
|
||||
// Returns base64(nonce(24) || ciphertext) as the YAW/2 spec requires.
|
||||
func SignalingBox(plaintext []byte, recipientPub, senderPriv *[32]byte) string {
|
||||
var nonce [24]byte
|
||||
rand.Read(nonce[:])
|
||||
ct := box.Seal(nonce[:], plaintext, &nonce, recipientPub, senderPriv)
|
||||
return base64.StdEncoding.EncodeToString(ct)
|
||||
}
|
||||
|
||||
// SignalingOpen opens a sealed box from a sender.
|
||||
func SignalingOpen(b64box string, senderPub, recipientPriv *[32]byte) ([]byte, error) {
|
||||
raw, err := base64.StdEncoding.DecodeString(b64box)
|
||||
if err != nil || len(raw) < 24 {
|
||||
return nil, errors.New("invalid box")
|
||||
}
|
||||
var nonce [24]byte
|
||||
copy(nonce[:], raw[:24])
|
||||
out, ok := box.Open(nil, raw[24:], &nonce, senderPub, recipientPriv)
|
||||
if !ok {
|
||||
return nil, errors.New("box open failed")
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
```
|
||||
|
||||
### 2c. X25519 keys must be derived from the Ed25519 identity
|
||||
|
||||
YAW/2 requires that signaling boxes are sealed using X25519 keys derived from
|
||||
the Ed25519 identity — not independently generated. This is the libsodium
|
||||
`crypto_sign_ed25519_pk_to_curve25519` / `crypto_sign_ed25519_sk_to_curve25519`
|
||||
conversion.
|
||||
|
||||
Add to `Identity`:
|
||||
|
||||
```go
|
||||
import "filippo.io/edwards25519"
|
||||
|
||||
// CurvePublicKey returns the X25519 public key derived from this identity.
|
||||
// Used for sealing signaling boxes (YAW/2 §3).
|
||||
func (id *Identity) CurvePublicKey() *[32]byte {
|
||||
edPoint, _ := new(edwards25519.Point).SetBytes(id.PublicKey)
|
||||
mont := edPoint.BytesMontgomery()
|
||||
var out [32]byte
|
||||
copy(out[:], mont)
|
||||
return &out
|
||||
}
|
||||
|
||||
// CurvePrivateKey returns the X25519 private key derived from this identity.
|
||||
func (id *Identity) CurvePrivateKey() *[32]byte {
|
||||
// Ed25519 private key is 64 bytes: scalar(32) || pubkey(32)
|
||||
// The X25519 scalar is the clamped SHA-512 first half.
|
||||
h := sha512.Sum512(id.privateKey[:32])
|
||||
h[0] &= 248
|
||||
h[31] &= 127
|
||||
h[31] |= 64
|
||||
var out [32]byte
|
||||
copy(out[:], h[:32])
|
||||
return &out
|
||||
}
|
||||
```
|
||||
|
||||
Add `filippo.io/edwards25519` to `go.mod` (it's a Go standard library
|
||||
dependency already pulled in transitively by `golang.org/x/crypto` — just
|
||||
import it directly).
|
||||
|
||||
---
|
||||
|
||||
## Step 3 — `internal/proto`: wire format updates
|
||||
|
||||
### 3a. PeerID is now hex
|
||||
|
||||
```go
|
||||
// PeerID is lowercase hex of the 32-byte Ed25519 public key (64 chars).
|
||||
type PeerID string
|
||||
|
||||
func (p PeerID) Short() string {
|
||||
// YAW/2 short id: first 16 hex chars, grouped in 4s: "a1b2 c3d4 e5f6 0718"
|
||||
s := string(p)
|
||||
if len(s) < 16 {
|
||||
return s
|
||||
}
|
||||
return s[0:4] + " " + s[4:8] + " " + s[8:12] + " " + s[12:16]
|
||||
}
|
||||
```
|
||||
|
||||
### 3b. Every application message needs a `mid`
|
||||
|
||||
YAW/2 requires a `mid` (random 16-byte hex) on every DataChannel message for
|
||||
deduplication in the full mesh (§8).
|
||||
|
||||
```go
|
||||
type ChatMessage struct {
|
||||
Mid string `json:"mid"` // add this — random 16-byte hex
|
||||
ID string `json:"id"` // can keep for internal use
|
||||
From PeerID `json:"from"`
|
||||
// ... rest unchanged
|
||||
}
|
||||
```
|
||||
|
||||
Generate `mid` the same way as message IDs: `hex.EncodeToString(randomBytes(16))`.
|
||||
|
||||
### 3c. Signaling payload types (new)
|
||||
|
||||
Add a new group of types for the signaling layer — these go *inside* the sealed
|
||||
boxes exchanged over the WebSocket anchor connection:
|
||||
|
||||
```go
|
||||
// SignalingKind identifies the kind of sealed signaling payload.
|
||||
type SignalingKind string
|
||||
|
||||
const (
|
||||
SigOffer SignalingKind = "offer"
|
||||
SigAnswer SignalingKind = "answer"
|
||||
SigCandidate SignalingKind = "candidate"
|
||||
SigBye SignalingKind = "bye"
|
||||
)
|
||||
|
||||
// SignalingPayload is the JSON plaintext sealed inside a crypto_box.
|
||||
type SignalingPayload struct {
|
||||
Kind SignalingKind `json:"kind"`
|
||||
SDP string `json:"sdp,omitempty"` // offer / answer
|
||||
Cand string `json:"cand,omitempty"` // trickle ICE candidate line
|
||||
Mid string `json:"mid,omitempty"` // media stream id for candidate
|
||||
MLine int `json:"mline,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
### 3d. Anchor WebSocket wire types (new)
|
||||
|
||||
The WebSocket connection to the anchor speaks its own JSON protocol (§5).
|
||||
Add these alongside the existing IPC types:
|
||||
|
||||
```go
|
||||
// AnchorMsgType identifies anchor WebSocket messages.
|
||||
type AnchorMsgType string
|
||||
|
||||
const (
|
||||
AnchorChallenge AnchorMsgType = "challenge"
|
||||
AnchorJoin AnchorMsgType = "join"
|
||||
AnchorJoined AnchorMsgType = "joined"
|
||||
AnchorPeerJoin AnchorMsgType = "peer-join"
|
||||
AnchorPeerLeave AnchorMsgType = "peer-leave"
|
||||
AnchorTo AnchorMsgType = "to"
|
||||
AnchorFrom AnchorMsgType = "from"
|
||||
AnchorNoPeer AnchorMsgType = "no-peer"
|
||||
)
|
||||
|
||||
// AnchorMessage covers all WebSocket frames to/from the anchor.
|
||||
type AnchorMessage struct {
|
||||
Type AnchorMsgType `json:"type"`
|
||||
Nonce string `json:"nonce,omitempty"` // challenge nonce, hex
|
||||
ID string `json:"id,omitempty"` // peer hex id
|
||||
Net string `json:"net,omitempty"` // hashed network name
|
||||
Sig string `json:"sig,omitempty"` // ed25519 sig, hex
|
||||
Peers []string `json:"peers,omitempty"` // joined response
|
||||
To string `json:"to,omitempty"` // sealed relay target
|
||||
From string `json:"from,omitempty"` // sealed relay sender
|
||||
Box string `json:"box,omitempty"` // base64 crypto_box
|
||||
}
|
||||
```
|
||||
|
||||
### 3e. DataChannel `hello` type (new)
|
||||
|
||||
The first message on every opened DataChannel is a mandatory identity
|
||||
confirmation (§6). Add:
|
||||
|
||||
```go
|
||||
// HelloMessage is the first message sent on the "yaw" DataChannel.
|
||||
// The sig binds this identity to the specific DTLS session.
|
||||
type HelloMessage struct {
|
||||
Type string `json:"type"` // always "hello"
|
||||
ID string `json:"id"` // hex pubkey
|
||||
Nick string `json:"nick"` // alias
|
||||
Caps []string `json:"caps"` // capability list, e.g. ["chat","file"]
|
||||
Sig string `json:"sig"` // hex ed25519 sig over bind string (see below)
|
||||
}
|
||||
|
||||
// HelloBindString returns the bytes the hello signature covers:
|
||||
// "yaw/2 bind" || localDTLSFP(32 bytes) || remoteDTLSFP(32 bytes)
|
||||
func HelloBindString(localFP, remoteFP []byte) []byte {
|
||||
buf := []byte("yaw/2 bind")
|
||||
buf = append(buf, localFP...)
|
||||
buf = append(buf, remoteFP...)
|
||||
return buf
|
||||
}
|
||||
```
|
||||
|
||||
### 3f. File transfer: dedicated channel, no chunk type in main channel
|
||||
|
||||
Remove `FileChunk` from `PeerMessage` — chunks go over a separate binary
|
||||
DataChannel labeled `f:<xid>`. Keep `FileOffer`, `FileResponse`, and add
|
||||
`FileDone`:
|
||||
|
||||
```go
|
||||
type FileDone struct {
|
||||
Mid string `json:"mid"`
|
||||
Xid string `json:"xid"` // transfer id
|
||||
SHA256 string `json:"sha256"` // hex, for receiver to verify
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 4 — `internal/mesh/peer.go`: replace with pion/webrtc
|
||||
|
||||
This is the largest single change. The current file (raw TCP dial/accept,
|
||||
manual ECDH, encrypt/decrypt loop) is replaced entirely.
|
||||
|
||||
The new shape:
|
||||
|
||||
```go
|
||||
// ConnectToPeer initiates a WebRTC session with a peer.
|
||||
// Called when we are the offerer (our id < their id, lexicographically).
|
||||
func ConnectToPeer(peerID proto.PeerID, m *Mesh, anchor Anchor) error {
|
||||
pc, err := webrtc.NewPeerConnection(webrtc.Configuration{
|
||||
ICEServers: []webrtc.ICEServer{{URLs: []string{"stun:stun.fnlr.se:3478"}}},
|
||||
})
|
||||
// create "yaw" DataChannel
|
||||
dc, _ := pc.CreateDataChannel("yaw", &webrtc.DataChannelInit{Ordered: ptr(true)})
|
||||
// wire up ICE candidate trickle → seal → send via anchor
|
||||
pc.OnICECandidate(func(c *webrtc.ICECandidate) { /* seal and send */ })
|
||||
// create offer, set local description, seal it, send via anchor
|
||||
offer, _ := pc.CreateOffer(nil)
|
||||
pc.SetLocalDescription(offer)
|
||||
// seal offer → anchor.SendTo(peerID, sealedOffer)
|
||||
// ... dc.OnOpen → sendHello, dc.OnMessage → handleMessage
|
||||
}
|
||||
|
||||
// HandleAnswer processes a sealed answer received from the anchor.
|
||||
func HandleAnswer(sealed string, fromID proto.PeerID, pc *webrtc.PeerConnection, ...) error {
|
||||
// open the nacl box, unmarshal SignalingPayload, set remote description
|
||||
}
|
||||
```
|
||||
|
||||
Key things pion handles for you that you were going to write manually:
|
||||
- All ICE candidate gathering (host + server-reflexive via STUN)
|
||||
- The DTLS handshake and key material
|
||||
- SCTP framing over UDP
|
||||
- DataChannel reliable/ordered delivery
|
||||
|
||||
Key things you still write:
|
||||
- Sealing/opening signaling payloads with `nacl/box` before sending to anchor
|
||||
- The `hello` confirmation on DataChannel open (mandatory per §6)
|
||||
- Verifying the hello signature against DTLS fingerprints
|
||||
- Deciding who offers (smaller id offers — one `strings.Compare` call)
|
||||
|
||||
---
|
||||
|
||||
## Step 5 — `internal/nat`: delete it
|
||||
|
||||
Remove the package entirely. ICE does what this package was going to do, and
|
||||
does it correctly for both host-to-host and NAT-traversal cases. The only
|
||||
"NAT" configuration you provide is the STUN server URL passed to
|
||||
`webrtc.NewPeerConnection`.
|
||||
|
||||
Update `cmd/daemon/main.go` to remove the `nat.Run(...)` goroutine and the
|
||||
`-relay` flag.
|
||||
|
||||
---
|
||||
|
||||
## Step 6 — `cmd/relay` → `cmd/anchor`
|
||||
|
||||
Rename the binary and replace its internals. The anchor is a WebSocket server,
|
||||
not a TCP server. It never reads the content of `box` fields.
|
||||
|
||||
Rough structure:
|
||||
|
||||
```go
|
||||
// cmd/anchor/main.go
|
||||
// Uses golang.org/x/net/websocket or nhooyr.io/websocket
|
||||
|
||||
// Per-connection state
|
||||
type client struct {
|
||||
id string // hex peer id, set after join
|
||||
net string // hashed network name
|
||||
ch chan []byte // outbound message queue
|
||||
}
|
||||
|
||||
// On "join": verify ed25519 sig over (nonce || net), register (net, id) → client
|
||||
// On "to": look up (net, to) → forward {type:"from", from:senderID, box:...}
|
||||
// On disconnect: broadcast {type:"peer-leave", id:...} to net members
|
||||
```
|
||||
|
||||
The anchor also runs (or points to) a STUN server. The simplest approach is
|
||||
to run `coturn` on your Hetzner VPS in STUN-only mode alongside the anchor
|
||||
binary, or use a public STUN server during development (`stun:stun.l.google.com:19302`).
|
||||
|
||||
The network name is never stored in plaintext:
|
||||
|
||||
```go
|
||||
func hashNetName(name string) string {
|
||||
h := sha256.Sum256([]byte("yaw2-net:" + name))
|
||||
return hex.EncodeToString(h[:])
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 7 — `internal/ipc`: minor additions
|
||||
|
||||
The IPC layer mostly survives. Add two new commands:
|
||||
|
||||
```go
|
||||
// Connect via anchor (replaces direct TCP connect)
|
||||
CmdJoinNetwork IpcMsgType = "join_network" // fields: network_name (plaintext)
|
||||
CmdLeaveNetwork IpcMsgType = "leave_network"
|
||||
|
||||
// New event when a peer's DataChannel opens and hello is verified
|
||||
EvtSessionReady IpcMsgType = "session_ready" // fields: peer_id, nick
|
||||
```
|
||||
|
||||
Remove `CmdConnect` (direct TCP dial) — there's no direct dialing in YAW/2,
|
||||
only joining a named network via the anchor.
|
||||
|
||||
---
|
||||
|
||||
## Order to implement
|
||||
|
||||
Do these in order — each step produces something testable before moving to the next.
|
||||
|
||||
1. **Crypto changes** (Step 2) — unit-testable, no network involved. Write a
|
||||
test that round-trips a `SignalingBox` seal/open and confirms the hex id format.
|
||||
|
||||
2. **Proto additions** (Step 3) — add types, confirm it compiles.
|
||||
|
||||
3. **Anchor server** (Step 6) — build this first so you have something to connect
|
||||
to. Test with `websocat` or a browser `WebSocket` console.
|
||||
|
||||
4. **WebRTC peer connection** (Step 4) — start with two daemons on the same LAN,
|
||||
anchor running locally. Confirm ICE succeeds and the `hello` bind check passes.
|
||||
|
||||
5. **Delete nat, update ipc, update daemon main** (Steps 5 + 7) — cleanup after
|
||||
the above works.
|
||||
|
||||
6. **File transfer** — once chat works, add the `f:<xid>` DataChannel flow.
|
||||
|
||||
---
|
||||
|
||||
## What interoperability actually means in practice
|
||||
|
||||
Once this is done, your Go daemon and your friend's web/Tauri/Python client can
|
||||
be in the same named network. They'll connect to the same anchor WebSocket,
|
||||
exchange sealed offers/answers, and open DataChannels directly to each other.
|
||||
The `hello` message format, the signaling box format, and the DataChannel
|
||||
message types are the shared surface — as long as those match §5–§9 of the
|
||||
YAW/2 spec, the implementations are interoperable regardless of language.
|
||||
|
||||
The reference STUN and signaling endpoints in the spec are `stun.fnlr.se:3478`
|
||||
and `wss://fnlr.se/...` (path TBD) — coordinate with your friend on the final
|
||||
WebSocket path before wiring it in.
|
||||
Reference in New Issue
Block a user