diff --git a/PROTOCOL.md b/PROTOCOL.md new file mode 100644 index 0000000..ea36d1b --- /dev/null +++ b/PROTOCOL.md @@ -0,0 +1,381 @@ +# YAW/2.0 β€” Protocol Specification + +**Version:** `yaw/2.0` Β· **Status:** πŸ”’ **LOCKED** (frozen wire β€” interop baseline) +Β· clean break from WASTE 1.x. + +> **This document is frozen.** Independent implementations interoperate against +> this exact wire and the live server. Do **not** change 2.0. New protocol work β€” +> currently forward-secret signaling β€” lives in +> [yaw2.1-protocol.md](yaw2.1-protocol.md), motivated by +> [YIP-0001](proposals/yip-0001-forward-secret-signaling.md). See [README](README.md). + +YAW/2 is a small, trusted, peer-to-peer encrypted mesh β€” chat, presence, and file +transfer β€” that pierces NAT the way the modern web does (ICE) and rides a modern +encrypted transport (WebRTC DataChannels). A lightweight server (the **anchor**) +helps peers *find* and *introduce* each other but never carries their data. + +## 0. Design goals & decisions + +- **True peer-to-peer.** Data flows directly between peers. The anchor only does + signaling + STUN. *No relay (TURN).* Consequence accepted: peers learn each + other's IP on a direct link, and a minority of peers behind symmetric NAT will + fail to connect. +- **Modern transport.** WebRTC **DataChannels** β€” DTLS 1.2/1.3 (PFS, AEAD), + congestion control, multiplexed reliable streams. File transfer is just a stream. +- **Modern identity.** **Ed25519** identities. Trust is friend-to-friend: you talk + to a peer only if its public key is in your keyring (exchanged out-of-band). +- **Browser-first.** The reference client is a web app (WebRTC native). A **Tauri** + shell wraps it for desktop (Electron is dropped); a **Python CLI** (`aiortc`) + speaks the same protocol. +- **The server learns as little as possible.** Signaling payloads are sealed end + to end, so the anchor never sees SDP, candidate IPs, chat, or files β€” only that + two fingerprints are online in the same (hashed) network and exchanging blobs. + +## 1. Terminology + +| Term | Meaning | +|------|---------| +| **node / peer** | A participant, identified by its Ed25519 public key. | +| **id** | Lowercase hex of the 32-byte Ed25519 public key (64 chars). The node's identity. | +| **short id** | First 16 hex chars of `id`, grouped in 4s, for human verification. | +| **keyring** | The set of peer `id`s you have accepted (trust). | +| **network** | A named group. Scoped on the server by `net = hex(sha256("yaw2-net:" + name))`, so the server never sees the name. | +| **anchor** | The server: a WebSocket **signaling** endpoint + a **STUN** server. | +| **session** | One established WebRTC PeerConnection between two peers. | + +## 2. Identity & trust + +- Each node has an **Ed25519** keypair. The 32-byte public key (hex) is its `id`. +- A node connects to another **only if that peer's `id` is in its keyring.** Keys + are exchanged out of band (paste the hex, a QR code, or `yaw://key/`). +- Human verification: compare **short id**s ("read me yours") before accepting. +- The keyring is the sole trust root. The anchor is *not* trusted to vouch for + identities; it cannot impersonate a peer (Β§7). + +## 3. Cryptography + +| Purpose | Primitive | +|---------|-----------| +| Identity / signatures | **Ed25519** (libsodium `crypto_sign_detached`) | +| Signaling confidentiality + sender auth | **X25519 + crypto_box** (XSalsa20-Poly1305). X25519 keys derived from the Ed25519 identity via `crypto_sign_ed25519_{pk,sk}_to_curve25519`. | +| Transport | **WebRTC DTLS** (ECDHE, AES-GCM/ChaCha20-Poly1305) over SCTP DataChannels. Per-session forward secrecy. | +| Hashes | SHA-256 (network scoping, file integrity) | + +> **Why the DTLS cert is *not* your Ed25519 key.** Browsers generate their own +> ephemeral cert for `RTCPeerConnection`; you cannot make it your identity key. +> Instead we **bind** the identity to the session: the SDP (which contains the +> DTLS certificate fingerprint) travels inside an Ed25519-authenticated sealed +> box, and peers re-confirm with a signed `HELLO` over both fingerprints once the +> channel opens (Β§6). The result is equivalent: the encrypted channel is provably +> to the holder of the trusted Ed25519 key. + +All implementations use **libsodium** (PyNaCl, libsodium.js, or a Rust binding) so +the signing and sealing are byte-identical across clients. + +## 4. Architecture + +``` + β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ anchor (server) ────────────────┐ + β”‚ WSS signaling (relays sealed blobs by id) β”‚ + β”‚ STUN udp/3478 (public-address discovery) β”‚ + β””β”€β”€β”€β”€β”€β”€β”€β–²β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–²β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ + β”‚ sealed offer/answer/cands β”‚ + β”Œβ”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β” + β”‚ peer A │◀═══════════▢│ peer B β”‚ + β”‚ web / cli / β”‚ WebRTC β”‚ web / cli / β”‚ + β”‚ tauri β”‚ DataChannel β”‚ tauri β”‚ + β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ (DTLS, P2P) β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ + direct, encrypted, no server in path +``` + +The anchor has **two** jobs and sees **no** user data: + +1. **Signaling (WebSocket, `wss://`)** β€” authenticates members of a network and + relays opaque sealed blobs between them by `id`. +2. **STUN (UDP/3478)** β€” standard STUN (RFC 5389), e.g. `stun::3478`, + used as an ICE server so peers learn their public (server-reflexive) address. + *STUN only β€” no TURN relay.* + +## 5. Signaling protocol (WebSocket, JSON) + +One WebSocket per node. All frames are **UTF-8 text** JSON objects with a `type` +(binary WebSocket frames are ignored). + +**Encoding conventions (apply everywhere unless stated):** + +- **hex** = lowercase, no separators. +- **base64** = standard RFC 4648 **with padding** (libsodium `base64_variants.ORIGINAL` + β€” *not* the url-safe / no-padding default). +- **`net`** = `hex(sha256(b))` where `b = utf8("yaw2-net:" + name)`; the name is taken + **verbatim** (case-sensitive, no normalization/trimming). +- **`id`** = `hex(ed25519_public_key)` (32 bytes β†’ 64 hex). +- Signature inputs are raw bytes (concatenated as written); signatures are Ed25519 + detached (64 bytes), hex-encoded. + +### 5.1 Join (authentication) + +``` +server β†’ { "v":"yaw/2.0", "type":"challenge", "nonce":"<32-byte hex>" } +client β†’ { "type":"join", + "id": "", + "net": "", + "sig": "" } +server β†’ { "type":"joined", "peers":[ "", ... ] } // current members of net +``` + +`sig` is over the **exact bytes** `nonce_raw || net_ascii`, where `nonce_raw = +hex_decode(nonce)` (32 bytes) and `net_ascii = utf8(net)` (the 64 ASCII hex +chars). The server verifies `sig` against `id`, then registers the socket under +`(net, id)`. A node may join only one `net` per socket. Bad signature β†’ the server +closes with code **4001**; a second connection for the same `(net, id)` displaces +the first with code **4002**. + +### 5.2 Presence + +``` +server β†’ { "type":"peer-join", "id":"" } +server β†’ { "type":"peer-leave", "id":"" } +``` + +Pushed to all members of the same `net` as peers come and go. + +### 5.3 Sealed relay + +``` +client β†’ { "type":"to", "to":"", "box":"" } +server β†’ { "type":"from", "from":"", "box":"" } +``` + +The server forwards `box` verbatim to the socket registered for `(net, to)`, +stamping the real `from`. **The server cannot read `box`.** If `to` is offline, +the server replies `{ "type":"no-peer", "to":"" }`. + +### 5.4 Sealed payload (inside `box`) + +`box = crypto_box(plaintext, nonce, recipient_x25519_pub, sender_x25519_priv)` +(X25519 + XSalsa20-Poly1305), serialized as **`base64(nonce(24) || mac(16) || +ciphertext)`** (i.e. the 24-byte nonce prepended to libsodium's combined-mode +output; PyNaCl's `Box.encrypt(msg, nonce)` already produces exactly this). The +X25519 keys are derived from the Ed25519 identities (Β§3); the recipient uses the +sender's, taken from the `from` id. `plaintext` is JSON: + +``` +{ "kind": "offer" | "answer" | "candidate" | "bye", + "sdp": "", // for offer/answer (candidates embedded; see Β§6) + "cand": "", "mid":"0", "mline":0 } // optional trickle (Β§6) +``` + +Because the box is authenticated by the sender's identity key, a received offer's +SDP β€” **including the DTLS fingerprint** β€” is bound to that identity. + +## 6. Connection establishment + +Both peers are joined to the same `net` and **each has the other's `id` in its +keyring**. (Untrusted `id` β†’ ignore, or hold for manual accept.) + +**Who offers:** the peer with the lexicographically **smaller `id`** is the +*offerer* (deterministic; avoids glare). + +``` +A = offerer (smaller id) B = answerer +─────────────────────────────── ─────────────────────────────── +pc = RTCPeerConnection({iceServers:[stun]}) pc = RTCPeerConnection({iceServers:[stun]}) +dc = pc.createDataChannel("yaw") pc.ondatachannel = … +createOffer; setLocalDescription +WAIT for ICE gathering complete ◀── candidates embedded in SDP (non-trickle) +seal(offer.sdp) ─────────"to B"────────────────▢ verify from∈keyring; setRemoteDescription + createAnswer; setLocalDescription + WAIT for ICE gathering complete +verify; setRemoteDescription ◀──"to A"─seal(answer.sdp) + ICE connectivity checks (host + srflx) β†’ DTLS handshake + "yaw" DataChannel opens on both sides +───────────────── identity confirm (mandatory) ───────────────── +each side, on open, sends on "yaw": + { "type":"hello", "id":"", "nick":"…", "sig":"" } +verify (below). Mismatch β†’ close the connection. +``` + +**ICE is non-trickle (baseline).** After `setLocalDescription`, wait until ICE +gathering is `complete`, then send the SDP with candidates embedded. Sending extra +`{kind:"candidate"}` messages is **optional** and additive; receivers MUST accept +candidates from the SDP and SHOULD accept trickled ones. Gather **host + +server-reflexive** (STUN) candidates; no TURN. If ICE fails (both behind symmetric +NAT) the session is abandoned (relay is optional, Β§8.4). + +**DataChannels are in-band negotiated** (`negotiated:false`): the offerer creates +`"yaw"`; the answerer receives it via `ondatachannel`. ⚠️ *A received channel may +already be `open` when `ondatachannel`/event fires β€” send your `hello` both on the +`open` event **and** immediately if `readyState === "open"`, or you will deadlock.* + +**Identity confirm β€” exact bytes.** Let `lfp`/`rfp` be the **raw 32-byte** SHA-256 +DTLS fingerprints parsed from the local/remote SDP `a=fingerprint:sha-256 …` lines +(strip the colons, hex-decode). Implementations MUST use `sha-256` fingerprints. + +- **Sender** signs `B = utf8("yaw/2 bind") || lfp || rfp` (its *own* local then + remote) and sends `sig = hex(ed25519_sign(B))` in `hello`. +- **Verifier** reconstructs the sender's bytes β€” which are the verifier's **remote + then local** β€” i.e. checks `ed25519_verify(sig, utf8("yaw/2 bind") || rfp || lfp, + peer_id)` **and** that `peer_id` equals the expected (keyring) id. Either check + failing β‡’ close. + +After `hello` verification the session is **trusted and live**. + +## 7. Why this is safe against a malicious anchor + +- The anchor relays only **sealed, sender-authenticated** blobs, so it cannot read + or forge SDP/candidates, and cannot inject its own DTLS fingerprint (that would + require an Ed25519 signature it cannot produce). +- The `hello` confirmation re-binds the live channel to both DTLS fingerprints + under each identity's signature. +- Therefore a hostile anchor can: see who is online in a `net`, see *that* two ids + exchange blobs, drop/delay messages, and learn timing. It **cannot**: read or + alter chat/files, MITM the channel, learn candidate IPs, or recover the network + name (only confirm a guess of it). + +## 8. Application protocol (over the `yaw` DataChannel) + +The `yaw` channel is reliable + ordered. Each DataChannel message is one +UTF-8 JSON object (DataChannels are message-framed β€” no length prefix needed). +Unknown `type`s and unknown fields are ignored (forward compatibility). In v1 +(full mesh, Β§8.4) there are no duplicates, so **`mid` is optional**; it becomes +**required only when relay (`hops`) is used**, as a random 16-byte hex id for dedup. + +| type | fields | meaning | +|------|--------|---------| +| `hello` | `id, nick, sig` (+ optional `caps[]`) | identity confirm (Β§6); first message | +| `presence` | `online:bool, nick` | online/away | +| `chat` | `room, text, ts` | group message to a room (default `#main`) | +| `pm` | `text, ts` | private message (this link only) | +| `file-offer` | `xid, name, size, sha256` | offer to send a file | +| `file-accept` | `xid` | accept an offer | +| `file-cancel` | `xid, reason` | decline / abort | +| `file-done` | `xid, sha256` | sender finished; verify hash | +| `bye` | β€” | graceful close | + +`ts` is Unix milliseconds (advisory). `room` names are app-defined strings. + +### 8.4 Group delivery (v1 = full mesh) + +In v1 each peer connects **directly to every other** peer in the network (full +mesh of sessions). `chat`/`presence` are sent to **all** open sessions; `pm` to one. +Each message is received once per session, so no dedup is needed (and `mid` may be +omitted). Dedup matters only once relay is enabled below. + +> *Forward-compatible relay (optional, v1.1):* messages may carry `hops` (int, ≀4). +> A node receiving a message with `hops>0` whose `mid` is new MAY re-send it to its +> other peers with `hops-1`, restoring connectivity across pairs that couldn't form +> a direct session. v1 senders set `hops:0` (no relay). + +## 9. File transfer (over a dedicated DataChannel) + +Files ride their own channel so a large transfer never blocks chat. + +``` +sender receiver +file-offer {xid,name,size,sha256} ──"yaw"───────▢ (user accepts) + ◀──"yaw"──── file-accept {xid} +open DataChannel label="f:" (ordered,binary) +stream raw chunks (default 64 KiB), honoring +bufferedAmountLowThreshold for backpressure ───▢ append to file; running sha256 +close "f:" after last chunk +file-done {xid, sha256} ────────"yaw"───────────▢ verify sha256; success/failure +``` + +- Chunk size: **64 KiB** default β€” but never exceed the session's negotiated + `a=max-message-size` (SDP); clamp down if the peer advertises a smaller limit. +- Integrity: SHA-256 over the whole file, sent in the offer and re-asserted in + `file-done`; the receiver verifies before accepting the file. +- The transport (DTLS) already encrypts; no extra app-layer file encryption. +- Either side may `file-cancel {xid}`; the data channel is closed. + +## 10. Reference parameters + +| Parameter | Value | +|-----------|-------| +| Protocol version | `yaw/2.0` | +| Signaling | `wss:////signal` (WebSocket) **[deployed & verified]** | +| STUN | `stun::3478` (coturn, STUN-only, **deployed & verified**) | +| Network scope | `net = hex(sha256("yaw2-net:" + name))` | +| Identity | Ed25519; `id = hex(pubkey)` (64 chars) | +| Signaling seal | libsodium `crypto_box`, `base64_ORIGINAL(nonce(24)||mac(16)||ct)` | +| Bind (sign) | `utf8("yaw/2 bind") || local_fp || remote_fp` (raw 32-byte fps) | +| Signaling close codes | 4001 = auth failed Β· 4002 = displaced by reconnect | +| DataChannel (control) | label `yaw`, reliable, ordered | +| DataChannel (file) | label `f:`, reliable, ordered, binary | +| File chunk | 64 KiB | +| Default room | `#main` | + +## 11. Security considerations + +- **IP exposure (by design).** On a direct session each peer sees the other's host + (LAN) and server-reflexive (public) addresses. The *anchor* does not (sealed + signaling). To reduce LAN leakage, a client MAY gather srflx-only candidates at + some connectivity cost. +- **Symmetric-NAT pairs may not connect** (no TURN). Optional app-relay (Β§8.4) or a + future opt-in TURN can recover these. +- **Signaling metadata.** The anchor learns presence and the contact graph within a + `net`, plus timing. It does not learn names, content, or IPs. +- **Signaling boxes are not forward-secret** (static X25519). They carry only + short-lived SDP/candidates; the *session* keys are DTLS-ephemeral (PFS). A future + revision may use ephemeral signaling keys. +- **Trust bootstrapping is out of band.** Compromise of the keyring exchange (e.g. + accepting a wrong `id`) defeats the system β€” verify short ids. +- **Replay.** The join `nonce` is single-use (server-issued per connection); DTLS + prevents transport replay; `mid` dedups app messages once relay is enabled. + +## 12. Differences from YAW/1 (WASTE) + +| | YAW/1 (WASTE-faithful) | YAW/2 | +|--|--|--| +| Identity | RSA + SHA-1 fingerprint | Ed25519, key = id | +| Transport crypto | Blowfish-PCBC (legacy) | WebRTC DTLS (AEAD, PFS) | +| Handshake | custom 30-step | ICE + DTLS + signed bind | +| NAT traversal | none (needs reachable peer) | ICE/STUN (true P2P) | +| Server role | rendezvous *directory* | signaling + STUN, sealed | +| Topology | flood mesh w/ TTL | direct full mesh (relay optional) | +| Transport library | hand-rolled sockets | WebRTC (browser/aiortc) | + +## 13. Open questions / future + +- Opt-in **TURN** for symmetric-NAT pairs (breaks "no relay" β€” explicit choice). +- **Gossip relay** (Β§8.4 `hops`) for partial-connectivity resilience. +- **Ephemeral signaling keys** for forward-secret signaling. +- **Post-quantum**: hybrid X25519+ML-KEM once WebRTC/libsodium support is routine. +- **Room key distribution** (anchor optionally serves a network's member ids to + ease group bootstrapping, still keyring-gated). + +## 14. Implementing & testing against the live server + +The reference infra is **live**: STUN `stun::3478` and signaling +`wss:////signal` (both deployed & verified). To interop: + +1. Open the WebSocket, complete the `join` (Β§5.1), and you'll see `peers` / + `peer-join` β€” that alone confirms your Ed25519 join signature is correct. +2. Pick a shared **network name** with your test partner; both hash it identically + (Β§5 conventions). Then run the connection flow (Β§6). +3. Cross-check against the reference client: run `cli/spike_peer.py ` + (Python/aiortc) β€” it will dial your implementation and chat/transfer a file. + +**Three gotchas that break naΓ―ve implementations (learned the hard way):** + +- **base64 variant.** The seal is **standard padded** base64, not libsodium's + default url-safe/no-padding. Using `to_base64(x)` without + `base64_variants.ORIGINAL` produces an unopenable box. +- **bind byte order on verify.** The sender signs `prefix||local||remote`; the + verifier must reconstruct `prefix||remote||local` (its remote = the sender's + local). Getting this backwards verifies nothing. +- **answerer DataChannel open-race.** The received `"yaw"` channel is often already + `open` when you get it; send your `hello` on `readyState==="open"` too, not only + the `open` event, or both sides wait forever. + +> **Versioning.** 2.0 is **locked**. The one known weakness β€” signaling is not +> forward-secret (Β§11) β€” is intentionally left as-is here so interop is stable. The +> fix is specified separately as **[yaw/2.1](yaw2.1-protocol.md)** and motivated in +> **[YIP-0001](proposals/yip-0001-forward-secret-signaling.md)**; 2.1 peers fall +> back to 2.0 so both versions interoperate. + +--- + +*Implement against Β§5–§9; everything else is rationale. Clients MUST interoperate +at the signaling JSON, the sealed-payload format, the identity-confirm `hello`, and +the application message types.* \ No newline at end of file diff --git a/cmd/anchor/main.go b/cmd/anchor/main.go index 28c6e5e..679fdf4 100644 --- a/cmd/anchor/main.go +++ b/cmd/anchor/main.go @@ -170,9 +170,8 @@ func (a *anchor) handleWS(w http.ResponseWriter, r *http.Request) { log.Printf("anchor: join: bad sig from %s", r.RemoteAddr) continue } - // Sig covers nonce || net (both as raw bytes decoded from hex/plaintext). - netBytes, _ := hex.DecodeString(msg.Net) - signed := append(nonce, netBytes...) + // Β§5.1: sig covers nonce_raw || net_ascii (net as 64-char hex UTF-8 string) + signed := append(nonce, []byte(msg.Net)...) if !ed25519.Verify(ed25519.PublicKey(pubBytes), signed, sigBytes) { log.Printf("anchor: join: sig verification failed for %s", msg.ID[:min(8, len(msg.ID))]) continue diff --git a/cmd/tui/main.go b/cmd/tui/main.go index c978364..858ec2d 100644 --- a/cmd/tui/main.go +++ b/cmd/tui/main.go @@ -290,8 +290,8 @@ func (m model) applyEvent(evt proto.IpcMessage) model { msg := evt.Message e := entry{ from: m.aliasOf(msg.From), - body: msg.Body, - at: msg.SentAt, + body: msg.Text, + at: time.UnixMilli(msg.Ts), fromMe: msg.From == m.localID, } m.messages[msg.Room] = append(m.messages[msg.Room], e) diff --git a/internal/anchor/client.go b/internal/anchor/client.go index 6eeb856..f62b9cc 100644 --- a/internal/anchor/client.go +++ b/internal/anchor/client.go @@ -81,8 +81,8 @@ func runOnce(ctx context.Context, anchorURL, netHash string, id *crypto.Identity if err != nil { return fmt.Errorf("bad challenge nonce: %w", err) } - netBytes, _ := hex.DecodeString(netHash) - sig := id.Sign(append(nonceBytes, netBytes...)) + // Β§5.1: sig covers nonce_raw || net_ascii (64-char hex string as UTF-8) + sig := id.Sign(append(nonceBytes, []byte(netHash)...)) sendCh <- proto.AnchorMessage{ Type: proto.AnchorJoin, ID: string(id.PeerID()), diff --git a/internal/ipc/ipc.go b/internal/ipc/ipc.go index da8be19..2a3c7f4 100644 --- a/internal/ipc/ipc.go +++ b/internal/ipc/ipc.go @@ -16,7 +16,6 @@ import ( "net" "time" - "github.com/google/uuid" "github.com/waste-go/internal/invite" "github.com/waste-go/internal/netmgr" "github.com/waste-go/internal/proto" @@ -49,9 +48,11 @@ func handleClient(conn net.Conn, mgr *netmgr.Manager) { writeCh := make(chan []byte, 128) done := make(chan struct{}) + writerDone := make(chan struct{}) // Writer goroutine. go func() { + defer close(writerDone) w := bufio.NewWriter(conn) for line := range writeCh { line = append(line, '\n') @@ -140,30 +141,63 @@ func handleClient(conn net.Conn, mgr *netmgr.Manager) { send(errMsg("send_message: not joined to any network")) continue } - msg := &proto.ChatMessage{ - Mid: randomHex(16), - ID: uuid.NewString(), - From: n.Identity.PeerID(), - To: cmd.To, - Room: cmd.Room, - Body: cmd.Body, - SentAt: time.Now(), - } - payload, err := json.Marshal(proto.PeerMessage{Type: proto.MsgChat, Chat: msg}) - if err != nil { - continue - } + ts := time.Now().UnixMilli() if cmd.To != nil { - n.Mesh.SendTo(*cmd.To, payload) + // DM β†’ spec "pm" type: flat {type, mid, text, ts} on the wire + mid := randomHex(16) + wire, err := json.Marshal(proto.PeerMessage{ + Type: proto.MsgPm, + Mid: mid, + Text: cmd.Body, + Ts: ts, + }) + if err != nil { + continue + } + n.Mesh.SendTo(*cmd.To, wire) + // Store locally with dm: room convention + local := &proto.ChatMessage{ + Mid: mid, + From: n.Identity.PeerID(), + To: cmd.To, + Room: "dm:" + (*cmd.To).Short(), + Text: cmd.Body, + Ts: ts, + } + n.Mesh.SaveMessage(local) + n.Mesh.Emit(proto.IpcMessage{ + Type: proto.EvtMessageReceived, + NetworkID: n.ID, + Message: local, + }) } else { - n.Mesh.Broadcast(payload) + // Group chat β†’ spec "chat" type: flat {type, mid, room, text, ts} + mid := randomHex(16) + wire, err := json.Marshal(proto.PeerMessage{ + Type: proto.MsgChat, + Mid: mid, + Room: cmd.Room, + Text: cmd.Body, + Ts: ts, + }) + if err != nil { + continue + } + n.Mesh.Broadcast(wire) + local := &proto.ChatMessage{ + Mid: mid, + From: n.Identity.PeerID(), + Room: cmd.Room, + Text: cmd.Body, + Ts: ts, + } + n.Mesh.SaveMessage(local) + n.Mesh.Emit(proto.IpcMessage{ + Type: proto.EvtMessageReceived, + NetworkID: n.ID, + Message: local, + }) } - n.Mesh.SaveMessage(msg) - n.Mesh.Emit(proto.IpcMessage{ - Type: proto.EvtMessageReceived, - NetworkID: n.ID, - Message: msg, - }) case proto.CmdGetState: send(stateSnapshot(mgr)) @@ -222,6 +256,7 @@ func handleClient(conn net.Conn, mgr *netmgr.Manager) { close(done) close(writeCh) + <-writerDone // wait for writer to flush before conn.Close() fires log.Printf("ipc: UI client disconnected") } diff --git a/internal/mesh/peer.go b/internal/mesh/peer.go index 8d3a2d7..b8a77ea 100644 --- a/internal/mesh/peer.go +++ b/internal/mesh/peer.go @@ -2,10 +2,13 @@ package mesh import ( + "crypto/rand" "encoding/hex" "encoding/json" "log" "strings" + "sync" + "time" "github.com/pion/webrtc/v3" @@ -21,7 +24,7 @@ type Anchor interface { } // WireDataChannel sets up open/message/close handlers on a "yaw" DataChannel. -// Must be called before the DataChannel opens. +// Safe to call whether the channel is already open or not (Β§6 open-race). func WireDataChannel( dc *webrtc.DataChannel, pc *webrtc.PeerConnection, @@ -31,7 +34,8 @@ func WireDataChannel( ) { sendCh := make(chan []byte, 64) - dc.OnOpen(func() { + var once sync.Once + doOpen := func() { log.Printf("peer: DataChannel open with %s", peerID.Short()) // Send hello β€” bind our identity to this DTLS session. @@ -69,7 +73,13 @@ func WireDataChannel( } } }() - }) + } + + dc.OnOpen(func() { once.Do(doOpen) }) + // Β§6 gotcha: answerer's DC may already be open when OnDataChannel fires. + if dc.ReadyState() == webrtc.DataChannelStateOpen { + once.Do(doOpen) + } dc.OnMessage(func(msg webrtc.DataChannelMessage) { if msg.IsString { @@ -152,8 +162,30 @@ func handleDCMessage(data []byte, from proto.PeerID, localID *crypto.Identity, m func dispatchPeerMessage(msg proto.PeerMessage, from proto.PeerID, m *Mesh) { switch msg.Type { + case proto.MsgChat: + chat := &proto.ChatMessage{ + Mid: midOrRandom(msg.Mid), + From: from, + Room: msg.Room, + Text: msg.Text, + Ts: msg.Ts, + } + m.SaveMessage(chat) + m.Emit(proto.IpcMessage{Type: proto.EvtMessageReceived, Message: chat}) + + case proto.MsgPm: + // Private message β€” reconstruct as ChatMessage for IPC/storage using dm: room. + chat := &proto.ChatMessage{ + Mid: midOrRandom(msg.Mid), + From: from, + Room: "dm:" + from.Short(), + Text: msg.Text, + Ts: msg.Ts, + } + m.SaveMessage(chat) + m.Emit(proto.IpcMessage{Type: proto.EvtMessageReceived, Message: chat}) + case proto.MsgFileListReq: - // Peer wants our file list β€” reply directly on their send channel. files := m.ScanShareDir() resp, err := json.Marshal(proto.PeerMessage{ Type: proto.MsgFileListResp, @@ -165,7 +197,6 @@ func dispatchPeerMessage(msg proto.PeerMessage, from proto.PeerID, m *Mesh) { m.SendTo(from, resp) case proto.MsgFileListResp: - // Received a remote peer's file list β€” forward to IPC subscribers. if msg.FileListResp != nil { m.Emit(proto.IpcMessage{ Type: proto.EvtFileList, @@ -174,11 +205,13 @@ func dispatchPeerMessage(msg proto.PeerMessage, from proto.PeerID, m *Mesh) { }) } - case proto.MsgChat: - if msg.Chat != nil { - m.SaveMessage(msg.Chat) - m.Emit(proto.IpcMessage{Type: proto.EvtMessageReceived, Message: msg.Chat}) - } + case proto.MsgFileOffer: + m.Emit(proto.IpcMessage{ + Type: proto.EvtIncomingFile, + PeerID: peerIDPtr(from), + Offer: &proto.FileOffer{Xid: msg.Xid, Name: msg.Name, Size: msg.Size, SHA256: msg.SHA256}, + }) + case proto.MsgPeerGossip: if msg.Gossip != nil { log.Printf("mesh: gossip from %s: %d peer hints", from.Short(), len(msg.Gossip.Peers)) @@ -187,19 +220,24 @@ func dispatchPeerMessage(msg proto.PeerMessage, from proto.PeerID, m *Mesh) { log.Printf("mesh: ping from %s", from.Short()) case proto.MsgPong: log.Printf("mesh: pong from %s", from.Short()) - case proto.MsgFileOffer: - if msg.FileOffer != nil { - m.Emit(proto.IpcMessage{ - Type: proto.EvtIncomingFile, - PeerID: peerIDPtr(from), - Offer: msg.FileOffer, - }) - } default: log.Printf("mesh: unknown message type %q from %s", msg.Type, from.Short()) } } +// midOrRandom returns mid if non-empty, otherwise generates a random 16-byte hex string. +// Ensures every stored message has a unique mid even from peers that don't send one. +func midOrRandom(mid string) string { + if mid != "" { + return mid + } + b := make([]byte, 16) + if _, err := rand.Read(b); err != nil { + return hex.EncodeToString([]byte(time.Now().String())) + } + return hex.EncodeToString(b) +} + func dtlsFingerprints(pc *webrtc.PeerConnection) (local, remote []byte) { if ld := pc.LocalDescription(); ld != nil { local = fingerprintFromSDP(ld.SDP) diff --git a/internal/proto/proto.go b/internal/proto/proto.go index 4bbfad6..3a670d2 100644 --- a/internal/proto/proto.go +++ b/internal/proto/proto.go @@ -35,40 +35,64 @@ type MsgType string const ( MsgChat MsgType = "chat" + MsgPm MsgType = "pm" // private message, Β§8 MsgPeerGossip MsgType = "peer_gossip" MsgFileListReq MsgType = "file_list_req" MsgFileListResp MsgType = "file_list_resp" - MsgFileOffer MsgType = "file_offer" - MsgFileResp MsgType = "file_response" - MsgFileDone MsgType = "file_done" + MsgFileOffer MsgType = "file-offer" // Β§9, hyphenated per spec + MsgFileAccept MsgType = "file-accept" + MsgFileCancel MsgType = "file-cancel" + MsgFileDone MsgType = "file-done" MsgPing MsgType = "ping" MsgPong MsgType = "pong" ) +// PmMessage is a private message sent directly over a single peer link (Β§8 "pm"). +// The sender/receiver are implicit from the DataChannel; no room or from fields on the wire. +type PmMessage struct { + Text string `json:"text"` + Ts int64 `json:"ts"` // Unix milliseconds +} + // PeerMessage is the top-level container sent over the "yaw" DataChannel. +// The spec types (hello, chat, pm, file-offer …) are flat JSON objects; we +// embed the fields directly using inline structs where needed, but for structured +// types we include the payload pointer. Unknown fields are ignored (forward compat). // File chunks go over a separate binary DataChannel labeled "f:". type PeerMessage struct { Type MsgType `json:"type"` - // Only one of these will be set, depending on Type. - Chat *ChatMessage `json:"chat,omitempty"` + // chat / pm fields (flat on the wire per spec Β§8) + Mid string `json:"mid,omitempty"` // optional dedup id; required when relay hops > 0 + Room string `json:"room,omitempty"` // chat only + Text string `json:"text,omitempty"` // chat and pm + Ts int64 `json:"ts,omitempty"` // chat and pm (Unix ms) + + // Non-spec extensions (unknown types are silently ignored by other impls) Gossip *PeerGossip `json:"gossip,omitempty"` FileListResp *FileListResp `json:"file_list_resp,omitempty"` - FileOffer *FileOffer `json:"file_offer,omitempty"` - FileResp *FileResponse `json:"file_response,omitempty"` - FileDone *FileDone `json:"file_done,omitempty"` - Seq *uint64 `json:"seq,omitempty"` // for ping/pong + + // file transfer (Β§9) β€” fields are flat on the wire + Xid string `json:"xid,omitempty"` + Name string `json:"name,omitempty"` + Size int64 `json:"size,omitempty"` + SHA256 string `json:"sha256,omitempty"` + + // file-done / file-cancel / file-accept just need xid (already above) + Reason string `json:"reason,omitempty"` // file-cancel + + Seq *uint64 `json:"seq,omitempty"` // ping/pong } -// ChatMessage is a message to a room or a DM. +// ChatMessage is a group chat message (wire type "chat", Β§8). +// Also used internally for persisting PMs after they are received. type ChatMessage struct { - Mid string `json:"mid"` // random 16-byte hex, for deduplication (YAW/2 Β§8) - ID string `json:"id"` // internal uuid, kept for local use - From PeerID `json:"from"` - To *PeerID `json:"to,omitempty"` // nil = broadcast to room - Room string `json:"room"` - Body string `json:"body"` - SentAt time.Time `json:"sent_at"` + Mid string `json:"mid,omitempty"` // optional dedup id (required when relay hops > 0) + From PeerID `json:"from,omitempty"` // set by receiver from DC context; not on wire for pm + To *PeerID `json:"to,omitempty"` // internal only β€” not transmitted; set for DMs + Room string `json:"room"` + Text string `json:"text"` + Ts int64 `json:"ts"` // Unix milliseconds } // PeerGossip shares known peer addresses. @@ -95,27 +119,13 @@ type FileListResp struct { Files []FileEntry `json:"files"` } -// FileOffer initiates a file transfer. +// FileOffer is used internally when emitting EvtIncomingFile to the IPC layer. +// On the wire, file-offer fields are flat inside PeerMessage (xid/name/size/sha256). type FileOffer struct { - Mid string `json:"mid"` // dedup id - Xid string `json:"xid"` // transfer id, used as DataChannel label "f:" - Filename string `json:"filename"` - SizeBytes int64 `json:"size_bytes"` - SHA256 string `json:"sha256"` // hex -} - -// FileResponse accepts or declines a FileOffer. -type FileResponse struct { - Mid string `json:"mid"` - Xid string `json:"xid"` - Accepted bool `json:"accepted"` -} - -// FileDone signals that all chunks have been sent. Receiver verifies SHA256. -type FileDone struct { - Mid string `json:"mid"` Xid string `json:"xid"` - SHA256 string `json:"sha256"` // hex + Name string `json:"name"` + Size int64 `json:"size"` + SHA256 string `json:"sha256"` } // ── DataChannel hello (YAW/2 Β§6) ───────────────────────────────────────────── diff --git a/internal/store/store.go b/internal/store/store.go index 5511485..bdf61e6 100644 --- a/internal/store/store.go +++ b/internal/store/store.go @@ -57,10 +57,11 @@ func (s *Store) Close() error { // SaveMessage persists a chat message. Duplicate mids are silently ignored // (INSERT OR IGNORE), so calling this more than once is safe. func (s *Store) SaveMessage(msg *proto.ChatMessage) error { + sentAt := time.UnixMilli(msg.Ts).UTC() _, err := s.db.Exec( `INSERT OR IGNORE INTO messages (mid, room, from_peer, body, sent_at) VALUES (?, ?, ?, ?, ?)`, - msg.Mid, msg.Room, string(msg.From), msg.Body, msg.SentAt.UTC(), + msg.Mid, msg.Room, string(msg.From), msg.Text, sentAt, ) return err } @@ -96,12 +97,12 @@ func (s *Store) RecentMessages(room string, limit int) ([]proto.ChatMessage, err var m proto.ChatMessage var from string var sentAt time.Time - if err := rows.Scan(&m.Mid, &from, &m.Body, &sentAt); err != nil { + if err := rows.Scan(&m.Mid, &from, &m.Text, &sentAt); err != nil { return nil, err } m.From = proto.PeerID(from) m.Room = room - m.SentAt = sentAt + m.Ts = sentAt.UnixMilli() msgs = append(msgs, m) } // Reverse so oldest-first. diff --git a/internal/store/store_test.go b/internal/store/store_test.go index f6e85f5..0f2fa49 100644 --- a/internal/store/store_test.go +++ b/internal/store/store_test.go @@ -16,11 +16,11 @@ func TestRoundTrip(t *testing.T) { defer st.Close() msg := &proto.ChatMessage{ - Mid: "aabbccdd00112233", - From: proto.PeerID("deadbeef"), - Room: "general", - Body: "hello world", - SentAt: time.Now().UTC().Truncate(time.Second), + Mid: "aabbccdd00112233", + From: proto.PeerID("deadbeef"), + Room: "general", + Text: "hello world", + Ts: time.Now().UTC().Truncate(time.Second).UnixMilli(), } if err := st.SaveMessage(msg); err != nil { @@ -40,7 +40,7 @@ func TestRoundTrip(t *testing.T) { t.Fatalf("got %d messages, want 1", len(msgs)) } got := msgs[0] - if got.Mid != msg.Mid || got.Body != msg.Body || got.Room != msg.Room { + if got.Mid != msg.Mid || got.Text != msg.Text || got.Room != msg.Room { t.Fatalf("message mismatch: %+v", got) } } @@ -79,11 +79,11 @@ func TestRecentMessagesOrdering(t *testing.T) { base := time.Now().UTC().Truncate(time.Second) for i := range 5 { st.SaveMessage(&proto.ChatMessage{ - Mid: string(rune('a'+i)) + "000000000000000", - From: "deadbeef", - Room: "general", - Body: string(rune('a' + i)), - SentAt: base.Add(time.Duration(i) * time.Second), + Mid: string(rune('a'+i)) + "000000000000000", + From: "deadbeef", + Room: "general", + Text: string(rune('a' + i)), + Ts: base.Add(time.Duration(i) * time.Second).UnixMilli(), }) } @@ -96,7 +96,7 @@ func TestRecentMessagesOrdering(t *testing.T) { } // Must be oldest-first. for i := 1; i < len(msgs); i++ { - if msgs[i].SentAt.Before(msgs[i-1].SentAt) { + if msgs[i].Ts < msgs[i-1].Ts { t.Fatalf("messages not in ascending order at index %d", i) } } diff --git a/test-network.sh b/test-network.sh index eef1e83..5260577 100755 --- a/test-network.sh +++ b/test-network.sh @@ -25,6 +25,21 @@ DATA_ROOT="/tmp/waste-test" rm -rf "$DATA_ROOT" mkdir -p "$DATA_ROOT/bin" +# Kill any leftover processes from a previous run holding our fixed ports. +for port in "$ANCHOR_PORT" "$ALICE_IPC" "$BOB_IPC" "$CHARLIE_IPC"; do + pid=$(lsof -ti tcp:"$port" 2>/dev/null || true) + [ -n "$pid" ] && kill -9 $pid 2>/dev/null || true +done +# Wait until all ports are actually free before proceeding. +for port in "$ANCHOR_PORT" "$ALICE_IPC" "$BOB_IPC" "$CHARLIE_IPC"; do + n=0 + while lsof -ti tcp:"$port" >/dev/null 2>&1; do + sleep 0.1 + n=$(( n + 1 )) + [ "$n" -gt 30 ] && echo -e "${RED}port ${port} still in use after 3s${RESET}" >&2 && break + done +done + # ── cleanup ─────────────────────────────────────────────────────────────────── PIDS=() cleanup() { @@ -35,7 +50,6 @@ cleanup() { done wait 2>/dev/null || true echo -e "${DIM}data left at: ${DATA_ROOT}${RESET}" - echo -e "${DIM} inspect: sqlite3 /tmp/waste-test/alice/messages.db${RESET}" echo -e "${DIM}done.${RESET}" } trap cleanup EXIT INT TERM @@ -80,7 +94,7 @@ pretty() { message_received) local from body room to_field from=$(echo "$line" | jq -r '.message.from[:8] // "?"' 2>/dev/null) - body=$(echo "$line" | jq -r '.message.body // ""' 2>/dev/null) + body=$(echo "$line" | jq -r '.message.text // ""' 2>/dev/null) room=$(echo "$line" | jq -r '.message.room // ""' 2>/dev/null) to_field=$(echo "$line" | jq -r '.message.to // ""' 2>/dev/null) if [ -n "$to_field" ]; then @@ -221,24 +235,6 @@ wait_port "$BOB_IPC" "bob" wait_port "$CHARLIE_IPC" "charlie" echo "" -# ── subscribe ───────────────────────────────────────────────────────────────── -echo -e "${DIM}subscribing to IPC event streams…${RESET}" -subscribe "$ALICE_COLOR" "alice " "$ALICE_IPC" -subscribe "$BOB_COLOR" "bob " "$BOB_IPC" -subscribe "$CHARLIE_COLOR" "charlie" "$CHARLIE_IPC" -sleep 0.3 # let state_snapshot lines arrive - -# ── resolve peer IDs ────────────────────────────────────────────────────────── -# Each daemon knows its own id from the state_snapshot. -ALICE_ID=$(peer_field "$ALICE_IPC" '.local_peer.id') -BOB_ID=$(peer_field "$BOB_IPC" '.local_peer.id') -CHARLIE_ID=$(peer_field "$CHARLIE_IPC" '.local_peer.id') - -log "$ANCHOR_COLOR" "ids" "alice = ${ALICE_ID:0:16}…" -log "$ANCHOR_COLOR" "ids" "bob = ${BOB_ID:0:16}…" -log "$ANCHOR_COLOR" "ids" "charlie = ${CHARLIE_ID:0:16}…" -echo "" - # ── join network ────────────────────────────────────────────────────────────── echo -e "${DIM}────────────────────────────────────────────────────────${RESET}" echo -e "joining all peers to network: ${BOLD}${NETWORK_NAME}${RESET}" @@ -250,6 +246,24 @@ sleep 0.2 ipc "$BOB_IPC" "$JOIN" sleep 0.2 ipc "$CHARLIE_IPC" "$JOIN" +sleep 0.3 + +# ── resolve peer IDs ────────────────────────────────────────────────────────── +# Peer IDs are network-scoped (derived identity), so we query AFTER joining. +ALICE_ID=$(peer_field "$ALICE_IPC" '.local_peer.id') +BOB_ID=$(peer_field "$BOB_IPC" '.local_peer.id') +CHARLIE_ID=$(peer_field "$CHARLIE_IPC" '.local_peer.id') + +log "$ANCHOR_COLOR" "ids" "alice = ${ALICE_ID:0:16}…" +log "$ANCHOR_COLOR" "ids" "bob = ${BOB_ID:0:16}…" +log "$ANCHOR_COLOR" "ids" "charlie = ${CHARLIE_ID:0:16}…" +echo "" + +# ── subscribe ───────────────────────────────────────────────────────────────── +echo -e "${DIM}subscribing to IPC event streams…${RESET}" +subscribe "$ALICE_COLOR" "alice " "$ALICE_IPC" +subscribe "$BOB_COLOR" "bob " "$BOB_IPC" +subscribe "$CHARLIE_COLOR" "charlie" "$CHARLIE_IPC" echo -e "${DIM}waiting for ICE / DataChannel setup (up to 10s)…${RESET}" sleep 6 @@ -318,19 +332,25 @@ echo -e "${DIM}───────────────────── echo -e "file listing" echo -e "${DIM}────────────────────────────────────────────────────────${RESET}" +alice_nets=$(echo '{"type":"get_state"}' | timeout 2 nc 127.0.0.1 "$ALICE_IPC" 2>/dev/null \ + | grep '"type":"state_snapshot"' | head -1 \ + | jq -r '[.networks[].network_name] | join(", ")' 2>/dev/null || echo "?") +echo -e "${DIM} alice networks: [${alice_nets:-none}]${RESET}" + for peer_ipc in "$ALICE_IPC" "$BOB_IPC" "$CHARLIE_IPC"; do if [ "$peer_ipc" = "$ALICE_IPC" ]; then peer_name="alice" elif [ "$peer_ipc" = "$BOB_IPC" ]; then peer_name="bob" else peer_name="charlie"; fi - result=$(echo '{"type":"get_file_list"}' \ - | timeout 2 nc 127.0.0.1 "$peer_ipc" 2>/dev/null \ - | grep '"type":"file_list"' | head -1) + raw=$(echo '{"type":"get_file_list"}' \ + | nc -q 2 127.0.0.1 "$peer_ipc" 2>/dev/null || true) + result=$(echo "$raw" | grep '"type":"file_list"' | head -1 || true) if [ -n "$result" ]; then count=$(echo "$result" | jq '.files | length' 2>/dev/null || echo "?") - files=$(echo "$result" | jq -r '.files[].name' 2>/dev/null | tr '\n' ' ') + files=$(echo "$result" | jq -r '[.files[].name] | join(" ")' 2>/dev/null || true) echo -e " ${BOLD}${peer_name}${RESET}: ${count} file(s) β€” ${files}" else - echo -e " ${RED}${peer_name}: no file_list response${RESET}" + types=$(echo "$raw" | jq -r '.type' 2>/dev/null | tr '\n' ',' | sed 's/,$//' || true) + echo -e " ${RED}${peer_name}: no file_list response${RESET} ${DIM}(got: ${types:-nothing})${RESET}" fi done sleep 0.5 @@ -349,15 +369,15 @@ echo -e "${DIM}───────────────────── echo -e "verifying persistence (SQLite)" echo -e "${DIM}────────────────────────────────────────────────────────${RESET}" for peer in alice bob charlie; do - db="$DATA_ROOT/$peer/messages.db" - if [ -f "$db" ]; then + db=$(ls "$DATA_ROOT/$peer/messages-"*.db 2>/dev/null | head -1 || true) + if [ -n "$db" ] && [ -f "$db" ]; then total=$(sqlite3 "$db" "SELECT COUNT(*) FROM messages;" 2>/dev/null || echo "?") group=$(sqlite3 "$db" "SELECT COUNT(*) FROM messages WHERE room='general';" 2>/dev/null || echo "?") dms=$(sqlite3 "$db" "SELECT COUNT(*) FROM messages WHERE room LIKE 'dm:%';" 2>/dev/null || echo "?") peers_count=$(sqlite3 "$db" "SELECT COUNT(*) FROM peers;" 2>/dev/null || echo "?") echo -e " ${BOLD}${peer}${RESET}: ${total} total (${group} group, ${dms} DM), ${peers_count} known peers" else - echo -e " ${RED}${peer}: no messages.db found${RESET}" + echo -e " ${RED}${peer}: no messages-*.db found${RESET}" fi done diff --git a/test-tui.sh b/test-tui.sh index 8ada116..764ba60 100755 --- a/test-tui.sh +++ b/test-tui.sh @@ -19,6 +19,19 @@ CYAN='\033[0;36m'; DIM='\033[2m'; BOLD='\033[1m'; RESET='\033[0m' rm -rf "$DATA_ROOT" mkdir -p "$DATA_ROOT/bin" +# Kill any leftover processes holding our fixed ports. +for port in "$ANCHOR_PORT" "$ALICE_IPC" "$BOB_IPC" "$CHARLIE_IPC"; do + pid=$(lsof -ti tcp:"$port" 2>/dev/null || true) + [ -n "$pid" ] && kill -9 $pid 2>/dev/null || true +done +for port in "$ANCHOR_PORT" "$ALICE_IPC" "$BOB_IPC" "$CHARLIE_IPC"; do + n=0 + while lsof -ti tcp:"$port" >/dev/null 2>&1; do + sleep 0.1; n=$(( n + 1 )) + [ "$n" -gt 30 ] && break + done +done + PIDS=() cleanup() { for pid in "${PIDS[@]:-}"; do