design for engine-internal audio routing rewrite and multi-engine/multi-UI coordination

proposals/architecture-overview.md

@@ -0,0 +1,148 @@
+# OALS Architecture Proposal — Overview
+
+**Status:** Proposal, not implemented, not yet reviewed  
+**Author:** Jonathan Reichardt  
+**Version:** Draft 0.1  
+**Last updated:** 2026-05-31  
+**Audience:** OALS contributors (current and future)  
+**Reading order:** this overview, then the three documents in the order given in [§2](#2-reading-order-and-why).  
+**Prerequisites:** general familiarity with the OALS codebase and with OAN (`OpenAudioNetwork/`) at the level of `common/packet_structs.h` and `netutils/LowLatSocket.h`.
+
+## Abstract
+
+This document is the entry point for a three-part architecture proposal that I
+am putting forward for OALS. The three parts together describe a proposed
+rework of how a single engine processes audio, how possibly multiple devices coordinate
+into a cluster, and the wire-level protocol that makes cluster-wide atomic
+reconfiguration possible. None of the three parts has been implemented. The purpose of this overview is
+to frame the three together — what each covers, the order they are best read
+in, the terms they share.
+
+## 1. The three documents
+
+### [engine-audio-pipeline.md](engine-audio-pipeline.md) — Engine Audio Routing & Processing
+
+How one engine turns inputs into outputs. The proposal replaces today's
+linked-list `AudioPipe` chain with a directed acyclic graph of processing
+nodes, separates the engine into a network-aware I/O matrix layer and a pure
+DSP graph layer with a clean lane boundary between them, and introduces
+plugin/processing delay compensation (PDC) at the boundaries where it
+matters. The proposal is contained to the inside of one engine; nothing in
+this part touches the wire protocol, peer discovery, or cluster coordination.
+
+### [cluster-coordination.md](cluster-coordination.md) — Cluster Coordination & Transport
+
+How one or multiple engines, UIs, and IO boards hold together as a single coordinated state (or show so to say). The
+proposal introduces the concept of a *cluster blueprint* — a single canonical desired-state
+description that lives at an elected *coordinator* and is projected into
+per-engine slices. It defines how structural edits and parameter edits are
+routed differently (atomic transactions for the former, direct unicast for
+the latter), how metering flows, how nodes are placed across engines, and
+what happens when the coordinator dies. This part assumes the per-engine
+model from the audio-graph proposal.
+
+### [state-coordination-protocol.md](state-coordination-protocol.md) — OAN State Coordination Protocol
+
+The wire-level protocol that gives the cluster proposal its atomic
+reconfiguration guarantee. OSCP is a single-leader, broadcast-with-
+correlation protocol that runs alongside the existing OAN audio, control,
+mapping, and clock-sync streams on a single L2 segment. It provides
+authority election, atomic transactional commits with sample-accurate apply
+timestamps, late-joiner sync, and partition handling. This part is the
+foundation that the cluster proposal stands on, but it is best read *after*
+the cluster proposal — the cluster proposal motivates the protocol, the
+protocol then explains the mechanism in detail.
+
+## 2. Reading order and why
+
+The three documents have a layered dependency that runs from the bottom up,
+but they read better in roughly that order *as a story* rather than top-down:
+
+1. **[engine-audio-pipeline.md](engine-audio-pipeline.md) first.** It is the most self-contained of the
+   three. The DAG, the lanes, the I/O matrix split, the PDC — all stand on
+   their own without knowing anything about multi-engine clusters or about
+   how blueprints are distributed.
+2. **[cluster-coordination.md](cluster-coordination.md) second.** Once the per-engine model is
+   in mind, the cluster questions make sense: how do nodes get placed across
+   engines, what does cross-engine wiring look like, where does cluster-wide
+   show state live. The cluster proposal references the audio-graph proposal
+   in several places (e.g. the per-engine RCU swap, the lane boundary).
+3. **[state-coordination-protocol.md](state-coordination-protocol.md) last.** The cluster proposal names the property it
+   needs — "atomic cluster-wide reconfiguration at a sample-accurate
+   instant" — and OSCP is the
+   protocol that delivers it.
+
+
+## 3. Shared glossary
+
+These are the terms that recur across all three documents. Each document also
+has its own glossary for terms it introduces specifically; the entries below
+are defined once here so the three docs do not have to redefine them.
+
+- **OAN** — OpenAudioNetwork. The raw-Ethernet audio transport submodule
+  (`OpenAudioNetwork/`) on which the entire OALS system runs. All
+  inter-process traffic in OALS — audio, control, mapping, clock sync,
+  cluster coordination — is carried over OAN EtherTypes; there is no IP-layer
+  transport anywhere in the system. New use cases get new EtherTypes.
+- **EtherType** — the 16-bit value at offset 12 in an Ethernet frame that
+  identifies the upper-layer protocol. OAN currently uses 0x0681 (audio),
+  0x0682 (discovery), 0x0683 (control), 0x0684 (clock sync); proposals
+  introduce 0x0685 (state coordination, OSCP) and 0x0686 (metering).
+- **Peer / UID** — any device participating in an OAN segment. Each peer has
+  a 16-bit UID, unique on the segment, used as the address in OAN's
+  `LowLatHeader`. UID 0 is broadcast.
+- **Engine** — a process that runs one audio graph plus its I/O matrix. The
+  authoritative owner of its own graph state and DSP. The current
+  implementation lives under `engine/`.
+- **UI / control surface** — an operator-facing process (today: the Qt-based
+  `coreui`). Sends parameter edits direct to engines, sends structural edits
+  via the coordinator, subscribes to metering data from engines.
+- **IO board / stage box** — a peer that contributes audio inputs and
+  outputs to the system but does no DSP.
+- **Block** — one chunk of audio processed in one go. 64 mono float samples
+  (`AUDIO_DATA_SAMPLES_PER_PACKETS`). At 96 kHz that is ~667 µs of audio per
+  block; the whole engine ticks at block rate.
+- **Cluster** — the set of engines, UIs and IO boards on one OAN segment,
+  working together on one show.
+- **Coordinator** — the role that holds the cluster blueprint and projects
+  it into per-engine snapshots. Elected via OSCP. Never on the audio data
+  path. Defined in detail in the cluster-coordination proposal.
+- **Blueprint** — the canonical cluster-wide desired-state description that
+  the coordinator holds and distributes. Carried as the opaque payload
+  inside OSCP transactions.
+- **`apply_at_us`** — a sample-accurate future clock instant at which all
+  peers simultaneously apply a committed change. The mechanism that makes
+  cluster reconfiguration atomic.
+
+## 4. What is NOT in these proposals
+
+For clarity about scope:
+
+- **Show-file persistence (saved presets, scene memory, recall).** These
+  are application-layer concerns above OSCP. A UI may load a show file and
+  propose it as a blueprint, but the format and storage of show files are
+  not part of any of these three proposals.
+- **Plugin authoring conventions** beyond the changes to the
+  engine-side `Node` interface implied by the audio-graph proposal. The
+  current plugin ABI under `plugins/loader/` will need migration; the
+  migration is mentioned where relevant but not designed in detail.
+- **Remote operation across routed networks, web UIs, or any non-OAN
+  transport.** The proposals retain OALS's hard rule that all inter-process
+  transport is raw Ethernet via OAN. A future gateway peer could bridge
+  OAN to an IP-layer protocol, but no such gateway is part of these
+  proposals.
+- **Auto-migration of nodes after engine death.** State recovery is out of
+  scope; operators recreate orphaned nodes manually.
+
+## 5. Status and how to read them
+
+All three documents are first-pass proposals. None has been reviewed; none
+has been implemented. The decisions described in them are decisions I am
+*recommending*, not decisions that are set in stone. Where significant
+alternatives were considered, the alternatives and the reason for the
+recommended choice are called out explicitly; smaller choices are explained
+inline as proposal rationale.
+
+Each document carries its own "Open Questions" section listing items that
+are genuinely unresolved. Comments, counter-proposals, and pushback on any
+of it are the point of putting them out for review.