design for engine-internal audio routing rewrite and multi-engine/multi-UI coordination

[jonathan-reichardt]

Proposes replacing the current singly-linked AudioPipe chain with a realtime optimized directed acyclical audio graph — explicit nodes/ports/mono-lanes, topologically-sorted execution, atomic sample-accurate reconfiguration, and properprocessing delay compensation. Adds a cluster coordination layer on top so multiple engines and multiple UIs (FOH + MON, laptop + tablet, engine pools) can share one logical show with a declarative full-state blueprint, sample-accurate cross-engine commits. All inter-process transport stays on raw Ethernet — coordination rides OAN State Coordination Protocol (OSCP) on a new EtherType, and metering gets its own EtherType too.

[Moi78]

So I read it all through. First of all, nice work I must say, there are a lot of good ideas in this proposal.

Audio Engine design

I am not 100% in for the engine redesign, well as described. I do agree however that we need to free the hot processing path from mutexes and increase thread-safety (which is the most important optimization we can do right now) and move toward a tree/DAG shaped processing graph instead of a set linked lists, which effectively would allow easier sidechain and "weird" internal routing schemes and enhance flexibility. The only concern I have is the parralelis-ability of the processing chain. On later versions of the engine, I want to make use of every available CPU cores for processing (need to find a clever way to sync them without much overhead, the IO matrix could help) distributing pipes accross CPU cores (e.g. I have 64 pipes on a 4 cores CPU, so I'll use 4 pools of 16 pipes for processing).

I like the IO Matrix idea, it would replace the specific I/O related pipe elements and getting networking and latency concerns out of the way of the processing chain. We must limit the summing points to the absolute minimum needed though, to avoid unnecessary compensation computations.

Btw, I must correct something that was written in the document but in fact wrong. Even though the quantum of data is a network packet, if the packet is going to the same host it doesn't goes through the network stack, it goes immediately to the local audio buffers. That wouldn't be needed anymore if we're going for a tree/DAG.
I like the way of seeing thing as in the analog world, a set of linked elements that makes a processing chain, hence the whole Pipe & PipeElem system. I'd like to keep that scheme, and tweak it to allow a less linear shaped processing structure.

OSCP Protocol

I really like that idea. I agree with most of its content. I do not have much to say other that I'd like to keep metering in the control packets area. I do not think that using a whole new EtherType for metering is useful. The other thing is that metering is computed by a special PipeElem, which allow the user to place the measurement point wherever he wants and the plugin gives feedback the same way the Comp would. The downside of that is that you need to have special ids and addresses for these datas.

We could try to implement a part of it for the pipe creation step, as part of a proof of concept for the OSCP protocol.

Misc

YES! I want to allow OAN Audio packets to transport multiple streams at a time, mostly to decrease and optimise the number of PPS on the network and avoiding overloading risks. That raises a big problem on the IO board : synchronisation amongst the grouped streams and finding a way to limit overhead and added latency because for instance the samples of channels 1/2 are waiting for the channels 3/4 to finish conversion. Maybe playing a bit with Zephyr's k_pipe primitves could make it work.

What do you think?

[jonathan-reichardt]

@Moi78
Yeah regarding the parallelization of the audio processing chain, im definitely with you! But this design should be at least not closing the door to that. Sure it doesn't support parallel execution right now, but it should allow building on it in the future, without much changes to the single core version.

We must limit the summing points to the absolute minimum needed though, to avoid unnecessary compensation computations.
Well the latency computation in this model is only on declared latency of processing nodes, and only happens once there is a processing node added anywhere. Also at first I'd limit the audio processing nodes / plugins to classical 0 sample stuff like gain eq, gate, comp, etc. Only once we get into the terrain of linear phase EQ, limiter with lookahed or true peak and other stuff, that needs to look at multiple samples in the future, before being able to process the current one, then we have a fixed declared latency of N samples... and have to align parallel paths.
This has nothing to do with jitter, or the time the computation of the processing actually takes.

The other thing is that metering is computed by a special PipeElem, which allow the user to place the measurement point wherever he wants and the plugin gives feedback the same way the Comp would. The downside of that is that you need to have special ids and addresses for these datas.

From a capability perspective we are definitely on the same page here, we just have to figure out the implementation details on how to do it.

YES! I want to allow OAN Audio packets to transport multiple streams at a time, mostly to decrease and optimise the number of PPS on the network and avoiding overloading risks. That raises a big problem on the IO board : synchronisation amongst the grouped streams and finding a way to limit overhead and added latency because for instance the samples of channels 1/2 are waiting for the channels 3/4 to finish conversion. Maybe playing a bit with Zephyr's k_pipe primitves could make it work.

I can't follow you too deeply - let's discuss this in more detail. What I can say is that it seems to me that there is quite some work to be done in the whole clocking domain, before we can actually start with many other things. I have some ideas, and maybe right or wrong, assumptions on what is necessary here - let's discuss this as well!

I mean in general this is just to paint some kind of a traget picture, where we might land one day... the whole "when do we do what, in which order, and is X needed before Y" is a whole different discussion.