SwanWeave's 'spatial' module is doing almost no spatial work

The SE-MoE module in SwanWeave contributes 91% of the model’s FAD advantage over SmartDJ but only 5% of its CRW advantage. A module named “Spatial Edit Mixture-of-Experts” is doing almost no spatial work in the ablation. The spatial quality is carried by pretraining and the task expert.

What SwanWeave is

SwanWeave is a one-stage diffusion-transformer that edits First-Order Ambisonics (FOA) spatial audio from a source recording plus a text instruction. One forward pass, edited target out. The use case is production: an engineer moves, repositions, or modifies sound events in a 4-channel FOA mix without re-rendering through a multi-stage pipeline. Its baselines (ZETA, AudioEditor, SDEdit, SmartDJ) are all prior instruction-guided audio editors, most working in stereo or general audio rather than native FOA.

Figures and their conditions

Metrics in the main results table are reported “on average over edit types,” spanning “over ten single-operation and compound tasks” (the paper’s phrasing). The ablation table uses the same averaging condition. Only the per-task rows — angle motion, distance motion — give task-specific numbers. Speed: SwanWeave’s 1.17s is an average over edit types. SmartDJ’s multi-stage 10.73s is explicitly on “compound-edit samples” only.

Reported by 2609.04975 s· scroll →

Conditions around the comparison

scroll →

1.17s

inference speed

2.72

LSD of SDEdit, average

4.23s

inference speed of SmartDJ, average

6.70

FD of SmartDJ, angle motion

Ablation decomposition

The paper reports four ablations from the full model. For each metric $j$, the marginal effect of component $m$ is $\Delta_m^{(j)} = M_j^{\text{w/o } m} - M_j^{\text{full}}$, and the fraction of the total advantage over SmartDJ attributable to $m$ is:

$$f_m^{(j)} = \frac{M_j^{\text{w/o } m} - M_j^{\text{full}}}{M_j^{\text{SmartDJ}} - M_j^{\text{full}}}$$

Substituting the supplied figures for the MoE ablation:

Metric	Full	w/o MoE	SmartDJ	$\Delta_{\text{MoE}}$	$f_{\text{MoE}}$
FD ↓	5.47	6.84	8.47	1.37	0.457
FAD ↓	1.16	2.72	2.87	1.56	0.912
KL ↓	1.67	2.34	2.69	0.67	0.657
LSD ↓	1.07	1.66	2.56	0.59	0.421
GCC ↓	12.39	14.05	20.32	1.66	0.209
CRW ↓	18.80	19.51	33.07	0.71	0.050
FSAD ↓	0.31	0.29	0.32	−0.02	−0.200

The FAD row makes the naming misleading. FAD (Fréchet Audio Distance) measures distributional fidelity of the generated audio’s spectral content. The MoE accounts for 91% of the SwanWeave–SmartDJ gap on FAD. CRW tells the opposite story. The paper uses it as a spatial-channel consistency measure; the MoE’s contribution there is 5%. On FSAD the MoE is actually slightly negative — removing it improves the score from 0.31 to 0.29, a 0.02 gain.

Pretraining is the single largest quality lever. FD jumps from 5.47 to 8.49 without it, a larger swing than MoE, task expert, or SPO produce individually. The base model saw 300K steps on 975K samples before the editing head was added, which explains the magnitude.

What the speed numbers do and do not support

The 1.17s figure averages over all edit types. The only compound-edit-specific speed in the table is SmartDJ multi-stage at 10.73s. That gives a ratio of 10.73/4.23 = 2.54 — multi-stage SmartDJ costs roughly 2.5× a single pass on the same compound task. SwanWeave’s compound-edit speed is not reported separately. The 1.17s is an upper bound on it only if all edit types take the same time, which is not stated.

Implications

The ablation already showed it: spatial quality comes from the training distribution and the task expert, not the routing architecture. The MoE is a spectral-fidelity module. Swap it or simplify it and spatial accuracy holds. Pretraining is not optional — skipping it costs roughly 35% of FD quality (8.49 vs. 5.47) and 64% of FAD quality (3.15 vs. 1.16).

The BEM HRTF work we published earlier found that spatial rendering quality is gated by the HRTF source more than by the rendering pipeline. SwanWeave’s ablation is the generator-side analogue: the training distribution gates spatial output more than the architecture does.

For AuraLinter, if we integrate a one-stage FOA editing model into the DSP generation loop, the quality ceiling is set by the pretraining corpus, not by which routing strategy we expose. The agent’s job is to write good instructions; the spatial fidelity is already baked in.

What cannot be concluded

The CLAP score is 0.21 for the full model, 0.21 for w/o MoE, and 0.21 for w/o SPO. Three identical values to two decimal places. Meanwhile w/o task expert drops to 0.18 and w/o pretrain to 0.16. Either CLAP is insensitive to these architectural changes — the metric may be saturating — or the reported precision masks small differences. The paper does not report confidence intervals or per-segment CLAP, so these explanations cannot be distinguished.

---

Method: this note was drafted by qwen/qwen3.8-27b from two sources — the arXiv abstract for 2609.04975 and this studio’s own published measurements, linked above. Before publication an automated gate re-checked every extracted claim against the source document (120 claim(s) and 120 quantity(ies) verified) and every claim-shaped number in the draft against that evidence (14 derived from it, 5 row(s) supplied from our own tables). The studio has not re-run 2609.04975’s benchmarks; figures attributed to it are its own.

FAQ

Can I use the w/o MoE variant for spatial editing tasks?

The w/o MoE model still beats SmartDJ on every spatial metric in the table (GCC 14.05 vs. 20.32, CRW 19.51 vs. 33.07). If your priority is spatial accuracy and you need to reduce inference cost or model complexity, the MoE is the component to cut. You will lose spectral fidelity (FAD goes from 1.16 to 2.72) but retain the spatial advantage.

Is 1.17s enough for real-time spatial audio editing?

The paper does not state the sample rate, clip length, or hardware for the 1.17s measurement. "On average over edit types" with no further qualification leaves open whether this is per-clip or per-second, and whether it includes VAE encode/decode. Treat it as a relative comparison point against the baselines (4.23s, 32.28s, 56.26s, 119.97s) rather than an absolute latency guarantee.

Does the "first one-stage multi-task" claim hold given SmartDJ has a one-stage mode?

SmartDJ's one-stage mode (4.23s, reported in the ablation) handles compound edits in a single pass. The paper's "first" claim is scoped to "instruction-guided 3D FOA spatial audio editing" specifically. SmartDJ operates in the general audio domain; SwanWeave's claim is that it is the first to do this natively in FOA with multi-task instruction following in one stage. The scope matters: it is not "first one-stage audio editor."

Which metric should I trust most for spatial quality?

GCC and CRW are the two channel-consistency metrics the paper reports. GCC (Group Channel Coherence) measures inter-channel correlation structure; CRW measures channel-wise energy ratio preservation. They disagree on where the MoE matters (GCC: 21% MoE contribution; CRW: 5%). If you are verifying spatial output against a reference, check both. A model can preserve channel energy ratios (CRW) while distorting the inter-channel phase structure (GCC), or vice versa. The multiphysics DSP lab has FOA channel decomposition tools if you need to inspect W/X/Y/Z relationships in generated output.