Low-Latency WebAudio DSP for Voice AI

When building real-time Voice AI applications—whether streaming speech from cloud endpoints like ElevenLabs or running local neural vocoders like kNN-VC (used in nymic)—managing audio graph scheduling in the browser is critical to prevent clicks, pops, and underruns.

In this field note, we break down the Web Audio API patterns for sample-accurate PCM chunk scheduling, procedural room impulse generation, and low-latency DSP processing.

---

1. Gap-Free Sample-Accurate PCM Scheduling

Cloud speech APIs typically return chunked PCM or MP3 buffers over WebSockets. Scheduling these chunks using simple setTimeout or naive callbacks causes audible gaps because JS timer jitter exceeds the ~2.9 ms duration of standard 128-sample Web Audio frames.

Instead, track an absolute playhead position in AudioContext.currentTime space:

const ctx = new AudioContext({ sampleRate: 44100 });
let playhead = 0;
const LEAD_BUFFER_SEC = 0.15; // Jitter buffer lead time

function schedulePcmChunk(float32Array: Float32Array) {
  const buffer = ctx.createBuffer(1, float32Array.length, 44100);
  buffer.copyToChannel(float32Array, 0);

  const source = ctx.createBufferSource();
  source.buffer = buffer;
  source.connect(ctx.destination);

  // If schedule fell behind, re-anchor with small lead
  if (playhead < ctx.currentTime + 0.02) {
    playhead = ctx.currentTime + LEAD_BUFFER_SEC;
  }

  source.start(playhead);
  playhead += buffer.duration; // Advance playhead gap-free
}

---

2. Procedural Schroeder Reverb Tails ($RT_{60}$)

Instead of loading heavy static .wav impulse responses for room acoustics, we generate procedural convolution buffers on-the-fly. The energy decay envelope for a target reverberation time $RT_{60}$ is given by:

$$E(t) = \text{Noise}(t) \cdot e^{\frac{-6.91 \cdot t}{RT_{60}}}$$

function generateSchroederIR(ctx, rt60Seconds) {
  const sampleRate = ctx.sampleRate;
  const length = Math.floor(sampleRate * rt60Seconds);
  const impulse = ctx.createBuffer(2, length, sampleRate);
  
  for (let channel = 0; channel < 2; channel++) {
    const channelData = impulse.getChannelData(channel);
    for (let i = 0; i < length; i++) {
      const t = i / sampleRate;
      const decay = Math.exp((-6.91 * t) / rt60Seconds);
      channelData[i] = (Math.random() * 2 - 1) * decay;
    }
  }
  return impulse;
}

---

3. Recommended Studio Audio Gear

For testing low-latency microphone DSP and binaural synthesis in our studio:

• Behringer UMC1820 Audio Interface — 8-preamp 18x20 USB I/O used for testing multi-channel Web Audio DSP graphs.
• Shure MV7+ Dynamic Podcast Microphone — Hybrid USB-C/XLR mic for testing real-time voice synthesis and formant filtering.
• Audio-Technica ATH-M50X Monitor Headphones — Reference headphones for evaluating Schroeder RT60 reverb tails.

Test your live microphone or synthesized voice against our interactive Voice Synthesis DSP Sandbox playground, featuring real-time spectrograms, parametric EQ, and Schroeder reverb tails.

---

4. Building the Future of Voice

Achieving true conversational latency in the browser requires moving past basic HTTP streams and embracing the low-level control that the Web Audio API provides. As voice AI models like ElevenLabs and localized neural vocoders improve, integrating seamless DSP directly into web applications opens up transformative, immersive experiences that feel instantaneous and alive.

FAQ

Why do WebAudio WebSocket audio streams stutter with setTimeout?

JavaScript event loop timers (setTimeout/setInterval) have a 4ms-15ms execution jitter. Web Audio processes audio in 128-sample frames (~2.9ms at 44.1kHz). If a timer fires 5ms late, the audio queue starves, causing audible clicks. Absolute time scheduling using AudioContext.currentTime is required.

How does procedural Schroeder reverb work without IR audio files?

A Schroeder reverberator shapes white noise using an exponential decay envelope e^(-6.91 * t / RT60). Generating this impulse response procedurally in JavaScript avoids fetching heavy 5MB impulse WAV files over the network.