Frequency-domain inputs and outputs
Let the STFT observation be . Here is the frequency bin, is the frame and is the microphone count. For outputs, define the demixing matrix with the following orientation:
Each row already contains the conjugate-transposed coefficients in this convention. The code therefore applies the matrix directly, without conjugating it again.
Array shapes
Removing the time axis from a single-frame function makes its state interface easier to follow.
| Quantity | Array | Shape |
|---|---|---|
| Observation | x | (F, M) |
| Demixing matrix | W | (F, N, M) |
| Observation covariance | covariance | (F, M, M) |
| Output | y | (F, N) |
State carried between frames
An exponential average of observation outer products updates the observation covariance as follows. The forgetting factor satisfies .
The following example updates that covariance and applies a supplied demixing matrix. It does not contain a learning rule for the matrix and cannot learn source separation by itself. The code requires Python and NumPy.
import numpy as np
def process_frame(x, W, covariance, beta=0.98):
"""x: (F, M), W: (F, N, M), covariance: (F, M, M)."""
if not 0.0 <= beta < 1.0:
raise ValueError("beta must satisfy 0 <= beta < 1")
outer = x[:, :, None] * x[:, None, :].conj()
covariance = beta * covariance + (1.0 - beta) * outer
y = np.einsum("fnm,fm->fn", W, x)
return y, covarianceInitial state
This example uses five frequency bins, two microphones and two outputs. Identity initialization makes the output equal to the observation. The input checks connectivity and shapes; it is not audio for evaluating separation quality.
F, M, N = 5, 2, 2
x = np.ones((F, M), dtype=np.complex128)
W = np.broadcast_to(np.eye(N, M), (F, N, M)).copy()
covariance = np.zeros((F, M, M), dtype=np.complex128)
y, covariance = process_frame(x, W, covariance)
print(y.shape, covariance.shape)Streaming boundaries
Reset the demixing matrix, statistics, and STFT and inverse-STFT buffers when moving to a different recording. Carry those states across chunks of the same recording. Define how the final chunk is padded and how the output is trimmed.
To check online behavior, compare inputs with the same past and different futures. Use the same permitted lookahead and check whether earlier outputs change when samples beyond it change. Inspect centered STFT and normalization for unintended access to future samples.
Records to retain for evaluation
Save input and output channel ordering, scale restoration, the warm-up interval and per-frame processing time. Associate audio quality metrics and timing with the same input identity. See Managing audio signal processing experiments for recording execution conditions.