Magnitude-squared coherence: what it measures, and how many averages it needs

Coherence answers one question at every frequency: is there a linear relationship between two channels, in the presence of noise? It is the tool you reach for when a measurement disagrees with a prediction, because it separates two very different explanations — the signal genuinely is not there, or the measurement is contaminated.

It is also the most commonly misread function in the whole of signal processing, because of one fact that most software never mentions: with a single average, coherence is exactly 1.0 at every frequency, no matter what the two channels contain. The analyser below demonstrates that in one edit, and shows what changes when you start averaging.

The definition, and the one number that matters

Magnitude-squared coherence is the squared correlation between the two channels computed separately in each frequency bin, after averaging over several segments:

S_xx and S_yy are the auto-spectra of the two channels and S_xy is the cross-spectrum. Because the three quantities are estimated from the same segments, the result is bounded by 1 and describes the fraction of the power in each channel that is linearly related through a common cause at that frequency.

Set the number of averages to 1 in the tool above and look at the plot: it is a flat line at exactly 1.0, with two unrelated noise channels. It is not an approximation of 1 — it is exactly 1, because with one segment S_xy is a single product and |X conj(Y)|² equals |X|²|Y|² identically. Coherence without averaging is an empty measurement that always looks perfect, which is why any software that hides the number of averages is hiding the only thing that makes the result meaningful.

What averaging buys you

Averaging K segments makes the estimate converge towards the true coherence, with two consequences that should be quoted alongside every coherence plot.

A bias of about 1/K. Two completely unrelated channels still read a mean coherence near 1/K rather than zero — 0.0156 with 64 averages, which is what the tool reports for unrelated noise. Subtracting that bias, γ²_true ≈ (γ² − 1/K) / (1 − 1/K), is the debiased estimate plotted as the lighter line.

A significance threshold. Under the null hypothesis — no coherence at all — the estimate follows a Beta(1, K−1) distribution, so the level that only 5% of noise bins should exceed is:

Averages K	95% threshold	Bias 1/K	Record at 1024-point segments, 50% overlap
2	0.950	0.500	32 ms
4	0.632	0.250	53 ms
8	0.348	0.125	96 ms
16	0.181	0.0625	181 ms
32	0.092	0.0313	352 ms
64	0.046	0.0156	693 ms
128	0.023	0.0078	1376 ms

The right-hand column is why coherence is never free. Reaching a 0.1 threshold needs about 30 averages; the record grows with K, and a longer record means the measurement is only valid if the system stayed stationary for its whole duration. A coherence measured over 1.4 seconds of a system that drifts will be low for reasons that have nothing to do with noise.

One more sanity rule: with unrelated channels, about 5% of bins should land above the threshold line purely by chance. If a third of the bins are above it, the channels are not unrelated — something common is leaking between them, and that is usually a measurement problem rather than a discovery.

What coherence does not measure

It does not measure similarity, it measures linearity. Set the delay on channel B to a few samples with no noise and no tone, and the coherence stays essentially at 1.0 while the two waveforms are visibly offset. A pure delay is a linear filter, so coherence is blind to it. In the tests behind this page a 4-sample delay with a Hann window gave 0.9968 — not exactly 1, because the DFT shift theorem applies to a circular shift and the segment edges do not shift with it. That shortfall is entirely about the segment boundary: with a rectangular window, which does not attenuate those edge samples, the same delay costs 0.9711.

It does not prove linearity either. Feed the analyser a channel that is a cubic function of the other — y = 3x³ — and the coherence still reads about 0.57, because a cubic has a large linear component. Coherence below 1 always has three possible explanations (noise, non-stationarity, or genuine nonlinearity) and the plot alone does not tell you which.

It says nothing about which channel leads. For that you need the cross-spectrum phase, plotted below the coherence plot. There the delay shows up clearly: a linear ramp whose slope is the time difference. Watch the sign convention — with S_xy = E[X conj(Y)] a positive slope means channel B lags channel A, and reversing the conjugacy flips it. Verify against a known delay before trusting the direction, because half the references you will read use the other definition.

Why real coherence drops

1. Uncorrelated noise in either channel. It is the clean case. Noise with power σ²_n in one channel against a common signal of power σ²_c gives γ² = σ²_c/(σ²_c+σ²_n): equal powers give 0.5, ten times more noise gives 0.09. The tool reproduces that within a couple of hundredths with the default settings.
2. Non-stationarity. If the transfer function changes during the record, the segments do not share a common answer and averaging drives coherence down. No amount of extra averaging fixes it; a shorter record does.
3. Nonlinearity. As above, the linear part still coheres, so the reading sits somewhere in the middle rather than collapsing to zero.
4. Truncation and window leakage. A strong tone leaks into neighbouring bins through the window, so bins a few steps away from a tone are dominated by its skirt rather than by the local signal. Comparing coherence near a loud tone with coherence in a quiet region is comparing two different things.
5. Low-frequency drift. A 1/f process or thermal drift produces a slowly varying common component that dominates the low bins; detrending removes the mean but not the trend. A linear detrend, or a shorter segment, is often needed before the low-frequency coherence means anything.

A recipe worth following

1. Choose the segment length from the frequency resolution you need, remembering that a Hann window's main lobe is 4 bins wide.
2. Use 50% overlap as a default: it doubles the number of averages for a given record without making neighbouring segments too similar.
3. Detrend every segment. Remove the window-weighted mean, not the plain mean: sum(w · x) is the quantity that a windowed DFT turns into bin 0, and subtracting the plain mean leaves a residue there. Set the detrend checkbox off with a shared DC offset and watch the 0 Hz bin jump to 1.0.
4. Take at least 16 averages, and prefer 32 or more if the record can be stationary for that long.
5. Draw the 95% threshold on the plot and remember that 5% of bins will cross it by chance.
6. Report K with the plot. A coherence curve without its number of averages cannot be interpreted, and debiasing it invents precision that the estimate does not have.

Five mistakes worth avoiding

1. Quoting coherence without the number of averages. The estimate's bias and its noise both scale with 1/K, so the number is part of the result.
2. Reading a single-average coherence as real. It is 1.0 by construction, at every frequency, for any two signals.
3. Treating low coherence as a broken measurement. Sometimes it is the answer: the two signals genuinely are not linearly related at that frequency.
4. Expecting a delay to lower coherence. It shows up in the phase, not in the coherence.
5. Applying window amplitude corrections to coherence. They cancel in the ratio. Applying them anyway is harmless but signals that the maths is not understood.

Summary

Coherence is a bias-corrected, averaged, bounded measure of the linear relationship between two channels per frequency bin. Use enough averages for the threshold you need, subtract the 1/K bias before believing small numbers, draw the significance line, and read the phase alongside it when the question is about timing rather than similarity. And verify your sign convention against a known delay at least once.

Values shown are engineering aids rather than measurements; see the disclaimer, and the tool index for the rest of the series.