Chapter 7

Allan deviation and clock stability

You built a clock. You logged its fractional frequency $y(t)$ for a week. Now your collaborator asks the only question that matters: how stable is it? The obvious answer — "compute the standard deviation" — is wrong, and not by a little: for the noises real clocks have, the standard deviation is not even a number. This chapter is about the minimal surgical fix, the Allan deviation $\sigma_y(\tau)$, and about the remarkable fact that its shape versus averaging time is a fingerprint of the noise type.

Throughout, $y(t) = \delta\nu(t)/\nu_0$ is the fractional frequency of chapter 6 — dimensionless, e.g. $y = 10^{-15}$ means the clock currently ticks a part in $10^{15}$ fast. Its one-sided PSD $S_y(f)$ (ordinary frequency $f$ in Hz) is the language we already speak; the Allan deviation will turn out to be a second view of the same $S_y(f)$.

Accuracy is not stability

Before doing any statistics, be sure which question you are answering. "How good is this clock?" hides two independent questions. Split the frequency of a real oscillator as

$$ \nu(t) \;=\; \nu_0\,\big[\,1 + \epsilon + y(t)\,\big], $$

where $\nu_0$ is the true (nominal) frequency the clock is supposed to produce, $\epsilon$ is a constant systematic offset, and $y(t)$ is the fluctuating part, defined so that it averages to zero. Accuracy is a statement about $\epsilon$: how far the clock's average frequency sits from the truth. Stability is a statement about $y(t)$: how much the frequency wanders about its own average, wherever that average happens to be. These are independent virtues — a clock can be any combination of stable/unstable and accurate/inaccurate — and conflating them is the classic beginner's error in time and frequency.

A dartboard makes the distinction impossible to forget. Every frequency measurement is a dart; the bull's-eye is the true frequency. Bias $\epsilon$ moves the whole cluster off-center; instability spreads the cluster out. Four throwers, four reputations: a tight cluster away from the bull's-eye (stable but not accurate — a good quartz oscillator years after its last calibration), a loose cluster away from it (neither), a loose cluster centered on it (accurate on average but not stable), and a tight cluster dead center (stable and accurate — the clock everyone wants).

Now look at what the statistic this chapter builds actually sees. The Allan deviation (two sections down) is made of differences of adjacent averages, so a constant $\epsilon$ cancels exactly: $\sigma_y(\tau)$ measures stability and is completely blind to accuracy. That blindness is a feature — you can measure a clock's stability by beating it against itself or an equal, without owning anything more accurate — and a trap: a beautifully flat ADEV at $10^{-16}$ says nothing whatsoever about whether the frequency is right. The dartboard's stable-but-not-accurate thrower has a superb Allan deviation.

The asymmetry between the two virtues sets the design rule of metrology. Accuracy is fixable after the fact: measure $\epsilon$ once against a better reference and correct for it — that is calibration. Instability is not fixable: you cannot subtract a random wander you cannot predict. Hence "stability first", and hence primary standards: the atomic-clock loop of chapter 10 ties $\nu_0$ to an atomic transition, so that $\epsilon$ is bounded by physics rather than by the date of the last calibration.

Why the ordinary variance fails

Chapter 5 ended on a warning: for noise steeper than white, the variance is not a property of the process but of how long you watched it. Clocks are where that warning bites hardest, because every real oscillator eventually shows flicker ($S_y \propto 1/f$) and random-walk ($S_y \propto 1/f^2$) frequency noise. The variance is the integral of $S_y(f)$ over all $f$, and for these spectra the integral diverges at the low-frequency end. A longer record admits lower Fourier frequencies, so the sample variance just keeps growing.

Watch it happen. Below, three synthetic clocks tick once per second: one with white FM noise, one with flicker FM, one with random-walk FM (each generated once and then analyzed cumulatively). For each we plot the sample standard deviation of the first $N$ points, as a function of $N$.

So "the variance of my clock" is simply not a number for flicker or random-walk noise — it depends on how long you cared to measure. Quote it and the only thing you have characterized is your patience. Any honest stability measure must (a) converge for the noises clocks actually have, and (b) admit that stability depends on time scale: a clock can be superb over one second and terrible over a day. Both requirements point the same way: measure fluctuations at a chosen time scale $\tau$, and make the statistic blind to the slow divergent stuff below $1/\tau$.

The two-sample fix

Here is what a frequency counter actually gives you: the average fractional frequency over consecutive gate intervals of length $\tau$,

$$ \bar y_k \;=\; \frac{1}{\tau}\int_{k\tau}^{(k+1)\tau} y(t)\,dt . $$

David Allan's 1966 move: instead of the variance of the $\bar y_k$ about their mean, take the mean square of the differences of adjacent pairs,

$$ \sigma_y^2(\tau) \;=\; \tfrac12\, \big\langle (\bar y_{k+1} - \bar y_k)^2 \big\rangle . $$

That is the whole definition. Differencing adjacent averages is the entire trick: a constant offset in $y$ cancels exactly, and slow wandering — the divergent low-frequency content — cancels almost exactly, because two neighboring gates see nearly the same value of it. The two-sample difference is a high-pass filter in disguise (we will make that literal in the last section), and it is precisely the divergent DC end that it removes. The factor $\tfrac12$ is a normalization: for white FM the $\bar y_k$ are independent, so $\langle(\bar y_{k+1}-\bar y_k)^2\rangle = 2\,\mathrm{var}(\bar y)$, and the $\tfrac12$ makes the Allan variance agree with the classical variance of the gate averages in the one case where the classical variance is honest. The square root $\sigma_y(\tau)$ is the Allan deviation — "ADEV" on every clock plot you will ever see.

The picture below is the definition drawn on the data: the time axis is chopped into gates of length $\tau$, each gate is collapsed to its average $\bar y_k$ (the staircase), and $\sigma_y(\tau)$ is the RMS step height between adjacent stairs, divided by $\sqrt2$.

One practical refinement, free of new ideas: with $N$ samples you can slide the pair of gates along one sample at a time instead of jumping gate-by-gate, reusing the data in every position — the overlapping Allan deviation, which is what allan.js (and every clock-metrology package) computes, because it squeezes far more statistical confidence out of the same record without changing the expected value.

The $\sigma_y(\tau)$ fingerprint

Now the payoff. Compute $\sigma_y(\tau)$ for every $\tau$ from one gate up to a quarter of your record, and plot it log–log. Write each power-law patch of the frequency-noise spectrum as $S_y(f) = h_\alpha f^{\alpha}$ — chapter 5's $h/f^\alpha$ levels, with the subscript naming the slope — and each noise type produces a straight line with its own characteristic slope. Every PSD in the table is the fractional-frequency one, $S_y(f)$ — the "PM" rows are phase noises seen through chapter 6's identity $S_y = (f/\nu_0)^2 S_\varphi$, which is how a flat phase spectrum becomes $h_2 f^2$ here:

Read a measured ADEV curve from left to right and you are reading the clock's biography: which noise dominates at each time scale. The demo below mixes all five ingredients into one $2^{16}$-sample record ($\tau_0 = 1$ s) and computes the overlapping ADEV live. The dashed lines are the theoretical contribution of each ingredient at its current slider setting; the solid curve is the ADEV actually measured from the synthetic record. Raise each slider and watch its regime lift the measured curve — the curve always hugs whichever dashed line is on top.

Reading a real clock plot

Here is the shape you will see in essentially every optical-clock paper. At short $\tau$ the curve falls as $\sigma_y(\tau) = a/\sqrt{\tau}$ — white FM, typically quantum projection noise from interrogating a finite number of atoms; a good lattice clock might have $a = 3\times10^{-16}$ at 1 s. Averaging longer keeps helping… until it doesn't: around $\tau \sim 10^4\,$s the curve flattens onto a flicker floor, say $1\times10^{-18}$, set by whatever $1/f$ process the clock owns (blackbody shifts, lattice light shifts, the cavity). Past the floor, curves often turn up again as random walk from slow thermal or environmental drifts takes over. The floor is the clock's headline number, and the white-FM prefactor decides how long you must average to reach it.

The white-FM prefactor is not a new quantity — it is the sensitivity of chapter 8 wearing fractional-frequency clothing. White FM with one-sided level $h_0$ gives (next section)

$$ \sigma_y(\tau) = \sqrt{\frac{h_0}{2\tau}} \qquad\Longrightarrow\qquad a = \sqrt{h_0/2}, $$

so quoting "$3\times10^{-16}$ at one second" and quoting a white noise floor of $h_0 = 2a^2 = 1.8\times10^{-31}\,\mathrm{Hz}^{-1}$ in $S_y$ are the same statement. Chapter 8 will read the same number as a per-$\sqrt{\mathrm{Hz}}$ sensitivity.

Allan meets the PSD

ADEV and $S_y(f)$ are not rival descriptions; the Allan variance is a weighted integral of the PSD. Averaging over a gate and differencing adjacent gates is a linear, time-invariant operation on $y(t)$, so it has a transfer function, and

$$ \sigma_y^2(\tau) \;=\; \int_0^{\infty} S_y(f)\, \big|H_\tau(f)\big|^2\, df, \qquad \big|H_\tau(f)\big|^2 \;=\; \frac{2\sin^4(\pi\tau f)}{(\pi\tau f)^2}. $$

Plot that weight and the "high-pass in disguise" claim becomes a picture — in fact it is better than a high-pass, it is a bandpass: $|H_\tau|^2$ rises as $f^2$ at low frequency (drift rejection), falls off above $1/\tau$ (the averaging), and peaks near

$$ f_{\mathrm{peak}} \approx \frac{0.37}{\tau} \approx \frac{1}{2\tau}. $$

This is the reason the fingerprint works: $\sigma_y(\tau)$ at gate time $\tau$ mostly reports $S_y$ near Fourier frequency $1/2\tau$, so as $\tau$ grows the ADEV scans down the spectrum, and each power law in $S_y(f)$ maps onto a power law in $\tau$. Doing the integral exactly for each one-sided power law $S_y = h_\alpha f^\alpha$ gives the standard conversion table:

Note the white-PM row needs the high-frequency cutoff $f_H$ of your measurement system: $S_y \propto f^2$ keeps growing, and the bandpass only tames it up to the bandwidth you actually recorded. This foreshadows chapter 9, where the same "sensor = filter function acting on the PSD" logic — with fancier $|H(f)|^2$ — is the master formula for qubit dephasing and LIGO alike.

The estimator in code

The overlapping estimator that powers every plot on this page is ten lines. The only cleverness is a cumulative sum, so each gate average is a subtraction instead of an inner loop (this is Allan.adev from allan.js):

// y[0..N-1]: fractional frequency samples every tau0 seconds.
// c[k] = y[0] + ... + y[k-1]  (cumulative sum with a leading zero)
const c = new Float64Array(N + 1);
for (let i = 0; i < N; i++) c[i + 1] = c[i] + y[i];

// overlapping Allan variance at averaging factor m  (tau = m * tau0)
let sum = 0, count = 0;
for (let j = 0; j + 2 * m <= N; j++) {          // slide one sample at a time
  const a = (c[j + m] - c[j]) / m;              // average of gate starting at j
  const b = (c[j + 2 * m] - c[j + m]) / m;      // average of the next gate
  sum += (b - a) * (b - a);
  count++;
}
const adev = Math.sqrt(sum / (2 * count));      // sigma_y(m * tau0)

Exercises