Operators, Observables & Dirac Notation

For thirteen chapters we have described quantum systems with wavefunctions, a complex number ψ(x) attached to every point in space. That picture hides a deeper structure. Recall the oscillator: any state of the spring can be built by adding up its rungs, each multiplied by a complex number. Objects that can be added together and scaled by numbers are what mathematicians call vectors. The great leap made between 1925 and 1932 by Heisenberg, Born, Jordan, Dirac and John von Neumann was to take this literally: a quantum state is a vector, an arrow in an abstract space of possibilities, and a wavefunction is just one way of listing its components.

Dirac gave the arrow its modern costume. He wrote a state as |ψ⟩ and called the symbol a ket; the label inside is only a name, so |n⟩ can stand for the oscillator's rung n and |↑⟩ for an electron whose spin points up. Kets live in a Hilbert space, named after the Göttingen mathematician David Hilbert: a vector space with complex components and a rule for measuring lengths and angles. Its dimension is the largest number of states some measurement could tell apart perfectly: two for an electron's spin, infinitely many for a particle's position.

Two rules make kets useful. Superposition is simply vector addition: if |a⟩ and |b⟩ are possible states, so is α|a⟩ + β|b⟩ for any complex numbers α and β, the linearity of chapter 10's Schrödinger equation in vector clothing. And every physical state has length exactly one, a condition called normalization that makes probabilities add up to 1. The smallest interesting case has two dimensions: every state of an electron's spin, a photon's polarization or a qubit is a column of just two complex amplitudes. That two-row column is the workbench for this whole session.

To compare two arrows in ordinary space you use the dot product, which multiplies matching components and adds the results: largest when the arrows point the same way, zero when they are perpendicular. Complex components need one extra twist: the components of one arrow must be complex-conjugated, every i turned into −i, and Dirac's notation builds that twist in. His bra, written ⟨φ|, holds the same components as |φ⟩, laid on their side as a row and each replaced by its complex conjugate. Place a bra against a ket and you get a bracket, ⟨φ|ψ⟩, a single complex number. Split the word bracket in two, drop the c, and you have Dirac's joke, unveiled in a three-page paper in 1939.

This number, the inner product, measures overlap: how much of the state |ψ⟩ lies along the direction |φ⟩. A state's overlap with itself, ⟨ψ|ψ⟩, is the squared length of its arrow; the conjugation guarantees that this is a positive real number, which normalization sets to 1. And states with zero overlap are called orthogonal, the Hilbert-space word for perpendicular. Orthogonal states are perfectly distinguishable: spin up and spin down, say, or two different rungs of the oscillator.

Now the payoff: Born's rule of chapter 9 in Dirac's clothing. Suppose a system is in the state |ψ⟩ and a measurement has an outcome whose state is |φ⟩. Call the overlap c = ⟨φ|ψ⟩; the probability of that outcome is |c|², the squared magnitude of the overlap. Light gives the oldest demonstration. In 1809 Étienne-Louis Malus found that polarized light meeting a polarizer turned at angle θ to its polarization keeps a fraction cos²θ of its intensity. Send the light through a polarizing filter one photon at a time and each photon passes whole or is absorbed whole: cos θ is the overlap between the photon's polarization state and the filter's, and cos²θ is the probability that it passes.

In chapter 10 an operator was a rule that turns one function into another. In Dirac's grammar an operator, written with a hat as in Â, turns one ket into another, and it is linear: it respects sums and multiples. In a finite-dimensional Hilbert space every such rule is a matrix, a square grid of numbers that multiplies the column of amplitudes. The central postulate of this chapter is that every observable, every measurable quantity from energy to spin, is represented by an operator of one special kind, which answers two questions before any experiment is done: which results are possible, and which states would give each result with certainty.

The special kind is called Hermitian, after the French mathematician Charles Hermite, whose polynomials shaped last session's oscillator wavefunctions. Flip a matrix across its main diagonal, so that rows become columns, and replace every entry by its complex conjugate; the result is the adjoint, written † and read 'A-dagger'. A Hermitian operator equals its own adjoint: its diagonal entries are real, and each entry above the diagonal is the complex conjugate of its mirror image below. Hermitian does not mean real: the matrix with rows (0, −i) and (i, 0) is perfectly Hermitian, as Part 5 will show.

Why insist on this? Because it delivers three guarantees, the first of which Hermite himself proved in 1855. The eigenvalues of a Hermitian operator, the numbers a for which Â|a⟩ = a|a⟩ has a non-zero solution, are always real, as every reading on a dial must be. Eigenvectors belonging to different eigenvalues are orthogonal, so different results are perfectly distinguishable. And the eigenvectors form a complete set of perpendicular axes for the whole space, a result called the spectral theorem. The measurement postulate then reads: the only possible results of measuring an observable are the eigenvalues of its operator. The energies of the box and the oscillator were eigenvalues of the Hamiltonian all along.

An eigenvector of an observable is called an eigenstate, the quantum version of a quantity with a definite value: in the eigenstate |a⟩, a measurement of A returns a every time, with no spread at all. Because the eigenstates of a Hermitian operator form a complete set of perpendicular axes, any state can be expanded along them: |ψ⟩ = c₁|a₁⟩ + c₂|a₂⟩ + c₃|a₃⟩ and so on, one complex coefficient per result. Each coefficient is an overlap, cₙ = ⟨aₙ|ψ⟩, and the probability of the result aₙ is |cₙ|². Because the axes are perpendicular and the arrow has length one, these probabilities add up to exactly 1: Pythagoras, promoted to the foundations of physics.

You have done this before without the vocabulary. Expanding a wavefunction in the stationary states of a box or a spring was an expansion in the eigenbasis of the Hamiltonian, and the squared coefficients were the probabilities of each energy. One state can be expanded in many eigenbases, one per observable, just as one arrow has different components along differently oriented axes. Rotating the axes changes the description, never the arrow, and a state with a definite value of one observable is generally a superposition with respect to another.

Repeat a measurement on many identically prepared copies and the results average to the expectation value: each possible result aₙ multiplied by its probability, |cₙ|², then all added up. Dirac's notation packs the same quantity into a sandwich, ⟨ψ|Â|ψ⟩: let the operator act on the ket, then take the overlap with the original state, with no need to find a single eigenvector. Worked example 2 does exactly this for an electron spin.

Now let us build the simplest quantum instrument there is. Take an electron's spin and choose as axes the two states a z-measurement can return: |↑⟩, the column with entries 1 and 0, and |↓⟩, the column with entries 0 and 1. A general state is a column of two complex amplitudes, α for up and β for down, with |α|² + |β|² = 1. In 1927 Wolfgang Pauli, building spin into Schrödinger's theory, represented its three components by three two-by-two matrices: σz has rows (1, 0) and (0, −1), σx has rows (0, 1) and (1, 0), and σy has rows (0, −i) and (i, 0). Spin along each axis is ħ/2 times the matching matrix.

Every Pauli matrix is Hermitian with eigenvalues +1 and −1 only, so spin measured along any axis is +ħ/2 or −ħ/2, never anything in between: the stark two-valuedness Otto Stern and Walther Gerlach saw in 1922, when a beam of silver atoms split in two, the story of chapter 17. The eigenstates of σz are |↑⟩ and |↓⟩. Those of σx are equal-weight superpositions, |+x⟩ = (1/√2)(|↑⟩ + |↓⟩) and |−x⟩ = (1/√2)(|↑⟩ − |↓⟩), while those of σy carry a relative factor of i or −i: a phase a z-measurement can never see, yet one that transforms the statistics of a y-measurement.

The same matrices run quantum computers, where they go by X, Y and Z and the two states are written |0⟩ and |1⟩. X swaps the two amplitudes, a quantum NOT; Z flips the sign of the second. Programmers also use the Hadamard matrix, (1/√2) times the matrix with rows (1, 1) and (1, −1), which is Hermitian too, with eigenvalues +1 and −1. But not every useful matrix is an observable. The phase gate S, with rows (1, 0) and (0, i), has eigenvalues 1 and i: a perfectly good operation on a qubit, but no dial can ever read i. Gates transform states; observables are menus of answers. X, Y, Z and the Hadamard happen to be both.

With ordinary numbers, 3 × 5 equals 5 × 3. With matrices, the order of multiplication can change the answer, and quantum mechanics is built on the difference. Put σx first and σz second and the product, σxσz, is the matrix with rows (0, −1) and (1, 0); reverse the order and you get rows (0, 1) and (−1, 0), its exact negative. A book on your desk shows the same effect: give it a quarter turn about a vertical axis, then a quarter turn about a horizontal one, and compare with the same turns in the opposite order. The book ends up facing a different way, because rotations do not commute, and spin operators are, at heart, the generators of rotations.

The commutator of  and B̂, written with square brackets as [Â, B̂], is the product in one order minus the product in the other, ÂB̂ − B̂Â: zero exactly when the order makes no difference. For spin, the commutator of σx with σy is 2i times σz, and likewise around the cycle x, y, z. Here is the physics: two observables share a complete set of eigenstates, and so can have definite values together, exactly when their operators commute. Such observables are compatible, like the energy and total angular momentum of a hydrogen atom. Incompatible ones cannot be pinned down together: no state is an eigenstate of both σx and σz, so no electron has a definite spin along both x and z.

Commutators also set the price of incompatibility. In 1929 Howard Percy Robertson proved, for any two observables, that the product of their spreads, ΔA·ΔB, is at least half the magnitude of the average of their commutator. Where that average is zero the bound is zero too; where it is not, the state itself forbids both quantities from being sharp at once, however perfect the laboratory. In 2007 Valentina Parigi, Alessandro Zavatta and Marco Bellini in Florence, with Myungshik Kim of Queen's University Belfast, added a single photon to a beam of thermal light and then removed one, or did the reverse, and showed by quantum tomography that the two orders leave measurably different states. Mwahaha — measurably! Last chapter's ladder operators, caught failing to commute.

For a particle on a line, the star observables are position and momentum, and the Hilbert space has infinitely many dimensions, one for each point. The position operator x̂ is simple: multiply ψ(x) by x. The momentum operator is the surprise, p̂ = −iħ d/dx, which in words says: take the slope of the wavefunction at every point and multiply by −iħ. The factor i makes the operator Hermitian, which a bare slope operator is not; the minus sign makes a wave travelling toward positive x carry positive momentum; and ħ converts wiggliness into units of momentum.

Why the slope? Try it on the plane wave e^(i·k·x), a complex wave that corkscrews along the x-axis with wavelength λ = 2π/k. Its slope is i·k times the wave itself, so p̂ returns the same wave multiplied by −iħ × i·k = ħk. The plane wave is a momentum eigenstate with eigenvalue p = ħk = h/λ: de Broglie's relation from chapter 6, now built into the machinery. A standing sine wave, sin(kx), is not an eigenstate but an equal superposition of waves running left and right, with momenta +ħk and −ħk. And p̂² divided by 2m reproduces the curvature term of chapter 10's Hamiltonian, −(ħ²/2m) d²/dx².

Now the commutator. Apply x̂p̂ − p̂x̂ to any wavefunction f. The first term gives −iħ times x times the slope of f. The second needs the slope of the product x·f, which by the product rule is f plus x times the slope of f, all multiplied by −iħ. Subtract, and everything cancels except iħ·f. Hence the canonical commutation relation, x̂p̂ − p̂x̂ = iħ. Max Born and Pascual Jordan made it the cornerstone of matrix mechanics in September 1925; it is carved on Born's gravestone in Göttingen as pq − qp = h/2πi. Earle Kennard proved Δx·Δp ≥ ħ/2 in 1927, and Robertson's inequality derives it from this one commutator. Uncertainty is not a failing of clumsy instruments; it is a theorem about operators that refuse to commute.