The discrete cosine transform as a change of basis
The discrete cosine transform (DCT) represents a block as a weighted sum of cosine patterns. In video coding the input is usually a prediction residual: the source block minus its prediction. For many smooth residuals, most of the energy ends up in a few coefficients, which helps the quantization and entropy-coding stages that follow.
Why change basis
Neighboring pixels tend to be alike. Written as pixels, a smooth row is eight numbers of about the same size, each as important as the next. Written as amounts of eight cosine patterns, the row below is one large number, one medium one and six that are nearly zero.
How it works
In two dimensions the patterns are products of a horizontal and a vertical cosine, 64 of them for an 8×8 block, running from flat (top left) to a fine checkerboard (bottom right). The top-left coefficient (DC) represents the block's average level; the other 63 describe variation around it. Pick a block and move the slider: it rebuilds the block from only the first coefficients in zigzag order, which runs broadly from low frequencies to high.
In this model the gradient reaches 99% of its detail energy with three coefficients, and the soft edge reaches 98% with ten. In the noise example, keeping 32 coefficients retains about half the detail energy: unlike the smooth examples, its energy is spread across many coefficients. A repeating or directional texture has structure the transform can use.
The steps in a real encoder:
- Transform the residual, not the picture. The input is the difference between the source block and its prediction.
- Rows, then columns. The two-dimensional transform is a one-dimensional one applied in each direction.
- Integer arithmetic. Codecs specify integer approximations of the cosines so that every decoder reconstructs the same values. The core of H.264's 4×4 forward transform, as x264 implements it, uses only additions, subtractions and doublings.
- Quantize the coefficients. See Quantization and the quantization parameter (QP).
Trade-offs
- It compresses nothing by itself. The transform still produces 64 coefficients. The orthonormal DCT used here is reversible and keeps the sum of squared values; see Block energy. Codec integer transforms add their own scaling and rounding. The saving comes afterward: quantization reduces precision and entropy coding represents the resulting levels compactly.
- It suits smooth residuals. Sharp edges and noise often spread across many coefficients, depending on an edge's direction, position and shape. Quantizing those coarsely can show as ripples beside an edge, called ringing.
- Block boundaries can show. Quantizing neighboring transform blocks can leave different errors on either side of a boundary, making the block grid visible. Codecs use tools such as deblocking filters to reduce this.
- Size is a choice. A large transform often represents a smooth residual with fewer nonzero coefficients; a small one limits how far the error around an edge extends. See Choosing a transform size.
Sources
Read 2026-11-13, at the commits linked.
- x264 at commit c24e06c (GitHub mirror):
sub4x4_dct, the 4×4 forward transform core. - Not taken from an encoder or from a source read for this page: everything else here, which is standard transform mathematics. The numbers in the figures are computed by the page itself.