Four kinds of redundancy
Video compresses because most of it is predictable or goes unnoticed. It helps to sort the waste into four kinds. Neighboring pixels are alike (spatial). Consecutive frames are alike (temporal). Some values occur far more often than others (statistical). Some detail a viewer may not notice (perceptual). The first three are redundancy: they can be exploited without changing the decoded picture. The fourth is irrelevance. Discarding it is lossy, but lossy coding can also remove detail that is visible.
Four kinds at a glance
Compare the predictions
The picture is one frame of a cartoon cat, 32×32 pixels at 8 bits each. Choose a prediction: each pixel is predicted, and only the difference is left to code. Flat predicts mid-gray everywhere, which removes nothing; it only re-centers the values. Left neighbor predicts each pixel from the decoded pixel to its left. Previous frame predicts it from the same position in the frame before. The cat had moved only slightly between the two frames, a third of a pixel, so they differ only along its outline and features.
Step sets how coarsely each difference is rounded before it is coded:
coded value = round(difference / step) decoded difference = coded value × step
At step 1 every difference survives exactly. At step 8 a difference of 13 is coded as 2 and decoded as 16, an error of 3. As the step grows, the number of distinct values left to code shrinks and the error usually grows. Step stands in for quantization, which later pages apply after a transform.
In the middle panel mid-gray is zero, lighter is positive and darker is negative. The histogram shows the decoded differences, so its bars lie at multiples of the step.
Common values can be represented more cheaply than rare ones. Modeled bits per pixel uses the histogram to estimate an ideal average, treating each pixel's coded value as independent and assuming the decoder already knows how often each value occurs. Mean squared error is the average of the squared differences between source and decoded pixels; zero means an exact match.
Leave Step at 1 and switch between the three predictions. Each choice starts from the same source picture, so the modeled bit counts are alternatives, not savings that add up.
For this cat:
- Statistical. With Flat at step 1, the modeled figure is 5.00 bits per pixel against the source's 8, with a constant prediction and nothing lost. The picture uses 84 of the 256 possible values, some far more often than others.
- Spatial. With Left neighbor the figure is 2.93: 69% of the differences are zero.
- Temporal. With Previous frame it is 2.24: 75% are zero.
- Perceptual. All three are exact at step 1. At step 8 the figures are 3.14, 2.08 and 1.44, with mean squared error 5.05, 5.05 and 1.30. Whether an error of that size is noticed is not something this demo measures.
For this cat and rounding rule, Flat and Left neighbor decode to the same picture at every step. The prediction changed how many bits the model needs, not the result.
Which tool removes which
| Kind | What repeats | Tools |
|---|---|---|
| Spatial | Neighboring pixels | Intra prediction, the transform |
| Temporal | Consecutive frames | Motion compensation, reference frames |
| Statistical | Common values | Entropy coding |
| Perceptual | Detail that may go unnoticed | Quantization, quantization matrices, adaptive quantization |
The tools in the last row control where and how much error is introduced; they do not guarantee that it goes unnoticed.
The kinds overlap. A prediction is aimed at spatial or temporal redundancy, but what it changes is the statistics of what is left, as the histogram shows. The transform is a spatial tool too. The hybrid codec loop shows how the tools fit together.
What to carry forward
- Each tool in this course is aimed at one or more of these four.
- Prediction and entropy coding can preserve the picture exactly. Quantization introduces error; whether that error is visible depends on the picture and the viewing conditions.
- A modeled bit count is not a bitrate.
Sources
Read 2026-11-14.
- ITU-T Recommendation H.264, the 05/2003 edition, clause 0.5 (an overview of the design, not an integral part of the Recommendation) and subclause 0.5.4, read as extracted text: intra coding exploits spatial statistical dependencies and inter coding temporal ones, the transform removes spatial correlation from the prediction residual, quantization is irreversible and "typically discards less important visual information", and the result is entropy coded. This supports the table of tools; the clause does not sort redundancy into these four kinds. Later editions may number these clauses differently.
- C. E. Shannon, A Mathematical Theory of Communication (1948), sections 6, 7 and 9, read as extracted text: the entropy of a set of probabilities, its form when successive symbols are independent, and the theorem that no code for a noiseless channel averages fewer bits per symbol than the source's entropy, though a code can come arbitrarily close. Modeled bits per pixel is that entropy, computed from the histogram.
- No encoder source was read for this page; it makes no claim about a particular encoder.
- Not checked: the four-way classification itself and the statements about what a viewer notices, which are given as generally described and not taken from a reference; the rest of H.264 and the other specifications.