encoding.info
You are hereSourcePredictionTransformEntropy codingFiltersFramesDecisionsRate controlPerceptionQualityBitstreamSettingsEncoder map
Page 4 of 4 in Pictures as numbers

What the eye misses: contrast sensitivity and masking

Pictures as numbers · concept · added 2026-11-14

In Brief:
Lossy tools aim their error where viewers are least likely to notice it. Two properties of vision guide them: fine detail is harder to see, and detail already in a picture can hide an error added on top of it.

A codec's lossy tools are aimed at what viewers are least likely to notice. Two properties of vision come up again and again in this course. Sensitivity to contrast varies with the size of the detail, and falls away for fine detail. And detail already in a picture can hide an error added on top of it, which is called masking.

Sensitivity depends on detail size

Contrast is how far a pattern's light and dark depart from its average. For the sine-wave gratings of the experiments below it is (brightest − darkest) / (2 × average), with all three as amounts of light, not pixel values. Contrast sensitivity is one over the smallest contrast a viewer can detect. Detail size is counted in cycles per degree: how many pairs of stripes fit in one degree of the visual field. The same picture has more cycles per degree from farther away.

Conceptual illustration: stripes get finer to the right and fainter toward the top. Where they fade into gray depends on your screen, its brightness and how far away you sit, and a screen does not reproduce contrast exactly. The contrast formula is about light; this picture varies screen gray values and is not calibrated. It is a picture to look at, not a measurement.

Campbell and Robson (1968) measured two observers with sine-wave gratings. Under their conditions sensitivity was highest near 4 cycles per degree and fell for both finer and coarser stripes. In the picture above you may see the faintest visible stripes reach higher in the middle than at either edge, and the place may shift as you move toward or away from the screen. The exact boundary depends on the display and the viewing setup; the picture is not a vision test.

Hide the same error in two places

Two patches, each 160 pixels square: one flat gray, one an invented fur texture. Noise sets the size of a random error, and the same error, pixel for pixel, is added to both. Mean squared error is the average squared difference between a patch and its noisy copy. Variance is how much a patch's own pixels differ from their average; it is zero for the flat patch.

Drag Noise down to 0, then raise it slowly while comparing each patch with its noisy copy. Does the same added error become noticeable at the same setting on both backgrounds?

8
← 0: none24: strong →
Flat
Flat, noisy
Fur
Fur, noisy
Mean squared error: flat
Mean squared error: fur
Variance: flat
Variance: fur
Simplified demo: the fur and the noise are invented. The noise is one field of whole numbers between minus and plus the Noise setting, added unchanged to both patches; nothing is clipped at any setting. The demo measures that the two errors are equal. Whether they look equal is your own observation, and it depends on your screen and distance. The patches are drawn slightly larger or smaller than 160 pixels depending on the screen, which changes how the finest noise looks.

At Noise 8 the mean squared error is 21.8 in both patches. The measure cannot tell them apart, at this setting or any other. What differs is the patch underneath: the fur's variance is 501 against 0.

Legge and Foley (1980) measured the effect with gratings in place of fur. A grating of 2 cycles per degree needed more contrast to be detected on top of a second, high-contrast grating, most of all when the second grating's stripes were close to its own in size. With a very faint second grating the opposite happened, and detection got easier. Masking is not a rule that a busy area hides everything.

Which tools lean on which

Idea from visionCoding tool
Sensitivity varies with detail sizeQuantization matrices
Color detail is resolved less finelyChroma subsampling
Some errors are less visible on some backgroundsVariance-based adaptive quantization

A quantization matrix is only the mechanism for weighting frequencies; its weights decide the behavior. The variance-based adaptive quantization modes covered in this course use block energy, a relative of the variance above, as a rough guide to where error can go. High variance is not the same as masking: one hard edge has high variance and is not dense texture. Other perceptual choices need their own explanation and are not modeled by this noise experiment; x264's psychovisual term, for one, raises the cost of losing texture energy.

These tools can use simplified assumptions about perception. They do not guarantee that an error will be invisible for a particular picture, viewer, screen or distance.

What to carry forward

Sources

Read 2026-11-14.