encoding.info
You are hereSourcePredictionTransformEntropy codingFiltersFramesDecisionsRate controlPerceptionQualityBitstreamSettingsEncoder map
Page 1 of 4 in Measuring quality

Why quality metrics disagree

Measuring quality · concept · added 2026-11-15

You'll want firstDistortion measures
In Brief:
A full-reference quality metric compares a decoded picture or video with its source and gives a score. Different metrics measure different kinds of change, so they can rank the same encodes differently. A score estimates one aspect of quality; none replaces checking what viewers see.

A full-reference quality metric compares two pictures, the source and the decoded encode, and gives one number. Different metrics look at different things, so they can rank the same encodes differently, and none of them is a viewer. Knowing what each one measures tells you which disagreements to expect.

Four errors of the same size

The cat is damaged four ways, each tuned to nearly the same mean squared error (MSE): the average of the squared pixel differences. Peak signal-to-noise ratio (PSNR) is that error on a decibel scale, so the four score nearly the same. Structural similarity (SSIM) compares the two pictures window by window and gives 1 when they are identical. The version here is x264's: one scale, luma only, 49 overlapping 8×8 windows on this picture. The last panel has one cell per window, 7 by 7, not one per pixel: white is 1, darker is lower.

25
← 9: small error144: large error →
Source
Damaged
Error, ×4
SSIM per window
DamagePSNR, dBSSIMLowest window
Brightness shift
Noise
Blur
Coarse quantization
MSE
PSNR
SSIM
Simplified demo: the cat, the noise pattern and the four kinds of damage are invented. Each strength is the closest to the target that a search over the rounded 8-bit picture found, so the four are within 0.1 of each other, not equal. PSNR and SSIM follow x264's arithmetic for 8-bit luma; x264 starts its windows two pixels in from the picture's edge and the demo starts at the edge. Coarse quantization is the course's 4×4 transform with no prediction. The highlighted row is the damage shown. In the error panel mid-gray is no change, lighter is brighter than the source. No viewer is measured.

At an error of 25:

Which of the four looks worst is for you to judge from the Damaged panel; neither number settles it.

What each one measures

PSNR
Each pixel against the pixel in the same place.
SSIM
Each window's mean, spread and pattern; windows overlap.
VMAF
Gray: three families of features per frame. Teal outline: a trained model that turns them into a score.
Conceptual illustration: the grids are smaller than the real windows.
PSNR = 10 × log10(255² / MSE)

SSIM of one window =
  (2·a·b + C1) × (2·cov + C2)
  / ((a² + b² + C1) × (va + vb + C2))

Here 255 is the peak of an 8-bit sample. In the second formula a and b are the two windows' means, va and vb their variances and cov their covariance; C1 and C2 are small constants that keep flat windows from dividing by zero. The first factor compares brightness and the second compares contrast and pattern. A picture's SSIM is the mean over its windows. Implementations differ in window, weighting, scale and edge handling, so another tool's SSIM need not match these digits.

Video Multi-Method Assessment Fusion (VMAF) is a different kind of thing: a model trained on viewers' scores turns several features of each frame into one number, for one viewing setup.

The model read for this page and its viewing setup

The model read for this page, vmaf_v0.6.1, takes six features in three families: visual information fidelity at four scales, a detail loss measure and a measure of the difference between adjacent frames. Its output is clipped to 0 to 100. It was trained for a 1080-line television watched from three screen heights, and its documentation warns against comparing a score computed at 1080 lines with one computed at 480. Other models target other viewing setups, and a score is not a percentage of detail kept.

In x264

x264 can print both measures of its own output, and it warns when one is taken with psychovisual optimization or adaptive quantization set the wrong way for that metric. The warning is about benchmark setup, not arithmetic: to learn how your normal settings behave, measure those settings.

The options, the warning and the two tunings

The options are --psnr and --ssim, and SSIM is also printed in decibels as −10 × log10(1 − SSIM). If psychovisual optimization is on, x264 logs that the results "will be invalid", and the same for PSNR with adaptive quantization on or SSIM with it off; it tells you to use --tune psnr or --tune ssim when benchmarking. The psnr tuning turns both tools off; the ssim tuning turns psychovisual optimization off and sets adaptive quantization to its auto-variance mode. With those tools on, each score still measures the decoded picture. When you report a measurement, say which settings it was made with.

Trade-offs

Sources

Read 2026-11-15 and 2026-11-16, at the commits linked.