encoding.info
You are hereSourcePredictionTransformEntropy codingFiltersFramesDecisionsRate controlPerceptionQualityBitstreamSettingsEncoder map
Page 5 of 6 in Entropy coding

Context modeling

Entropy coding · concept · added 2026-11-14

In Brief:
A coder is only as good as its probabilities. Context modeling picks each value's probability from what was coded nearby and keeps adapting it. On this page's picture that lowered the modeled cost from 0.89 to as little as 0.62 bits per symbol.

An entropy coder is only as good as the probabilities it is given, and context modeling is how a codec supplies them. In the scheme shown here, each value is sorted into a context by what kind of value it is and what was coded nearby. Choosing the context says which probability estimate to use; adaptation changes that estimate after each value coded with it. The decoder keeps the same estimates, because it has decoded the same values.

Neighbors predict a flag

A flag here is a symbol with two possible values, zero or nonzero. This page's flags are the cat's: one per pixel, zero where the pixel equals the pixel to its left.

Neither nonzero
Left nonzero
Above nonzero
Both nonzero
Measured on the cat's flags from the first page of this unit, whose rule for the first pixel of each row applies here: it is compared with the pixel above, and the first pixel of all with mid-gray. Each tile is one context: the flag above, the flag to the left, and, outlined, the flag being coded. Gray is zero and teal is nonzero; the outlined square is filled gray in proportion to how many of that context's flags are zero. A neighbor outside the picture counts as zero. The cat is invented.

Over the whole picture 69% of the flags are zero, and a coder that knows only that needs 0.89 bits per flag. Sorted by their two neighbors, the flags fall into four groups with very different shares, so a coder with one estimate per group can price each flag closer to what it is worth. These percentages summarize the finished picture. The demo does not give them to the decoder.

Choose what the coder knows

Model chooses how the same 1,024 flags are priced, row by row. The fixed probability is the picture's own share of zeros, as if the decoder had been told it. An adapting probability starts at 50% and learns; with neighbor contexts there are four of them, one per tile. For each flag the model picks the context from flags already coded, prices the flag with the estimate as it stands, and only then moves that estimate toward the flag:

estimate += (target − estimate) / N
target = 100% after a zero,
         0% after a nonzero

Adaptation sets the fraction 1/N. At 1/16, an estimate of 50% that sees a zero becomes 53.1%. The slider is switched off for the fixed model.

1/16
← 1/4: fast, jumpy1/128: slow, steady →
Flags
Estimate
Cost
Modeled bits per flag
Modeled total bits
Costliest flag, bits
Simplified demo: the cat, the contexts and the update rule are this page's, not any codec's. In Estimate, white represents a 100% chance of a zero and black a 0% chance. In Cost, white represents zero cost and black 6 bits or more. Estimates are kept between 2% and 98%, so no flag costs more than 5.64 bits. Modeled bits are ideal costs, −log2 of the probability the model gave to what happened; no arithmetic coder is run, and nothing is counted for the fixed model's one probability, which a decoder would have to be told. The adapting models assume the decoder shares the order the flags are taken in, the context rule, the starting estimates and the update rule. Modeled bits are sums for this one sequence, not a bitrate and not a measurement of how far the flags can be compressed.

For this cat:

The fixed-share figure treats every flag as having the same probability, independent of the rest. The cat's flags are not like that: earlier flags give clues about later ones. Adapting, and choosing a context by the neighbors, use some of those clues, so the modeled cost can fall below the fixed-share figure without contradicting it.

Contexts in x264

x264's arithmetic coder keeps one byte per context: one of 64 probability states and which of the two values is currently the likely one. After each context-coded decision the byte is replaced from a table, x264_cabac_transition, which usually moves the estimate toward the value just coded. Repeated likely values stop at the strongest adapting state, and at even odds, coding the value that is not marked as likely swaps which one is. There are 460 contexts, or 1,024 for 4:4:4 video.

At the start of every slice each context is set from the slice's quantization parameter (QP):

initial value =
  clip(((m × QP) >> 4) + n, 1, 126)

with m and n from tables: one set for intra slices and three for the others, chosen by the library parameter cabac-idc. x264 converts the result into the state and the likely value.

Some coded-block flags, which say whether a block has any nonzero levels, use a similar four-way context from the blocks to the left and above. The base context and the handling of a missing neighbor depend on the block type. The flag for whether one coefficient of a 4×4 block is nonzero uses the coefficient's position in the scan.

libaom adapts by a rule like the demo's: update_cdf moves a two-valued symbol's probability 1/16 of the way at first, 1/32 once the context has coded 16 symbols and 1/64 after 32.

Trade-offs

Sources

Read 2026-11-14 and 2026-11-15, at the commits linked.