Why some symbols cost less than a bit
An entropy coder is the last stage of the loop: it turns the levels and the prediction information into bits, and the decoder gets every value back exactly. It saves bits by spending few on values that are likely and many on values that are not. The ideal price of a value that occurs with probability p is −log2(p) bits, so the ideal price of any value that is more likely than not is less than one bit.
The price of a symbol
A symbol is one value handed to the entropy coder. On this page it is one flag: is this difference zero or not? An outcome that is given probability p has an ideal cost of
ideal cost = −log2(probability) bits
At probability 1/2 that is 1 bit, at 1/4 it is 2 bits, and at 0.9 it is 0.15 bits. A real code need not spend that many bits on that particular outcome: with the codewords 0 and 1, a flag costs one bit whether it was the 90% outcome or the 10% one. Weighting the ideal costs by how often each outcome occurs gives the entropy. For symbols that do not depend on each other, that is the lower limit on a lossless code's average bits per symbol.
Change how often zero occurs
The slider sets the share of flags that are zero. The field shows 100 flags with that share, and the plot marks each kind of flag's ideal cost on the curve −log2(p). A code that gives every flag its own codeword spends at least one whole bit on each; the last readout is that baseline. All four readouts are in bits per flag. The slider starts at 69%, the cat's share rounded to a whole percent.
By the formula:
- At 50%, nothing is saved. Each kind of flag has an ideal cost of 1 bit and the average is 1.00.
- At 69%, the ideal cost of a zero is 0.54 bits and of a nonzero 1.69. The average is 0.89 bits per flag.
- At 99%, they are 0.014 bits and 6.64. The average is 0.081 bits per flag, about a twelfth of one bit each.
The unlikely symbol pays for the likely one's discount. The average still falls, because the expensive symbol is rare.
The price list in x264
x264 keeps these prices in a table, x264_cabac_entropy, in units of 1/256 bit: one pair of entries, likely value and unlikely value, for each of the 64 probability states of its arithmetic coder. At even odds both entries are 256, one bit. In the strongest state a probability estimate can adapt to, the likely value is priced at 7, which is 0.027 bits, and the unlikely one at 1,449, which is 5.66 bits. The entries are estimates rounded to 1/256 bit.
The table is used to estimate bits before coding. The bits themselves come from the coder, which Arithmetic coding covers; the page after it compares it with variable-length codes.
What to carry forward
- An ideal cost belongs to a probability, not to a value. The same zero is worth 1 bit at 50% and 0.014 bits at 99%.
- The probability is the coder's assumption. By the same formula, coding the cat's flags as if 90% were zero would average 1.13 bits per flag, more than one bit each.
- A fraction of a bit cannot be written on its own. Getting below one bit per symbol takes either a codeword shared by several symbols, as in variable-length codes, or no codewords at all, as in arithmetic coding.
- An entropy figure is a bound for its model only. These figures treat the flags as independent. Context modeling uses a different model and gets a lower figure for the same flags.
Sources
Read 2026-11-14 and 2026-11-15, at the commit linked.
- x264 at commit c24e06c (GitHub mirror):
x264_cabac_entropy, its even-odds entries, its strongest states, the 1/256 scaling,FIX8, the transition table, in which the first state pair never changes and no other state leads to it. - The entropy bound is Shannon's, cited on Four kinds of redundancy, where sections 6, 7 and 9 of the 1948 paper were read as extracted text. It was not read again for this page.
- Not checked: how the table's figures were derived, or the H.264 specification's own probabilities; x265, libaom and SVT-AV1 price tables, which the estimating page takes up for x265 and libaom.