An AI model can store a value using a small integer while interpreting that integer through a scale and an offset. If you multiply the stored integers as though they were the original numbers, even a perfectly representable example can produce the wrong answer. The problem can appear before any discussion of model accuracy or rounding error.

The ONNX Runtime guide describes an affine relationship between an integer code and its represented value: subtract the zero point, then multiply by the scale [4]. ONNX also documents an integer matrix-product operator with explicit input zero points [1]. Those extra values participate in the meaning of the calculation; they are not optional labels for someone reading a chart.

We will follow three numbers through two equivalent weight encodings. Every chosen input lies exactly on its stated grid. That lets us isolate the arithmetic of offsets and scales, then identify where rounding, output conversion and real implementation limits would enter. The calculations are original teaching examples, checked with exact rational arithmetic, rather than measurements from a deployed model.

Separate a stored code q from its reconstructed real value x̂=s(q−z), where s>0 is a scale and z is the integer code assigned to zero [3][4]. An integer dot product over raw q values generally differs from a dot product over the reconstructed values. The distinction remains even when the original inputs are exactly representable.

The documentation scope here is ONNX MatMulInteger version 10 and QuantizeLinear/DequantizeLinear version 13, alongside the retrieved ONNX Runtime overview [1][2][3][4]. The operator pages contain multiple revisions, so their selected contracts are identified explicitly. No behavior from newer blocked, float8 or other low-bit variants is assumed.

Our original example uses scalar scales, uint8 activation codes and int8 weight codes. We derive a centered integer accumulator, expand its correction terms and reconstruct the result. Python integers and Fraction checked the arithmetic; an ONNX graph, execution provider or model was not run. Storage dtype, intermediate arithmetic and output representation therefore remain separate statements throughout.

Imagine integer codes as numbered marks on a ruler. A scale of one half makes neighboring codes represent values half a unit apart. A zero point of 128 says that mark 128 represents zero. Mark 126 then represents −1, and mark 130 represents +1. The integer labels are positive in all three cases, while the represented values are negative, zero and positive.

The ruler analogy describes this example’s evenly spaced values. It does not mean that every quantization format uses the same rule. The selected ONNX linear operators document a scale, a zero point and their associated shapes and types [2][3]. Keep that complete description with the stored codes.

Converting an arbitrary source value onto such a grid may require rounding or limiting it to an endpoint. The documented version-13 quantizer uses nearest-even rounding for the scaled input and saturation to the output type’s range [2]. Our chosen numbers need neither adjustment, so the coming mismatch cannot be blamed on a difficult halfway case or an out-of-range input.

With s_x=1/2 and z_x=128, reconstruction gives x̂=(q_x−128)/2. Thus q_x=126,128,130 represents −1,0,1 respectively. The zero point is an integer code in the quantized representation, not an additional real-valued bias applied after reconstruction.

For the selected int8/uint8 QuantizeLinear-13 contract, write the conversion as q=clip(roundEven(x/s)+z), with clipping to the declared integer range. The source specifies nearest-even rounding of x/s, matching scale and zero-point shapes, and the same type for the zero point and output [2]. DequantizeLinear-13 then applies the affine reconstruction and preserves the input tensor’s shape [3].

The uint8 range is 0 through 255; int8 uses −128 through 127 in this contract [2]. These ranges constrain stored codes, not the sign of the reconstructed real value. All coordinates below satisfy their ranges and have integral x/s before the offset, so neither tie handling nor clipping affects our computed outputs.

Take an input list x=(−1, 0, 1) and a weight list w=(1, −1/2, 1/2). A dot product multiplies matching positions and adds the results. Using the original values gives (−1×1)+(0×−1/2)+(1×1/2)=−1/2. This is the reference answer for our small example.

Encode the inputs with scale 1/2 and zero point 128. Their stored codes are (126, 128, 130). Encode the weights with scale 1/4 and zero point 0. Their codes are (4, −2, 2). Reconstructing each list with its own rule returns exactly the values we started with.

The two scales differ, and that difference is intentional. Each input step represents half a unit; each weight step represents a quarter. A product of their centered codes will consequently carry a scale of one eighth. We have supplied every value needed for this calculation, including the sign and origin of each representation. Nothing depends on a hidden calibration set or an unspecified model layer.

Let x=(−1,0,1), w=(1,−1/2,1/2), s_x=1/2, z_x=128, s_w=1/4 and z_w=0. Exact encoding gives a=(126,128,130) in uint8 and b=(4,−2,2) in int8. The original real dot product is y=Σx_i w_i=−1/2.

Here x̂=x and ŵ=w coordinate by coordinate. This equality is a property of the selected inputs and grids, not a general dequantization guarantee. Subtracting the declared origins gives a−z_x=(−2,0,2) and b−z_w=(4,−2,2). Their integer products can be accumulated before applying the common factor s_xs_w=1/8.

We treat the operation as a one-row by one-column product with a summed dimension of three. Scalar zero points avoid additional broadcasting questions; MatMulInteger-10 also documents broader zero-point shapes, which this example does not exercise [1]. There is no bias term, activation function, output clipping or hidden rescaling in the reference dot product.

Multiply the stored codes directly. The first pair contributes 126×4=504, the second contributes 128×−2=−256, and the third contributes 130×2=260. Their sum is 508. These are ordinary, correct integer multiplications.

Multiplying 508 by the combined scale of 1/8 gives 63.5. That is far from the reference answer, −0.5. The multiplication itself did not fail. The calculation used the inputs’ code labels without first accounting for where their zero lies. A large positive code can stand for a small negative value when its origin is 128.

This is why adding the scales to a raw integer result is not enough. Both the spacing and the origin belong to the representation. Our example contains no lost source precision, so increasing the number of stored bits would not fix the particular mistake of omitting an already-declared zero point. The correction is to perform the intended affine calculation with the available metadata.

The uncentered integer sum is R=Σa_i b_i=126·4+128·(−2)+130·2=508. Scaling this raw sum yields s_xs_wR=508/8=63.5. The intended reconstructed-value product is −0.5, so the discrepancy is 64 under the declared inputs.

There is no contradiction with exact representability. R computes a different expression from Σ(a_i−z_x)(b_i−z_w). Scaling cannot remove an additive origin term that remains inside the product. This is a representation-interpretation error, rather than a rounding residual.

The example is useful when comparing intermediate records: the label “integer dot product” does not identify whether offsets were already applied. A stored raw sum, a centered accumulator and a scaled real output can all be numerically different while referring to related stages. They need explicit meanings before equality or error is assessed. In particular, an int32 storage label alone does not tell us which expression produced its contents.

Subtract 128 from the three input codes. They become (−2, 0, 2). The weights already use zero point 0, so their centered codes remain (4, −2, 2). Matching products are now −8, 0 and 4, which add to −4.

Restore the combined scale: −4×1/8=−1/2. That agrees exactly with the original dot product. Each part has a specific job. Subtracting the origins gives the correct signed displacement, multiplication combines those displacements, addition accumulates them, and the product of the scales supplies the result’s real-value spacing.

The chosen operator reference makes this distinction visible by accepting input zero points and specifying an int32 output [1]. Our arithmetic demonstrates the relationship without claiming that a particular runtime follows this sequence instruction by instruction. Implementations may organize equivalent work differently. Whatever their organization, an intermediate named as the centered accumulator must represent the same mathematical quantity if it is to support this reconstruction.

Define A=Σ(a_i−z_x)(b_i−z_w). For the declared codes, A=(−2)·4+0·(−2)+2·2=−4. Because both scales are constant over i, Σx̂_i ŵ_i=s_xs_wA=(1/8)(−4)=−1/2.

This factoring follows directly by substituting x̂_i=s_x(a_i−z_x) and ŵ_i=s_w(b_i−z_w) into the real-valued dot product [3][4]. It reconstructs the dot product of the represented inputs. Equality to the pre-quantization reference additionally uses the fixture’s x̂=x and ŵ=w.

MatMulInteger-10 specifies 8-bit integer input types, optional input zero points and an int32 output [1]. Our calculations use wider exact host integers as an oracle, and all stated intermediate values fit within int32. The arithmetic therefore does not test a backend’s overflow behavior, instruction fusion or accumulation order. In particular, subtracting 128 from a uint8 code is a signed mathematical operation here, not a claim that subtraction should wrap within uint8.

Replace the real input list with (0, 0, 0), keeping its scale and zero point unchanged. The correct stored codes are now (128, 128, 128). Subtracting the origin gives three zeros, so every centered product and their sum is zero.

The raw code calculation instead gives 128×4+128×−2+128×2=512. Multiplying by 1/8 produces 64. A calculation that returns 64 for this declared zero input has not preserved the intended meaning of the representation. The example reveals the same missing-origin term without needing a complicated reference result.

The integer code 0 is also available, but under this input rule it represents (0−128)×1/2=−64. Filling a code array with integer zeros would therefore not represent the real zero list used here. The ONNX Runtime guide highlights the role of a representable zero in operations such as zero padding [4]. Our small check illustrates that meaning; it is not a test of a particular network’s padding implementation.

For x=(0,0,0), the declared quantizer returns a=(128,128,128). With unchanged b=(4,−2,2), the centered accumulator is A=0. The raw sum is R=128·Σb_i=128·4=512, giving an incorrect scaled raw value of 64.

This check distinguishes a represented real zero from the integer bit pattern for zero. Under s_x=1/2,z_x=128, q_x=0 reconstructs to −64. Any padding or initialization intended to mean real zero must respect the encoding contract, rather than assuming that the numerically smallest unsigned code has that meaning [4].

The example does not prove that every zero-input response of a whole model should be zero. A real model can include biases and other operations. Our reference is the explicitly bias-free dot product, with all other processing excluded. That boundary makes the zero-input comparison a valid check of this local expression, while preventing it from being mistaken for a universal model-output requirement.

Keep the same real weights and their quarter-unit scale, but change their zero point from 0 to −3. The new codes are (1, −5, −1). Subtracting −3 from each code gives (4, −2, 2), exactly the centered weight list we used before.

The raw integer dot product changes. Using the original input codes, it becomes 126×1+128×−5+130×−1=−644. A different raw number is expected because we changed the weight-code labels. The represented weights have not changed, so the correctly interpreted real dot product must remain −1/2.

This is a useful consistency question: does a calculation depend on the represented values or accidentally on the arbitrary code origin? Our two encodings both stay within their stated int8 range. For these selected values, moving the code labels and their zero point together preserves the represented weights. The statement is conditional on preserving valid codes; it does not say that any origin shift will fit every possible weight into the same finite range.

Choose z′_w=−3 while holding s_w=1/4 and w fixed. The corresponding int8 codes are b′=(1,−5,−1)=b−3. Then b′−z′_w=b, so ŵ is unchanged. Centered accumulation still gives A=−4, and reconstruction still gives −1/2.

The raw sum is now R′=126·1+128·(−5)+130·(−1)=−644. Comparing R with R′ without their respective origins would compare different code-coordinate systems. Their unequal values do not establish unequal represented dot products.

More generally, shifting codes and zero point by the same integer c preserves q−z wherever both old and new codes satisfy the required type range. This is an algebraic invariance of the affine representation, not a claim about optimal calibration, representable range coverage or an operator’s preferred quantization scheme. The example deliberately changes only the weight origin, so the effect remains traceable through every term of the calculation.

You can obtain the centered sum by expanding the same multiplication. Start with the raw products, subtract the input origin times the sum of weight codes, subtract the weight origin times the sum of input codes, then add back the number of terms times both origins. The last addition is needed because multiplying two subtractions includes a product of their origins.

For the shifted-weight example, the four contributions are −644, +640, +1152 and −1152. They add to −4, the same centered accumulator found by subtracting before every multiplication. This route is a rearrangement of the calculation, not a new approximation.

With weight zero point 0, the last two contributions disappear. In our original example, only 508−512 remains. That can simplify the expression, but an algebraic simplification does not by itself establish a faster runtime. The actual work depends on how an implementation stores inputs, reuses sums, handles shapes and executes the operator.

For n terms, expand A=Σa_i b_i−z_xΣb_i−z_wΣa_i+nz_xz_w. With b′=(1,−5,−1), Σa_i=384, Σb′_i=−5, n=3 and z′_w=−3, the terms are −644, −128(−5)=640, −(−3)384=1152 and 3·128·(−3)=−1152. Their sum is −4.

The final two terms happen to cancel in this particular fixture because Σa_i=nz_x. They are not generally removable when z_w≠0. The expansion itself shows the dependence on the actual input-code sum; changing that sum changes the third term while the fixed n,z_x,z_w term remains unchanged.

Setting z_w=0 eliminates both terms algebraically, leaving A=Σa_i b_i−z_xΣb_i. For fixed weights and fixed input zero point, that correction is also fixed. If the input zero point changes dynamically, its value must change accordingly. These are identities under stated conditions, not measurements of a kernel or a universal claim that zero-centered weights always yield a specific speed advantage.

The centered accumulator is −4, while its reconstructed result is −1/2. Suppose the next stored output uses scale 1/4 and zero point 10. The code for −1/2 is then 8, because (8−10)×1/4=−1/2.

We now have three useful numbers: −4 for the centered accumulator, −0.5 for its represented real result, and 8 for a newly encoded output. None can be substituted for another without its interpretation rule. In particular, writing the accumulator directly into the output-code field would skip a change of scale and origin.

This output happens to lie exactly on the new grid and within the chosen unsigned range. A general output conversion can require rounding and saturation under its own documented rules [2]. The extra conversion is separate from the integer accumulation described by MatMulInteger [1]. Naming the destination representation makes it possible to decide whether a mismatch comes from the dot product, the reconstruction or the later encoding.

Choose an output encoding s_y=1/4,z_y=10 in uint8. The represented result is ŷ=s_xs_wA=−1/2. Exact re-encoding gives q_y=ŷ/s_y+z_y=(−1/2)/(1/4)+10=8, and reconstruction gives s_y(q_y−z_y)=−1/2.

Equivalently, q_y=(s_xs_w/s_y)A+z_y=(1/2)(−4)+10 for this exactly representable case. In the general integer-output conversion, apply the destination’s specified rounding and saturation as well [2]. This article does not assume a fixed-point multiplier implementation or a particular rescaling instruction sequence.

MatMulInteger’s int32 accumulator output should therefore not be silently described as already carrying a new uint8 output zero point [1]. DequantizeLinear-13 separately documents zero-point behavior for int32 reconstruction [3]. Our conceptual reconstruction uses scale 1/8 and zero origin for A; the later uint8 encoding uses scale 1/4 and origin 10. These are two different representation boundaries with different metadata.

The earlier factor of 1/8 could be applied after adding because every product used the same input scale and the same weight scale. If the scale changes between terms being added, each term may carry a different real-value weight.

Consider a separate two-term construction. Both centered integer products equal 1, but the first carries scale 1/8 and the second scale 1/4. Their real sum is 1/8+1/4=3/8. Adding the integer products first and applying only the first scale gives 2×1/8=1/4, which is a different result.

A statement such as “per-axis scales” therefore needs the axis and the operation. Scales can vary between separate output channels while remaining common across the terms inside each individual dot product. The selected ONNX linear operators document per-tensor and per-axis forms [2][3]. Our two-term construction explains why scale placement matters mathematically; it does not assert that every such arrangement is supported by the integer operator or a particular execution provider.

The identity ŷ=s_xs_wA assumes s_x and s_w do not vary over the contraction index. If instead product term i has scale factor c_i, the general expression is Σc_i(a_i−z_x,i)(b_i−z_w,i). A single factor may leave the sum only under an appropriate common-factor condition.

Take centered products t=(1,1) with c=(1/8,1/4). Then Σc_i t_i=3/8, while (1/8)Σt_i=1/4. There cannot be one fixed factor k reproducing the weighted sum for all t: t=(1,0) requires k=1/8, while t=(0,1) requires k=1/4. A factor fitted after seeing one particular t would not supply a common representation rule.

Per-output-channel variation does not automatically violate this condition; a scale may be constant within each contraction and differ between its outputs. The relevant dimension matters. The version-13 quantize/dequantize contracts describe axis-associated scale shapes [2][3], but this example is an algebraic boundary, not validation of any broader MatMulInteger quantization-granularity combination.

Our reference inputs survived encoding exactly. A correct centered calculation therefore recovered their dot product exactly. Real quantization can change values through rounding or saturation, so the reconstructed inputs may differ from the originals even when their codes are interpreted correctly [2][4].

These are different checks. First, verify that the arithmetic agrees with the represented inputs and their metadata. Then compare that represented-value result with the original-value reference. A failure in the first check points to interpretation or computation; a difference in the second can reflect the approximation introduced by the chosen representation.

For a complete model, even matching one local calculation does not establish unchanged outputs or task performance. The ONNX Runtime guide describes comparing corresponding weights and activations when investigating quantization differences [4]. Keep the reference, tensor identity and processing stage aligned. A local arithmetic example explains what to compare, but it cannot replace an evaluation of the actual model and the inputs that matter to its use.

Let x̂ and ŵ denote reconstructed inputs without assuming equality to x and w. Correct affine integer arithmetic establishes ŷ=Σx̂_iŵ_i. The remaining representation error relative to the original dot product is ŷ−y=Σ(x̂_iŵ_i−x_iw_i). This error is distinct from an implementation returning a value inconsistent with x̂ and ŵ.

The worked fixture has x̂=x and ŵ=w, so its representation error is zero. The incorrect scaled raw sum still differs by 64, identifying a separate semantic mistake. No conclusion about arbitrary tensor distributions, clipping frequencies or accumulated network error follows from that deliberately exact case.

The Runtime guide notes that quantization can affect accuracy and provides comparison methods for corresponding weights and activations [4]. Such comparisons require matching the same computational quantity and boundary; an integer code should not be directly compared with a real activation as though they shared units. Output conversion and later model operations introduce additional boundaries that must be identified before attributing the observed difference.

A useful record names the input codes, their integer types, their scales and zero points, the dimension being summed, the accumulator’s meaning and any output conversion. For our example, that record explains why 508, −4, −0.5 and 8 appear at different stages without treating them as interchangeable answers.

A graph’s representation also matters. ONNX Runtime distinguishes quantized operators from a pattern that inserts quantize and dequantize operations around tensors [4]. Seeing an 8-bit tensor somewhere in a graph does not reveal every intermediate dtype or show that all operations happen directly on raw 8-bit codes. Read the relevant operator contract and the surrounding conversions.

The record also needs to identify where its scales came from. The Runtime guide distinguishes parameters obtained using calibration inputs from parameters calculated dynamically [4]. For this article, the scales are declared constants, chosen to make the arithmetic visible. In another system, a code array and a scale copied from a different calculation may not describe the same represented values. Keep the matching metadata with the data.

The result of this article is a reproducible interpretation, not a speed or accuracy rating. Subtract the appropriate origins, preserve each term’s scale, identify the accumulated quantity and name the next representation. Those steps let a reader ask a sharper question about a real system: which precise quantity does this number represent, and which evidence shows that the implementation computed it?

The complete local contract includes code dtypes and shapes, scale/zero-point values and shapes, contraction index, centered-accumulator semantics, arithmetic range and any destination quantizer. In the declared fixture, raw sum R=508, centered A=−4, reconstructed ŷ=−1/2 and final q_y=8 describe four different stages.

ONNX Runtime’s QOperator and QDQ descriptions distinguish an operator-oriented graph from quantize/dequantize boundaries around tensors [4]. Neither a low-bit storage label nor the presence of one quantization node proves a whole-model arithmetic policy. Backend fusion, actual kernel selection, throughput and accuracy require their own observed evidence.

Parameter provenance matters as well. The Runtime guide describes static quantization parameters obtained using calibration inputs and dynamic activation parameters computed during execution [4]. This distinction concerns how s and z are obtained, rather than changing the affine reconstruction formula. If an activation zero point changes, a previously cached correction based on that zero point cannot be reused unchanged without additional justification. The same stored integer tensor paired with different metadata can denote different real values. Our scales and origins are declared constants, with no calibration procedure or claim that they are optimal for any learned distribution.

Retain the named specification versions and the exact test inputs when making a comparison. Our source-backed affine definitions support the original algebra, while the exact three-term fixture verifies its numbers. The general lesson is that precision, origin, spacing, accumulation and output encoding are separate parts of the contract. A meaningful mismatch report states which part was compared and keeps the corresponding representation metadata with the result.