Suppose a program processes several sentences together. The sentences have different lengths, so the shorter ones receive extra padding positions to fit the chosen batch layout. Those positions are bookkeeping, not additional words in the sentence. A mask tells the attention operation which positions belong in its calculation. Hugging Face's glossary demonstrates this with a tokenizer that marks attended positions with 1 and padding with 0 [3]. The numbers have a defined meaning in that particular interface.

Now imagine moving the calculation to a different library function. Copying the same mask can change which positions are allowed, even when the words and the mask's shape stay the same. One interface uses True to mean include this position; another uses True to mean ignore it. A float mask can mean something different again. We will examine one small calculation rather than a complete language model. Its values are invented and easy to recompute. The purpose is to understand the mask's contract before trusting a result that looks plausible. The specific API comparison below uses PyTorch 2.15 documentation, not an unversioned promise about every implementation.

Padding makes variable-length inputs compatible with a chosen batch representation, but padded positions should not automatically participate in the intended attention calculation. Hugging Face's documented BertTokenizer example emits a binary attention_mask with 1 for attended positions and 0 for padding [3]. That field is an input contract, not a universal encoding shared by every attention API. Treating the name mask as a complete specification is therefore insufficient.

This article follows an original single-row attention calculation through three representations: an allow-oriented Boolean mask, an exclusion-oriented Boolean padding mask and an additive floating-point bias. It uses the retrieved PyTorch 2.15 definitions for scaled_dot_product_attention and MultiheadAttention [1][2]. The worked values do not come from trained weights, token embeddings or a measured model response. They isolate a correctness boundary that can otherwise be hidden by a fluent final answer: the same stored bits can implement opposite logical policies when moved between named API arguments. A valid migration needs the meaning, dtype and shape contract, rather than a cast that merely makes the function accept the tensor.

Attention starts by comparing a query with keys associated with available positions. For our purposes, the comparison produces one score per position. Those scores are scaled and adjusted as required, then a softmax operation turns them into nonnegative weights that sum to one. The weights combine the corresponding values. PyTorch's reference formulation shows that order explicitly: calculate scores, add the attention bias, apply softmax, then multiply by the values [1]. A mask can change which positions receive weight before the final combination.

To make the calculation visible, imagine one query and four scalar values: [2, 4, 6, 0]. A scalar is just one number; real attention values normally have more coordinates. Assume all four already-scaled scores are zero and that dropout is disabled. Equal scores receive equal softmax weights. None of these numbers represents a word or a factual confidence level. We have chosen them so the effect of including or excluding the fourth position is easy to see. The example examines this one attention row, not the output of a full transformer with learned projections and later processing.

For one query row, scaled dot-product attention computes scores from QK-transpose, applies the relevant scale and attention bias, normalizes with softmax and forms a weighted combination of V. The PyTorch reference formulation makes the additive bias precede normalization [1]. Under our assumptions, the output is the sum over positions j of softmax(score+bias)_j times V_j. We consider a single scalar value coordinate and set dropout to zero so that the normalization is inspectable.

Choose already-scaled scores s=[0,0,0,0] and values v=[2,4,6,0]. These are invented analytical inputs, not a recorded model layer. Equal scores imply weights of one quarter when every position remains available. The toy deliberately separates the score vector from the value vector: a zero value does not imply an excluded score. MultiheadAttention additionally has its own learned projection and combination structure [2], so the scalar result below must not be presented as the full module's output for arbitrary embeddings. Its role is narrower: it exposes the effect of the mask at the attention normalization boundary.

With all four positions included, the equal weights are one quarter each. The output is (2 + 4 + 6 + 0) divided by 4, which equals 3. The fourth value contributes zero to the sum, but its position still takes one quarter of the available weight. If that fourth position is padding and should be excluded, the three real values instead receive one third each. Their output is (2 + 4 + 6) divided by 3, which equals 4.

This is why setting a padded value to zero is not the same operation as excluding the position. The zero can leave the normalization denominator changed. The example follows the documented softmax-before-value-combination order [1]; it does not claim that every padded position in a real network has a zero value. Nor does it predict a particular change to a chatbot's final sentence. It establishes a smaller, useful fact: an apparently harmless placeholder can alter the numerical attention result unless the intended exclusion is represented correctly. A test that checks only whether padding vectors are zero can miss that distinction.

Without exclusion, softmax([0,0,0,0])=[1/4,1/4,1/4,1/4], so the scalar output is 3. Excluding the fourth score from support gives weights [1/3,1/3,1/3,0] and output 4. The padded value's zero removes its numerator contribution to the weighted sum but does not, by itself, remove the exp(0) term from the softmax denominator. The mask changes the support of the normalized distribution, not merely the values being combined [1].

The example demonstrates a mechanism rather than a universal padding representation. Real projection layers, biases and positional handling need their own analysis; we have stipulated the values and scores. A useful reference check therefore compares the intended allowed positions and the resulting attention row, not only the input value at a pad index. It should also distinguish zero score from excluded score: a zero logit can receive substantial normalized weight. This distinction is especially useful when a refactor returns finite, well-shaped tensors that are nevertheless numerically different because an unwanted position still participates in normalization.

For the four-position example, the intended rule is simple: use positions one, two and three; ignore position four. In PyTorch 2.15's scaled_dot_product_attention, a Boolean attn_mask uses True for a position that may take part. The logical row is therefore [True, True, True, False]. The same documentation explicitly contrasts this with MultiheadAttention's key_padding_mask, where True marks a key to ignore [1][2]. For that argument, the corresponding padding row is [False, False, False, True].

The two rows express the same intention through opposite conventions. Sending the second row unchanged to the first function would express the opposite intention in this toy: only the fourth position remains available, giving output 0 instead of 4. This is a meaning error, not a subtle rounding difference. Do not apply an inversion simply because a tensor is called a mask, however. First identify the exact destination argument and its definition. A correct conversion starts with the intended allowed positions, then encodes them for the destination, while also satisfying that argument's separate shape requirements.

The named PyTorch 2.15 contracts have opposite Boolean polarity. Functional scaled_dot_product_attention interprets attn_mask=True as participation, whereas MultiheadAttention interprets key_padding_mask=True as exclusion [1][2]. For our logical allowed set {1,2,3}, an SDPA allow row is [T,T,T,F], while an MHA key-padding row is [F,F,F,T]. The documentation explicitly warns about this difference. MultiheadAttention's Boolean attn_mask also uses its documented blocked-position convention; do not generalize from the parameter's shared spelling across APIs [2].

Passing the exclusion-oriented row directly as the SDPA allow row retains only the fourth toy value, yielding 0. The remedy is a semantic conversion, with dtype and shape handled deliberately, not a universal rule to negate every mask. In particular, bitwise operations on integer 0/1 arrays should not be confused with logical inversion of a Boolean condition. Define a clear allowed-position predicate first, then encode its truth values in the destination convention. Even after polarity is correct, the MHA wrapper and raw SDPA function are not interchangeable complete layers: their projection and interface responsibilities remain distinct.

A floating-point mask in the documented scaled-dot-product function is added to the attention scores [1]. That makes a numeric 0 different from a Boolean False. Adding zero leaves a score unchanged; it does not exclude the position. In the reference formulation, exclusion is represented by negative infinity before softmax, while allowed positions receive no added bias. This describes the named operation's mathematical convention, not an instruction to copy arbitrary masks between unrelated libraries.

Return to the desired row of three allowed positions and one padding position. For this additive convention, [0, 0, 0, negative infinity] represents the intended exclusion. Simply converting [True, True, True, False] to [1.0, 1.0, 1.0, 0.0] does something else: it raises the first three scores and leaves the padding score present. The next calculation shows the result. The useful question is therefore not only what numbers are in the mask. Ask what operation the receiving function performs with them. A value that looks like an off switch in one representation can be an ordinary numerical bias in another.

The documented SDPA float-mask path adds the tensor to the score bias, while the Boolean path marks disallowed positions with negative infinity [1]. Thus Boolean participation and additive score modification are different contracts. An additive bias of zero preserves a score; a finite negative bias reduces relative weight but does not make that position mathematically absent from softmax support. The exact dtype and supported shapes still follow the API definition.

For the toy row, the intended additive bias is [0,0,0,-infinity]. Casting the allow-oriented Boolean mask to floating point instead yields [1,1,1,0], which is a finite preference rather than exclusion. This error can survive a shape check and produce plausible finite output. It also illustrates why a 2D tokenizer mask should not be treated as a ready-made low-level additive bias merely because its numerical values are available [3]. The high-level model may perform its own conversion. When crossing that abstraction boundary, inspect the destination's semantic contract and translate the logical allowed set instead of assuming a dtype cast preserves the operation.

Use the same zero scores and values [2, 4, 6, 0]. Adding the mistaken float mask [1, 1, 1, 0] produces scores [1, 1, 1, 0]. Softmax gives each of the first three positions a weight of e divided by (3e + 1), where e is approximately 2.718. The padding position gets 1 divided by (3e + 1), or about 10.9 percent. It has not disappeared.

The resulting weighted value is 12e divided by (3e + 1), approximately 3.563. That differs from both the correctly masked value 4 and the completely unmasked value 3. These three results come from changing only the stated mask rule in an invented example. They are not measured chatbot scores or percentages of factual accuracy. The calculation makes a debugging problem visible: the wrong mask can move an output in a believable direction without implementing the intended exclusion. PyTorch's documented additive-bias and softmax ordering supplies the mechanism [1]; the numbers and comparison here are original teaching calculations.

For s=[0,0,0,0] and incorrect float bias b=[1,1,1,0], the exponentiated scores are [e,e,e,1]. Their normalization constant is 3e+1. Consequently the padding weight is 1/(3e+1)≈0.109232 and the scalar output is (2e+4e+6e)/(3e+1)≈3.563073. Correct exclusion instead gives 4. The calculation follows the reference order of additive bias then softmax then value aggregation [1], with dropout absent throughout.

This example isolates a semantic error from numerical tolerance. A roughly 0.437 difference from the intended output is not explained by an unspecified optimized kernel when the declared mathematical operation itself differs. Conversely, the specific magnitude is not a prediction for another query, score vector or value dimension. Keep the assumptions attached to the result. The checked calculation provides a small reference case for reasoning about a future implementation, but no PyTorch model or backend was executed for this article. A real migration should exercise its installed version with controlled tensors and compare the intended operation under a defined tolerance.

Padding asks whether a position belongs to the actual sequence. A causal restriction asks whether a query may use a later position. A real token can be disallowed because it lies in the future, even though it is not padding. PyTorch's square causal-mask example uses a lower-triangular allowed pattern [1]. These are separate reasons for excluding a position, and a complete intended rule may need both.

For an invented four-position sequence, suppose the first three positions are real and the fourth is padding. A query at the second position may use positions one and two in a simple causal self-attention pattern. It may not use the real third position yet, and it may not use the padded fourth. The logical allowed row is [True, True, False, False]. Under our equal-score values, its output would be (2 + 4) divided by 2, or 3. This happens to equal the earlier unmasked four-position result, but for a different reason. Equal outputs in one small example do not prove the masks implement the same policy.

A padding predicate identifies valid source positions, whereas a causal predicate constrains each query-key relationship by sequence order. Their logical intersection describes an intended combined allowed pattern in the simple square self-attention case. PyTorch's SDPA reference uses a lower-triangular causal mask for that case and documents separate behavior for nonsquare alignment [1]. Do not extend a square drawing to cached decoding with unequal query and key lengths without checking the actual contract.

With valid positions 1–3 and padded position 4, the second query's simple causal allowed set is {1,2}. Our equal-score scalar values then yield 3. The coincidence with the fully unmasked four-position average is useful: output equality for one input is not a proof of policy equivalence. Compare the allowed relationships themselves and use cases that distinguish the intended rules. The logical intersection is an explanatory construction, not permission to supply arbitrary combinations of explicit masks and causal flags. The retrieved SDPA documentation states constraints on those arguments [1]; the installed runtime's supported path must govern an actual implementation.

A mask needs to line up with the positions it describes. In the low-level operation, one axis identifies queries and another identifies the keys they can use. If several sequences or attention heads are processed together, additional axes may be involved. PyTorch documents the shapes each argument accepts and how some masks are shared across other dimensions [1][2]. Sharing a mask across an axis is useful only when that is the intended rule.

For a small sketch, draw four columns for key positions and one row for each query. Mark allowed cells before converting anything into True, False or numeric bias. Then ask which parts may legitimately be shared across a batch. Two sentences with different padding lengths may need different padding information. A pattern describing future-token access may vary across query rows. This drawing does not replace API documentation, but it exposes a transpose or accidental sharing error more clearly than a tensor's total element count. A mask can have an acceptable shape and still encode the wrong sentence, query position or convention.

SDPA documents an attention-weight shape ending in query length L and key length S, with batch and head dimensions preceding them. Its mask must broadcast to that shape [1]. MultiheadAttention documents different argument contracts: a batched key_padding_mask has shape (N,S), while its attn_mask can be (L,S) or (N×num_heads,L,S) [2]. These shapes belong to named interfaces; their similar-looking dimensions should not be freely substituted.

Begin with an explicit logical relation allowed[batch,head,query,key], even if the implementation later represents shared dimensions compactly. Decide which axes genuinely share a rule, then encode the allowed relation using the supported broadcast pattern. Check a batch whose members have different valid lengths so unintended sharing is visible. Also distinguish key exclusion from any separate handling of padded query outputs: a key-padding argument does not by its name establish every downstream output convention. The proposed inspection is a reasoning aid, not a claim that any constructed tensor was run here. Retain both semantic orientation and axis labels in the implementation record.

For a concrete batch check, let the first member have three valid keys and one padded key, while the second member has four valid keys. Reusing the first member’s padding row across both would incorrectly exclude the second member’s fourth key. The shared shape can conceal this semantic mismatch. Conversely, a shared causal pattern may be appropriate when the query-key ordering matches, while the per-member padding information still differs. This is an invented relationship check, not a claim about the behavior of a tested batching system.

A useful comparison changes one thing at a time. Keep the example scores and values fixed, then compare the intended allowed positions and their encoded masks. Disable dropout for a deterministic arithmetic reference. PyTorch's functional attention documentation says dropout follows its explicit probability argument, including during evaluation unless the caller supplies zero [1]. A surrounding evaluation setting should not be assumed to override that function argument automatically.

There is also a boundary case: what should happen if a row excludes every position? Our ordinary softmax calculation has no remaining allowed weight to normalize. A real implementation needs documented handling or a guard for that case; this article does not claim every backend returns the same value. Finally, PyTorch notes that optimized floating-point implementations can differ numerically [1]. Use an appropriate comparison tolerance for the installed setup, while still checking the logical mask exactly. Small numerical differences and reversed inclusion rules are different problems. Neither should be used as an excuse to ignore the other.

For a controlled mask check, hold Q, K, V, scale, dtype and intended layout fixed, and set SDPA dropout_p=0 explicitly. The functional documentation warns that it applies dropout according to that argument [1]. Otherwise a stochastic change can obscure the semantic comparison. The reference calculations here omit projections and dropout deliberately; they establish expected scalar mathematics, not a full-module equivalence result.

An all-excluded row is another necessary boundary. In the naive mathematical softmax representation with every score set to negative infinity, each exponent is zero and the normalization denominator is zero. That observation does not establish the return behavior of every optimized backend; an application should use supported handling and exercise the relevant case. PyTorch also documents numerical differences between kernels and possible nondeterminism under some configurations [1]. A real verification record therefore needs the installed version, backend, precision and tolerance, together with exact allowed-position checks. A single plausible output is weak evidence when shape acceptance, semantic correctness and floating-point reproducibility are separate properties.

A useful handoff begins with a sentence: these are the positions each query is allowed to use. Next record the named API and argument, the data type and the shape. Only then write down the actual mask values. The documented PyTorch and tokenizer examples show why all of those details matter [1][2][3]. A file called attention mask does not preserve enough meaning by itself, especially when somebody later changes the implementation.

Our four-position example leaves several checks that fit on paper. Including the zero-valued padding gives 3; excluding it gives 4; reversing the Boolean convention leaves only padding and gives 0; using the mistaken float bias gives about 3.563. Each result follows from a different declared operation. Keep these examples labelled as invented when using them in a discussion or review. For a real change, reproduce the intended relation in the installed system and record the observed result. The practical skill is reading a mask as a rule about relationships, not assuming its name or familiar zeros and ones make that rule obvious.

The durable artifact is a mask contract: logical allowed relation, exact function and parameter, dtype, shape/broadcast rules, version and any causal or padding assumptions. Pair that contract with a small discriminating example and separately recorded runtime evidence. The primary documentation demonstrates opposite Boolean polarity and additive float semantics [1][2], while the tokenizer example supplies another high-level convention [3]. None can safely be inferred from the word mask alone.

The original arithmetic gives four distinct reference outputs: 3 for all four equal-score positions, 4 for exclusion of the fourth, 0 for the inverted SDPA inclusion rule and approximately 3.563073 for the incorrect float cast. The second-query causal example also gives 3, reminding reviewers that output coincidence does not establish equal policies. These results were calculated analytically with an ordinary numerical check, not obtained from a trained network. Preserve that distinction in future reuse. A successful API migration must keep the intended attention relationships while satisfying the destination's representation contract; plausible text generation or a passing shape check cannot establish that by itself.