A training example contains a question and an answer. Should the model receive a training penalty for predicting the question's words, the answer's words, or both? That choice changes what the reported loss summarizes. It does not require removing the question from the example. The model can still read the question as context while the training objective scores only its assistant reply. TRL's supervised fine-tuning documentation provides an assistant-only loss option for conversational data and a separate completion-only option for prompt-completion data. [1]

The distinction is easy to miss when a dashboard shows one small loss number. We will use two invented probability tables for the same short sequence. One table looks better when all six target tokens are averaged; the other looks better when only the two answer targets are averaged. No training or model benchmark is performed here. The calculation uses ordinary unweighted next-token cross-entropy with natural logarithms and no label smoothing. Those assumptions let us isolate the effect of selecting targets. They also give a practical question to ask of any training log: which tokens contributed to this number, and what did the averaging rule count?

Instruction-tuning data can contain prompt context and assistant responses within the same causal sequence. Selecting supervised target positions defines which prediction errors enter the objective. It does not, by itself, remove prompt tokens from the forward input. TRL v0.23.1 documents assistant_only_loss for conversational data and completion_only_loss for prompt-completion data, including their combination [1]. The distinction concerns the loss support, not the mere presence of text in a serialized conversation.

We analyze two stipulated next-token probability tables on the same sequence, under class-index cross-entropy with no class weights or label smoothing. Loss uses natural logarithms and a mean over selected targets. The examples are analytical; no tokenizer, training loop or model inference is executed. Our mechanism references retain their individual package versions and do not certify a compatible installation. With these boundaries fixed, switching from all six targets to two assistant targets reverses the ordering of the two tables by mean loss. The reason is a changed objective and denominator, which should be visible in the experiment record before a lower logged value is interpreted as a better assistant.

Picture a teacher reading a student's question and answer but grading only the answer. The question remains necessary for deciding whether the answer fits. In supervised language-model training, prompt tokens can similarly stay in the model's input while their own target positions are excluded from the loss. TRL's collator documentation shows input IDs, an attention mask and labels as separate outputs, and uses the label value -100 for targets excluded by the completion or assistant mask. [1] The labels decide what is scored; the input still supplies the conversation.

An attention mask answers another question: which available input positions can contribute through attention? A causal language model predicts from the permitted earlier context, so the prompt can influence the assistant response. [3][5] Setting an ignored target label does not automatically change that visibility. Nor does assistant-only loss identify a special set of assistant-only weights. Choosing which parameters are trainable is a separate configuration; the same trainer documentation discusses adapter training independently. [1] Keeping context, supervised targets and trainable parameters separate helps explain why an apparently small label change can alter the learning objective without deleting the prompt or selecting a different model architecture.

The forward inputs, attention policy and loss labels are separate objects. TRL's documented collator returns input_ids and labels separately, setting labels outside the requested assistant/completion region to -100 while preserving the corresponding input tokens [1]. The GPT-2 interface separately documents attention_mask and ignored language-model labels [5]. Therefore changing label support does not imply changing the attention graph or removing the prompt from the sequence.

In a causal forward pass, later assistant predictions can condition on permitted prompt representations [3]. Their losses can consequently propagate through computations that used that context. Ignoring a prompt position's own target term does not establish that every prompt-related activation or parameter receives zero gradient. Likewise, selecting assistant targets is independent of whether the optimizer updates the full model or an adapter subset; TRL describes PEFT adapter training as a separate integration [1]. A clear training configuration specifies all three choices: accessible context, scored targets and trainable parameters. Conflating them can lead to a model seeing less useful context than intended or to misleading assumptions about which weights a masking option will update.

Use this abstract seven-token sequence: BOS, P1, P2, P3, P4, A1, A2. BOS marks the beginning, P1 through P4 are prompt tokens, and A1 and A2 are assistant tokens. The prediction at BOS is compared with P1; the prediction at P1 is compared with P2; and so on. In particular, the prediction at P4 is compared with A1, while the prediction at A1 is compared with A2. There are six next-token targets. These symbols stand for token IDs, with no extra role markers, padding or end token in this small example.

If only the answer is scored, retain the A1 and A2 target terms. The prompt can still provide their earlier context. The model is not allowed to see A2 when predicting A2 simply because the full training sequence is stored in one array; causal attention restricts the visible history. [3] Some model interfaces perform the target shift internally. The cited Transformers GPT-2 interface explicitly documents that behavior. [5] Shifting labels manually and then using an interface that shifts them again would change which target matches each prediction. Inspect the actual input, labels and model contract together, rather than assuming that array positions alone reveal the intended prediction task.

Let x=[BOS,P1,P2,P3,P4,A1,A2], indexed 0…6. The six scored prediction/target pairs before masking are logits at positions 0…5 against x1…x6. Thus the logits at P4 predict A1, and the logits at A1 predict A2. Under assistant-only selection, those last two aligned terms remain. The construction contains no additional chat-control, end or padding tokens; it isolates the alignment convention rather than reproducing a real tokenizer's conversation format.

The versioned GPT-2 interface states that labels are shifted inside the model and that -100 labels are ignored [5]. Under that convention, an unshifted label vector[-100,-100,-100,-100,-100,A1,A2] becomes four ignored targets followed by A1,A2 after next-token alignment. Input_ids remains x. The causal context prevents the target's future position from being visible to its predicting position [3]. An additional manual shift would misalign the targets for that interface. Other models or custom loss functions can own shifting differently, so identify the responsible component before inspecting the mask. A label-selection check that ignores alignment can accidentally discard the first assistant target or score a neighboring control token instead.

For the simple objective used here, a target token contributes minus the natural logarithm of the probability assigned to that token. Giving the required token probability 0.99 produces a small loss, about 0.01005. Probability 0.20 produces about 1.60944. The penalty is larger when the required token receives less probability. PyTorch documents cross-entropy in terms of logits and the corresponding normalized class probabilities; our table starts from probabilities only to make the arithmetic visible. [2] These probability values should not be confused with the raw logits expected by that API.

Call the first invented table A. At each of the four prompt targets, it gives the correct token probability 0.99. At each of the two assistant targets, it gives the correct token probability 0.20. Table B gives the prompt targets probability 0.50 and the assistant targets probability 0.60. Their per-target losses are therefore approximately 0.01005 and 1.60944 for A, versus 0.69315 and 0.51083 for B. The tables assign probabilities to the same required targets. We have not generated text from them or supplied distributions for every vocabulary item; the assigned target probabilities are enough to calculate these particular negative-log terms.

With a class-index target y, unit class weight and no label smoothing, the per-target negative-log-likelihood is l=-ln p(y|prefix). PyTorch's CrossEntropyLoss computes the corresponding quantity from unnormalized logits and documents its equivalence to LogSoftmax followed by NLLLoss [2]. We stipulate normalized probabilities for the required class to expose the loss calculation. Passing such probabilities directly as the logits argument would define a different computation; the table is not an API-input example.

For probability table A, p(correct)=0.99 at each of four prompt targets and 0.20 at each of two assistant targets. For table B, the corresponding values are 0.50 and 0.60. Therefore the prompt/assistant loss pair is approximately(0.0100503,1.6094379) for A and(0.6931472,0.5108256) for B. Each specified probability is feasible as one component of a categorical distribution, with remaining mass assigned to other classes. We do not infer top-one accuracy from these values or claim they came from trained models. They are sufficient for the selected-target loss terms and allow a clean comparison of how supervision support changes the aggregate.

Average all six target terms for A. Four small prompt losses plus two larger answer losses total about 3.25908; divide by six to obtain 0.54318. For B, the corresponding total is about 3.79424, giving a mean of 0.63237. Under that all-target average, A has the lower loss. Its very strong probabilities on the four prompt targets outweigh its weaker probabilities on the two answer targets. Nothing about the text generation has changed; we are still evaluating the same two invented tables.

Now score only A1 and A2. A's mean is 1.60944, while B's is 0.51083. B has the lower assistant-target loss. The ordering reverses because the objective now focuses on the reply instead of including prompt reconstruction. This does not establish that assistant-only training is always superior. It establishes that the two averages measure different selected prediction tasks. When comparing training runs, retain the masking rule and evaluation target set beside the loss. A lower number under one rule cannot automatically be interpreted as better performance on the answer tokens measured by another rule. The calculation shows exactly where that inference can fail.

For A, the all-target sum is 4(-ln0.99)+2(-ln0.20)=3.2590772, so its six-target mean is 0.5431795. For B, 4(-ln0.50)+2(-ln0.60)=3.7942400, giving 0.6323733. Thus L_all(A)<L_all(B). The prompt contribution dominates the difference because four prompt targets receive substantial weight in the unweighted six-position average.

Restrict support to the two assistant targets and divide by two. Then L_assistant(A)=-ln0.20=1.6094379 and L_assistant(B)=-ln0.60=0.5108256, so L_assistant(B)<L_assistant(A). The reversal occurs without changing either probability table. It demonstrates objective mismatch rather than an observed training effect or proof that one mask is universally preferable. TRL describes logged cross-entropy over non-masked tokens [1], making the masking and reduction definitions part of the meaning of that metric. For a controlled comparison, evaluate the same retained targets under the same reduction even if the training objectives differ, and separately assess the generated behavior that the application actually needs.

For table A, the two assistant losses add to about 3.21888. Dividing by the two scored targets gives the correct assistant mean, 1.60944. If someone instead replaces the four prompt losses with zero but still divides by all six positions, the result is about 0.53648. That much smaller number is not the same mean. Its denominator counts four positions that supplied no loss. PyTorch's class-index ignore_index behavior averages over non-ignored targets in this unweighted case. [2] The averaging rule is as important as which terms were removed.

The value -100 is an instruction for the target-handling convention, not a token the model should produce and not a request to remove a vocabulary class from every prediction. The remaining assistant targets still compare their required token against the model's output distribution. [2][5] Also count how many valid targets remain after formatting and any length processing. If none remain, the selected-target average has a zero denominator. That is a data or configuration case to handle explicitly; it is not evidence of a perfect answer with zero error. A visible valid-target count makes that problem easier to detect than a loss value shown alone.

For A, assistant-target loss sum is 2(-ln 0.20)=3.2188758. The selected-target mean divides by two and yields 1.6094379. Zeroing the four excluded prompt terms but dividing by six yields 0.5364793, a different normalization. PyTorch's class-index ignore_index convention excludes ignored targets from the unweighted mean denominator [2]. When class weights are supplied, the documented denominator is weighted; our unit-weight example should not be copied unchanged into that setting.

The ignore sentinel selects target terms. It does not remove an output class from the softmax normalization at other supervised positions, nor is it a token ID to insert into input_ids. The GPT-2 label convention and TRL collator documentation keep that distinction explicit [1][5]. A sample with no active aligned targets provides no defined selected-target mean under the simple formula, so verify valid-target counts and handle such records before interpreting training statistics. This is a mathematical zero-denominator condition, not a claim about a particular library's returned exceptional value. Check target support after the actual preprocessing path, including alignment and any truncation that could eliminate the intended response.

Consider another small example using two conversations. The first has one scored answer token with loss 1.0. The second has three scored answer tokens, each with loss 0.2. Pool all four token losses and the mean is (1.0 + 0.2 + 0.2 + 0.2) divided by four, or 0.4. First average within each conversation and then average those two values, and the result is (1.0 + 0.2) divided by two, or 0.6. The same errors produced different summaries because the conversations received different weights.

Under the pooled token mean, the longer answer contributes three of the four terms. Under the mean of conversation means, each conversation gets half the total weight. Neither label should be shortened to simply average loss when the distinction matters. A training setup can deliberately choose a weighting scheme, but its evaluation and logging need to describe it. PyTorch offers mean, sum and unreduced outputs, while TRL describes its logged loss over non-masked tokens. [1][2] Keep the reduction boundary clear when combining batches or comparing runs with different answer lengths. A changed mix of short and long answers can change the aggregate even when the individual examples have not improved.

Let example one have a single active target with loss 1.0 and example two have three active targets with loss 0.2 each. The global token mean is (1.0+3×0.2)/4=0.4. The equally weighted mean of per-example means is (1.0+0.2)/2=0.6. In the former, the examples receive weights one-quarter and three-quarters; in the latter, each receives one-half. This original calculation separates the normalization choice from the target mask itself.

The documented CrossEntropyLoss reductions operate on the provided loss elements, and the unweighted class-index mean excludes ignored targets [2]. A trainer can introduce additional aggregation, sample weighting or accumulation behavior. Do not infer its global objective solely from the word mean or from one local batch value. TRL's token-based metric description identifies the intended logged population, but implementation/version and aggregation boundaries still belong in a reproducible record [1]. When batches contain different active-token counts, an unweighted average of batch means need not equal the pooled token mean. Compare sums and valid counts over the same evaluation population, or explicitly state the alternative weighting being used.

Imagine a conversation with four messages: user question U1, assistant reply R1, user follow-up U2 and assistant reply R2. Suppose a training example declares the first three messages as its prompt and R2 as its completion. Selecting all assistant replies in the whole conversation can include both R1 and R2. Selecting only the declared completion includes R2. Applying both selections keeps the assistant tokens that are also inside the completion, which in this example are the tokens of R2. The exact retained token boundaries still depend on the conversation format.

The distinction explains why a role name and a dataset boundary are not interchangeable. R1 is an assistant message, but it is also part of the later prompt. It can remain useful context even when the objective scores only R2. TRL v0.23.1 documents conversational assistant-only training, prompt-completion training and their compatibility. [1] Before choosing either option, identify the intended learning example: are earlier assistant turns also targets, or are they only the history for the final reply? Inspect the masks on an actual multi-turn record. A one-question, one-answer sample may look correct under several policies and fail to reveal the difference.

Define a four-message record U1,R1,U2,R2, where U denotes user content and R assistant content. If the full conversation is treated as a language-modeling example, an assistant-role selection can retain targets in R1 and R2. Now define a prompt-completion partition with prompt=(U1,R1,U2) and completion=R2. Completion-only selection retains the latter region; intersecting it with the assistant selection also retains R2. These statements refer to declared content regions, with actual control-token inclusion determined by the template and generated masks.

TRL v0.23.1 documents assistant_only_loss for conversational datasets and completion_only_loss for prompt-completion datasets, and explicitly supports their combination [1]. Its collator applies the available completion and assistant masks to labels. The earlier assistant message remains in input context even when its own targets are excluded by the completion boundary. A meaningful inspection therefore includes a multi-turn record with prior assistant history, rather than only a single user/assistant exchange where the policies may coincide. Record the intended intersection and verify that the final aligned labels match it; the role field alone does not define which portion of the dataset was designated as completion.

A chat template converts role-tagged messages into the tokenizable format expected by a model. Different models use different markers and arrangements. [4] That means an assistant mask must match the formatted tokens, not just a character range guessed from the original message text. The cited TRL version requires a template capable of returning an assistant-token mask through its generation markers. [1] A setting named assistant-only cannot establish that a particular template marks every intended reply correctly; inspecting a small prepared batch makes the boundary visible.

Look at the first and last scored token in each reply, whether earlier assistant turns are included, and whether padding or control markers have the intended labels. Also check the formatting path. The Transformers template guide warns about duplicating special tokens when formatted text is tokenized again; its documented approach disables additional special tokens in that separate tokenization step. For training on complete conversations, it usually recommends leaving the extra generation prompt off. [4] These checks catch different mistakes: wrong response boundaries, duplicated formatting and an unwanted start-of-next-reply marker. They are easier to diagnose before a long run than from an unexpected loss curve afterward.

Template serialization establishes the actual token sequence on which assistant and completion masks operate. Transformers v4.49.0 documents model-specific chat formats, while TRL v0.23.1 requires assistant-mask support through the generation/endgeneration template markers for its assistant-only feature [1][4]. Inspect produced input_ids, labels and relevant masks after formatting, tokenization and collation. Verify the intended first response token, end/control-token handling, earlier assistant turns and padding exclusion at the aligned loss boundary.

The template guide also distinguishes direct tokenized output from formatting to text and tokenizing later. In the latter path, it specifies add_special_tokens=False to avoid adding special tokens already supplied by the template; it usually recommends add_generation_prompt=False for training complete conversations [4]. These are documented paths, not interchangeable flags to apply without regard to the selected model. Preserve the template revision and the prepared example with the experiment configuration. If the mask is empty, shifted or attached to a differently serialized conversation, changing the optimizer cannot repair the intended supervision boundary. Resolve the representation and label agreement first, then interpret training metrics under that verified contract.

A useful record states which targets are scored, how they are aligned with predictions, how their losses are weighted and averaged, and how many valid targets remain. Save the data format, template, tokenizer and model versions alongside the trainable-parameter choice. Then evaluate comparable runs on the same chosen targets and weighting rule. TRL's documentation distinguishes loss, token accuracy and other logged quantities, so keep their definitions rather than treating them as different names for answer quality. [1] A next-token objective measures probabilities assigned to provided targets; it does not by itself show how a complete generated answer will behave.

There is also a difference between scoring a supplied answer and asking the model to write one. During the next-token calculation, each target is evaluated using the provided earlier tokens as context. During generation, the model produces its own earlier tokens. If those differ, the later context differs too. This follows from the next-token setup [3]. A loss on the supplied answer and an assessment of the freely generated answer therefore observe different paths. Keeping both results can reveal a problem that either measurement alone leaves hidden.

The two invented tables leave a memorable check. A has all-target mean loss about 0.543, below B's 0.632, but its answer-only mean is about 1.609, above B's 0.511. The contradiction disappears once the graded tokens are named. Use that habit when reading a real training result: identify the population behind the number before deciding what improved. Generated answers still need task-appropriate evaluation, including whether they follow the question and preserve relevant conditions. Loss selection defines the learning signal. Clear target boundaries and honest evaluation make that signal interpretable instead of letting one smaller number stand in for every property of the assistant.

An interpretable experiment records target support, shift ownership, reduction and weighting, valid-target counts, dataset partition, template/tokenizer/model revisions and trainable parameter scope. When comparing runs trained with different objectives, evaluate a common declared target population with the same normalization. TRL describes its cross-entropy and token-accuracy metrics over non-masked tokens [1]; those metrics are evidence about their defined prediction task, not substitutes for end-to-end generation evaluation.

Teacher-forced next-token loss conditions each answer target on the provided preceding answer tokens. Free generation instead supplies the model’s own preceding outputs. Once an earlier generated token differs, the later conditional context can differ from the one used for the recorded target loss. This is a consequence of the next-token conditioning described in the task guide [3], not a claim that every deviation must compound. A controlled target-loss comparison and a generated-response evaluation should therefore preserve their separate protocols. Report the common evaluation targets and normalization for the former, and the generation settings and task judgments for the latter.

Our fixed probability tables demonstrate the objective change precisely: L_all(A)=0.5431795<L_all(B)=0.6323733, while L_assistant(A)=1.6094379>L_assistant(B)=0.5108256. The separate normalization example yields 0.4 for the pooled token mean and 0.6 for equal example weighting. These are exact consequences of the stated masks, probabilities and reductions. They do not predict which training policy will produce the most useful model. A subsequent evaluation should assess generated responses on the application's tasks and preserve the same provenance. Naming the supervised positions and denominator turns a loss curve into a result that can be compared and investigated, while leaving broader behavior to measurements designed to evaluate it.