A speech-recognition report shows a word error rate of 200 percent. That looks impossible if you read it as the percentage of reference words that are wrong. But word error rate, usually shortened to WER, counts a different quantity. It adds substitutions, deletions and insertions, then divides by the number of words in the reference transcript. Inserted words contribute errors without adding reference words to the denominator. The result can therefore exceed 100 percent. [1]

Our example reference is “open gate”. The invented hypothesis, meaning the transcript being scored, is “please open the blue gate now”. Both reference words appear in order, but four extra words have been inserted. Four errors divided by two reference words gives 2, written as 200 percent. Nothing about that arithmetic requires a broken calculator. We will use written transcript pairs to examine alignment, normalization, averaging and empty references. No audio was transcribed and no speech model was evaluated. The purpose is to make the score understandable before it is used to compare systems. A number becomes much easier to interpret once its reference, counting rule and preprocessing are visible alongside it.

Word error rate is a normalized edit count, not a bounded fraction of reference positions marked incorrect. With a nonempty reference, WER=(S+D+I)/N, where N is the reference-token count and S, D and I are substitution, deletion and insertion counts from the chosen word alignment [1]. Insertions increase the numerator without increasing N. Consequently, WER is not generally bounded above by one.

For the original pair reference “open gate” and hypothesis “please open the blue gate now”, a minimum unit-cost word alignment has two matches, four insertions and no substitutions or deletions. Thus N=2 and WER=4/2=2. This article computes explicit text examples with unit edit costs and declared tokenization. It does not report a speech-model run, an acoustic experiment or an installed scorer compatibility test. We use retained JiWER 4.0.0 source and documentation to distinguish particular implementation conventions from the nonempty-reference formula. The subsequent examples show why preprocessing, corpus aggregation, segment boundaries and silence handling belong to the measurement definition. Those choices can change a score while the underlying transcript strings remain unchanged, so they must be preserved when interpreting a claimed improvement.

The reference is the transcript you have chosen as the expected wording. The hypothesis is the transcript you want to compare with it. A substitution replaces a reference word with another word. A deletion leaves out a reference word. An insertion adds a hypothesis word without a matching reference word. Matching words are sometimes called hits. If we call the hit count H, then the number of reference words is H+S+D. Insertions are not part of that total. The JiWER implementation uses this reference-side denominator. [3]

The direction of comparison matters. If the reference says “open gate” and the hypothesis says only “open”, one of two reference words is missing, giving 50 percent. Reverse those strings and the reference now contains one word, with one extra word in the hypothesis, giving 100 percent. The edit count is one in both directions, but the denominator changes. This is why an evaluation file should label references and hypotheses explicitly. An accepted reference also needs to represent the task you intend to score. A transcript with different spelling or annotation conventions can create mismatches that belong to the comparison protocol rather than proving that the recognizer failed to hear a sound.

The denominator is reference length, equivalently N=H+S+D for the selected alignment. An inserted hypothesis token has no corresponding reference position, so it contributes to I but not N. The retained JiWER word-processing code accumulates these operation counts and, for a nonempty total reference, divides S+D+I by H+S+D [3]. This makes the direction of normalization explicit even though unit-cost Levenshtein distance itself is symmetric.

For reference [open,gate] and hypothesis [open], distance is one deletion and WER=1/2. Reversing the arguments gives one insertion over a one-token reference, so WER=1. Both examples use the same edit costs and token boundaries. Reversing labels therefore does not preserve the normalized result. Retain reference identity, annotation conventions and the argument mapping in an evaluation record. A reference is the declared target for this comparison; its presence does not prove that every annotation is acoustically correct or that another valid transcript would be identical. This distinction matters when evaluating formatting, numbers, names or alternative acceptable wording. The metric compares the supplied token sequences under the declared rules, rather than independently adjudicating the audio or validating the reference creation process.

Consider the invented reference “turn left at the next street” and hypothesis “turn right at next street now”. Each has six words, but equal length does not make the words line up position by position. A useful minimum-edit alignment matches “turn”, substitutes “right” for “left”, matches “at”, deletes “the”, matches “next street”, and inserts “now”. There are four matching words, one substitution, one deletion and one insertion. The total is three errors over six reference words, or 50 percent.

If you simply compare the fourth, fifth and sixth positions, the missing “the” shifts later words and makes the comparison misleading. Alignment allows the shared sequence to line up across the deletion. JiWER describes its measures as using minimum edit distance, and its retained source records matching, replacement, deletion and insertion spans. [2][3] Our example uses a separate local calculation with a cost of one for each edit. A record should retain the alignment or enough information to reproduce it, especially when error categories are used to diagnose a system. The total score alone does not show which word changed, which disappeared, or which extra phrase appeared. Those details are often the most useful part of investigating an actual transcript.

Use reference tokens [turn,left,at,the,next,street] and hypothesis tokens [turn,right,at,next,street,now]. Under unit substitution, deletion and insertion costs, a minimum alignment yields H=4, S=1, D=1 and I=1. The operations are left→right, deletion of the reference token the, and insertion of hypothesis token now. Hence N=H+S+D=6 and WER=3/6=0.5. A positional mismatch count would fail to account for the shift caused by deletion.

Our independently implemented dynamic-programming recurrence minimizes the number of unit edits, then reconstructs an alignment with a documented tie order. JiWER's primary documentation describes minimum-edit scoring, and its versioned source obtains Levenshtein opcodes before accumulating operation counts [2][3]. We did not import that source or claim identical tie-breaking for every possible sequence. Multiple optimal alignments can sometimes share a minimum total while differing in their detailed explanation. Retaining scorer version, edit conventions and alignment output is therefore useful when comparing substitution, deletion and insertion breakdowns. For this original pair, the displayed alignment directly accounts for every token and the independent minimum distance is three. The calculation concerns the stipulated text; it does not establish why a recognizer might produce it from audio.

Return to “open gate” and “please open the blue gate now”. Match “open” with “open” and “gate” with “gate”. The unmatched words are “please”, “the”, “blue” and “now”: four insertions. The hypothesis has six words, the reference has two, and at least four length-changing edits are required to bridge that difference. Four insertions achieve it, so the alignment is minimal. The result is four errors per two reference words, not four wrong reference positions.

Dividing by the six hypothesis words would instead produce 66.67 percent, but that is a different denominator and is not the WER definition used here. Clipping the result to 100 percent would also discard information about the extra output. Hugging Face's explanation explicitly notes that insertion errors can make WER exceed one. [1] Keep the raw count and denominator visible: I=4 and N=2 explain the 200 percent figure directly. A very high score can be a reason to inspect a transcript, reference or evaluation setup, but its magnitude alone does not identify the cause. Our invented pair demonstrates valid arithmetic; it does not diagnose hallucination, microphone quality, language mismatch or any particular speech model.

The original two-token reference and six-token hypothesis have a length difference of four. Under unit edit costs, every insertion or deletion changes sequence length by one and substitution leaves it unchanged, establishing a distance lower bound of four. Matching open and gate while inserting please, the, blue and now achieves that bound. Therefore H=2, I=4, S=D=0 and WER=2 exactly.

The denominator remains N=2. Substituting hypothesis length would yield 4/6, which is a different statistic; capping WER at one would alter the measure. The absence of a universal upper bound follows because arbitrarily many additional hypothesis tokens can be inserted while a nonempty reference length stays fixed [1]. This mathematical possibility should be separated from an empirical failure diagnosis. Large insertion counts might arise in many circumstances, and a text-only score cannot establish their acoustic or system cause. Retain the actual operation counts and inspect the paired material. In particular, do not reinterpret 1−WER as a probability guaranteed to lie between zero and one: for this example it is −1. A label containing the word rate does not by itself imply a bounded probability model.

Take the written reference “Turn left.” and hypothesis “turn left”. If our stated rule is to split on spaces and compare tokens exactly, “Turn” differs from “turn” and “left.” differs from “left”. There are two substitutions over two reference words, giving 100 percent WER. Now define another rule for this example: lowercase both strings and remove the full stop before splitting. Both become the same two words, so the score becomes zero. The recognizer did not improve between those calculations. We changed what the evaluation treats as equivalent.

Normalization can be useful when the intended question concerns words rather than capitalization or punctuation. It can also remove information that matters for a finished transcript. Hugging Face discusses evaluating normalized text while retaining formatted output for readers. [1] The choice should be declared before comparing systems and applied consistently to references and hypotheses. Do not assume a scorer silently makes those choices for you. JiWER 4.0.0's default word pipeline removes repeated spaces and surrounding spaces, then creates word lists; its separate standardization pipeline additionally specifies transformations such as lowercasing and contraction expansion. [4] Our full-stop removal is an explicit original rule, not a claim that either listed default automatically removes every punctuation mark.

Define raw tokenization for the original pair as space splitting with exact string equality. Reference [Turn,left.] versus hypothesis [turn,left] gives S=2, N=2 and WER=1. Define a second, explicitly chosen transform T that lowercases and removes the period in this pair before tokenization. Then T(reference)=T(hypothesis)=[turn,left], giving WER=0. Only the scoring representation changed; no model output was regenerated.

Hugging Face's course distinguishes orthographic output from normalized evaluation [1]. Preserve the original strings, transformation order and resulting tokens so a comparison can be reconstructed. The pinned JiWER 4.0.0 default pipeline is RemoveMultipleSpaces, Strip and ReduceToListOfListOfWords. Its separate standardization pipeline includes lowercasing, common-English contraction expansion and other named processing steps [4]. Neither list should be embellished with an assumed general punctuation-removal stage. Our constructed transform is separate from those library defaults. Normalization defines an equivalence policy and can also change token counts, so a score change is not automatically evidence of better acoustic recognition. Choose the policy for the evaluation purpose, record it for both sides, and evaluate output formatting separately when punctuation, casing or readability is part of the application requirement.

Imagine two recorded utterances. The first has two reference words and one error: 50 percent. The second has eighteen reference words and one error: about 5.56 percent. If you average those two percentages equally, you get approximately 27.78 percent. If you add their errors and reference words first, you get two errors divided by twenty words, or 10 percent. These results use the same two transcripts but answer differently weighted questions.

The JiWER implementation totals hits and edit counts across the paired sentences before computing its nonempty-reference WER. [3] This corresponds to the pooled 10 percent result in our example. It is also a reference-length-weighted average of the individual rates when all individual references are nonempty. A report calling 27.78 percent the mean utterance WER can still describe a useful quantity, but it should identify that averaging choice. Otherwise, a reader may compare it directly with a pooled corpus WER and mistake different weights for a recognizer improvement. Keep per-utterance word and error counts as well as the final number. They show whether a long recording or a collection of very short clips dominates the chosen summary.

Let the original per-utterance records be (N1,E1)=(2,1) and (N2,E2)=(18,1), where E=S+D+I. The unweighted mean of utterance rates is (1/2+1/18)/2=5/18≈0.27778. The pooled ratio is (E1+E2)/(N1+N2)=2/20=0.1. For nonempty individual references, the pooled ratio equals ΣNi·WERi/ΣNi, so it weights reference words rather than treating every utterance as equally influential.

The retained JiWER implementation accumulates counts across aligned pairs and applies the formula to their totals [3]. We use that source to identify its aggregation convention, not to claim every evaluation tool has the same reporting default. Our text pairs are “go now” versus “go”, and an eighteen-word reference containing blue versus the same sentence containing green. Each contributes exactly one edit. Retain pairing, reference lengths, error counts and any corpus exclusions. A macro average, a pooled ratio and a subgroup score can each be legitimate summaries of explicitly defined questions; silently exchanging them creates an uninterpretable comparison. Empty-reference handling needs additional care because an individual zero-length reference has no ordinary per-utterance denominator, even when the overall corpus denominator is positive.

An empty reference has no words. Directly inserting that zero into the ordinary WER denominator would require division by zero. A tool therefore needs a stated policy if it supports such cases. JiWER 4.0.0 defines zero error when both the reference and hypothesis are empty. When the reference is empty but the hypothesis contains words, it returns the insertion count in this all-empty-reference case. [2][3] Under that convention, an empty reference paired with “music playing” produces a value of two because both words are insertions.

This is a documented implementation rule, not a literal calculation of two divided by zero. It is also different from adding an empty-reference segment to a larger corpus with a positive reference-word total. If we add those two inserted words to our earlier twenty-word corpus, the total error count rises from two to four while the reference denominator remains twenty. The pooled result becomes 20 percent. Keeping silent segments can therefore expose unwanted output that would disappear if those segments were dropped. Our example uses an empty written reference; it does not establish that a real recording is silent or that every recognizer will produce the invented phrase. Those are separate facts that an audio evaluation must verify.

For an isolated empty reference, the ordinary (S+D+I)/N expression has N=0 and is undefined. JiWER 4.0.0 explicitly defines its all-empty-reference branch as WER=num_insertions; the README gives zero for two empty strings and insertion counts for nonempty hypotheses [2][3]. Thus our constructed empty reference with hypothesis [music,playing] yields the library-defined value 2. This is an edge convention, not a finite value obtained by evaluating 2/0.

The source branch tests total reference-word count across the processed pairs [3]. If the earlier corpus contributes N=20 and E=2, adding an empty-reference pair with two hypothesis tokens increases I by two while leaving total N positive. The ordinary aggregate branch then yields 4/20=0.2. Applying an arbitrary per-utterance zero-denominator replacement before averaging would describe another policy. Preserve the empty-reference convention, inclusion rules and scorer version in the report. The pinned implementation's operative branch and versioned README are the evidence used here; an older-looking exception description elsewhere should not be assumed to override them. No acoustic silence detection was performed. The example establishes how declared text pairs affect counts, while real silence labels still require a suitable reference-creation process.

Take two tiny invented segments. Their references are “red” and “blue”. Their hypotheses are an empty string and “red blue”. Scored as separate pairs, the first segment has one deletion. The second contains the correct “blue” plus an inserted “red”. The total is two errors over two reference words, or 100 percent. The words exist somewhere in the combined output, but one appears in the wrong segment.

Now join each side into a single string before scoring. The combined reference is “red blue” and the combined hypothesis is also “red blue”. The score becomes zero. That does not repair the segment assignment. It removes the boundary from the comparison. JiWER provides separate contiguous transformation options, while its ordinary processing aligns the corresponding transformed sentence pairs. [3][4] Choose the boundary policy according to the task. A transcript-wide word comparison may be useful, while a caption or segmented-recording evaluation can also need correct local placement. Preserve segment identifiers and pairing instead of assuming that concatenation is harmless housekeeping. This example concerns text boundaries only; it makes no measurement of timestamps, speaker attribution or caption delay.

Construct paired references [[red],[blue]] and hypotheses [[],[red,blue]]. Independent pair alignment gives one deletion in the first pair and one insertion plus one hit in the second. Therefore total E=2, N=2 and pooled WER=1. Concatenating reference tokens and hypothesis tokens before alignment yields [red,blue] on both sides, so the concatenated score is zero. The difference is caused entirely by the allowed alignment boundary.

JiWER 4.0.0's standard processing iterates over corresponding transformed sentence pairs, while its contiguous pipelines include ReduceToSingleSentence [3][4]. Those are distinct scoring preparations. Removing a boundary permits matching across it; it does not demonstrate that the original segment outputs were correctly assigned. Retain segment pairing and the concatenation policy when comparing systems. If timing, speaker ownership or local caption placement matters, define an appropriate evaluation for that requirement rather than assuming a global word edit count covers it. Our constructed record demonstrates only a boundary-sensitive edit result. It does not implement a diarization or latency metric, and it does not establish which boundary policy is universally best. The correct policy depends on the question the evaluation was designed to answer.

Use the five-word reference “do not open the gate”. One hypothesis is “do open the gate”, which deletes “not”. Another is “do not open gate”, which deletes “the”. Each has one deletion over five reference words, so each receives 20 percent WER under our rules. Yet the first changes the instruction's direction, while the second retains its prohibition with less natural grammar. The equal counts do not make the two transcripts equally useful.

This follows from what the metric counts. The basic word-edit score assigns the same unit cost to either deletion in our example; it does not read the sentence and assign a larger penalty to a word carrying negation. Looking at the alignment gives the reviewer information that the total has compressed away. Keep WER for its stated role as a reproducible token-error summary, then examine task-relevant errors when they matter. Names, numbers, punctuation or specific commands may warrant separate checks depending on the application. Our example makes that limitation visible without defining a universal severity scale. A score can be mathematically correct and still be insufficient for deciding whether a transcript communicates the intended message.

For the original reference tokens [do,not,open,the,gate], deleting not or deleting the gives E=1 and N=5 under unit edit costs. Both hypotheses therefore have WER=0.2. The first removes negation; the second removes an article. This is a direct counterexample to treating equal word-edit rates as equal semantic consequence. The counting model does not encode a task-specific cost for losing negation.

The conclusion is about the metric's information boundary. An alignment preserves lexical error identity, whereas its scalar edit total discards much of that detail. Additional measures or review criteria can be defined for an application's important information, but they require their own clear targets and validation. Do not silently change edit costs and continue comparing the result as if it were the same ordinary WER. Likewise, normalized text can conceal formatting differences that remain relevant to reading or interpretation. Preserve raw output and diagnostic alignments alongside a summary. This original example neither measures a model's semantic understanding nor ranks specific commercial recognizers. It establishes why a lower token-edit summary is one piece of evidence, and why equal summaries can leave meaningful differences between candidate transcripts unresolved.

A useful WER record starts with the paired reference and hypothesis text. Add the tokenization and normalization rules, segment boundaries, scorer version and handling of empty references. Keep the matching-word, substitution, deletion and insertion counts, the reference-word total, and the method used to combine utterances. These details explain how a reported number was produced and make later changes easier to assess. The retained implementation sources show why defaults and edge cases are part of that recipe. [2][3][4]

Our examples now explain several results that can initially look surprising. Four inserted words over a two-word reference gives 200 percent. The same written strings can move from 100 percent to zero after an explicit formatting transformation. Two utterances can produce a 27.78 percent mean of individual rates and a 10 percent pooled rate. Removing a segment boundary can hide misplaced words, and two equal deletion counts can alter meaning differently. None of those observations requires a model benchmark. They are consequences of declared scoring choices and original text pairs. When you encounter a speech-recognition score, reconstruct those choices first. Then compare like-for-like evaluations and inspect the errors relevant to the task the transcript must serve.

Retain original paired strings, tokenization, ordered transforms, segment policy, scorer identity and empty-reference convention. Store operation counts H,S,D,I, the reference denominator and aggregation weights. For each comparison, identify which parts of this protocol stayed fixed. The primary JiWER source separates preprocessing, per-pair alignment, count accumulation and final metric calculation [3][4]; those stages should remain distinguishable in an evaluation record.

The exact constructed results are WER=2 for the insertion example, 1 versus 0 for the raw/normalized formatting pair, 5/18 versus 1/10 for macro and pooled utterance aggregation, and 1 versus 0 for paired versus concatenated segments. The negation and article-deletion examples both score 1/5 while differing in meaning. These examples establish arithmetic and interpretive boundaries rather than empirical system quality. A WER improvement becomes interpretable when the underlying task, data and scoring protocol are identified; changing the denominator or preprocessing is not automatically a recognition gain. Use the metric to summarize the declared token comparison, retain enough evidence to reproduce it, and assess additional requirements with measures suited to those requirements. That gives the score a precise role without asking it to certify every useful property of a transcript.