A storage result can show more completed operations per second without showing that each operation finishes sooner. That is not a contradiction. A system can keep several requests in progress and finish more of them overall, while the time each request spends in the system stays the same or increases. SNIA describes this latency-versus-throughput trade-off when discussing queue depth. NVMe is designed to support parallel command submission and completion, but that ability does not tell you how many independent requests your application has available. A large peak result therefore needs its workload conditions before it can explain a shorter wait in your own task. [1][5]

The useful question is what must finish before the application can continue. In an imagined task, each read might reveal where the next read should go. Another task might already know the addresses of many independent reads. Those are different workloads even if they use the same drive. The examples below deliberately separate a request’s completion time, the number of requests in progress, and the total bytes completed per second. They are calculated models, not tests of an SSD. The goal is to identify which measurement explains the work that must finish before the application can continue.

Completion rate and per-operation response time are distinct observables. A workload with more independent operations in flight can achieve greater aggregate throughput without reducing the latency of an individual operation. SNIA’s queue discussion explicitly identifies the environment-dependent trade-off between outstanding work, per-I/O latency, and throughput. NVMe provides parallel submission and completion paths, but protocol capability alone does not establish application concurrency, sustained media performance, or a particular latency distribution. A peak IOPS claim is consequently incomplete for an application-level decision unless the associated workload, concurrency, and measurement boundary are retained. [1][5]

Consider two hypothetical dependency structures. In one, an operation’s result determines the next address, limiting that stream to one outstanding request. In another, many addresses are already known and requests can overlap. Neither structure is a claim about a named application; they isolate why the same hardware may be exercised differently. This article develops closed-loop numerical models in which a completed request is immediately replaced, then distinguishes those assumptions from benchmark configuration and observed behavior. It does not infer controller channel count, endurance, cache durability, or real application speed from the arithmetic. Those would require separate evidence at the appropriate boundary.

IOPS counts input/output operations completed per second. Byte throughput counts how much data those operations transfer per second. A request’s latency measures time between stated starting and finishing events, while outstanding work counts requests that have started but have not completed at the chosen boundary. Microsoft’s DiskSpd results keep byte totals, bandwidth, operation counts, and latency statistics as separate outputs. That separation matters: one large transfer and one small transfer each count as an operation, but they do not carry the same amount of data. A result stated only in IOPS leaves the request size unanswered. [3][4]

Queue depth needs a similar label. A program can be configured to allow a certain number of outstanding requests, but the configured limit is not automatically a measurement of how many were present throughout the run. The operating system and device also have their own queues. Linux’s block-layer documentation describes software staging and hardware dispatch queues rather than one universal queue shared by every counter. When someone reports a depth of eight, ask where that number came from: a load-generator setting, an observed application count, or a particular lower-level queue. Those descriptions cannot be substituted for each other without checking their scope. [2][3]

IOPS is an operation-completion rate; bandwidth is a byte-completion rate. Latency is elapsed time across a defined operation boundary, while concurrency describes operations outstanding within a defined system boundary. DiskSpd reports per-thread, per-target byte, bandwidth, and IOPS statistics, with latency histograms collected separately when requested. These quantities are related but not interchangeable. If all counted operations transfer the same B bytes, multiplying their completion rate by B gives byte throughput at that same boundary. With mixed sizes, the appropriate total is completed bytes divided by duration, or an operation-weighted size combined with the matching operation rate. [3][4]

A configured outstanding-I/O limit is a control parameter rather than an observed time average. DiskSpd describes issuing operations up to its configured limit and replacing them as completions arrive; scheduling and processing still occur between those events. Linux separately documents software staging and hardware dispatch structures, demonstrating why a queue label must include its layer. The number of allowed commands, submitted commands, waiting requests, and commands currently executing are not automatically identical. Preserve that distinction before using a queue figure in a rate calculation or interpreting it as a direct count of physical flash operations. [2][3]

Change the imagined workload so four independent reads start together. Assume the system can complete all four after the same 200 microseconds and immediately start the next four. This grants enough parallel capacity to preserve each request’s completion time, a condition that a queue setting alone cannot establish. Every repeating 0.0002-second cycle now completes four operations, so the rate is 4/0.0002 = 20,000 IOPS. With the same 4 KiB size, the calculated throughput becomes 81.92 MB/s. Each request still waited 200 microseconds. The total increased fourfold, while the individual wait did not decrease.

That is a possible reason for a high-concurrency result to exceed a one-request result, but it is not a promise of fourfold scaling on a real drive. Requests share resources, and the extra load can change their completion times. SNIA explicitly describes that dependence on the storage environment. An application also needs independent work to overlap; changing the benchmark’s request count does not rewrite another application’s dependencies. Read the result as evidence about the tested combination of concurrency and workload. A separate measurement is needed before saying that one interactive operation, file-opening task, or dependent read became quicker. [1][5]

For the second hypothetical system, four independent requests are submitted simultaneously and each completes exactly 200 μs later. All four slots are refilled immediately and the cycle repeats. The completion rate is 4/0.0002 = 20,000 IOPS, with 4,096-byte requests yielding 81,920,000 B/s. The model’s fourfold throughput increase coexists with unchanged per-request response time. This is a constructive example of aggregate progress differing from individual latency. Preserving W requires enough resources under the increased concurrency. If shared resources delay completion instead, the denominator changes and the fourfold rate calculation must be revised.

Increasing a configured depth alone does not establish that invariant response time. SNIA notes that more outstanding operations can increase load on interconnects and storage media and therefore increase per-I/O latency. Application dependencies also bound the concurrency exposed to the stack. Consequently, comparing a low-depth run with a high-depth run answers how that workload behaves under different offered concurrency, not necessarily how quickly a dependent operation finishes. Evidence of improved application responsiveness would require a measurement of the relevant application boundary, with its actual dependency and scheduling behavior retained. Interface support for parallel work is a capability statement, not that measurement. [1][5]

For a third invented case, keep eight reads outstanding, but suppose each now takes 400 microseconds. Again, all eight finish together and are replaced immediately. Eight completions every 0.0004 seconds still give 20,000 IOPS and 81.92 MB/s for 4 KiB reads. Compared with the four-request case, twice as many operations are in progress and each waits twice as long, while total throughput is unchanged. The extra occupancy represents more unfinished work inside the measurement boundary, while the rate of finished work stays the same. Looking at both numbers reveals that trade-off immediately.

A comparison that celebrated the bigger queue number would miss the lack of extra completed work. A comparison that looked only at MB/s would miss the longer waits. SNIA’s general guidance describes why lower queue depth can trade some throughput for lower latency, depending on the environment. The sensible question is which result matches the task’s goal: a batch transfer may emphasize total completion, while a dependent request can be held up by its own delay. Even then, evaluate the actual distribution of delays and the full workload, because these deliberately constant-latency examples leave out variation. [5]

Now define a periodic closed-loop batch of eight requests, each with W = 400 μs and immediate replacement at completion. Its λ = 8/0.0004 = 20,000 IOPS matches the four-request, 200 μs case, as does its fixed-size byte throughput. The declared outstanding count doubles and response time doubles, so the quotient is unchanged. The arithmetic checks the relationship among the declared cycle, occupancy and completion rate. To identify whether the longer W came from queue residence, changed service behavior, or another stage, an investigation would need timing at those internal boundaries.

SNIA’s queue discussion supports the qualitative possibility of greater outstanding work increasing latency without a commensurate application benefit. The numerical comparison therefore illustrates why queue depth should be evaluated with both throughput and latency, rather than maximized as a standalone objective. Do not replace the model’s continuously occupied slots with a configured maximum and assume that the same quotient describes a real trace. An observed average, a limit, and a periodically sampled instantaneous count are different quantities. The workload’s dependency structure and latency distribution also determine whether equal aggregate throughput is acceptable for the application. [3][5]

Microsoft’s DiskSpd documentation gives a specific meaning to its outstanding-request option: the usual -o setting applies per thread and per target. Its -t option sets threads for each target. Imagine a configuration with one target, two threads per target, and eight outstanding requests per thread. Its configured request allowance is 1 × 2 × 8 = 16, not eight. With two targets and the same per-target settings, it becomes 32. These are configured allowances rather than measured queue occupancy. A target can be a file, drive, or partition, so two targets need not mean two physical SSDs. [3]

The documentation also describes fixed-thread and other outstanding-operation controls, so one shortcut does not cover every configuration. Even the number of completions processed together is controlled separately from the number of outstanding requests. This is why a result described only as “queue depth eight” can be ambiguous. Keep the actual tool version and full configuration with the result, including thread and target counts. DiskSpd’s XML output records system and test configuration for this purpose. That record makes it possible to understand what load was offered instead of assuming the label names one universal drive setting. [3][4]

In the documented DiskSpd interface, -o specifies outstanding I/Os per thread per target, while -t specifies per-target threads. A hypothetical one-target configuration with -t2 and -o8 therefore permits 16 outstanding operations across its two threads. Two targets under the same per-target controls permit 32 across four threads. These are configuration-derived counts, not assertions about instantaneous or time-averaged occupancy. Targets can be files, physical drives, or partitions, so target count does not establish physical-device count. The arithmetic applies to the stated option combination and must not be generalized across other ways of constructing threads and outstanding limits. [3]

DiskSpd also documents -F for fixed thread counts, -O for outstanding operations per fixed thread, and -oc for completion dequeue batching independently of the outstanding limit. Those controls demonstrate why an isolated depth number lacks sufficient scope. Preserve the tool version, target mapping, and complete configuration when interpreting an existing result; the option names describe different controls, even when their numerical values happen to match. Microsoft’s results documentation identifies system configuration and full test configuration in XML output. That provenance is necessary to distinguish an interface’s command capacity from the actual workload that generated a reported completion rate. [3][4]

An application’s outstanding work is not necessarily sitting in one device queue. Linux’s multi-queue block layer describes software staging queues, hardware dispatch queues, and requests held until hardware resources become available. It can also combine adjacent requests before dispatch. Those documented mechanisms show why operation counts can depend on where you look. If two higher-level pieces of work are combined lower down, a counter at one boundary need not agree one-for-one with a counter at another. Naming the layer gives that disagreement a concrete explanation to investigate. [2]

Queues and data buffers are different things as well. SNIA distinguishes command/response queues from buffers holding data in flight. A configured count of requests does not by itself tell you the number of bytes in a buffer or the amount of drive cache. Likewise, a queue does not guarantee that completions return in the same order as submissions: Linux explicitly says higher layers must handle completion ordering. For the reader comparing results, the consequence is practical. Preserve whether the reported quantity concerns application requests, block-layer requests, or device commands, rather than treating all three as interchangeable descriptions of a physical line of waiting reads. [2][5]

Linux blk-mq distinguishes software staging structures from hardware dispatch contexts. It can route eligible requests toward hardware, retain requests when resources are unavailable, and merge adjacent work at the staging layer. Its documentation also states that a block request may contain one or more BIOs. These transformations mean that event counts and queue populations must be interpreted at their own boundaries. A userspace operation counter, a block-layer request counter, and a controller-command counter cannot be assumed to describe identical populations. For the documented Linux path, the mapping between those populations is part of the interpretation of each counter. [2]

SNIA separately distinguishes queues containing command/response information from in-flight data buffers. Therefore queue depth is not directly a buffer-byte count or cache-capacity measurement. Linux also warns that completion order is not guaranteed by the block layer or device protocols and must be handled above them. The presence of an ordered submission list should not be interpreted as proof of serial execution or ordered completion. For analysis, retain the event definition, layer, aggregation scope, and time interval alongside each count. Without those details, apparent disagreement between tools may reflect different populations rather than a faulty device or an impossible performance result. [2][5]

Consider two invented sets of 1,000 completed requests. In the first set, every request takes 0.2 milliseconds. In the second, 990 requests take 0.1 milliseconds and ten take 10.1 milliseconds. Both sets have an average of 0.2 milliseconds: the second total is 990 × 0.1 + 10 × 10.1 = 200 milliseconds, divided by 1,000. Yet one percent of the second set waits much longer than any request in the first. These are constructed latency lists, separate from the earlier repeating-workload models. The sum of their response times is not necessarily the elapsed run time because operations can overlap.

Microsoft’s DiskSpd results can report average and percentile latency statistics rather than reducing the run to one mean. Its documentation also warns that high percentiles need enough completed operations to be meaningful: an extreme tail can otherwise depend on only a handful of observations. Keep the sample count, operation type, and duration beside the latency result. A read/write mixture should not be described as though its combined average proved identical behavior for both kinds of request. This does not create a universal acceptable latency target. It gives you enough context to judge whether the measurement answers the application’s actual concern. [4]

Two hypothetical 1,000-operation populations illustrate why the mean is insufficient. Population A has 1,000 latencies of 0.2 ms. Population B has 990 at 0.1 ms and ten at 10.1 ms. Both sum to 200 ms and therefore have a 0.2 ms arithmetic mean, although B contains a one-percent group at 10.1 ms. No percentile interpolation convention is assumed or reported for these lists. Their summed response times are not treated as wall-clock runtime, since concurrent operations can have overlapping durations. The second population concentrates its long waits in ten operations, information that the shared mean removes.

DiskSpd collects latency histograms per thread and target and can report per-operation-type averages and percentile summaries. Microsoft explicitly connects extreme-percentile interpretation to sample size and test duration, noting when there are too few observations to distinguish higher percentiles from the maximum. Consequently, a reported tail statistic should retain operation count, read/write scope, configuration, and duration. A useful comparison also explains its aggregation, rather than assuming that an overall average describes each target or operation class. This evidence can characterize the observed run, but it does not by itself establish a service guarantee or predict all future workload conditions. [4]