Guide / Software Engineering
“It Should Be Fast” Is Not a Requirement
Turn vague performance expectations into testable scenarios, with explicit workloads, measurement boundaries and targets that reflect real user needs.
The sentence “it should be fast” can survive months of planning because nobody has to disagree with it. Developers imagine a responsive interface. Operations imagine a nightly job finishing before work starts. A customer imagines getting an answer before losing their train of thought. The agreement disappears as soon as someone asks whether the delivered system passed.
A useful performance requirement connects an activity to a person, an environment and an observable result. It also admits what the team does not yet know. Precision means defining the question well enough to test it; adding an arbitrary millisecond threshold merely makes an ambiguous promise look scientific.
A requirement becomes useful when two people can agree on whether it passed.
Identify the waiting person
Start by describing a concrete situation. A catalogue manager filters products to correct missing identifiers. A reader opens an article on a phone. An overnight import prepares tomorrow’s inventory. All involve speed, but the waiting has different consequences.
The manager may repeat a filter dozens of times during one task, making interaction delay cumulative. The reader may abandon an initial load but tolerate a later background download. The overnight import may take considerable time without causing trouble, provided it completes before a known operational deadline.
Ask what happens when the system is slow. Does somebody wait without feedback, retry an operation, miss a deadline or perform duplicate work? That consequence helps choose the measurement boundary. Measuring a server response alone is insufficient when the user still waits for rendering, a worker or a dependent request.
Document the beginning and end of the observed interval. “From selecting the filter to the updated results becoming usable” is clearer than “query time.” It prevents one team from reporting database execution while another measures the complete interaction.
Turn adjectives into scenarios
The Software Engineering Institute’s work on quality attributes treats quality concerns as something to reason about explicitly in architecture. A practical application is to describe the stimulus, operating conditions, expected response and measurement rather than leaving an adjective attached to the whole product.
For example, a proposed catalogue scenario could specify a signed-in editor, a representative dataset, a particular filter and a defined device class. The result must become usable within a target agreed after observing the workflow. The target is a decision the team still needs to make, not a number borrowed from an unrelated application.
Use placeholders honestly during discovery. Writing “target pending baseline measurement and user review” is better than pretending a guessed figure is an established need. Give the unresolved decision an owner and a date so it does not become permanent ambiguity.
The same technique works for reliability and usability. “Easy to recover” can become a scenario involving an interrupted import, a visible failure report and a documented restart procedure. “Accessible” needs the relevant tasks, standards and verification methods, rather than a single unchecked checkbox in a specification.
Separate the target from the evidence
A target expresses an acceptable outcome. A measurement describes what happened under specified conditions. They should be recorded separately because they change for different reasons. Improving a benchmark does not justify moving the target, and revising the business need does not rewrite past results.
A small requirement record can include:
- The user task and the consequence of delay.
- The dataset, device and network assumptions.
- The measured interval and the tool used to observe it.
- The acceptance target and who approved it.
- The conditions that would require another measurement.
This is enough structure for many ordinary projects. A larger system may need separate load profiles, service objectives and capacity planning, but it still benefits from the same clarity about what is being promised.
Avoid hiding distribution behind one average. A typical response and an unusually slow response can have different operational importance. If a percentile is used, specify the sample, observation window and excluded cases. A percentile without those details can be as vague as “fast.”
Build a representative workload
A benchmark against ten clean records may prove that the implementation starts. It says little about a catalogue containing missing fields, long descriptions, many categories and uneven supplier batches. Performance tests need a workload that exercises the relevant path.
Representativeness does not require copying private production data. A synthetic dataset can preserve the shapes that matter: record counts, field lengths, relationship counts and distributions of exceptional cases. Document which properties it represents and which it omits.
For an import, include both accepted and rejected rows. Validation and reporting may dominate work when the input is messy. For search, include queries that return many results and queries that return none. For a page, measure a cold visit separately from a repeat visit if caching materially changes the experience.
These distinctions also make optimization more honest. A cached response can be excellent for repeat visitors while the first visit remains poor. Both observations can be true. Naming the scenario prevents an improvement in one from being presented as a universal improvement.
Measure the experience that the requirement actually promises.Negotiate when the evidence is inconvenient
Suppose the first measurement misses the proposed target. There are several possible responses: improve the implementation, reduce the workload, change the interaction or renegotiate the expectation. Choosing among them requires understanding the user’s underlying need.
A long operation may be acceptable with a useful progress state and safe cancellation. That does not make it faster, but it may solve the actual problem of uncertainty. Conversely, an animated spinner does not satisfy a deadline needed by a downstream business process.
This is where requirements and architecture decisions meet. An architectural choice constrains which quality targets are practical. The team should make that trade visible instead of quietly changing the benchmark until the preferred implementation passes.
Record the decision in terms of consequences. If the initial release supports smaller imports, state the supported limit and what happens above it. A clear rejection with a useful explanation is safer than an undocumented threshold that fails unpredictably.
Keep acceptance separate from diagnosis
An acceptance check should answer whether the agreed behavior holds. A diagnostic benchmark helps explain why it does or does not hold. Confusing the two can produce brittle tests that fail because an internal implementation changed even though the user experience improved.
For instance, a requirement may concern the time until a usable article appears. Diagnostics can separately measure HTML transfer, font loading and rendering. The diagnostic breakdown guides engineering work; the acceptance boundary remains tied to what the reader needs.
Likewise, do not require every developer laptop to reproduce a production benchmark exactly. Keep functional checks deterministic where possible, and run environment-sensitive measurements in a documented setting. Report uncertainty rather than converting noisy local timings into a strict claim about the deployed system.
A failed measurement should preserve enough context to investigate: revision, dataset version, environment and relevant output. Without those, teams often rerun the same test until it passes and mistake variability for a fix. The testing essay explores why the ability to detect meaningful failure matters more than a reassuring green result.
Leave a requirement someone can revisit
A useful requirement outlives its first implementation. When the catalogue doubles, the device mix changes or a new integration joins the workflow, its assumptions should be easy to find. Otherwise the team inherits a number without knowing which promise it represented.
Give significant quality requirements review triggers. These can be concrete changes rather than calendar ceremonies: a larger supported dataset, a new region, a different rendering strategy or a new dependency in the critical path. Re-measure when those conditions change.
The finished record should let a new colleague reproduce the check and explain its purpose. They should know what passed, under which conditions, and what the result does not establish. That is a stronger deliverable than a sentence everyone agrees with but nobody can test.