Reliable Software Starts Before the Code: Turning Vague Requirements into Things You Can Test

“Recover quickly” is an excellent way to let several people agree while imagining different systems.

One person means the dashboard opens again. Another means new measurements are arriving. A third means all missing historical records have been restored. Everyone can approve the requirement, and the disagreement only becomes visible during an outage.

I am increasingly suspicious of adjectives that make a system sound good without saying how anyone could tell. Reliable, fast, available and secure can express legitimate priorities. They are poor acceptance criteria on their own.

The useful work is turning the priority into a situation we can reproduce, an outcome we can observe and a boundary we can defend. That work changes implementation decisions. It also makes a failed test informative instead of merely disappointing.

Write the situation before choosing the number

Consider the forest-monitoring architecture exercise described in my architecture essay. Connectivity can disappear for 24 hours while devices continue collecting measurements. This is an academic example, not a report of a deployed service.

An initial requirement might say: after connectivity returns, accumulated measurements should be recovered within a defined window while normal acquisition continues.

That is better than “recover quickly.” It is still incomplete. Was the device powered throughout the outage? How much data was generated? Does the connection remain stable after returning? Does recovery refer to one device or an entire fleet reconnecting at once?

Write down the operating conditions before arguing about the window. Otherwise, two teams could run radically different tests and both claim to have validated the same requirement. The target might be identical; the workload would not be.

For a first test, I might assume one powered device, a fixed sampling rate, a known starting buffer state and a stable connection after restoration. Those are deliberately narrow conditions. Naming them lets someone challenge the simplification instead of discovering it in a test script months later.

Choose an observable finish line

A requirement becomes much more useful when it identifies who can observe success. The device knows that it sent a batch. The receiving service knows that it accepted a request. The application may know that the records are durable and queryable. These observations are related, but they are not interchangeable.

For this example, I would choose durable acceptance of every expected measurement as the recovery boundary. A missing sequence number should prevent the test from declaring success. A duplicate transmission should not create a duplicate measurement in the application’s view.

This is a proposed acceptance rule. It does not imply that a particular transport guarantees those properties. The implementation has to establish them, and the test has to look at the boundary where they matter.

The clock also needs a start event. “Within two hours” could begin when a cable is connected, when a device detects connectivity or when the first successful request reaches the receiver. I would measure from a recorded restoration event in the test harness and document any detection delay included in the result.

If the stopwatch starts at different moments, the same number does not describe the same promise.

Separate the requirement from its rationale

Here is an illustrative acceptance statement, with a two-hour target chosen for discussion:

Proposed acceptance statement: For one device collecting at the specified test rate, after a 24-hour network interruption with power maintained, all buffered measurements must be durably accepted within two hours of connection restoration, while acquisition continues at that rate.

This example still needs supporting definitions for “specified test rate” and “durably accepted.” Attach those definitions rather than keep expanding one sentence until nobody can read it.

The rationale belongs alongside the requirement. Why two hours? Which activity is harmed if recovery takes longer? Is it a contractual boundary, a user need, a resource constraint or simply an early assumption? A precise but arbitrary number can be just as misleading as an adjective.

NASA’s requirements checklist is useful here: it emphasizes clarity, verifiability, assumptions and intelligible rationale. I borrow those disciplines without pretending a small software project needs the full documentation apparatus of a space programme.

For an unresolved target, I would name an owner and the evidence needed to settle it. An explicit uncertainty with a next action is better than a confident value that nobody believes.

Design the test while the requirement is still cheap to change

Before building the recovery mechanism, I would sketch the test in plain language. Generate records with stable identifiers, disconnect the network, continue acquisition, restore connectivity, and compare expected records with durable records at the receiver. Continue generating new records throughout catch-up.

The expected dataset must exist independently of the mechanism being tested. Reading the sender’s final queue and concluding that an empty queue proves completeness would reuse the implementation’s own belief as the verdict. A queue can be empty because records were delivered or because they were discarded.

A small test ledger could record the following:

Decide what evidence a failure should leave behind. A single red result is frustrating when the problem could be insufficient bandwidth, lost records, duplicate handling or an incorrect timer. Preserving the observations makes the next engineering decision easier.

This is where testability starts influencing architecture. If we cannot observe durable acceptance, perhaps the interface lacks an acknowledgement we need. If generated identifiers cannot be reconciled, perhaps identity is underspecified. The test sketch exposes missing design work before the test suite exists.

Expand the scenario deliberately

Once the basic case passes, change one condition at a time. Interrupt connectivity again during recovery. Restart the device with unacknowledged records. Reduce useful throughput. Reconnect several devices together. Fill the buffer near its limit before the outage begins.

These are separate scenarios with potentially different acceptance criteria. Treating them as one giant “reliability test” makes it difficult to know what coverage actually exists.

A restart test, for example, adds a persistence question. A fleet reconnection test adds contention. A constrained-link test checks whether the recovery window remains feasible. Each has a reason to exist, and each may expose a different trade-off.

The SEI’s Quality Attribute Workshop approach uses stakeholder scenarios to identify and refine architectural concerns. The practical lesson I take is to involve the people who care about the outcome before the scenario becomes a technical fixture nobody feels able to question.

I would ask the consumer of the measurements what happens when a gap remains. Perhaps an explicit missing-data marker is acceptable for one analysis. Perhaps another use requires stopping a calculation. The answer changes how failure should appear, even when the storage implementation stays the same.

Numbers can hide ambiguity too

A response-time target such as “under 200 milliseconds” looks precise, but still needs a workload, measurement boundary and treatment of slow requests. Is it a maximum, a percentile or an average? Does it include network time? What happens during a dependency failure?

This is an illustrative example, not a recommended latency threshold. The number itself matters less than the agreement it represents. I would rather have a modest target with an honest test than an impressive target measured under conditions nobody encounters.

Reliability requirements have the same trap. “Zero data loss” needs a failure model and a scope. A test covering a network interruption while power remains available cannot establish behavior after physical destruction of the device. Naming that boundary is useful engineering, not an excuse for poor quality.

I also want the implementation and the requirement to evolve together. If a change increases sampling frequency, the old recovery test may still pass while representing a workload the product no longer uses. Trace the scenario to the relevant configuration, and review the relationship when either side changes.

Leave room for an honest failure

A good requirement gives the team permission to discover that its chosen design is insufficient.

If the target can be reinterpreted after every failed run, the test becomes a ceremony.

Record the agreed conditions, keep the result reproducible and distinguish a product decision to change the target from an engineering claim to have met it. Both can be reasonable. They should leave different evidence.

The most useful moment in requirements work is often the moment agreement breaks. Someone points out that “recovered” meant something else, or that the assumed bandwidth is unavailable. Finding that disagreement before implementation is progress. It is much cheaper than shipping two different interpretations behind one reassuring adjective.

Sources

1. NASA: How to Write a Good Requirement — checked 2026-10-11
2. SEI: Quality Attribute Workshops, Third Edition — checked 2026-10-11