Guide / AI & Automation
Give the Agent a Definition of Done, Not a Vibe
Make repository tasks verifiable with acceptance criteria, focused context, meaningful tests, visual review and clear boundaries for a production release.
“Finish the site without breaking anything” expresses a reasonable intention. It leaves a coding agent with several unanswered engineering questions: which repository is authoritative, which behavior is approved, what counts as a regression, and whether completion includes deployment. A capable agent can investigate, but investigation is more reliable when the desired outcome has observable boundaries.
The useful work is to turn that intention into a small acceptance contract. It should identify the result, the constraints that matter and the evidence needed to call the task complete. It should leave routine implementation choices to the agent while making consequential assumptions visible.
An agent can optimize for completion only when completion is observable.
Describe the result in the product
For a change to this publication, a concrete outcome might be that the archive lists every public article from oldest to newest, with the same global number used on topic pages. That describes behavior a reviewer can inspect. “Improve chronology” does not specify whether the problem is sorting, dates, numbering or presentation.
Add a counterexample. A draft must not shift the numbers of public articles. Two articles with equal publication timestamps must retain deterministic ordering. These cases clarify the intended model without prescribing every line of code.
Avoid defining success only as a file modification. Asking the agent to edit an archive component can be useful orientation, but the defect may originate in a shared helper. The product behavior should remain the acceptance boundary even if investigation identifies a different implementation location.
OpenAI’s Codex modernization example uses explicit validation scenarios and parity evidence to make a substantial code change assessable. The general lesson applies to a much smaller task: specify how the result will be checked before treating a completed edit as a completed change.
Point to sources of truth
A repository often contains several descriptions of the same system: current code, older planning documents, tests, generated output and deployment notes. They do not all have equal authority. An agent needs to distinguish a historical proposal from behavior that is intentionally running today.
For this site, the publication gate and editorial helpers define visibility and numbering. Content frontmatter supplies dates and taxonomy. The deployment workflow defines the release path. An old PRD can explain intent, but it should not silently override the current implementation when the owner requests a surgical revision.
Useful context names these files and explains what they govern. It also identifies protected boundaries: existing article copy, approved page hierarchy, Research PDFs and the exact Search Console verification file. This reduces the chance that a well-intended cleanup removes something whose purpose is not obvious from its filename.
Keep context specific. A long list of generic instructions to be careful adds less value than one statement that provider-owned files must survive deployment. The latter can become an assertion; the former cannot.
Establish a baseline before editing
An agent should know whether it started from a clean tree and which revision it inspected. Existing modifications may belong to the user. A failing test may predate the task. Without a baseline, the final report can mistakenly attribute old problems or unrelated changes to the current work.
For a content expansion, record existing slugs, dates and file hashes. That makes preservation testable. For a visual revision, capture the relevant page at representative widths. For a behavior change, reproduce the failure with a meaningful example before changing the implementation.
The baseline need not become permanent product code. Temporary evidence can live in an ignored working directory. Durable regression checks should remain in the repository when they protect an important contract.
This approach also helps after an interrupted session. A concise record of the starting state, completed changes and remaining checks is more useful than asking a new session to infer progress from a collection of modified files.
Ask tests to prove the boundary
Tests should detect the failure the task is intended to prevent. A test that repeats the implementation’s sorting expression may confirm that two copies of the same mistake agree. A stronger check asserts the visible order against an independently defined fixture, including ties and excluded entries.
Use different layers for different questions. Unit tests can exercise publication boundaries. Build checks can inspect sitemap coverage. Browser tests can verify visible numbering, focus behavior and overflow. No single layer establishes all three.
A green test suite is evidence within its coverage. It does not establish that a remote host applied every header or that a deployment transferred the expected files. Those claims need checks at the relevant boundary.
Completion evidence should come from the place where the promise is made.
For a deployment task, that place includes the public site. For a local-only task, it may be the running application and a reviewable diff. Define the scope explicitly so the agent neither stops too early nor performs an external action the owner did not authorize.
Include visual checks for visual work
A screenshot can reveal a cropped face, an overflowing heading or a broken hierarchy that a text assertion misses. The agent should inspect the actual rendered page at widths relevant to the design, including a narrow mobile viewport when the content is public-facing.
Visual review should also include interaction. A table of contents may look correct while its active state stops updating. A PDF reader may render its first page while keyboard controls or cleanup fail. A navigation label can appear clear in a screenshot yet have an ambiguous accessible name.
Preserve console errors and failed requests as evidence. They often explain a visual defect more directly than speculative CSS changes. When the page depends on lazy loading, trigger the interaction that loads the dependency rather than assuming a successful initial page visit covers it.
The AI coding article explores this broader distinction between plausible implementation and demonstrated behavior. The useful agent workflow includes both implementation and examination of its consequences.
Review the diff as a separate activity
After the tests pass, inspect what changed. Look for unrelated formatting, generated files, credentials, dependency churn and deleted assets. These can remain hidden inside a successful build because the build only answers whether the resulting project works under its checks.
A deletion deserves particular care. Search imports, dynamic references, generated pages, tests and documentation. A file with no direct import may still be served from a public path or used as a fixture. Absence from one text search is insufficient evidence of obsolescence.
The agent should be able to explain each group of changes in terms of the requested result. If a refactor has no connection to the task or a verified defect, it probably belongs in another change. Small, coherent diffs make both review and recovery easier.
This does not mean every task needs a ceremonial review process. The depth should match the consequence. Replacing a static asset requires less analysis than changing remote deletion rules, but both benefit from checking that the diff matches the intent.
Make release constraints executable
For this site, release constraints include deploying only built output, preserving provider-owned files and reading credentials from GitHub Secrets. Those rules belong in scripts and workflow configuration, where a violation can stop the release. Repeating them in a prompt is useful context but weaker enforcement.
The definition of done should name the required pipeline stages and the final evidence: commit, push, successful CI, successful deployment and live checks when those actions are authorized. It should also distinguish an external blocker from unfinished local work.
If the provider’s proxy omits a response header, an agent with file-transfer access cannot honestly claim to have fixed the proxy configuration. It can complete the code work, demonstrate the remaining issue and identify the precise provider action needed. That is a bounded result, not a reason to fabricate success.
A good task handoff ends with what changed, how it was verified and any unresolved boundary. The owner should be able to inspect the result without reconstructing the agent’s entire conversation. That is what turns autonomous work into something another person can trust and maintain.