How to maintain software quality in an agentic world

The cost of producing software has fallen sharply in the last two years. The cost of being answerable for it hasn't moved at all.

The gap between those two costs is the problem. Most teams adopting agentic development still run a quality process designed for the old ratio, where writing the code was the expensive part and reviewing it was cheap by comparison. Invert the ratio and the process stops working. There's now more output than anyone can read carefully, and careful reading was the control.

We build our own product site, Kalebtec Websites, this way. Below is the machinery we run on that repository, and the reasoning behind each piece. None of it is exotic. Most of it is older than the tooling it now guards against.

Plausible is the failure mode

The popular version of what goes wrong is misleading. Agents don't typically produce code that's obviously broken. Obviously broken code is the easy case: it fails a test or a build, and someone notices in ten seconds.

What they produce is code that's plausible. It compiles, it reads well, it uses your naming conventions, and it's subtly wrong about something a reviewer would have to already know to catch. The same is true of prose. Ask an agent for a page about your guarantee and it'll write a confident, fluent paragraph about a guarantee one degree more generous than the one you can honour.

Human review is a poor defence against plausibility, and it degrades as volume rises. So the control has to move. You constrain the environment that produces the output, and you make the things you care about mechanically checkable.

One: the rules live in the repository

Our operating rules live in a single AGENTS.md at the repository root: commit format, the design-token rule, kebab-case filenames, server-components-by-default, the accessibility floor, the list of commands an agent may not run without authorisation. Every harness reads that one file. The tool-specific config files are pointers to it and never duplicate it.

It sounds like housekeeping. If your rules live in one vendor's configuration format, your standards only hold for the people using that vendor this month, and a rule you changed six months ago is still being followed in three other places. Worse, an agent can't follow a rule that lives only in a senior engineer's head.

The useful test: could a competent stranger, with no access to your team, produce work that passes review by reading only what's in the repository? If not, whatever they're missing is currently held up by your attention alone.

Two: anything you care about has to be a gate

Written rules are necessary and not enough. An agent follows a written rule most of the time, which means the rule doesn't hold.

So the standards that matter are machinery. On this codebase that means:

• A local gate that must be green before any push: ESLint, tsc --noEmit, Prettier, and a full production build.
• Accessibility scanned by axe on every route in the sweep, at two viewport widths and in both colour schemes, with any serious or critical violation failing the build outright.
• A targeted contrast guard on top of the full-page scan, scoped to the elements whose contrast we previously repaired. It asserts that the rule reached each element and that the element passes. A general scan can pass while one specific fix is undone.
• Behavioural tests for what a static check cannot see: the mobile drawer keeping focus inside across twelve tab presses and restoring it on Escape, the consent banner firing no analytics request before a choice is made.
• CI that runs with every environment variable unset. No Stripe, no analytics, no backoffice, no auth. The application is built to no-op without them, and a test asserts that it does, so no gate can ever depend on a live third party.

One detail matters more than it looks. A git hook enforces our consultation requirement, and we chose that on purpose. A git hook binds every agent, every harness and every human who commits to the repository. A hook in one CLI binds whoever happens to be using that CLI. As more of your contributors stop being people, that covers less and less of the work.

When our gate was a string constant

Our own history has an example.

We had a blog post about WCAG 3.0 that wasn't cleared to publish. It lived in a source file that nothing imported, and the file exported a string constant named publishGate whose value explained the conditions that had to be met first. Everyone involved understood the arrangement, and it held.

It held because everyone understood it. Nothing enforced it. The moment anyone imported that file, whether an agent tidying up unused modules or a person wiring up the blog index, the post would have gone live. The string explaining why it shouldn't have would have shipped alongside it as dead code.

That post now lives in a content collection with a holdReason field. The collection's beforeChange hook refuses to publish any document that has one on file, whatever the status field says, so the condition now travels with the post. It's still unpublished, for the reasons below.

Three: the repository is the memory

An agent starts every session with no recollection of why anything is the way it is. Left to itself, it'll re-derive a decision you made deliberately six months ago and land on the other side of it, with a well-argued rationale. The reasoning that ruled the alternative out isn't anywhere it can read.

We keep architecture decision records for exactly this: one decision per file, amended in place, currently over 200 of them. The test for whether something is an ADR is a single question. Would a reader six months from now need it to avoid re-deriving a choice, or to avoid contradicting something we have published?

What keeps this usable is knowing what isn't an ADR: bug fixes, merge narratives, which gates went green, test-failure triage, defects fixed in passing. Git already holds all of it, in a form that can't drift. Ours replaced an append-only log that had reached 9,923 lines of decisions mixed with merge narrative, and nobody could find anything in it.

The second test, when the first is close: does it constrain future work? A fix doesn't. A rule does.

Four: catalogue what you have promised

Most engineering teams don't have this layer at all. For us it turned out to matter most.

Agents write marketing copy now. Marketing copy contains promises. In Spain, where we're domiciled, a promise made to a consumer in commercial communication binds. A garantía comercial binds the trader on the terms in which it was offered, including its advertising, and where the guarantee document is less favourable than the advertising, the advertising wins. A promise you can't honour is a misleading practice, actionable without any customer having to prove loss.

So every customer-facing promise on our site is catalogued in one register: guarantees, refunds, prices and price locks, delivery windows, response times, every claim containing the words forever, unlimited, any time or never, ownership and export promises, and quality and accessibility claims. Adding a promise means updating the register in the same cycle, checking it against the promises already there for conflicts, and confirming a mechanism to honour it exists.

The last check matters most. If we publish a promise with no way to deliver it, we're exposed from the day it goes live.

Here's what the register caught. Our blog draft asserted that every site we build is built to WCAG 2.2 AA. Our own accessibility statement, written in a different session for a different purpose, described this site as partially conformant. Neither sentence was a lie and no individual step was careless. They couldn't both be true in the way a customer would read them, and nothing in an ordinary review process was going to put the two paragraphs side by side.

The register did. The post is held until a WCAG 2.2 AA check exists inside our client delivery process and the two claims are reconciled. Holding it costs us one post. Publishing it would have committed us to something we have not yet built the machinery to keep.

Make the escape hatch legible

Every gate needs a way past it, or people route around the gate itself and you lose the signal. The question is what the bypass costs.

Ours takes a reason. Skipping the decision-record requirement means setting an environment variable to a sentence explaining why this change isn't a decision, and that sentence is printed into the commit output where a reviewer sees it. The hook can't verify that anyone thought. It verifies the sentence that thinking leaves behind, and it makes every skip visible.

We chose a sentence over a boolean because a boolean flag gets set once and never unset.

What this costs

It's slower at the start. Writing down a standard you've been carrying implicitly is real work, and the first pass over an existing codebase will find contradictions you'd rather not have found.

It also has a failure mode of its own. Gates that fire spuriously get disabled, and a disabled gate protects nothing while still appearing on the list of things you do. Ours are deliberately small and deterministic so they don't fire spuriously.

In exchange you get checks that scale. The gates run at machine speed on every change, whoever or whatever made it, and they don't get tired at five on a Friday afternoon. Human attention goes to the decisions. Those were always the part worth paying for, and none of this tooling has come close to replacing them.

Agentic development raises the bar for engineering rigour. You can now generate more plausible work in an afternoon than a careful team could review in a week, so review can no longer be the control.

The site this describes is Kalebtec Websites, our productised website offer: published prices, a published scope, and a published accessibility audit that we failed parts of and printed anyway. It's built under every rule above, which is why we're comfortable writing them down in public.