Spec-Driven Agentic Engineering for Engineers

Every structural engineer already knows how to trust work they cannot fully re-check.

You don't re-derive every equation in the code. You don't hand-verify every number a finite element package returns. You don't re-run the concrete supplier's cylinder breaks. What you do instead is bound the uncertainty: you know roughly how wrong each input can be, you know your margins, you check the answer against a hand calc or a sanity limit, and you get a second set of eyes on the parts that matter. Trust, in engineering, is not faith. It is a system for catching the failures you can't personally inspect.

AI-assisted development has the same problem in a different costume. A language model writes code that compiles, passes lint, and looks right. Sometimes it is right. Sometimes it is confidently, quietly wrong, in a way that looks exactly like the times it was right. If you accept its output on vibes, you will ship the wrong thing eventually, and you won't know which time it was.

Most people respond to this by either trusting the model too much or refusing to use it at all. Engineers have a third option, because we already own the discipline for exactly this situation. The habits transfer. Here is how I applied them to a tool I now rely on every day, ETABuddy, and what each engineering habit turned into once it hit software.

Write the spec before the prompt

The first mistake is prompting before you know what "correct" means.

In design, nobody sizes a member before they know the loads, the code, and the governing case. The spec comes first. It is not bureaucracy, it is the thing that lets you tell later whether the answer is any good. Software with an AI in the loop is no different, and it is more tempting to skip, because the model will happily produce something plausible from a vague ask.

For ETABuddy, the spec was concrete before I wrote a single prompt. The tool diagnoses ETABS warnings. So: what is a correct diagnosis? It names the warning, states the likely root cause, and gives a fix an engineer can act on. What are the inputs? Real warning text, in the messy formats ETABS actually emits. What is out of scope? Anything that requires opening the model file. Writing that down changed what I built. A vague "help with ETABS warnings" would have wandered. A spec gave every later decision something to be measured against.

The rule is the same one you already follow: if you can't say what correct looks like, you are not ready to start.

Build a golden set

Once you know what correct looks like, you write down the cases where you know the answer.

Every engineer keeps a mental library of check problems: the cantilever whose tip deflection you can do in your head, the simple beam whose moment you know cold. When a new analysis method or a new tool shows up, you run it against the case you already know before you trust it on the case you don't.

That is a golden evaluation set. For ETABuddy I collected real warnings and wrote the known-correct diagnosis for each one by hand: the warning, the cause, the fix I would give a colleague. Not many at first, a couple dozen, but real ones, chosen because I already knew the answers. Every change to the tool got run against that set. If a "clever" improvement to the search made three golden cases worse, it wasn't an improvement, no matter how good the idea sounded.

This is the single highest-leverage habit, and it is the one non-engineers skip most often. Without a golden set you are tuning on feel, and feel is exactly what the model is best at fooling. With one, every change produces a number, and you argue with the number instead of with your own optimism.

Get an independent review, and make it blind

You do not check your own drawings and call it reviewed. Someone who didn't do the design looks at it, precisely because they don't share your blind spots.

The trap in AI-assisted work is grading the output with the same model, or the same person, that produced it. It agrees with itself. So I made the evaluation independent: a separate agent, given the golden answer and the tool's answer, asked to judge whether they match, with no stake in the result and no memory of having produced it. A blind third party.

That one move caught things I would have waved through. When the author and the reviewer are the same process, "close enough" wins. When the reviewer is blind to who wrote the answer and is only comparing against the known-correct case, close enough stops being good enough. It is the software version of handing your calcs to the engineer across the hall who has no reason to be kind to them.

Route the work by how hard it is

You don't send every question to the senior principal, and you don't hand a bridge to a co-op student. You match the reviewer to the difficulty of the problem. That is triage, and it is a design decision, not a slight.

Models come at different capability and different cost. A fast, cheap model is fine for the easy, high-volume steps: normalizing a warning, deciding which bucket a question falls in. The expensive, capable model earns its cost on the hard step: the actual diagnosis, where getting it wrong is the whole failure. Routing by difficulty means the tool spends capability where it changes the answer and saves it where it doesn't.

The engineering instinct here is exactly right. Effort is a budget. You spend it on the governing case. ETABuddy runs the same way: cheap models do the sorting, the capable model does the judgment, and the budget cap on the whole thing means a diagnosis can't silently run up a bill. A diagnostic tool with no cost ceiling is a tool you stop trusting the first surprising month.

Let the search run overnight

Some things you don't solve by thinking harder. You solve them by running the case a few hundred times and reading the results in the morning.

The hybrid search in ETABuddy has parameters: how much weight keyword matching gets versus embedding similarity, how the two scores get normalized so they're comparable, how results get fused. There is no first-principles value for these. You could guess, or you could measure. So I let it optimize overnight: try combinations, score each against the golden set, keep what wins. I woke up to a configuration I would not have guessed and could not have argued my way to, backed by numbers on cases I already trusted.

This is not exotic. It is a parametric study, the same thing you'd run to find the wall thickness that governs across a range of loads. The computer is patient and you are not. Set up the sweep, define the objective, and let it grind while you sleep. The only prerequisite is the golden set, again, because an overnight search without a trustworthy objective just finds the fastest way to fool your metric.

Where it still bites

None of this makes the model correct. It makes the wrongness catchable, which is a different and more honest claim.

The golden set only covers the cases I thought to include. A warning I've never seen, in a format I didn't anticipate, can still draw a confident wrong answer, and the blind evaluator can only judge against cases it has. So the set grows. Every time ETABuddy surprises me, the surprise becomes a new golden case, the same way a near miss on a project becomes a new line on your personal checklist. The system is not a proof of correctness. It is a ratchet that makes the tool a little harder to fool each time it fools me.

That reframing matters. Engineers don't claim their structures cannot fail. They claim the failure modes are understood and bounded. A tool built this way earns the same modest, defensible claim: not that it is always right, but that when it is wrong I will probably catch it, and the next version will be wrong less often. That is the most you can honestly promise about work you cannot fully inspect, and it is enough to build on.

Why the habits transfer

None of this is a software methodology I learned and applied to engineering. It is the engineering methodology I already had, applied to software.

Specs before work. Known cases before unknown ones. Independent review that doesn't share your blind spots. Effort spent where it governs. Parametric studies when intuition runs out. Every one of these is something a competent engineer already does, for the same reason: because the cost of being confidently wrong is high, and because you cannot personally inspect everything you are responsible for. That is the exact situation AI-assisted development puts you in, and it is the situation our profession is built around.

The people struggling with AI tools right now are mostly missing the discipline, not the technology. They have a powerful, plausible, occasionally-wrong assistant and no system for catching the "occasionally." Engineers have that system. We have had it for a century. The tools are new. The judgment is not.

ETABuddy is not impressive because it uses a language model. Plenty of things use language models. It is reliable because I pointed a hundred years of engineering discipline at it. That is the transferable part, and it is the part worth writing down.