What "Done" should mean for AI work

Every team has a definition of Done, written down or not. For most software it's some version of: code reviewed, tests pass, merged, shipped. When those things are true, the work is finished.

For AI features, all of that can be true while the feature is worse than it was yesterday.

Merged is not the same as working

Say an issue asks to make a support bot answer refund questions from the help center. An engineer improves the retrieval step and updates the prompt. The code is clean, reviewed and merged. The ticket moves to Done.

Did refund answers get better? Did shipping answers get worse? Is each answer now slower, or more expensive? The ticket can't tell you, because none of that is about the code. It's about the model's behavior, and that's measured somewhere else, if it's measured at all.

A better definition

For AI work, Done should mean:

1. The change is merged, like any other work.
2. The evals that cover it pass: the score is at or above the agreed baseline.
3. The slices that matter pass too, so an average doesn't hide a broken group of users.
4. Cost and speed stayed within budget.

None of this is new. Teams already argue about these numbers in reviews and retros. The difference is making them part of the definition, so nobody has to remember to check.

Making it stick

Definitions only work when the workflow enforces them. A checklist people tick by hand gets skipped on busy weeks, which is exactly when regressions slip in.

So the board itself should know. Work that's merged but not yet proven waits in its own column. When a passing eval run arrives, it moves to Done on its own. When a run fails, a bug appears with the evidence attached. People can still override it, with a written reason that stays in the history, because sometimes shipping is the right call.

That's what we built Evalban around: an Evaluating column between review and Done, and a definition of done that includes "its evals pass".

Read why we built it on the About page, or get started.