The Quality Loop Scorecard

// THE GATE - does anything check the work?

session 1

01 An automated check runs before human eyes. Something checks your agent's output before a person sees it. Or are you the entire QA department? 02 The pipeline can actually reject work. Rejected means the pipeline sends it back with specific findings - not logs a warning and waves it through anyway. 03 The work gets a rival pass. A rival, separate from the maker - different prompt at minimum, different model ideally - whose only job is to find what is wrong. As Addy Osmani puts it, a model is too nice grading its own homework. 04 Deterministic checks run first. Schemas, tests, banned patterns - before any LLM judge gets an opinion. Free, instant, and they never hallucinate a finding.

// MEASUREMENT - would you know if it got worse?

session 2

05 Fixtures exist. Saved inputs with known-good and known-bad outputs, for the prompts that matter. 06 Prompt edits are measured. Pass rates before and after - not shipped because the new prompt feels better in your head. 07 Production misses get pinned. A bad output that slipped through becomes a permanent fixture, so no future edit quietly brings it back. 08 Fixes ship with regression proof. A prompt is a balloon - push in here, bulge out there. The fix comes with proof it did not break what worked.

// BOUNDS - can it spin forever?

session 3

09 Rework is capped. A maximum number of reject-and-rework rounds, then escalation to a human. Otherwise the loop argues with itself on your API bill. 10 A budget can kill a run. A cost ceiling stops the loop no matter what the models still want to try. 11 No-progress means escalate. The same finding twice in a row triggers escalation - not another round against the same wall. 12 Every run ends in an explicit outcome. Pass, scrap, or hold - declared. Not just… stopping.

// SIDE EFFECTS & HUMANS - what happens when it acts?

sessions 1 & 3

13 Side effects are guarded. Publishing, committing, billed calls: intent log + idempotency key. A retry cannot double-publish. 14 Human approval is durable state. A hold the pipeline cannot pass - enforced by the system, not a sentence in a prompt.

// MEMORY - does the loop learn?

session 4

15 Findings survive across runs. The loop remembers what it caught last week - it does not re-fight the same defects forever.

// YOUR VERDICT

0–4

You are the loop.

Every check lives in your head. Honest at pair scale - but you can't scale you.

5–9

Team, no gates.

Agents produce faster than you check. That's a defect amplifier on a delay timer.

10–13

Team with gates.

The loop exists. Now it's a tuning problem - prompts and caps, measured not vibed.

14–15

Fleet-ready.

You've got the prerequisite. Scaling to many agents multiplies whatever this score is - for you, that's good news.

// GET THE FIX LIST

Every "no" above has a smallest working fix - a pattern, a steal-this prompt or config sketch, and an honest install cost. Drop your email and the full Fix List for all 15 checks lands in your inbox, including the rival prompt template and a deterministic grader starter. One send with the goods in it - no drip sequence sitting between you and the fixes.

Everyone's racing to run more agents. Your score doesn't improve when you parallelize it - it compounds. Quality first, then scale.

You've got agents shipping work.
Score the loop that checks it.