Evals and Golden Sets: Notes From Building My First Eval Harness

Why evals exist at all

LLM systems fail silently. You change one line in a prompt to fix case A, and cases B and C quietly break. Nothing throws a compiler error or a stack trace. The output still looks plausible. You find out three weeks later, usually from a user.

Traditional code has unit tests. Prompts without evals have vibes.

So the definition I’ve landed on:

We use evals to measure an AI system’s quality against a fixed standard, so that (1) we can improve deliberately, changing the prompt, model, or tools and keeping only what raises the score, and (2) we can catch regressions immediately when a change breaks something that used to work.

Improvement and regression-catching. Both jobs matter. Most people only think about the first one.

The golden set

A golden set is a set of pairs:

(fixed input, your definition of correct/good for that input)

Both halves matter.

Fixed inputs are what make comparison valid. The same cases run on every eval. If run #1 used different inputs than run #2, you can’t tell whether the score moved because your prompt improved or because the questions got easier. Fixed inputs are the controlled variable.

The definition of correct is the answer key, and each input carries its own. The form varies per case, it can be an exact expected answer, a reference example, a constraint checklist, or a rubric. Without it, the grader is judging blind.

Two concrete golden sets from my own builds:

• LinkedIn writing engine — golden set is 14 of my existing posts plus my definition of what makes them work, hand-ranked into TOP, MIDDLE, and BOTTOM buckets. More on this one below, it’s the deepest example I have.
• “Boring code” skill (keep generated code simple, obvious, junior-readable) — golden set is ~5 coding tasks, each paired with what boring code looks like for that task: plain function, no clever one-liners, obvious variable names, no unnecessary abstraction.

Same structure both times, completely different answer keys. The structure is what carries over from project to project.

The eval loop: Run → Grade → Decide

An eval is bigger than the golden set or the grader alone. It’s the whole repeatable loop.

1. Run. Fixed inputs go through the thing under test, a prompt, a model, an agent, a toolset. Outputs come out. Note: the thing under test can be any of these. Swap the model and re-run; swap the prompt and re-run; the harness doesn’t care.

2. Grade. Each output is scored individually, paired with its own answer key from the golden set. The grader doesn’t have to be an LLM. A code check counts, and so does an LLM judge, or both together. Line counts, banned-phrase regexes, and unit tests are graders too, and they’re cheaper and more reliable than a judge.

3. Aggregate + Compare + Decide. Per-output scores collapse into one number (average, pass rate). That number gets compared against previous runs:

• Score improved → keep the change
• Score fell → revert

Then feed the decision back into the prompt or system and run again. Without that feedback arrow, it’s a one-time quality check. With it, it’s an improvement engine.

Per-output scores stay valuable even after aggregation, “v2 improved overall but regressed on case 7” is how you debug. A single blended grade hides that.

Types of evals (organized by what the grader compares against)

The cleanest taxonomy I’ve found: classify evals by what the answer key is, because that determines what your golden set must contain.

1. Exact-match / ground-truth. Golden set: input → single correct answer. “Extract the invoice total” → ₹4,250. Grader is pure code. Cheapest and most reliable, use it wherever a hard right answer exists.

2. Rule / constraint-based. Golden set: input → checkable constraint list. No single correct answer, but clear rules: function under 20 lines, no nested ternaries, no banned phrases. Grader is code, linters, AST checks, regex. My boring-code skill lives mostly here.

3. Reference-based (similarity). Golden set: input → a known-good reference output. Grader measures closeness, usually an LLM judge, because “similar voice” isn’t string-matchable. My LinkedIn engine’s labeled posts are this type.

4. Rubric-based (LLM-as-judge). Golden set: input → rubric dimensions with score anchors. No reference output; the rubric is the definition of good. The subtlety: rubrics need anchors (what does a 2 vs a 4 look like, with examples), otherwise the judge drifts.

5. Pairwise / preference. Golden set: input → two outputs + a human label for which is better. More reliable than absolute scoring, LLM judges are better at “which is better” than “is this a 7 or an 8.”

6. Human evals. Same inputs, but the grader is a person. Slow and expensive, but it’s the ground truth that calibrates everything else.

Real systems mix types. My writing skill (“engaging, non-AI-sounding text”) is 2 + 3 + 4:

• Type 2 handles “non-AI-sounding” — AI text has detectable fingerprints you can ban with code: “delve”, “in today’s fast-paced world”, “it’s not just X, it’s Y”, suspiciously uniform sentence lengths. Cheap, deterministic gate.
• Type 3 handles voice, does it sound like my reference posts?
• Type 4 handles “engaging”, which is neither a constraint nor similarity. A post can match my voice and still be boring. That needs a rubric.

The layering rule: cheap code graders run first as a gate; only survivors go to the LLM judge. Never pay for a judge call on an output a regex could have rejected.

One recursive problem with types 3-5: who evals the judge? Answer: hold out a few cases you’ve scored by hand and check the judge agrees with you. My hand-labeled posts do double duty, golden set for the content engine, calibration set for the judge. The rest of this article is that calibration, in detail.

Golden set in practice: labels.md

Here is the golden set from the LinkedIn writing engine, the one referenced above, in full. It lives in a private repo, not this one, so the table below is reproduced directly so this article doesn’t depend on a file you can’t see.

golden-set/labels.md is a single markdown table ranking 14 of my own past LinkedIn posts into TOP, MIDDLE, or BOTTOM, with two extra columns: a one-line reason and the real engagement numbers. Filled by hand, no AI. This is the actual file:

Building it, and living with it through a judge calibration cycle, taught me three things a description of “golden set” never would have.

The label is judgment, not metrics. Engagement is recorded but explicitly demoted to a tiebreaker between adjacent buckets. The table proves why this matters: the highest engagement-rate post in the whole set, prompt vs. context vs. harness engineering at 4.9%, is only MIDDLE, while the TOP anchor, API gateway vs. application gateway, runs at just 1.1%. If engagement rate had been the label, the ranking would invert. Raw metrics and taste disagree, and the golden set exists to encode the taste.

The one-line reasons are diagnoses, not descriptions. They name the causal failure or success: “image is clear, but the message in image is not easy to understand” (AI agent evals, BOTTOM), “people want to remember it hence bookmarked, curious topic” (API gateway vs. application gateway, TOP), “lacks depth and actionable insights that adds value” (loop engineering, BOTTOM). These lines start as notes to myself, then later become the judge’s training signal.

Near-duplicate pairs teach granularity. The table has three posts on essentially the same topic, an AKS observability stack: the Azure-native version and the open-source version both land MIDDLE with the identical reason, “showcases observability stack, advance topic.” The hybrid version lands TOP, and its reason differs by exactly one clause: “better diagram.” Same topic, same depth, one variable changed, one bucket jump. That single-variable contrast is worth more to a judge than ten unrelated examples, because it isolates what actually moved a post between buckets.

The hard rules

These are the non-negotiables the system runs on, roughly in order of importance.

Author, judge, and blinding, enforced mechanically. Claude Code writes the caption and picks the media; Codex runs the judge. The author model never scores its own drafts, this is the same failure mode named in the FAQ above, “a model agreeing with itself,” except here the pipeline enforces it, before either model gets a chance to grade its own work. The judge never sees labels.md either. A small script reads the hand-filled labels, constructs only cross-bucket pairs (a TOP vs. TOP pair carries no agreement signal, so it’s invalid by rule and gets replaced), and passes the judge nothing but folder names. The session running the evaluation cannot leak a bucket it was never shown.

1. The golden set is built by hand, no AI in the loop. Collection, bucketing, and reasons are all manual. AI only enters afterward, to write the prompt that gets calibrated against the hand labels.
2. Judge change control. Any edit to the judge prompt, however small, invalidates the freeze and re-triggers the full agreement eval before that judge touches a real draft. “Looks small” is not an exemption.
3. Anchors are excluded from the math. Judge v3 embeds four labeled posts directly in the prompt as calibration examples. Those four are banned from every agreement measurement, because you cannot test a judge on its own training examples. Only held-out posts count.
4. Four pass thresholds, all required before freeze: at least 80% pairwise agreement with the hand-labeled buckets; at least 6 of 8 scoring dimensions must clear an absolute sanity floor; at most 3 of 15 verdicts may flip when the A/B slots are swapped (position bias); no dimension may swing more than 2 points across 3 repeated runs of the same post (consistency).
5. Disagreements are adjudicated in both directions. Every judge-vs-operator disagreement gets one line stating whether the rubric was wrong or the original label was wrong. The protocol forbids defaulting to blaming the rubric. The golden set stays just as open to correction as the judge does.
6. Engagement writes back. Seven or more days after posting, real performance is recorded and correlated against the judge’s score, closing the loop between predicted and actual quality.

Judge calibration: v1 to v3, and a named failure mode

The eval-stages post in this same writing series is proof that a plain prompt is a valid unit under test. The judge prompt itself went through the full run-grade-decide loop, with the inputs and label definitions held constant across versions:

v2 is the interesting failure. It scored worse than v1 despite being a revision meant to improve it, because its scores saturated: nearly every post landed in the same narrow band, so the numbers carried almost no separating power. This is a distinct failure mode from low agreement. A judge can look internally consistent, giving similar scores to similar-looking posts, while still being useless, because “similar-looking” and “actually good” are not the same thing. v3’s fix was structural: score posts relative to the embedded anchors instead of on an absolute scale, which gave the judge something to calibrate against.

The full v3 pass line: 12/15 pairwise agreement, 7/8 absolute sanity, 0 position-swap flips, 0 consistency spread across repeated runs. All four thresholds cleared before the judge was frozen for real use.

What the separation buys you, in two real verdicts

Two eval.json files from this system show what author/judge separation buys in practice, beyond the architecture diagram.

For the eval-stages post itself, the Codex judge placed its own candidates at MIDDLE, with the reason that the media had “too many boxes and arrows to match the TOP anchor’s instant contrast.” A self-judging author would not write that sentence about its own diagram. It takes an evaluator with no stake in the draft to say the visual is the problem.

For the golden-set-by-hand post, both candidates landed at TOP, and the pairwise verdict picked the more focused candidate over the more instructional one, noting that the loser “widened into a less target-specific example.” Both files also record that the A/B order was randomized and carried no signal, the position-bias rule from the pass thresholds, visible at the level of a single verdict, well before it ever shows up in an aggregate score.

The golden set is doing double duty here. It’s the ground truth the content engine writes toward, and separately, it’s the calibration set that tells you whether the judge grading that content can be trusted. So “who evals the judge” turns out to have a small, unglamorous answer: a hand-checked set you’re willing to argue with, held by someone willing to say the judge got it wrong.

What can be evaled

Anything with a definable input → output boundary. In an agentic solution that gives you a stack, with evals at each layer:

Prompt / skill. Everything above. A SKILL.md is a prompt artifact; it evals the same way.

Tool. Two distinct evals people conflate. First, the tool itself: does search_docs(query) return the right docs? Plain unit testing, often no LLM. Second, the interesting one, tool selection: given a task, does the agent pick the right tool with the right arguments? Golden set: task → expected tool call(s). Grader: code check on tool name and args. Wrong tool choice is a top agent failure mode, which makes this one of the highest-value evals.

Single agent. A new distinction appears: outcome vs. trajectory. Outcome: did the final state end up correct? Trajectory: was the path sensible, did it reach for the right tools in the right order without looping or burning wasted calls? Trajectory golden sets include an expected or acceptable step sequence, beyond just a final answer.

Multi-agent / agentic system. End-to-end evals (whole task in, final result judged) plus per-agent component evals. The operating rule: end-to-end evals detect, component evals diagnose. When e2e drops from 80% to 65%, layer scores tell you which component broke. Without them, a multi-agent failure is undebuggable.

The shape across all layers is a pyramid, same economics as the unit/integration/e2e test pyramid: many cheap deterministic evals at the bottom, fewer LLM-judged component evals in the middle, a small set of expensive end-to-end evals on top.

(Sections on my actual single-agent and multi-agent eval builds will land here as I ship them.)

Pitfalls I’m watching for (before they bite)

Score saturation, distinct from low agreement. A judge whose scores crowd into a narrow band can look consistent while carrying no real signal. It’s a different disease from a judge that simply disagrees with your labels, and it needs a different fix (relative, anchor-based scoring rather than more prompt tweaking). See the v1-v3 table above for the version where this actually happened to me.

Judge noise and drift. Run the same LLM judge on the same output five times, get different scores. Mitigations: pin the judge model version, low temperature, prefer pairwise over absolute, periodically re-check the judge against hand-labeled cases. Upgrading the judge model shifts all historical scores, re-baseline.

Variance vs. signal. With a 15-case golden set and non-deterministic outputs, 72% → 75% can be pure noise. Run each eval 3-5 times and look at the spread. Don’t celebrate improvements smaller than your run-to-run variance.

Overfitting to the golden set. Iterate a prompt against the same 15 cases for weeks and it gets great at those 15 and worse in general. Fix: a held-out set you evaluate rarely, plus rotating fresh cases in.

Golden sets are living artifacts. Every interesting production failure should become a new case. A golden set that never grows only tests yesterday’s failure modes.

Cost and latency budgets. An eval that costs $4 and 20 minutes per run gets skipped, and a skipped eval is worthless. This is the pyramid argument again.

Negative cases. Golden sets skew toward “what good looks like.” Also include cases where correct behavior is refusal or escalation, the agent should say “I can’t do this.” Agents that ace your eval but confidently do wrong things in production usually have this gap.

Tooling: build first, adopt later

Names worth knowing (verify currency, this space moves fast): promptfoo (config-driven, open source, prompt/skill-level), Braintrust / LangSmith (commercial platforms: runs, tracing, score history, datasets), Langfuse (open-source, self-hostable alternative), DeepEval (pytest-style library), Ragas (RAG-specific metrics), Inspect (UK AISI’s framework, strong on agentic/trajectory evals).

My approach: hand-roll the first harness, then adopt a tool. A minimal harness is genuinely small, a JSONL of golden cases, a loop that runs the system, graders as Python functions, scores to a results file, a diff against the last run. Maybe 150 lines. Building it teaches you what the platforms abstract, so later you can say “here’s what LangSmith does under the hood and here’s when I’d pay for it” instead of “I used LangSmith.”

What to avoid: adopting a heavy platform before running a single eval (you’ll configure instead of learn); BLEU/ROUGE for tasks they don’t fit (they measure translation/summarization overlap, near-useless for judging engagement or code quality); trusting any tool’s built-in “hallucination score” as ground truth without checking it against your own judgment on your own data.

What’s next

My build order, each step updating this article:

1. Hand-rolled harness for the boring-code skill (types 1+2, all code graders)
2. Add an LLM judge for the writing skill (types 3+4, with judge calibration) — done, this is the labels.md section above
3. Trajectory + outcome evals inside my agent runtime
4. Multi-agent component evals
5. Port one harness to promptfoo/Langfuse and write up the comparison

Changelog

• 2026-07 — Added the labels.md deep dive: the real 14-row table (reproduced here so this article doesn’t depend on a file outside this repo), the hard rules the judging system enforces (with a blind-judging-pipeline diagram), the v1-to-v3 judge calibration table with score saturation as a named failure mode, and two real eval.json verdicts showing what author/judge separation buys in practice.
• 2026-07 — Initial version: why evals exist, golden sets, the run-grade-decide loop, eval taxonomy, systems stack, pitfalls, tooling stance.