Verification and self-checking

Have the agent, or a separate checker, verify its work against the acceptance criteria before it commits an action or returns a result.

Check before you commit

A cheap way to raise reliability is to add a verification step: before the agent finalizes, it checks its own output against the requirements. Did it answer the actual question, does the result meet the constraints, is the action about to be taken the intended one.

A second set of eyes

Self-critique helps, but a separate verifier, a different model or a rule-based check, catches mistakes the original run is blind to. For high-stakes tasks, a dedicated checker that must approve the result before it ships is worth the extra call.

Verify against criteria, not vibes

Verification is only useful if it checks something concrete. Give the checker explicit acceptance criteria so "looks good" becomes "meets these conditions," which is what makes the check repeatable.

pythonimport json

RUBRIC = ("Score the answer from 0 to 5 on correctness, completeness, and "
          "whether it follows the format. Return JSON: "
          '{"score": <int 0-5>, "reason": "<short>"}.')

def judge(question, answer):
    resp = client.chat.completions.create(
        model="gpt-4o-2024-08-06",
        temperature=0,
        response_format={"type": "json_object"},
        messages=[
            {"role": "system", "content": RUBRIC},
            {"role": "user", "content": f"Q: {question}\nA: {answer}"},
        ],
    )
    return json.loads(resp.choices[0].message.content)["score"]  # 0..5