passmuster

Make LLM output pass muster. A tiny, model-agnostic verify-and-retry loop: generate → check → retry with feedback until the output passes — or you’re out of attempts. Zero runtime dependencies.

LLM output is often almost right — valid JSON but a missing field, a plausible answer that ignores the question, a plan with a TODO left in. passmuster lets you declare what “good” means as composable checks and makes the model keep trying — feeding each failure back so it can self-correct.

  attempt 1: FAIL -> schema
  attempt 2: FAIL -> no-todos, actionable
  attempt 3: PASS

  ok: true | usedAttempts: 3

Schema checks

Validate against any Standard Schema — Zod, Valibot, ArkType. No schema library is bundled.

Predicate checks

Any (async) function over the value. Return true to pass, or a reason to fail.

LLM-as-judge

Grade the output with a model — “did this answer the question?” — bring your own complete function.

Retry with feedback

Every failure is formatted and handed to the next attempt so the model can fix exactly what was wrong.

How it pairs with promptsize

promptsize measures your prompts at build time; passmuster verifies your model’s output at runtime. Two halves of keeping an LLM app honest.

Next steps

Quick start Wire a generate function and a few checks.

Checks schema, predicate, and LLM-judge checks in depth.

---

MIT · zero runtime deps ·   github.com/abdulmunimjemal/passmuster