Tagged “faithfulness”
-
Measuring a Prompt Change With Retrieval Held Still
Prompt edits only touch generation, so let retrieval vary and you measure noise. Freeze the contexts, compare paired, watch the judge's style bias.
-
Measuring Faithfulness, Not Just Answer Quality
An answer can be good and ungrounded at the same time. How to check every claim against the retrieved context, and where LLM judges fail.
-
Measuring Whether Your System Knows When to Refuse
Refusal is a classification problem with two error types. How to build the unanswerable set, score both cells, and catch fake refusals.
-
Validating an LLM Judge Before You Trust It
A judge is an instrument with unknown error until you measure it. The labelled probe set, the bias checks, and the re-validation triggers.
-
Answer Relevance Is Not Answer Correctness
An answer can be true, grounded, and still not answer the question. How to measure relevance separately, and why holistic scores miss it.